<< ST_deandixon Outlier-Tests ST_grubbs >>
samplestat >> samplestat > Outlier-Tests > ST_esd

ST_esd

Generalized Extreme Studentized Deviate test according to Rosner

Syntax

[outlierfree, outlier] = ST_esd(v, p, maxoutliers)
[outlierfree, outlier] = ST_esd(v, p, maxoutliers, side)
outlierfree = ST_esd(v, p, maxoutliers)
outlierfree = ST_esd(v, p, maxoutliers, side)

Arguments

v:

real vector of numerical sample values; at least three values are required

p:

statistical confidence level as a string or significance level alpha as a decimal value "95%", "99%", "99.9%", 0.05, 0.01 or 0.001

maxoutliers:

positive integer specifying the maximum number of suspected outliers; it must not exceed n-2

side:

test direction, "both" (default), "left" or "right"

outlierfree:

input vector with all detected outliers removed; unchanged if the test does not identify an outlier

outlier:

detected outliers in their original input order; [] if no outlier is detected

Description

ST_esd performs Rosner's generalized Extreme Studentized Deviate (ESD) test for one or more outliers. Unlike repeatedly applying the classical Grubbs test, the generalized ESD procedure evaluates a sequence of test statistics and matching critical values while controlling the overall significance level for up to maxoutliers suspected observations.

The procedure first removes the most extreme remaining observation and calculates Ri and λi for i = 1,...,maxoutliers. The number of outliers is then the largest index k for which Rk > λk. The first k observations removed during the sequence are classified as outliers. Observations removed only for calculating later test statistics are restored when their index is greater than k.

The test assumes that the non-outlying observations are independent and approximately normally distributed. A minimum sample size of n >= 3 is required. Rosner's critical-value approximation is most reliable for moderate and large samples; test power and calibration are restricted for very small samples. maxoutliers should be chosen before inspecting the results and should normally be small relative to the sample size.

Test directions: "both" removes the observation with the largest absolute deviation from the current sample mean. The critical probability uses α/(2*ni). This is Rosner's standard two-sided procedure. "left" removes only the current minimum. The critical probability uses α/ni. "right" removes only the current maximum. The critical probability uses α/ni.

The sample standard deviation is calculated with denominator ni-1. If all remaining values are identical at any step, no additional test statistic can be calculated and the sequence stops.

Test statistic at step i

\begin{eqnarray} n_i &=& n-i+1 \\ R_i &=& \frac{\max_j |x_j-\overline{x}_i|}{s_i} \quad\text{for a two-sided test} \\ R_i &=& \frac{\overline{x}_i-x_{min,i}}{s_i} \quad\text{for a left-sided test} \\ R_i &=& \frac{x_{max,i}-\overline{x}_i}{s_i} \quad\text{for a right-sided test} \end{eqnarray}

Critical value at step i

\begin{eqnarray} \lambda_i &=& \frac{(n_i-1)t_i}{\sqrt{(n_i-2+t_i^2)n_i}} \\ t_i &=& t_{1-\alpha/(2n_i),\,n_i-2} \quad \text{for a two-sided test} \\ t_i &=& t_{1-\alpha/n_i,\,n_i-2} \quad \text{for a one-sided test} \\ \text{with} \\ x_i &:& \text{test value} \\ n &:& \text{number of values} \\ s &:& \text{sample standard deviation} \\ \bar{x} &:& \text{arithmetic mean} \\ x_{max} &:& \text{max. value} \\ x_{min} &:& \text{min value} \\ t &:& \text{student factor} \\ \alpha &:& \text{statistical confidence level} \end{eqnarray}

Decision rule

Let k be the largest index i for which Ri > λi. The first k observations removed during the calculation sequence are classified as outliers. Equality does not lead to rejection. If no such index exists, no outlier is returned.

Examples

data = [0.4827129 0.3431706 -0.4127328 0.3843994 ..
-0.7107495 -0.2547306 0.0290803 0.1386087 ..
-0.7698385 1.0743628 1.0945652 0.4365680 ..
-0.5913411 -0.7426987 1.609719 0.8079680 ..
-2.1700554 -9.7361261 0.0069708 14.626386 ..
-2.5036545 -2.9046385];

// Test for up to three two-sided outliers at 95% confidence
[of, o] = ST_esd(data, "95%", 3)
// Expected outliers in original input order: [-9.7361261 14.626386]

// Test for up to three low outliers at 99% confidence
[of, o] = ST_esd(data, "99%", 3, "left")

// Test for up to two high outliers at 99.9% confidence
[of, o] = ST_esd(data, 0.001, 2, "right")

See also

Authors

Bibliography

Rosner, B. (1983). Percentage Points for a Generalized ESD Many-Outlier Procedure. Technometrics, 25(2), 165-172.

NIST/SEMATECH e-Handbook of Statistical Methods, Generalized ESD Test for Outliers.


Report an issue
<< ST_deandixon Outlier-Tests ST_grubbs >>