In educational research, data rarely behaves perfectly. Students’ attitude scores are often skewed, classroom sizes are small, and assessment tools frequently produce ordinal rather than interval data. When these realities collide with the assumptions of parametric tests – normality, equal variances, interval-level measurement – researchers need a reliable fallback. That’s precisely where non-parametric tests for related samples step in. Two of the most widely used options are the Sign Test and the Wilcoxon Matched-Pairs Signed-Ranks Test. Both are designed specifically for situations involving paired or matched observations, and neither requires the data to follow a normal distribution.
Table of Contents
- What are related samples?
- Why parametric assumptions often fail in educational settings
- The Sign Test
- Step-by-step procedure
- Strengths and limitations of the Sign Test
- The Wilcoxon Matched-Pairs Signed-Ranks Test
- Step-by-step procedure
- Educational application
- Sign Test vs. Wilcoxon: which one to use?
- Assumptions summary and conditions for use
- Conducting these tests in SPSS
- A note on statistical power and sample size
What are related samples?
Before diving into the tests themselves, it’s important to clarify what “related samples” means in research. Related (or dependent) samples are data points that are connected – most commonly because the same participants are measured twice (a pre-test and post-test design), or because participants have been deliberately matched based on characteristics like age, ability level, or prior performance. In educational research, this setup is extremely common. A teacher might measure student comprehension before and after a new instructional intervention. A researcher might compare scores of matched pairs of students from two different schools. The paired nature of the data is what defines a related-samples design, and it’s this very pairing that requires special statistical treatment.
Parametric tests like the paired t-test are the standard choice for such designs – but they come with conditions. They require the differences between pairs to be normally distributed and the data to be at least interval-level. When these conditions cannot be met – which happens regularly with small samples, ordinal scales, or non-normal distributions – non-parametric tests provide a flexible and valid alternative.
Why parametric assumptions often fail in educational settings
Educational data frequently violates the assumptions required for parametric testing. Likert-scale survey responses, attitude ratings, teacher effectiveness scores, and rank-based assessments are all ordinal – they have order but unequal intervals. Small class sizes mean sample sizes are often too small to reliably confirm normality. And in intervention studies, student performance distributions often show ceiling or floor effects, producing skewed data.
For nominal or ordinal scale measurements, the general rule is to use non-parametric statistics. Additionally, when outliers are present – as is common with diverse student populations – parametric means can be distorted, while non-parametric tests based on ranks or signs remain robust. Recognising when to step away from parametric assumptions is a core skill in sound educational research design.
The Sign Test
The Sign Test is one of the simplest non-parametric procedures available. It works by examining only the direction of change between paired observations – not the size of the change. For each pair, you calculate the difference (Post – Pre). If the result is positive, it gets a “+” sign; if negative, a “−” sign; and if zero (a tie), it is excluded from the analysis. The test then counts the number of positive and negative signs and uses the binomial distribution to determine whether the pattern of signs differs significantly from what would be expected by chance.
The null hypothesis of the Sign Test is that the median difference between paired observations is zero – meaning positive and negative differences are equally likely, each with a probability of 0.5. The sign test can also be used for dependent samples when the assumptions for a paired t-test are not met.
Step-by-step procedure
To carry out the Sign Test for related samples, follow these steps. First, compute the difference for each pair (Condition 1 minus Condition 2). Second, assign a positive or negative sign based on whether the difference is above or below zero; discard any tied pairs. Third, count the total number of non-tied pairs (this is your sample size n), and separately count the number of positive and negative signs. Fourth, for small samples (n ≤ 25), use the binomial distribution to find the critical value. For larger samples, the test statistic approximates a standard normal (Z) distribution. Finally, compare your test statistic to the critical value: if it falls in the rejection region, the null hypothesis is rejected, indicating a significant median difference between the two conditions.
Consider a concrete educational example: A researcher tests whether a peer-tutoring programme improves student scores on a reading comprehension task. Twenty students are tested before and after the programme. For 14 students, post-programme scores are higher (+), for 4 students they are lower (−), and 2 students show no change (ties, excluded). With n = 18 valid pairs and 14 positive signs, the researcher applies the binomial test to determine whether this preponderance of positive signs is statistically significant.
Strengths and limitations of the Sign Test
The Sign Test’s greatest strength is its simplicity and minimal assumptions. It does not assume a normal distribution of the data, making it ideal for ordinal data or continuous data that do not meet parametric assumptions. It requires only that pairs be dependent and that the differences have a continuous distribution. This makes it extremely versatile – applicable even when data is only ranked or when the researcher cannot assume symmetry in the distribution of differences.
However, this simplicity comes at a cost. The sign test does not detect the magnitude of the difference between the hypothesized value and is not as efficient as the t-test. By ignoring how large or small the differences are, the Sign Test discards potentially useful information, making it less statistically powerful. In practice, this means it is more likely to miss a real effect – particularly in small samples. Generally, Wilcoxon’s signed-ranks test is used instead of the sign test when the conditions for the t-test are not met since not only the signs are taken into account but also the ranking of the data, thereby giving more accurate results.
The Wilcoxon Matched-Pairs Signed-Ranks Test
Developed by Frank Wilcoxon in 1945, the Wilcoxon Matched-Pairs Signed-Ranks Test takes the Sign Test a step further. Rather than only recording whether a difference is positive or negative, it also accounts for the magnitude of the difference by ranking the absolute values of all differences. This means a large positive change and a small positive change are treated differently – the larger change receives a higher rank. The result is a considerably more powerful test than the Sign Test, while still avoiding the normality assumption required by the paired t-test.
Step-by-step procedure
The Wilcoxon test involves the following steps. First, compute the difference for each pair and exclude any pairs with a difference of zero. Second, rank all the non-zero differences by their absolute value, from smallest (rank 1) to largest. When ties occur, assign the average of the tied ranks. Third, restore the original sign (positive or negative) to each rank. Fourth, separately sum the positive ranks (W+) and the negative ranks (W−). The test statistic W is the smaller of these two sums. Fifth, compare W to a critical value from the Wilcoxon table (for small samples), or convert it to a Z-score for larger samples (n > 10). If the observed W is less than or equal to the critical value, the null hypothesis is rejected.
The test statistic for the Wilcoxon Signed Rank Test is W, defined as the smaller of W+ (sum of the positive ranks) and W- (sum of the negative ranks). If there is no systematic difference between the two conditions, we would expect W+ and W− to be roughly equal. A very small W indicates a lopsided distribution of signed ranks – strong evidence of a consistent direction of change.
Educational application
Take a study examining whether a flipped classroom model improves student performance on a mathematics assessment. Twenty students complete a pre-test, experience the flipped classroom intervention for a semester, then take a post-test. Because the scores are recorded on an ordinal rubric and the sample size is modest, normality cannot be assumed. The Wilcoxon test is applied. Differences between pre- and post-test scores are calculated, ranked by magnitude, and signed accordingly. If the sum of positive ranks (W+) vastly outweighs the sum of negative ranks (W−), the test indicates a statistically significant improvement attributable to the intervention.
This is directly comparable to the literacy example frequently cited in research: a research team assessing the effectiveness of a novel teaching method on improving children’s literacy levels can use pre- and post-intervention scores measured on an ordinal scale and apply the Wilcoxon test to determine whether the new method has a statistically significant effect.
Sign Test vs. Wilcoxon: which one to use?
Both tests are appropriate for related samples without parametric assumptions, but they are suited to different situations. The Sign Test should be used when you can only determine the direction of change between pairs (not rank the magnitude), when the data is purely ordinal with no meaningful numerical distance, or when the distribution of differences is not symmetric. If the distribution of differences can be shown to be not symmetric, then the sign test should be used.
The Wilcoxon test is the preferred option in most other cases. The Wilcoxon signed-rank test is more powerful than the sign test because it makes use of the magnitudes of the differences rather than just their signs. It requires that the distribution of differences be symmetric around a central value – an assumption that is generally reasonable for many educational datasets. When this condition holds, the Wilcoxon test is more likely to detect a true effect, making it the more efficient of the two.
A practical rule of thumb: if you can rank the differences meaningfully and have reason to believe the distribution of differences is roughly symmetric, use the Wilcoxon test. If you can only determine direction, or if the distribution is markedly asymmetric, use the Sign Test.
Assumptions summary and conditions for use
Both tests share the following core conditions: the data must come from related (dependent) samples; observations within each pair must be independent of other pairs; and the measurement must be at least ordinal. Beyond these, the Wilcoxon test adds the requirement that the distribution of pairwise differences be symmetric. Neither test requires normality of the original data, which is the central reason they are valuable in educational research contexts where such an assumption is frequently untenable.
It is also worth noting that parametric tests usually have more statistical power than their non-parametric equivalents – meaning one is more likely to detect significant differences when they truly exist with parametric tests. Non-parametric tests are therefore not a universal preference but a principled choice when parametric assumptions are genuinely violated.
Conducting these tests in SPSS
Both the Sign Test and the Wilcoxon Matched-Pairs Signed-Ranks Test are straightforwardly available in SPSS. Navigate to Nonparametric Tests and then Related Samples from the drop-down menu, add the paired variables to the test pairs list, and select the appropriate test type. SPSS output will include the test statistic, Z-value, and p-value. A p-value below 0.05 indicates that the null hypothesis of no median difference should be rejected at the conventional significance level, pointing to a statistically significant difference between the two related conditions.
For reporting purposes, the output typically shows the number of positive, negative, and tied differences (for the Sign Test), or the sums of positive and negative ranks with the W statistic and corresponding Z and p-values (for the Wilcoxon test). These results should be interpreted in context: statistical significance tells you whether a difference is unlikely to be due to chance, but the researcher must also evaluate the practical or educational significance of the finding.
A note on statistical power and sample size
A common concern with non-parametric tests is their reduced statistical power compared to parametric alternatives when all parametric assumptions hold. The wider applicability and increased robustness of non-parametric tests comes at a cost: a larger sample size may be required to draw conclusions with the same degree of confidence. In educational research, this is worth planning for at the design stage. If your sample size is very small and you already know parametric assumptions are violated, you are in a doubly challenging position. Collecting a reasonably sized sample – even 15 to 20 paired observations – substantially improves the test’s ability to detect real effects.
The Wilcoxon test has a particular advantage here: with a sample size of at least ten paired observations, the distribution of the test statistic approximates a normal distribution, allowing researchers to use a Z-score for significance testing. This makes it practical for most classroom or small-group studies common in educational settings.
What do you think? If you were designing a study to evaluate the impact of a new teaching strategy on student performance, under what conditions would you choose the Wilcoxon Matched-Pairs Signed-Ranks Test over the Sign Test – and would a small sample size change your decision? Consider also: how might the choice of measurement scale (ordinal vs. interval) in your data collection instruments influence which non-parametric test is most appropriate for your analysis?
References
- https://ogs.edu/when-to-use-parametric-versus-nonparametric-procedures-in-statistics-for-social-research/
- https://builtin.com/data-science/parametric-vs-nonparametric
- https://distancelearning.institute/research/statistical-tests-parametric-non-parametric/
- https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Mostly_Harmless_Statistics_(Webb)/13:_Nonparametric_Tests/13.02:_Sign_Test
- https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/sign-test/
- https://real-statistics.com/non-parametric-tests/sign-test/
- https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test
- https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/wilcoxon-sign-test/
- https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_nonparametric/BS704_Nonparametric6.html
- https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/how-to-conduct-the-wilcox-sign-test/
- https://www.sciencedirect.com/topics/mathematics/wilcoxon-signed-rank-test
- https://www.healthknowledge.org.uk/public-health-textbook/research-methods/1b-statistical-methods/parametric-nonparametric-tests
- https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/the-wilcoxon-sign-test-in-spss/
- https://en.wikipedia.org/wiki/Nonparametric_statistics
Leave a Reply