Statistical tests can help us to decide whether or not the results we have obtained could reasonably have occurred by chance alone.   Only if we can exclude chance as an explanation for our results, can we properly conclude, with reasonable certainty, that something interesting has happened.

For instance, suppose we find that fish sometimes prefer to school with individuals of the same species and sometimes with those of another species.   Say, in a total of N trials, they prefer one species X times and a second species Y times.

Some devil's advocate examining our results might argue that X trials out of N could have gone one way or the other by chance alone, even if the fish had no preference at all for schooling with one species rather than the other.

The devil's advocate's position is called the null hypothesis in statistics.   It is the hypothesis that chance could explain the results and consequently that any alternative hypothesis, such as a preference for a certain stimulus, is unnecessary.   Should we reject this null hypothesis or accept it?

The devil's advocate's argument that chance alone could explain the results is easily illustrated by tossing a coin.   Even if the coin we tossed was fair (nothing caused it to fall one way more than the other), we would probably not get equal numbers of heads and tails.   Of course, a small inequality of heads and tails would not make us reject the fairness of the coin.   A large difference, however, would make us unwilling to accept the coin as a fair one.   In other words, if the difference were large enough, chance alone would not seem reasonable as an explanation of the results.

How large a difference is enough?   An example can show how we make a decision to accept or to reject the hypothesis that a coin is fair.   Flip a coin 10 times.   Suppose the coin lands heads up 3 times and heads down 7 times.   To reach a decision about the coin's fairness, we need to know the chance of getting this result with a fair coin.   If the chance of getting 3 heads in 10 trials is small, then we could reasonably reject the hypothesis that the coin is fair.   Instead, we could reasonably conclude that something influenced the behavior of the coin.

The Sign Test lets us determine this probability of having X trials or comparisons go one way and Y go the other way by chance alone.   If this probability turns out to be small (scientists usually accept 5% as a small probability in such cases), then we can feel reasonably confident that our results could not have occurred by chance alone.   So we would reject the null hypothesis and instead tentatively accept our alternative hypothesis.   In the case of our coin tossing, we would accept the hypothesis that we had an unfair coin.   In the case of the schooling fish, we would accept the hypothesis that our subjects do in fact prefer to school with one species rather than another.

Notice that the Sign Test makes use of paired observations:   two observations or measurements made under the same conditions (the same subject, or two similar ones, in the same circumstances, as far as possible), except for one known difference.   Examples include (1) measurement of the same subject's responses to two different stimuli, (2) the winner of an encounter between two animals similar in all respects except that one is larger or is in its home cage, and (3) the number of times a particular display is performed by the dominant and subordinate animals in an encounter.

The Sign Test takes account of the direction of the difference between paired observations in an experiment or comparison.   The magnitude of the difference is ignored.

The magnitudes of the differences between paired observations can be important.   For instance, suppose we measure the number of minutes that a fish spent nearest each of the two stimuli, and we wanted to know whether the fish had different preferences for these two stimuli.   The measurements of time spent near each stimulus constitute paired observations in this experiment.   We could focus merely on the direction of the difference by determining which stimulus the fish spent more time near.   Suppose the fish spent more time near stimulus A in X trials and more time near stimulus B in Y trials.   Could we reasonably conclude that X results went one way and Y went the other way by chance alone? We could use the Sign Test to find out.

In this example, though, in addition to knowing the direction of the difference for each paired observation (in other words, which stimulus had the larger score), we also know the magnitude of the difference.   We know the difference between our measurements of the number of minutes spent nearest each stimulus (minutes nearest stimulus A minus minutes nearest stimulus B).

Suppose all the cases that went one way were small differences (the fish spent only slightly more time nearest stimulus B, say), but the cases that went the other way were often large differences (the fish spent much more time nearest stimulus A).   This situation might well arise if fish with strong preferences provided more predictable results than did those with weak preferences.   When the preference was strong, the fish always spent more time near stimulus A; when the preference was weak, the result was less predictable.   It makes sense that we should give the larger differences more emphasis in deciding whether or not chance alone can reasonably explain our results.

When we know the magnitude, as well as the direction, of differences between paired observations, then the Wilcoxon Test provides a better test of the null hypothesis than does the Sign Test.

Key ideas in this introduction to some simple statistics for experiments on behavior are

• the null hypothesis (devil's advocate's argument);
• the use of paired observations in experiments; and
• the direction and magnitude of the differences between paired observations.

If you understand these ideas, you should be able to decide whether the

Siegel, S.   Nonparametric statistics for the behavioral sciences.

Sokal, R. R., and F. J. Rohlf.   Biometry.