Idler and Kasl’s p Values: A Cautionary Lesson

Gary Smith, Ph.D.

Fletcher Jones Professor of Economics

Pomona College

Claremont, California 91711

telephone: 909-607-3135

fax: 909-621-8576

email: gsmith@pomona.edu

Abstract

Idler and Kasl’s study of elderly New Haven residents indicated that some Christians and Jews postponed their deaths until after the celebration of religious holidays. However, the correct p values are larger than they report and make their conclusions less convincing, especially for Jews.

key words: deathdate, mortality and religion, postponing death

Idler and Kasl’s p Values: A Cautionary Lesson

Based on a study of elderly New Haven residents, Idler and Kasl (1) report that some Christians and Jews evidently postponed their deaths until after the celebration of religious holidays. However, there are several technical problems: (a) their continuity correction is incorrect; (b) exact p values can be calculated from the binomial distribution; (c) two-sided p values should be used; (d) the sample deaths may not have been independent; and (e) there is no adjustment for the multiple, overlapping tests. Just taking (a) and (b) into account, the correctly calculated p values are larger than those reported and weaken their conclusions, particularly for the small sample of Jews studied.

Correct p-Value Calculations

Idler and Kasl compare the number of deaths 30 days before and 30 days after various Christian and Jewish holidays. If the timing of deaths are independent across individuals, the binomial distribution can be used to test the null hypothesis that a death that occurs during a 60-day interval has a 0.5 probability of occurring in either 30-day subinterval. The p value is the probability, if the null hypothesis is true, that a random sample would yield a success proportion that is this far or farther from the success probability given by the null hypothesis (2).

For example, among the 5 "more-observant" white Protestants who died within 30 days of Christmas, 2 died before the holiday and 3 afterward. Idler and Kasl calculate the p value to be 0.187 by using a normal approximation with an incorrect continuity correction. In this example, they add one-half to 3 and thereby approximate the probability of more than 3 deaths instead subtracting one-half from 3 in order to approximate the probability of 3 or more deaths. In samples this small, the binomial distribution should be used to calculate the exact p value, which is 0.500: when a fair coin is flipped 5 times, there is a 50 percent chance of 3 or more heads.

Two-Tail Tests

The use of one-sided or two-sided p values depends on whether the alternative hypothesis is one-sided or two-sided–here, whether the probability of death in the first half of the interval can plausibly be larger or smaller than 0.5, or whether we can rule out one possibility before we look at the data. Idler and Kasl argue that the probability may be less than 0.5 because people may be able to postpone death. The crucial question is whether, if the data show the opposite–that there are more deaths before holidays than after–we will dismiss the results as a statistical fluke, no matter how low the p value. Or will we consider the possibility that the anticipation of a holiday increases the probability of death? If we could be persuaded that the probability is larger than 0.5, we should use two-sided p values.

Idler and Kasl report one-sided p values. However, a number of studies (including theirs) find evidence of an increase in deaths before holidays. David Phillips has written several papers on this topic and is cited four times by Idler and Kasl. One of his papers (3) argues that a "symbolically meaningful occasion" can be either a lifeline (prolonging death) or a deadline (provoking death), and that females seem to view birthdays as lifelines while males view them as deadlines. If this is indeed a possibility, two-sided p values should be used.

Independence

Idler and Kasl’s calculated p values assume that individual deaths are independent and thereby constitute a random sample. This is a crucial assumption that should be considered seriously. There might be more deaths after Christmas than before simply because the weather is harsher. Or there might be more deaths among a particular group living in public housing because there was a heating-system failure, food poisoning, or an outbreak of a contagious disease.

One way to take into account seasonal patterns is to compare the number of deaths in a control group. Idler and Kasl report the number of Christian and Jewish deaths before and after holidays, but do not compare them statistically. Looking at all Christian holidays, 125 of 296 of the Christian deaths within 30 days of a holiday were before the holiday, as were 28 of 58 Jewish deaths; the two-sided p value is 0.480. For all Jewish holidays, 38 of 74 Jewish deaths and 202 of 387 Christian deaths were before the holiday; the two-sided p value is 0.994. Neither comparison is close to being statistically persuasive.

Conclusions

Tables 1 and 2 show the incorrect p values P reported by Idler and Kasl in their Tables 3 and 4 (they only report p values if there are fewer deaths before the holiday) and the correct one-sided p values P’ (assuming independence). The correct two-sided p values are equal to twice P’, unless P’ > 0.5.

For Christians and Christian holidays, Idler and Kasl report that 17 of 42 possible tests have one-sided p values less than 0.05. In fact, only 6 of 42 tests have correctly calculated 2-sided p values less than 0.05. It is unclear how this should be interpreted since many of the tests overlap; for example, Catholic/Christmas, Catholic/Christmas&Easter, and more-observant-Catholic/Christmas&Easter.

The corrected results for Jewish holidays are even more dramatic. Idler and Kasl conclude: "We find that the reduction in deaths before Jewish holidays occurs only among Jews [not Christians], particularly among the more observant Jews" (p. 1073) and that, "the timing of mortality among elderly Christians and Jews was closely linked to their own religious holidays" (p. 1076). In fact, the combined data for all Jews and all Jewish holidays show more deaths before holidays than after (38 versus 36). Looking at all 20 Jewish tests, the lowest p value reported by Idler and Kasl is 0.019 for 23 more-observant Jews who died within 30 days of any of 3 holidays (7 before and 16 after); the correct one-sided p value is 0.047. This should be interpreted cautiously since it is but one of 20 statistical tests for Jews and, in addition, a two-sided p value is more appropriate.

In fact, the most statistically persuasive result for the Jewish data (and the only one with a two-sided p value less than 0.05) is for Jewish females for all Jewish holidays, 24 before and 10 after, with a two-sided p value of 0.024; in this most statistically persuasive case, there were more deaths before the holidays, which contradicts Idler-Kasl’s postponement theory.

REFERENCES

1. Idler EL, Kasl SV. 1985. Religion, disability, depression, and the timing of death. American Journal of Sociology 1992; 97: 1052—1079.

2. Smith G, Introduction to statistical reasoning. McGraw-Hill: New York, 1998; p. 348.

3. Phillips D, Van Voorhees CA, Ruth TE. The birthday: Lifeline or deadline? Psychosomatic Medicine 1992; 54: 532—542.

Table 1 Deaths Among Christian and Jewish Yale Health and Aging Project Residents

Thirty Days Before and After Christian Holidays, 1982—1988

Table 2 Deaths Among Jewish and Christian Yale Health and Aging Project Residents

Thirty Days Before and After Jewish Holidays, 1982—1988