Idler and Kasl’s p Values: A Cautionary Lesson

Gary Smith, Ph.D.

Fletcher Jones Professor of Economics

Pomona College

Claremont, California 91711

telephone: 909-607-3135

fax: 909-621-8576

email: gsmith@pomona.edu

Abstract

Idler and Kasl’s study of elderly New Haven residents indicated that some Christians and Jews postponed their deaths until after the celebration of religious holidays. However, the correct p values are larger than they report and make their conclusions less convincing, especially for Jews.

key words: deathdate, mortality and religion, postponing death

Idler and Kasl’s p Values: A Cautionary Lesson

Based on a study of elderly New Haven residents, Idler and Kasl (1) report that some Christians and Jews evidently postponed their deaths until after the celebration of religious holidays. However, there are several technical problems: (a) their continuity correction is incorrect; (b) exact p values can be calculated from the binomial distribution; (c) two-sided p values should be used; (d) the sample deaths may not have been independent; and (e) there is no adjustment for the multiple, overlapping tests. Just taking (a) and (b) into account, the correctly calculated p values are larger than those reported and weaken their conclusions, particularly for the small sample of Jews studied.

Correct p-Value Calculations

Idler and Kasl compare the number of deaths 30 days before and 30 days after various Christian and Jewish holidays. If the timing of deaths are independent across individuals, the binomial distribution can be used to test the null hypothesis that a death that occurs during a 60-day interval has a 0.5 probability of occurring in either 30-day subinterval. The p value is the probability, if the null hypothesis is true, that a random sample would yield a success proportion that is this far or farther from the success probability given by the null hypothesis (2).

For example, among the 5 "more-observant" white Protestants who died within 30 days of Christmas, 2 died before the holiday and 3 afterward. Idler and Kasl calculate the p value to be 0.187 by using a normal approximation with an incorrect continuity correction. In this example, they add one-half to 3 and thereby approximate the probability of more than 3 deaths instead subtracting one-half from 3 in order to approximate the probability of 3 or more deaths. In samples this small, the binomial distribution should be used to calculate the exact p value, which is 0.500: when a fair coin is flipped 5 times, there is a 50 percent chance of 3 or more heads.

Two-Tail Tests

The use of one-sided or two-sided p values depends on whether the alternative hypothesis is one-sided or two-sided–here, whether the probability of death in the first half of the interval can plausibly be larger or smaller than 0.5, or whether we can rule out one possibility before we look at the data. Idler and Kasl argue that the probability may be less than 0.5 because people may be able to postpone death. The crucial question is whether, if the data show the opposite–that there are more deaths before holidays than after–we will dismiss the results as a statistical fluke, no matter how low the p value. Or will we consider the possibility that the anticipation of a holiday increases the probability of death? If we could be persuaded that the probability is larger than 0.5, we should use two-sided p values.

Idler and Kasl report one-sided p values. However, a number of studies (including theirs) find evidence of an increase in deaths before holidays. David Phillips has written several papers on this topic and is cited four times by Idler and Kasl. One of his papers (3) argues that a "symbolically meaningful occasion" can be either a lifeline (prolonging death) or a deadline (provoking death), and that females seem to view birthdays as lifelines while males view them as deadlines. If this is indeed a possibility, two-sided p values should be used.

Independence

Idler and Kasl’s calculated p values assume that individual deaths are independent and thereby constitute a random sample. This is a crucial assumption that should be considered seriously. There might be more deaths after Christmas than before simply because the weather is harsher. Or there might be more deaths among a particular group living in public housing because there was a heating-system failure, food poisoning, or an outbreak of a contagious disease.

One way to take into account seasonal patterns is to compare the number of deaths in a control group. Idler and Kasl report the number of Christian and Jewish deaths before and after holidays, but do not compare them statistically. Looking at all Christian holidays, 125 of 296 of the Christian deaths within 30 days of a holiday were before the holiday, as were 28 of 58 Jewish deaths; the two-sided p value is 0.480. For all Jewish holidays, 38 of 74 Jewish deaths and 202 of 387 Christian deaths were before the holiday; the two-sided p value is 0.994. Neither comparison is close to being statistically persuasive.

Conclusions

Tables 1 and 2 show the incorrect p values P reported by Idler and Kasl in their Tables 3 and 4 (they only report p values if there are fewer deaths before the holiday) and the correct one-sided p values P’ (assuming independence). The correct two-sided p values are equal to twice P’, unless P’ > 0.5.

For Christians and Christian holidays, Idler and Kasl report that 17 of 42 possible tests have one-sided p values less than 0.05. In fact, only 6 of 42 tests have correctly calculated 2-sided p values less than 0.05. It is unclear how this should be interpreted since many of the tests overlap; for example, Catholic/Christmas, Catholic/Christmas&Easter, and more-observant-Catholic/Christmas&Easter.

The corrected results for Jewish holidays are even more dramatic. Idler and Kasl conclude: "We find that the reduction in deaths before Jewish holidays occurs only among Jews [not Christians], particularly among the more observant Jews" (p. 1073) and that, "the timing of mortality among elderly Christians and Jews was closely linked to their own religious holidays" (p. 1076). In fact, the combined data for all Jews and all Jewish holidays show more deaths before holidays than after (38 versus 36). Looking at all 20 Jewish tests, the lowest p value reported by Idler and Kasl is 0.019 for 23 more-observant Jews who died within 30 days of any of 3 holidays (7 before and 16 after); the correct one-sided p value is 0.047. This should be interpreted cautiously since it is but one of 20 statistical tests for Jews and, in addition, a two-sided p value is more appropriate.

In fact, the most statistically persuasive result for the Jewish data (and the only one with a two-sided p value less than 0.05) is for Jewish females for all Jewish holidays, 24 before and 10 after, with a two-sided p value of 0.024; in this most statistically persuasive case, there were more deaths before the holidays, which contradicts Idler-Kasl’s postponement theory.

REFERENCES

1. Idler EL, Kasl SV. 1985. Religion, disability, depression, and the timing of death. American Journal of Sociology 1992; 97: 1052—1079.

2. Smith G, Introduction to statistical reasoning. McGraw-Hill: New York, 1998; p. 348.

3. Phillips D, Van Voorhees CA, Ruth TE. The birthday: Lifeline or deadline? Psychosomatic Medicine 1992; 54: 532—542.

Table 1 Deaths Among Christian and Jewish Yale Health and Aging Project Residents

Thirty Days Before and After Christian Holidays, 1982—1988

 Christmas Easter All Christian Holidays -30 +30 P P' -30 +30 P P' -30 +30 P P' Among Christians Males 25 38 .039 .065 33 42 .125 .178 58 80 .025 .037 Females 30 51 .007 .013 37 40 .326 .410 67 91 .024 .033 More Observant 25 41 .018 .032 32 40 .145 .205 57 81 .017 .025 Less Observant 30 48 .016 .027 38 42 .288 .369 68 90 .034 .047 Catholics 35 62 .002 .004 42 56 .066 .094 77 118 .001 .002 More Observant 17 30 .021 .039 23 32 .089 .140 40 62 .011 .019 Less Observant 18 32 .017 .032 19 24 .181 .271 37 56 .019 .031 White Protestants 9 11 .251 .412 17 10 .124 26 22 .333 More Observant 2 3 .187 .500 2 3 .187 .500 4 6 .171 .377 Less Observant 7 8 .302 .500 15 7 .067 22 15 .162 Black Protestants 10 16 .085 .163 11 14 .212 .345 21 30 .081 .131 More Observant 6 8 .212 .395 6 5 .500 12 13 .345 .500 Less Observant 4 8 .073 .194 5 9 .092 .212 9 17 .039 .084 For All Christians 55 89 .002 .003 70 82 .147 .186 125 171 .004 .004 Among Jews Males 6 6 .386 .613 5 9 .090 .212 11 15 .164 .279 Females 7 10 .166 .315 10 5 .151 17 15 .430 More Observant 5 5 .374 .623 4 7 .113 .274 9 12 .192 .332 Less Observant 8 11 .179 .324 11 7 .240 19 18 .500 For all Jews 13 16 .230 .356 15 14 .500 28 30 .348 .448

Table 2 Deaths Among Jewish and Christian Yale Health and Aging Project Residents

Thirty Days Before and After Jewish Holidays, 1982—1988

 Passover Rosh Hashanah Yom Kippur All Jewish Holidays -30 +30 P P' -30 +30 P P' -30 +30 P P' -30 +30 P P' Among Jews Males 5 11 .040 .105 6 7 .291 .500 3 8 .035 .113 14 26 .020 .040 Females 9 4 .133 8 3 .113 7 3 .172 24 10 .012 Less Observant 11 7 .240 11 7 .240 9 6 .304 31 20 .080 More Observant 3 8 .035 .113 3 3 .341 .656 1 5 .021 .109 7 16 .019 .047 For All Jews 14 15 .359 .500 14 10 .271 10 11 .330 .500 38 36 .454 Among Christians Males 35 32 .404 32 27 .301 34 24 .119 101 83 .105 Females 38 34 .362 32 31 .500 31 37 .198 .272 101 102 .444 .500 Less Observant 42 36 .286 34 27 .221 33 31 .450 109 94 .163 More Observant 31 30 .500 30 31 .397 .500 32 30 .450 93 91 .471 Catholics 43 46 .337 .416 43 43 .456 .543 47 39 .225 133 128 .402 White Protestants 16 9 .115 9 6 .304 7 10 .166 .315 32 25 .214 Black Protestants 15 11 .279 12 9 .332 11 9 .412 38 29 .164 For All Christians 73 66 .305 64 58 .325 65 61 .395 202 185 .208