Idler and Kasl’s p Values: A Cautionary Lesson
Gary Smith, Ph.D.
Fletcher Jones Professor of Economics
Pomona College
Claremont, California 91711
telephone: 9096073135
fax: 9096218576
email: gsmith@pomona.edu
Abstract
Idler and Kasl’s study of elderly New Haven residents indicated that some Christians and Jews postponed their deaths until after the celebration of religious holidays. However, the correct p values are larger than they report and make their conclusions less convincing, especially for Jews.
key words: deathdate, mortality and religion, postponing death
Idler and Kasl’s p Values: A Cautionary Lesson
Based on a study of elderly New Haven residents, Idler and Kasl (1) report that some Christians and Jews evidently postponed their deaths until after the celebration of religious holidays. However, there are several technical problems: (a) their continuity correction is incorrect; (b) exact p values can be calculated from the binomial distribution; (c) twosided p values should be used; (d) the sample deaths may not have been independent; and (e) there is no adjustment for the multiple, overlapping tests. Just taking (a) and (b) into account, the correctly calculated p values are larger than those reported and weaken their conclusions, particularly for the small sample of Jews studied.
Correct pValue Calculations
Idler and Kasl compare the number of deaths 30 days before and 30 days after various Christian and Jewish holidays. If the timing of deaths are independent across individuals, the binomial distribution can be used to test the null hypothesis that a death that occurs during a 60day interval has a 0.5 probability of occurring in either 30day subinterval. The p value is the probability, if the null hypothesis is true, that a random sample would yield a success proportion that is this far or farther from the success probability given by the null hypothesis (2).
For example, among the 5 "moreobservant" white Protestants who died within 30 days of Christmas, 2 died before the holiday and 3 afterward. Idler and Kasl calculate the p value to be 0.187 by using a normal approximation with an incorrect continuity correction. In this example, they add onehalf to 3 and thereby approximate the probability of more than 3 deaths instead subtracting onehalf from 3 in order to approximate the probability of 3 or more deaths. In samples this small, the binomial distribution should be used to calculate the exact p value, which is 0.500: when a fair coin is flipped 5 times, there is a 50 percent chance of 3 or more heads.
TwoTail Tests
The use of onesided or twosided p values depends on whether the alternative hypothesis is onesided or twosided–here, whether the probability of death in the first half of the interval can plausibly be larger or smaller than 0.5, or whether we can rule out one possibility before we look at the data. Idler and Kasl argue that the probability may be less than 0.5 because people may be able to postpone death. The crucial question is whether, if the data show the opposite–that there are more deaths before holidays than after–we will dismiss the results as a statistical fluke, no matter how low the p value. Or will we consider the possibility that the anticipation of a holiday increases the probability of death? If we could be persuaded that the probability is larger than 0.5, we should use twosided p values.
Idler and Kasl report onesided p values. However, a number of studies (including theirs) find evidence of an increase in deaths before holidays. David Phillips has written several papers on this topic and is cited four times by Idler and Kasl. One of his papers (3) argues that a "symbolically meaningful occasion" can be either a lifeline (prolonging death) or a deadline (provoking death), and that females seem to view birthdays as lifelines while males view them as deadlines. If this is indeed a possibility, twosided p values should be used.
Independence
Idler and Kasl’s calculated p values assume that individual deaths are independent and thereby constitute a random sample. This is a crucial assumption that should be considered seriously. There might be more deaths after Christmas than before simply because the weather is harsher. Or there might be more deaths among a particular group living in public housing because there was a heatingsystem failure, food poisoning, or an outbreak of a contagious disease.
One way to take into account seasonal patterns is to compare the number of deaths in a control group. Idler and Kasl report the number of Christian and Jewish deaths before and after holidays, but do not compare them statistically. Looking at all Christian holidays, 125 of 296 of the Christian deaths within 30 days of a holiday were before the holiday, as were 28 of 58 Jewish deaths; the twosided p value is 0.480. For all Jewish holidays, 38 of 74 Jewish deaths and 202 of 387 Christian deaths were before the holiday; the twosided p value is 0.994. Neither comparison is close to being statistically persuasive.
Conclusions
Tables 1 and 2 show the incorrect p values P reported by Idler and Kasl in their Tables 3 and 4 (they only report p values if there are fewer deaths before the holiday) and the correct onesided p values P’ (assuming independence). The correct twosided p values are equal to twice P’, unless P’ > 0.5.
For Christians and Christian holidays, Idler and Kasl report that 17 of 42 possible tests have onesided p values less than 0.05. In fact, only 6 of 42 tests have correctly calculated 2sided p values less than 0.05. It is unclear how this should be interpreted since many of the tests overlap; for example, Catholic/Christmas, Catholic/Christmas&Easter, and moreobservantCatholic/Christmas&Easter.
The corrected results for Jewish holidays are even more dramatic. Idler and Kasl conclude: "We find that the reduction in deaths before Jewish holidays occurs only among Jews [not Christians], particularly among the more observant Jews" (p. 1073) and that, "the timing of mortality among elderly Christians and Jews was closely linked to their own religious holidays" (p. 1076). In fact, the combined data for all Jews and all Jewish holidays show more deaths before holidays than after (38 versus 36). Looking at all 20 Jewish tests, the lowest p value reported by Idler and Kasl is 0.019 for 23 moreobservant Jews who died within 30 days of any of 3 holidays (7 before and 16 after); the correct onesided p value is 0.047. This should be interpreted cautiously since it is but one of 20 statistical tests for Jews and, in addition, a twosided p value is more appropriate.
In fact, the most statistically persuasive result for the Jewish data (and the only one with a twosided p value less than 0.05) is for Jewish females for all Jewish holidays, 24 before and 10 after, with a twosided p value of 0.024; in this most statistically persuasive case, there were more deaths before the holidays, which contradicts IdlerKasl’s postponement theory.
REFERENCES
1. Idler EL, Kasl SV. 1985. Religion, disability, depression, and the timing of death. American Journal of Sociology 1992; 97: 1052—1079.
2. Smith G, Introduction to statistical reasoning. McGrawHill: New York, 1998; p. 348.
3. Phillips D, Van Voorhees CA, Ruth TE. The birthday: Lifeline or deadline? Psychosomatic Medicine 1992; 54: 532—542.
Table 1 Deaths Among Christian and Jewish Yale Health and Aging Project Residents
Thirty Days Before and After Christian Holidays, 1982—1988
Christmas

Easter

All Christian
Holidays


30

+30

P

P'

30

+30

P

P'

30

+30

P

P'


Among Christians  

25

38

.039

.065

33

42

.125

.178

58

80

.025

.037



30

51

.007

.013

37

40

.326

.410

67

91

.024

.033



25

41

.018

.032

32

40

.145

.205

57

81

.017

.025



30

48

.016

.027

38

42

.288

.369

68

90

.034

.047



35

62

.002

.004

42

56

.066

.094

77

118

.001

.002



17

30

.021

.039

23

32

.089

.140

40

62

.011

.019



18

32

.017

.032

19

24

.181

.271

37

56

.019

.031



9

11

.251

.412

17

10

.124

26

22

.333



2

3

.187

.500

2

3

.187

.500

4

6

.171

.377



7

8

.302

.500

15

7

.067

22

15

.162



10

16

.085

.163

11

14

.212

.345

21

30

.081

.131



6

8

.212

.395

6

5

.500

12

13

.345

.500



4

8

.073

.194

5

9

.092

.212

9

17

.039

.084



55

89

.002

.003

70

82

.147

.186

125

171

.004

.004


Among Jews  

6

6

.386

.613

5

9

.090

.212

11

15

.164

.279



7

10

.166

.315

10

5

.151

17

15

.430



5

5

.374

.623

4

7

.113

.274

9

12

.192

.332



8

11

.179

.324

11

7

.240

19

18

.500



13

16

.230

.356

15

14

.500

28

30

.348

.448

Table 2 Deaths Among Jewish and Christian Yale Health and Aging Project Residents
Thirty Days Before and After Jewish Holidays, 1982—1988
Passover  Rosh Hashanah  Yom Kippur  All Jewish Holidays  
30  +30  P  P'  30  +30  P  P'  30  +30  P  P'  30  +30  P  P'  
Among Jews  

5  11  .040  .105  6  7  .291  .500  3  8  .035  .113  14  26  .020  .040  
Females 
9  4  .133  8  3  .113  7  3  .172  24  10  .012  
Less Observant 
11  7  .240  11  7  .240  9  6  .304  31  20  .080  
More Observant 
3  8  .035  .113  3  3  .341  .656  1  5  .021  .109  7  16  .019  .047  
For All Jews 
14  15  .359  .500  14  10  .271  10  11  .330  .500  38  36  .454  
Among Christians 

Males 
35  32  .404  32  27  .301  34  24  .119  101  83  .105  
Females 
38  34  .362  32  31  .500  31  37  .198  .272  101  102  .444  .500  
Less Observant 
42  36  .286  34  27  .221  33  31  .450  109  94  .163  
More Observant 
31  30  .500  30  31  .397  .500  32  30  .450  93  91  .471  
Catholics 
43  46  .337  .416  43  43  .456  .543  47  39  .225  133  128  .402  
White Protestants 
16  9  .115  9  6  .304  7  10  .166  .315  32  25  .214  
Black Protestants 
15  11  .279  12  9  .332  11  9  .412  38  29  .164  
For All Christians 
73  66  .305  64  58  .325  65  61  .395  202  185  .208 