Report to the Great Pumpkin

Research Fables from the Sisters Grinn, No. 4


Measures R US, Inc. , J. Grace, Project Manager

Most Esteemed Vine Vegetable:

Last month, you commissioned the NUR 301 class at the University of Rochester to design an evaluation research strategy, for the purpose of identifying the most sincere pumpkin patch available to you on Halloween. This memo constitutes the final report on this project.

Our first effort in your behalf was to define "sincerity". While this concept may seem perfectly clear to you and Linus, our plan to use multiple evaluators threatened the inter-rater reliability of the "I'll know it when I see it" strategy. To this end, we generated characteristics amenable to multiple triangulation. We defined a sincere pumpkin patch in terms of pumpkin shape, color, odor, and vine behavior. While our observational work groups developed appropriate measures for these attributes, our interviewers developed a group of open- ended interview questions to assess "inner pumpkin" sincerity (not to be confused with the pumpkin seed count.) Our survey researchers developed a questionnaire on global pumpkin patch sincerity and identified the appropriate populations to sample, and our records researchers developed a schedule for reviewing the written narratives of nursery school classes who had previously visited a pumpkin patch.

The review of nursery school records proved problematic. First, our researchers observed that, since only half of them had attended nursery school, a nursery school sample was unlikely to be representative of the entire population of humans, much less the opinions of Great Pumpkins like you. Moreover, since these narratives are typically written on the first day of nursery school after the pumpkin patch visit, children who got wet, chilled, and developed upper respiratory infections would be underrepresented in the data. Finally, we could not develop a strategy to account for missing photographic data in the likely event that some parents' or teachers' snapshots would be lost in processing.

Our questionnaire team quickly determined that pumpkins (with the obvious exception of your esteemed self) cannot fill out questionnaires; however, they felt that farmers and consumers would be valid informants. Accordingly, they planned to distribute a mailed questionnaire to these groups, hoping for a high rate of return. (It is of course possible that the habitues of the most sincere pumpkin patch may wish to keep that knowledge to themselves, and only brash Jack-o'lanterns-come-lately will respond.) The team designed a multiple item questionnaire, asking for ratings of pumpkin size, price, color, shape and the relative muddiness of the patch after rain. They are now pilot-testing this instrument to determine whether it has acceptable internal consistency and will submit it to a group of squash and gourd experts to establish content validity.

Our behavioral observation experts identified three aspects of sincerity:

  1. in a sincere pumpkin patch, the other pumpkins don't move away from a pumpkin that has fallen on hard times and succumbed to rot
  2. in a sincere pumpkin patch, pumpkins are mellow and don't race around in a crazed way
  3. in a sincere pumpkin patch, vines are organized to allow access, so no vine rises up to trip the unwary as they walk.


This group plans to observe pumpkin patches for total vegetable inactivity, which will be a reliable and valid measure of all three criteria. They plan to do these observations in pairs until inter-rater reliability is established, and they will return to a reference pumpkin patch every few days to assess any tendency for intrarater drift over time. For reasons of obvious bias, they regretfully conclude that Linus cannot be employed as an objective rater of pumpkin patches.

Given your example of obvious sentience, our interviewers determined that they could effectively interview individual pumpkins for sincerity. They noted, however, that they would dress as pumpkins to offset interspecies bias in response. Their questions would include "Why are you a pumpkin and not a squash?" and "What is the best thing about being a pumpkin?" On the assumption that all pumpkins want to be known as most sincere, they have developed two strategies for assessing social desirability. First, they plan to modify a polygraph to assess the respiration analog, photosynthesis: the greater the oxygen output, the more truthful the pumpkin The polygraph will be operated by a trained operator, and only readings from sessions on warm, sunny days will be used. A minority also argued for lining pumpkins up on a fence and threatening them with a shotgun if they did not tell the truth. Let me hasten to assure you and the local People for the Ethical Treatment of Pumpkins chapter that our vegetable subjects research review board would never allow this approach to be implemented. Moreover, the researchers who suggested this approach now recognize that it would bias pumpkin responses not toward the truth, but toward whatever answer the interrogator indicated was most desirable.

Finally, our pumpkinmetricians identified physical parameters of color, reflectance, size and shape for assessing sincerity. Since spherical pumpkins were felt to be most sincere, and pumpkins are frequently enjoyed in pi, the researchers determined that the height to circumference ratio of the ideal pumpkin would be 1:3.1415928. The pumpkinmetricians practiced measuring sample pumpkins with a commercial sincerity measure (otherwise known as a paper measuring tape) until they had acceptable inter-rater agreement. While the reliability of this measure was established beyond any question, the validity of this ratio as a measure of pumpkin sincerity remains in doubt: the pumpkinmetricians reported that the object most closely approximating the ideal ratio was not a pumpkin at all, but rather the head of one of the interview researchers.

Although, unfortunately, we were unable to complete our evaluation plan before Halloween, 1996, we are confident we are well positioned to fulfill your research needs for the 1997 Halloween season. We look forward to hearing from you then.

Very truly yours,

Jeanne Grace
Project manager, Measures R Us Inc.


Return to Research Fables List

© 1996 - University of Rochester School of Nursing