Nullifying Null Findings: Conducting Research That Is Inherently Interesting

• In Event: When Nothing Is Something: Null Results in Informal Learning Research

Abstract

Objectives: The purpose of this paper is to look at the question of null results in research as a function of the nature of the question that is asked in the research as well as the design of the study looking at that question. The question addressed in this presentation is: What is the nature of questions that produce inherently interesting results versus those that produce null results?

Perspectives: Having almost 50 years of experience in conducting research, I have had my share of crushingly disappointing moments when research studies yield null results. As I gained more experience as a researcher, I learned that one could avoid such moments by designing research that provided interesting results no matter what those results were.

Methods: A series of research studies (both by the author and taken from the literature) will be examined to determine the nature of questions that are inherently interesting versus those that produce disappointing null results (and are consequently difficult to publish).

Evidence: A set of eight research studies will be examined, four of which produced null results and four of which were “immune” to null results. An example of a null result study taken from my research had to do with varying the nature of the label that accompanied images on the Astronomy Picture of the Day website. It simply turned out to be the case that the differences among the various formats were not particularly strong. On the other hand, a study of how long visitors look at great works of art in museums was something that scholars in the psychology of aesthetics wanted to know, irrespective of what the amount of time was (27 seconds was the mean, 17 seconds was the median).

Conclusions: The basic finding here is that “if” questions tend to hold more potential for producing null results versus questions that ask, “how”, “what” and “why” questions. Examples of these are:

“If” questions:
• Does approach X lead to participant gains in Y? (Not interesting if the answer is “no”)
• Do long labels cause traffic jams in tight exhibitions? (Answer was no, not much interest in journals)

“How” question
• How do people make meaning in an exhibition that they know little about before entering?

“Why” question:
• Why do people tend to hang back from guided tour groups through museums? (It’s a question of wanting both structure – information from expert, and freedom -wanting to be able to go anywhere, not follow tour.)

“What: question:
• What motivations do people have for visting a particular exhibition?

Scientific or scholarly significance of the study or work: Although research that holds the potential for null results can be very important, and often the null results are worth communicating to a broader audience, the essence of a good research question is one where others want to know the answer to the question no matter what the answer is.

Author

• Jeffrey K. Smith, University of Otago