Orne, M.T. The Significance of unwitting cues for experimental outcomes: Toward a pragmatic approach. Annals of the New York Academy of Sciences, 1981, 364, 152-159.

The Significance of Unwitting Cues for Experimental Outcomes: Toward a Pragmatic Approach *

MARTIN T. ORNE

Unit for Experimental Psychiatry, Institute of Pennsylvania Hospital Philadelphia, Pennsylvania 19139
Department of Psychiatry, University of Pennsylvania Philadelphia, Pennsylvania 19139

THE CLEVER HANS PHENOMENON excited the scientific community of the day because it appeared to document a unique cognitive skill on the part of a horse. When Pfungst, l after extensive study, was able to document this fascinating case of apparently unwitting communication, the scientific community after an initial shock rapidly lost interest in what seemed like an almost trivial alternative explanation. It is well to remember that several phenomena, ranging from ESP to the inheritance of acquired characteristics, and from the especially powerful effects of new drugs- -which must be used while the special power persists--to some effects ascribed to biofeedback, have been shown to relate directly to the same process that explains the famous "talking horse." In these and many other instances, the effects of unwitting, subtle communications have been responsible for what was initially thought to be an entirely different phenomenon being subjected to scientific scrutiny. From the point of view of the scientific community, the Clever Hans effect has often been seen as a troublesome artifact which might readily mislead some other investigator.

It was not until the social psychology of research itself was recognized as requiring systematic inquiry that there has been some serious concern about the process of unwitting communication and its potential effects on experimental findings. 2,3 Unfortunately, to appreciate the


* The substantive research discussed here was supported in part by Grant MH 19156 from the National Institute of Mental Health and by a grant from the Institute for Experimental Psychiatry.

152

 

153 ORNE: TOWARD A PRAGMATIC APPROACH

Clever Hans phenomenon it is essential to recognize that an experiment, just like any other social interaction, does not occur in a vacuum. Accordingly, to evaluate the effect of subtle communication, it is necessary to understand the roles of the various participants and how they perceive them. †

In contrast to the experimental model of the physical sciences, which studies the effect of stimuli on inanimate objects, the psychological experiment involves an interaction of an experimenter with an active, sentient participant who has agreed to take part in an experiment and recognizes that his role is to appear as a passive responder. The participant, however, takes part for his own purposes, seeks to present himself in a desirable way, generally wants to assist the research, and tries to do the "right" thing. In some ways the psychological experiment is viewed by the subject as a problem-solving situation, where the rules of the game preclude his being given sufficient information about what is expected of him but where he is nonetheless supposed to respond appropriately by attending carefully to the various sources of information available to him.

I have tried to suggest that one way the psychological experiment may be viewed advantageously is as two conceptually distinct experiments: the first, the study that the investigator has designed and operationalized to test his hypothesis; the second, the study that the subject perceives he is participating in.5 What I have called the demand characteristics of the experiment are the sum total of cues available to the subject before the experiment, the instructions during the experiment, the covert communications during the experiment, and the nature of the procedure itself that communicate the experimental purposes and the desired behavior. 6 Clearly, unwitting communications are a major determinant of these demand characteristics. The extent to which the experiment designed by the investigator and the experiment participated in by the subject are the same will determine much of the ecological validity of the procedure and, therefore, the generalizability of the findings beyond the immediate experimental context.

It should be emphasized that it is not possible to design an experiment without demand characteristics, and it is unlikely that an experiment can be carried out without unwitting, subtle communications. It is, however, both possible and necessary to determine the kinds of effects that the demand characteristics of the situation will have for the subject in order to assess the adequacy of the experimental procedure--


† Goffman 4 has conceptualized this as frame analysis.

 

154 ANNALS NEW YORK ACADEMY OF SCIENCES

that is, the extent to which the subject perceives the study as the investigator intends him to perceive it. ‡

Once it is recognized that the effects of demand characteristics and the potential consequences of unwitting communication are problems that are likely to be with any working scientist for as long as he carries out research with man and that these cannot be controlled by statistical design, there has been a tendency to take one of two opposing points of view. The first is to view the whole matter as a tempest in a teapot, to argue that these are problems only when proper controls are not carefully carried out, that with well-designed studies and trained investigators there are no difficulties, that the cooperative, good subject is a myth, and any variance accounted for by any such individuals is corrected by others who, in the jargon of the day, show a "screw-you effect."7 The other point of view tends to be no less extreme in that it views with alarm the problems of experimental social psychology, seriously questioning whether any experiment is ever worth doing, using concepts like the Clever Hans phenomenon, demand characteristics, evaluation apprehension,7 and the like as spoiler variables rather than as conceptual tools to design more appropriate studies. Proponents of this view enjoy citing some of the more dramatic studies, as for example by Rosenthal, 2 Adair, 9 Silverman, l0 Page, 11 and myself, 12 put iconoclastic interpretations on the findings, and tend to argue against the experiment as a means of learning about the nature of man.

It is of course obvious that neither of these extreme views is acceptable. In evaluating the outcome of research, the problems inherent in working with subjects who like ourselves have an interest and, to some degree, a stake in the experiment will have varying degrees of effects on different experimental situations. I would like to emphasize that it is essentially impossible to predict the extent of these effects from an armchair. They must be evaluated empirically, using procedures along the lines of quasi-controls, suggested elsewhere. 6


‡ I cannot resist addressing the issue of deception at this point. The use of deception is neither inherently good nor inherently evil. The task of the investigator is to make certain that no one leaves his laboratory with more troubles than he brings, and I seriously doubt that many individuals in a nontherapeutic context will be helped by pointing out their infantile anger and other inadequacies even though the observations may be accurate in the sense that they are based on the best available data. By the same token, some investigators proudly proclaim that they need not worry about the subject's perception of the experiment since they always are truthful. However, it turns out that it matters less whether the investigator is truthful and more whether what he says is plausible. Many an experiment is done for what--from the subject's point of view--are arcane and entirely unlikely reasons, and the subject will accordingly disbelieve them, leading to potential discrepancies between the investigator's intent and the subject's perception, even though the former is always being truthful. Thus, in any experiment involving human participants, one needs to assess what the subjects perceive as the experimental purposes and their own appropriate role in the procedure.

 

155 ORNE: TOWARD A PRAGMATIC APPROACH

Though I have tended to talk about the effects of demand characteristics and subtle communications on experimental contexts, it should be clear that these are by no means the only contexts where effects of this kind can be documented. Since one of the would-be solutions proposed for the difficulties of the experiment is the naturalistic field study, it seems worthwhile to examine how these effects can play a powerful role in nonexperimental contexts. Field studies are important and desirable, but they are not a substitute for carefully controlled laboratory research. For example, it is far more difficult and vastly more expensive to simultaneously control several relevant parameters in the field than in the laboratory. From my perspective, an interplay between field and laboratory research in psychology and psychiatry is as important as the interplay between in vivo and in vitro studies in biology. As with laboratory settings, it must also be emphasized that no meaningful field investigation should be carried out without a thorough analysis and detailed understanding of the social roles of the participants in the particular social context under study.

For example, a "real life" context that shares a number of important aspects of the experimental situation is the one-to-one psychotherapeutic interaction. The patient, even more than the subject, is motivated to participate in the interaction for reasons of his own. In superficial contrast to the experiment, where the subject may hope to learn something about himself and psychology in general, the patient in psychotherapy hopes to obtain direct personal benefit. In the therapeutic situation, however, like in the experiment, the participant does not know precisely what to expect and has only general knowledge of the rules of the game. Though he is generally told some things about what to do, the patient typically believes that there is more that he has to discover and that it is very important for him to do the "right thing." The therapist, analogous to the experimenter in the experiment, defines the correct response of the patient by his own behavior--being pleased, displeased, bored, and so on.

By way of illustration, it is common that among a new group of psychiatric residents one or another male resident will come to supervision troubled because an attractive female patient has for some time been bringing in extremely lurid sexual dreams, which take up most of the hour and bring treatment to a virtual standstill. To remedy this difficulty is as simple as it is effective. One first discusses the resident's interest in his patient, exploring briefly the countertransference, and then simply instructs him that henceforth he should stop writing down any sexual dream material. Instead he should write assiduously whenever the patient begins to talk about her current life difficulties but put down the pencil as soon as dream material is brought up. By the end of

 

156 ANNALS NEW YORK ACADEMY OF SCIENCES

the next session usually, and certainly by the end of the next session thereafter, the sexual dreams simply cease to be an issue in treatment. This illustration documents that writing serves a function for the patient analogous to that of headnodding for Clever Hans!

Again, in the therapeutic setting I had observed that many patients--particularly those who did not know anyone else in psychotherapy--fail to do well in treatment because they do not understand the rules of the game. Accordingly, we developed a technique referred to as anticipatory socialization for treatment, designed to teach the individual how to be an effective patient. 13 This procedure was tested in two independent studies 14, 15 where it was shown that patients who had a single session of anticipatory socialization did significantly better in psychotherapy as assessed four months later.

What was communicated to the patient about his role and expected behavior was in these instances deliberate and purposive. He was chosen as the target of the anticipatory socialization interview because we assumed he would have a very high level of motivation to learn about the enterprise of psychotherapy. The modification of the patient's behavior made it more congruent with the therapist's expectations and probably served to modify the therapist's expectations and behavior, which in turn led to greater therapeutic improvement. The therapist then was responding to the patient's unwitting communication. Thus, it would appear that in the therapeutic context either the therapist or the patient is capable of altering the course of treatment by purposive behavior, which affects the unwitting behavior of the partner in the dyadic relationship.

Another context where unwitting communications are potent is the forensic context. Recently I have become interested in the use and (at times) abuse of hypnosis in attempts to enhance the recall of witnesses to or victims of crimes. There have been instances where hypnosis facilitates the recall of important information not previously available. The recall of several numbers of the license plate of the kidnapper's car in the case of the Chowchilla kidnapping of a bus load of children is a good example. Here, neither the bus driver, the authorities, nor the media had any notion of the correct license number. In other instances, however, where the authorities or the media had identified suspects, the probable impact of unwitting communication can be seen. Under such circumstances, witnesses who had previously been unable to identify assailants became able to do so when their memory was "refreshed by hypnosis." The key question here, however, is whether in such circumstances hypnosis serves to enhance actual recall or to create a pseudo memory where there is no recall. That is, by increasing the response to subtle cues, hypnosis helps to persuade the witness who has

 

157 ORNE: TOWARD A PRAGMATIC APPROACH

no recall that he had in fact seen the plausible suspect commit the crime.16

One example of such a situation is a court martial case that occurred several years ago in the Philadelphia Naval Base. Two sailors were sitting drinking coffee when a man walked by, pulled a gun, and fired at one of them. The victim fortunately jumped away and was only grazed. Subsequently the witness identified an individual as the assailant. The victim, however, was unable to identify this individual as his attacker. At a preliminary hearing, the witness again identified the defendant as the assailant whereas the victim said that he looked somewhat like the assailant but that he was not the right person.

The victim was then hypnotized in order to help him recall. After the first session he still had no memory of the original event. During the second session he relived the events and was given a posthypnotic suggestion to remember everything that happened. On awakening he was certain that he had actually seen the defendant attempt to kill him. During the court martial trial I testified about the problems of confabulation when hypnosis is used in this fashion, pointing out that the pressure to remember caused the victim to accept a plausible suspect as the assailant though in the wake state he had rejected this possibility. However, the pseudo memory created in hypnosis was then confused with his earlier recollection. The judge advocate excluded the testimony based on hypnotically enhanced recall and the defendant was acquitted. Within several weeks two reliable witnesses returned from overseas and corroborated the defendant's alibi, thus proving that he could not have been the assailant. Interestingly, the effects of the hypnosis persisted and the victim still accepted his pseudo memory as real one year later.

This case is one of a considerable number where the hypnotic process was used to alter the recollection of witnesses or victims. I bring it up in this context because it illustrates a rather frightening consequence of the Clever Hans phenomenon where, in conjunction with hypnosis, unwitting communication may cause an honest person to compellingly lie.

I have tried to show that powerful effects of subtle communication are ubiquitous and by no means limited to the laboratory experiment. Field settings such as the therapeutic context or the context of hypnotic interrogation may also bring about precisely the same kinds of consequences that have been described for psychological experiments. If anything, the effects are stronger and more durable in nonlaboratory settings. Clearly, the notion of leaving the laboratory in order to escape from the methodological problems inherent in Clever Hans or demand characteristic effects is naive and ill founded. By all means let us con-

 

158 ANNALS NEW YORK ACADEMY OF SCIENCES

duct field research, but we must be clear that we are exchanging one social psychological context for another, and the failure to understand or to specify that context is not likely to advance science.

With varying degrees of comfort, psychologists have learned to live with the need to use statistical tests in order to appropriately assess the outcomes of research. In an analogous way, we will need to become comfortable with the realization that any research involving human beings occurs in a social context and is never carried out in a vacuum. The nature of that social context and the subject's awareness of his particular social role will help determine his responsivity to the Clever Hans phenomenon, as well as other aspects of the demand characteristics inherent in the situation.

The fact that such effects exist does not mean that they determine the experimental outcome. The question of the impact of unwitting cues needs to be raised and considered in relation to each piece of research. It always represents an alternative hypothesis to be considered. While this involves a certain amount of effort, it need not paralyze us. Once we recognize that research findings are obtained with varying degrees of control and can therefore be relied upon with varying degrees of certainty, we will begin to develop a pragmatic point of view toward our observations. Certainly, if a research finding is to be the basis of decisions that seriously affect the health and welfare of man, or involve great expenditures of valuable resources, the level of certainty demanded should be far greater than if one were dealing with a purely academic exercise. Many would define science as the approximation of truth, and if we take such a point of view seriously, the realization that different levels of certainty are required, depending upon the purpose of the research, should not make us too uncomfortable. It should, in fact, ultimately lead to a pragmatic view of empirical psychological research as one approach to an understanding of the real world.

ACKNOWLEDGMENTS

I would like to thank my colleagues David F. Dinges and Kevin M. McConkey for their detailed substantive suggestions and Emily Carota Orne, William H. Putnam, and William M. Waid for their comments in the preparation of this manuscript.

REFERENCES

1. PFUNGST, O. 1965. Clever Hans (The Horse of Mr. Von Osten). [Edited by R. Rosenthal.] Holt, Rinehart & Winston. New York, N.Y.

 

159 ORNE: TOWARD A PRAGMATIC APPROACH

2. ROSENTHAL, R. 1966. Experimenter Effects in Behavioral Research. Appleton-Century-Crofts, New York, N.Y.

3. ORNE, M. T. 1962. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. Am. Psychol. 17: 776-783.

4. GOFFMAN, E. 1974. Frame Analysis: An Essay on the Organization of Experience. Harper Colophon Books, New York, N.Y.

5. ORNE, M. T. 1973. Communication by the total experimental situation: Why it is important, how it is evaluated, and its significance for the ecological validity of findings. In Communication and Affect. P. Pliner, L. Krames & T. Alloway, Eds. pp. 157-191. Academic Press, New York, N.Y.

6. ORNE, M. T. 1969. Demand characteristics and the concept of quasi-controls. In Artifact in Behavioral Research. R. Rosenthal & R. L. Rosnow, Eds. pp. 143-179. Academic Press, New York, N.Y.

7. MASLING, J. 1966. Role-related behavior of the subject and psychologist and its effects upon psychological data. In Nebraska Symposium on Motivation. D. Levine, Ed. pp. 67-103. Univ. of Nebraska Press, Lincoln, Neb.

8. ROSENBERG, M. J. 1969. The conditions and consequences of evaluation apprehension. In Artifact in Behavioral Research. R. Rosenthal & R. L. Rosnow, Eds. pp. 279-349. Academic Press, New York, N.Y.

9. ADAIR, J. G. 1973. The Human Subject: The Social Psychology of the Psychological Experiment. Little, Brown & Company, Boston, Mass.

10. SILVERMAN, I. 1968. Role-related behavior of subjects in laboratory studies of attitude change. J. Pers. Soc. Psychol. 8: 343-348.

11. PAGE, M. M. & A. R. LUMIA. 1968. Cooperation with demand characteristics and the bimodal distribution of verbal conditioning data. Psychonom. Sci. 12: 243-244.

12. ORNE, M. T. 1970. Hypnosis, motivation and the ecological validity of the psychological experiment. In Nebraska Symposium on Motivation. W.},. Arnold & M. M. Page, Eds. pp. 187-265. University of Nebraska Press, Lincoln, Neb.

13. ORNE, M. T. & P. H. WENDER. 1968. Anticipatory socialization for psychotherapy: Method and rationale. Am. J. Psychiat. 124: 1202-1212.

14. HOEHN-SARlC, R., J. C. FRANK, S. D. IMBER, E. H. NASH, A. R. STONE, & C. C. BATTLE. 1964. Systematic preparation of patients for psychotherapy. 1. Effects on therapy behavior and outcome. J. Psychiat. Res. 2: 267-281.

15. SLOANE, R., A. CRISTOL, M. PEPERNIK, & F. STAPLES. 1970. Role preparation and expectation of improvement in psychotherapy. J. Nerv. Ment. Dis. 150: 18-26.

16. ORNE, M. T. 1979. The use and misuse of hypnosis in court. Int. J. Clin. Exp. Hypnosis 27: 311-341.


The preceding paper is a reproduction of the following article (Orne, M.T. The significance of unwitting cues for experimental outcomes: Toward a pragmatic approach. Annals of the New York Academy of Sciences, 1981, 364, 152-159. Copyright 1981 New York Academy of Sciences, U.S.A.). It is reproduced here with the kind permission of the New York Academy of Sciences.