Withdrawal Designs

Strategies and Tactics of Behavioral Research

Third Edition

2

Strategies and Tactics of Behavioral Research

Third Edition

James M. Johnston Auburn University

Henry S. Pennypacker University of Florida

3

Published in 2009 by Routledge 270 Madison Avenue New York, NY 10016 www.psypress.com

Published in Great Britain by Routledge 27 Church Road Hove, East Sussex BN3 2FA

Copyright © 2009 by Routledge

Routledge is an imprint of the Taylor & Francis Group, an informa business

This edition published in the Taylor & Francis e-Library, 2010.

To purchase your own copy of this or any of Taylor & Francis or Routledge’s collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.

Cover design by Design Deluxe

All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.

Library of Congress Cataloging-in-Publication Data Johnston, James M.

Strategies and tactics of behavioral research / James M. Johnston and Henry S. Pennypacker, Jr. – 3rd ed.

p. cm. Includes bibliographical references and index. 1. Psychology—Research—Methodology. I. Pennypacker, H. S. (Henry S.) II. Title. BF76.5.J63 2008 150.72—dc22 2008019278

ISBN 0-203-83790-8 Master e-book ISBN

ISBN: 978–0–8058–5882–2 (hbk)

4

To Ogden Lindsley and Murray Sidman

Giants of our field upon whose shoulders we proudly stand

5

Contents

LIST OF BOXES

PREFACE

PART ONE THE NATURAL SCIENCE OF BEHAVIOR

1 SCIENCE AND SCIENTIFIC BEHAVIOR

Introduction

Scientists as Behaving Organisms

Science as the Behavior of Scientists

Control by the Subject Matter

Scientific Method

The Products of Science

Research Methods and Service Delivery

2 BEHAVIOR AS A SCIENTIFIC SUBJECT MATTER

The Evolution of Conceptions of Behavior

Toward a Scientifically Useful Definition of Behavior

A Working Definition of Behavior

Some Implications

3 ASKING EXPERIMENTAL QUESTIONS

The Nature of Experimental Questions

The Functions of Experimental Questions

PART TWO MEASUREMENT

4 SELECTING AND DEFINING RESPONSE CLASSES

Strategies of Selecting and Defining Response Classes

Tactics of Selecting and Defining Response Classes

5 DIMENSIONAL QUANTITIES AND UNITS OF MEASUREMENT

Introduction

Properties, Dimensional Quantities, and Units

Tactical Issues

6 OBSERVING AND RECORDING

6

Strategic Issues

Tactics of Observing and Recording

7 ASSESSING MEASUREMENT

Strategic Issues

Tactical Options

PART THREE DESIGN

8 BEHAVIORAL VARIABILITY

Strategic Issues

Sources of Behavioral Variability

9 STEADY STATES AND TRANSITIONS

The Steady-State Strategy

Steady States

Transitions

10 STRATEGIC ISSUES IN EXPERIMENTAL DESIGN

Experimental Design and Reasoning

Strategic Issues

Notation of Experimental Designs

11 CREATING EXPERIMENTAL DESIGNS

Introduction

Single Baseline Designs

Multiple Baseline Designs

Turning Designs into Experiments

PART FOUR INTERPRETATION

12 ANALYZING BEHAVIORAL DATA

Data Analysis Strategies

Graphical Analytical Tactics

Statistical Analytical Tactics

13 INTERPRETING EXPERIMENTS

Interpretive Behavior

Sources of Control

Generality

Evaluating Interpretations

GLOSSARY

7

REFERENCES

AUTHOR INDEX

SUBJECT INDEX

8

9

List of Boxes

BOX 1.1 Are Scientists Different? BOX 1.2 Rule-Governed versus Contingency-Shaped Behavior BOX 2.1 Inner “Causes” BOX 2.2 The Dead Man’s Test BOX 2.3 Is Thinking a Behavior? BOX 2.4 Traits and Colloquial Language BOX 2.5 Theory, Concepts, and Observability BOX 2.6 Parsimony BOX 2.7 Pure versus Quasi-Behavioral Research BOX 3.1 Research Styles BOX 3.2 Thematic versus Independent Research Styles BOX 3.3 Advocacy Research BOX 3.4 Experimental Questions versus Hypotheses BOX 3.5 The Null Hypothesis Game BOX 3.6 The Hypothetico-Deductive Method BOX 3.7 Ethical Considerations in Behavioral Research BOX 3.8 Serendipity BOX 4.1 Units of Analysis versus Units of Measurement BOX 4.2 Behavior, Response Classes, and Responses BOX 4.3 Another Kind of Response Class? BOX 4.4 Parent: “What Did You Do Today?” Child: “Nothing” BOX 4.5 Operational Definitions and Behavioral Measurement BOX 5.1 Frequency versus Rate BOX 5.2 A Tale of Two Frequencies BOX 5.3 How Many Dimensional Quantities Are There? BOX 5.4 Is Probability a Dimensional Quantity? BOX 6.1 How Do You Measure Slouching? BOX 6.2 What About Rating Scales? BOX 6.3 The Problem of Measurement Reactivity BOX 7.1 Reliability in the Social Sciences BOX 7.2 The Relationship Between Accuracy and Reliability BOX 7.3 Validity in the Social Sciences BOX 8.1 Free Will versus Determinism BOX 8.2 What is Inside the Organism? BOX 9.1 Measuring One Participant Many Times versus Many Participants Once BOX 9.2 One Data Point at a Time BOX 9.3 How Long Should Each Phase Last? BOX 10.1 Levels of Empirical Elegance BOX 10.2 Why Scientists Do Not Talk About Causes BOX 10.3 Why Psychology Likes Lots of Participants BOX 11.1 Risks of Between-Subject Comparisons BOX 11.2 Experimentum Crucis BOX 11.3 Do Experimental Designs Have to be Perfect? BOX 12.1 How Science Deals with Subjectivity BOX 12.2 Do We Need Data Analysis Police?

10

BOX 12.3 The Cumulative Recorder BOX 12.4 Does a Graphic Approach to Data Analysis Need Defending? BOX 13.1 When You Cannot Get There From Here BOX 13.2 Internal and External Validity BOX 13.3 Inferential Statistics as Interpretation BOX 13.4 Do Attitudes Toward Interpretation Vary Across Disciplines? BOX 13.5 Are We Preoccupied with Generality Across Individuals?

11

Preface

Our decision to write a third edition of Strategies and Tactics of Behavioral Research arose from our experiences, as well as those of many colleagues, in helping students to understand this material. We discovered many ways of improving our discussion of this approach to studying individual behavior, but we also saw that the audience for our text was changing. We had written the second edition primarily for doctoral students in psychology, education, and other academic fields specializing in the experimental study of behavior. However, we also observed considerable growth in Master’s programs, especially those preparing practitioners working toward newly established credentials in the field of Applied Behavior Analysis.

We have written this third edition, no longer accompanied by a readings volume, to meet these changing needs. Although the core content and chapter divisions of the second edition remain relatively untouched, we have discarded many of the secondary issues and digressions that encumbered discussions in the previous edition. Instead, we have focused on describing and explaining the primary material in a straightforward and relatively simple narrative. We have composed both sentences and text_indentgraphs with unwavering attention to the needs of student readers.

Because many students learning this material may be planning careers as practitioners rather than as researchers, this third edition considers the relevance of methodological procedures and decisions for the delivery of professional services. This is not a stretch, of course. Many methodological requirements of professional practice originated in behavioral research methods, and the fact that the Behavior Analyst Certification Boardfi mandates coursework in this area clarifies the need to address the role of research methods in service scenarios.

Aside from substantive and literary revisions, we have also added a number of features that will make the volume more effective as a textbook. New terms are now identified in bold face type and are formally defined in indented tinted boxes, as well as in the glossary at the end of the book. There are now many tables that summarize the main points of a discussion, and they are joined by considerably more figures, including figures adapted from journal articles. Chapter end matter now includes not only study guides, but a chapter summary, suggested readings, discussion topics, and exercises. This material is also available on an instructor’s website, www.psypress.com/behavioral-research, which further includes lecture outlines and test items.

In sum, although the chapter topics are unchanged from the second edition, this third edition otherwise provides a very different experience for student and instructor. Substantial improvements in clarity of exposition and the addition of new learning aids offer a more appealing learning opportunity, and instructors will find it easier to take advantage of students’ interests. Incorporating the methodological interests of practitioners into each chapter extends this appeal to a growing professional discipline, a field partly defined by its respect for the highest standards of scientific practice.

We would like to thank our many students and colleagues who have offered valuable feedback along the way. We would especially like to thank Ryan Zayac at Central Washington University, who prepared many of the supplementary materials, and Wayne Fuqua at Western Michigan University, who served as a reviewer.

Finally, as with the second edition, the first author has assumed primary responsibility for this edition, although with the active intellectual support and guidance of the second author. Our contributions to the foundations on which the third edition is based remain thoroughly intertwined.

—James M. Johnston

—Henry S. Pennypacker

12

13

Those who fall in love with practice without science are like a sailor who enters a ship without a helm or a compass, and who never can be certain whither he is going.

—Leonardo da Vinci

14

Part One The Natural Science of Behavior

15

Chapter One Science and Scientific Behavior

INTRODUCTION

SCIENTISTS AS BEHAVING ORGANISMS

Are Scientists Different?

The Three-Term Contingency

SCIENCE AS THE BEHAVIOR OF SCIENTISTS

Scientific Behavior

Examples of Methodological Choices

CONTROL BY THE SUBJECT MATTER

SCIENTIFIC METHOD

THE PRODUCTS OF SCIENCE

RESEARCH METHODS AND SERVICE DELIVERY

Research versus Practice

Role of Research Methods in Practice

The chief problem of science is the scientist

—D. L. Watson

16

Introduction

The scientific achievements of the 20th and now the 21st century have changed our lives in profound ways, and many people have come to revere science as an almost magical endeavor. We have grown confident that, given enough time and money, science can solve most of life’s problems, and we may be right. Those who devote their lives to doing research are held in high regard, and scientific careers are now rewarding not just professionally but financially.

And yet, most people do not understand how science really works. The average citizen does not have contact with the daily activities of scientists, so it is not surprising that it is hard to appreciate how scientific pursuits are different from everyday interests. A newspaper article about a scientific discovery inevitably stops short of explaining exactly how it was accomplished or describing the years of research that made the breakthrough possible.

Even researchers are likely to “miss the forest for the trees” as they focus on their own areas of interest. Most scientists are trained in the research literature and methods of their own specialties. They usually do not appreciate the underlying features of experimental methods common to all disciplines that make science a special way of learning about the world. There are some writers who specialize in studying science as an industry or enterprise, and a few others focus on science from a philosophical point of view. However, the critical essence of science—the features that are fundamental to its effectiveness—often eludes these writers too.

17

Scientists as Behaving Organisms

Are Scientists Different?

Understanding the essential features of a scientific search for nature’s secrets requires looking at the behavior of scientists is a particular way. This perspective is based on appreciating the fundamental processes underlying how human behavior actually works, which is itself the product of a field of scientific study. This point of view is very different from how we are taught by the culture to view human behavior. For instance, although we learn to talk about what is going on “in the scientist’s head,” this only distracts us from noticing more important relationships between scientists’ behavior and their daily work environments.

The key to understanding how science works lies in acknowledging that scientists are behaving organisms. As such, there is no evidence that scientists are generally different from other people. In other words, they are not any smarter or more logical than others who earn advanced degrees (Mahoney, 1976).

It is also important to recognize that the behavior of scientists, just like the behavior of all human and nonhuman animals, is as much a part of nature as any other scientific subject matter and can be approached with the same experimental tools. In fact, the scientific study of behavior over the past 100 years or so has revealed many now well-established laws about the variables that determine an organism’s behavior. This research has shown that, in addition to whatever genetic endowment each individual is born with, the major influence on behavior is each person’s moment by moment experiences as he or she goes through life.

Box 1.1

Are Scientists Different?

In a somewhat humorously disrespectful book, titled Scientist as Subject: The Psychological Imperative (1976), Michael Mahoney delights in puncturing many illusions about scientists. For instance, he argues that scientists are not more intelligent than others, often illogical in their work, often selective and biased in their treatment of data, passionate in their prejudices, frequently dogmatic in their opinions, sometimes selfish and ambitious in pursuing personal recognition and defending territory, often secretive about their findings, and fond of spinning “truths” in hypotheses and theories before the data warrant. His general point is that scientists are not special, but just like the rest of us.

This list of shortcomings should suggest that, however well science usually works, it can go awry. Although it is relatively uncommon, scientists are sometimes dishonest with themselves (when they interpret data in a way they know is incorrect) or with their peers (when they publish findings they know are false). Fortunately, science has some effective self-corrective mechanisms. In brief, scientific research includes a public component that keeps innocent bias and blatant dishonesty at a minimum. Scientists must publish complete reports of their methods, data, and analytical procedures before other scientists will pay any attention to their findings. Some of their colleagues who are interested in the same topic will repeat the experiments, which will either confirm the original conclusions or cast doubt on them and lead to still further experimental efforts to see what the truth really is.

Scientific ethics is an important part of graduate training. If a researcher is found to have broken the cardinal rule of honesty, there are a variety of sanctions that may be applied. These sanctions include being prevented from being considered for federal grants, being fired, and even being prosecuted under civil or criminal statutes.

The Three-Term Contingency

18

The interactions between each action or response and its environmental context involve biologically mediated processes called conditioning or learning. The laws of conditioning describe exactly how the relationship between an individual’s responses and the environmental events surrounding them affects his or her behavior in the future. It may be easiest to understand how learning works in terms of what is called the three-term contingency. In this context, a contingency refers to relationships between instances of behavior (responses) and their environmental antecedents and consequences. Figure 1.1 identifies the three terms that define the basic contingencies underlying all behavior. Environmental events that immediately precede responses are called antecedent events or stimuli, and those that follow responses are called consequent events. (These terms are often shortened to “antecedents”

Fig. 1.1. Schematic representation of the three-term contingency.

and “consequences.”) The contingencies involving these antecedent events, responses, and consequent events describe different relationships between a particular behavior or action and those features of the environment that precede or follow it. These relationships are termed respondents, operants, and discriminated operants.

Conditioning. The process of changing a behavior that involves interactions between responses and environmental events whose effects depend on the processes of reinforcement and punishment.

Learning. The relatively enduring changes in behavior that result from conditioning processes.

Contingency. A relationship between a class of responses and a class (or classes) of stimuli. Implies nothing about the nature of the relationship or its effects.

Three-term contingency. A set of functional relationships among distinct classes of antecedent stimuli, responses, and consequent stimuli that together constitute the model of how behavior is influenced by the environment.

Antecedent event. An environmental event that occurs immediately before a response. Used generically when it is not certain what function the event serves.

Consequent event. An environmental event that occurs immediately after a response. Used generically when it is not certain what function the event serves.

Respondent. A class of responses elicited by a particular unconditioned or conditioned antecedent stimulus.

Operant. A class of responses defined by a functional relation with a class of consequent events that immediately follow those responses.

Discriminated operant. A class of responses that are functionally related to classes of both antecedent and consequent stimuli.

The three-term contingency is a useful way of summarizing how behavior works because the biology of organisms, together with their unique life experiences, makes their behavior especially sensitive to certain kinds of environmental events. For example, humans are from birth especially sensitive to sweet-tasting substances, and their experiences with particular foods (e.g., cookies) can make such stimuli especially important. When a child’s behavior such as standing on a chair to reach the kitchen counter where the cookies

19

are kept leads to the consequence of eating the cookie, that behavior is more likely to occur again in the future. This instance of the three-term contingency is called positive reinforcement. If the same behavior resulted in a different kind of consequence such as falling off the chair or being scolded by a parent, the behavior might be less likely to occur again. This instance of the three-term contingency is called positive punishment. Of course, there are other kinds of contingencies as well.

Positive reinforcement. A class of procedures involving the occurrence of a stimulus immediately following responding that results in an increase in some aspect of the response class over baseline levels.

Positive punishment. A class of procedures involving the occurrence of a stimulus immediately following responding that results in a decrease in some aspect of the response class over baseline levels.

The relationship between instances of behavior and the antecedent side of the three-term contingency also influences behavior, although somewhat differently. If a certain behavior occurs when a particular environmental event is present and the behavior then produces a reinforcing consequence, that antecedent event comes to serve a sort of signaling function. The behavior (a discriminated operant) is then more likely to occur when similar antecedent events (called discriminative stimuli) are present than when they are not present. For instance, if you drive up to a store and see an “Open” sign on the door, you will usually get out of the car and go in because in the past such behavior has resulted in reinforcing consequences. Not surprisingly, you would be less likely to get out of the car and try to go in if such behavior has been followed by a punishing consequence (the door is locked) in the presence of a different antecedent (a “Closed” sign).

Discriminative stimuli. Stimuli that have acquired the function of setting the occasion for a behavior to occur. A behavior is more likely to occur in the presence of a discriminative stimulus than in its absence. Abbreviated SD.

Other antecedent stimuli have functions that depend less on the consequences of responding and more on an organism’s biology. For example, when a stimulus such as a puff of air contacts our eye, we blink, and it is difficult to avoid doing so. Even innocuous events paired with a puff of air elicit the same kind of blinking response. This behavior is an example of a respondent. (See Catania, 2007, for a more detailed treatment of the three-term contingency and the resulting classes of behavior.)

The repertoire of each of us at any point in our lives is largely the result of our history of contingencies like these. What we do or do not do, our skills, our emotions, and the unique features of our individuality are largely a function of the laws of conditioning. As with other laws of nature, these relationships are at work even if we are unaware of what is going on, and no one is exempt from them, even for a moment.

Over the years, scientific study of the relationship between behavior and the environment represented by the three-term contingency has been very fruitful. Researchers have not only learned a great deal about the basic components of conditioning, but they have also learned how to apply these fundamental principles to human behavior, especially under everyday circumstances. As a result, a still developing but powerful technology for changing behavior has emerged. This technology, called applied behavior analysis, is now used in diverse areas, including mental retardation, autism, brain injury, education, business, medicine, and sports (Austin & Carr, 2000).

Applied behavior analysis. A phrase that may refer to (a) the field of research that focuses on applying the findings of a basic science (the Experimental Analysis of Behavior) concerning fundamental processes of conditioning to address the need for changing behavior under everyday circumstances (b) the behavior change technology developed by applied researchers, or (c) the field encompassed by both applied research and delivery of the resulting technology.

20

21