D journal article by Clark and the following summary:

Media  Will  Never  Influence  Learning     Richard  E.  Clark     The  purpose  of  this  discussion  is  to  explain  and  sharpen  different  points  of  view   about  the  impact  of  media  and  attributes  of  media  on  learning,  motivation  and   efficiency  gains  from  instruction.  This  paper  is  an  attempt  to  summarize  my   arguments  about  the  research  and  theory  in  this  area  and  to  respond  to  Robert   Kozma’s  criticism  of  my  earlier  discussion  of  these  issues.  I  will  first  briefly   summarize  my  arguments  about  media  effects;  next  I  will  attempt  to   characterize  the  many  reactions  to  the  controversial  claim  that  media  do  not   influence  learning  or  motivation.  Finally,  I  will  respond  to  the  specific  criticisms   advanced  by  Robert  Kozma  this  issue.     INTRODUCTION     A  Brief  History  of  Media  Research     The  claim  of  “no  learning  benefits”  from  media  has  been  made  and  substantiated   many  times  in  the  past.  Many  researchers  have  argued  that  media  have   differential  economic  benefits  but  no  learning  benefits.  For  example,  in  the  first   Handbook  of  Research  on  Teaching,  Lumsdaine  (1963)  concluded  that  the   benefits  of  media  were  primarily  economic  and  that  their  use  was  “to  develop   the  technology  of  instructional  method”  (p.  669).  Mielke  (1968)  was  eloquent  on   the  topic  in  an  article  he  wrote  for  the  now-­‐defunct  Educational  Broadcasting   Review  (Mielke,  1968)  titled  “Questioning  the  Questions  of  ETV  Research.”  He   predicted  that  adequately  designed  research  on  the  learning  benefits  of  various   media  would  yield  no  significant  differences  between  treatments.  Another   example  of  the  argument  came  from  this  century’s  most  prolific  media  research   reviewer,  Wilbur  Schramm  (1977),  who  claimed  that  learning  is  influenced  more   by  the  content  and  instructional  strategy  in  a  medium  than  by  the  type  of   medium.  Levie  &  Dickie  (1973)  made  the  same  point  as  Schramm  in  their   chapter  on  media  and  technology  research  in  the  Second  Handbook  of  Research   on  Teaching.  Finally,  this  was  the  conclusion  I  reached  with  Gavriel  Salomon  in   our  review  in  the  third,  and  most  recent,  Handbook  of  Research  on  Teaching   (Clark  &  Salomon,  1986).  It  is  therefore  a  bit  of  a  mystery  why  my  restatement  of   the  claim  of  “no  differences  expected”  a  decade  ago  (Clark,  1983)  received  so   much  attention.  A  colleague  has  suggested  that  previous  discussions  of  this   argument  have  put  the  claim  in  very  tentative  terms  (as  befits  our  training  as   researchers)  and  left  the  door  open  to  media  effects  on  learning.  I  made  the   explicit  and  clear  claim  that  there  were  no  learning  benefits  possible  and  urged   that  we  not  continue  to  waste  effort  on  the  question  until  a  “new  theory”  was   developed.  I  intended  to  stimulate  discussion  and  I  was  not  disappointed.  Before   I  describe  the  reactions  however,  the  discussion  turns  to  a  brief  review  of  the   argument.     The  Important  Aspects  of  the  Learning  From  Media  Argument

My  early  articles  (Clark,  1983,  1985a)  claimed,  in  part,  that  media  are  “mere   vehicles  that  deliver  instruction  but  do  not  influence  student  achievement  any   more  than  the  truck  that  delivers  our  groceries  causes  changes  in  our  nutrition”   (1983,  p.  445).  The  articles  presented  evidence  in  support  of  the  hypothesis  that   instructional  methods  had  been  confounded  with  media  and  that  it  is  methods   which  influence  learning.  Further,  I  claimed,  that  any  necessary  teaching  method   could  be  designed  into  a  variety  of  media  presentations.  I  also  questioned  the   unique  contributions  of  media  attributes.  Gavriel  Salomon  and  others  (Salomon,   1979)  had  argued  that  it  was  not  the  medium  which  influenced  learning  but   instead  certain  attributes  of  media  that  can  be  modeled  by  learners  and  can   shape  the  development  of  unique  “cognitive  processes.”  Examples  of  media   attributes  are  the  capacity  of  television  and  movies  to  “zoom”  into  detail  or  to   “unwrap”  three-­‐dimensional  objects  into  two  dimensions.  The  problem  with  the   media  attribute  argument  is  that  there  is  strong  evidence  that  many  very   different  media  attributes  accomplish  the  same  learning  goal  (for  example,  there   are  a  variety  of  equally  effective  ways  to  highlight  details  other  than  zooming).  In   every  attempt  to  replicate  the  published  media  attribute  studies  (see  studies   cited  by  Clark,  1985c;  Clark  &  Sugrue,  1988),  a  number  of  very  different  media   attributes  served  the  same  or  similar  cognitive  functions.  This  point  is  critical  to   my  argument.  If  there  is  no  single  media  attribute  that  serves  a  unique  cognitive   effect  for  some  learning  task,  then  the  attributes  must  be  proxies  for  some  other   variables  that  are  instrumental  in  learning  gains.     A  Replaceability  Challenge     It  may  be  useful  to  apply  the  following  “armchair  experimental  criteria”  to  any   situation  where  it  appears  that  media  or  attributes  of  media  have  been   instrumental  in  fostering  learning  gains:  We  need  to  ask  whether  there  are  other   media  or  another  set  of  media  attributes  that  would  yield  similar  learning  gains.   The  question  is  critical  because  if  different  media  or  attributes  yield  similar   learning  gains  and  facilitate  achievement  of  necessary  performance  criteria,  then   in  a  design  science  or  an  instructional  technology,  we  must  always  choose  the   less  expensive  way  to  achieve  a  learning  goal.  We  must  also  form  our  theories   around  the  underlying  structural  features  of  the  shared  properties  of  the   interchangeable  variables  and  not  base  theory  on  the  irrelevant  surface  features.   I  challenge  Robert  Kozma  and  other  colleagues  in  this  area  to  find  evidence,  in  a   well  designed  study,  of  any  instance  of  a  medium  or  media  attributes  that  are  not   replaceable  by  a  different  set  of  media  and  attributes  to  achieve  similar  learning   results  for  any  given  student  and  learning  task.  This  replaceability  test  is  the  key   to  my  argument  since  if  a  treatment  can  be  replaced  by  another  treatment  with   similar  results,  the  cause  of  the  results  is  in  some  shared  (and  uncontrolled)   properties  of  both  treatments.  Of  course  it  is  important  for  instructional   designers  to  know  that  there  are  a  variety  of  treatments  that  will  produce  a   desired  learning  goal.  However,  the  utility  of  this  knowledge  is  largely  economic.   The  designer  can  and  must  choose  the  less  expensive  and  most  cognitively   efficient  way  to  represent  and  deliver  instruction.  It  cannot  be  argued  that  any   given  medium  or  attribute  must  be  present  in  order  for  learning  to  occur,  only   that  certain  media  and  attributes  are  more  efficient  for  certain  learners,  learning   goals  and  tasks.  This  allows  the  discussion,  and  our  mental  set  as  theorists,  to

shift  from  media  attributes  as  causal  in  learning  to  media  attributes  as  causal  in   the  cost-­‐effectiveness  of  learning.  While  this  may  seem  a  small  shift  in  the   representation  of  the  problem,  it  would  have  major  consequences  for   instructional  research,  theory  and  for  design.  Cognitive  instructional  theory  can   shift  to  a  concern  with  instructional  methods  that  support  the  structural   elements  of  cognitive  processing  during  learning  and  transfer.     What  is  an  Instructional  Method,  and  How  is  it  Different  From  a  Medium?     An  instructional  method  is  any  way  to  shape  information  that  activates,   supplants  or  compensates  for  the  cognitive  processes  necessary  for  achievement   or  motivation  (Salomon,  1979).  For  example,  students  often  need  an  example  to   connect  new  information  in  a  learning  task  with  information  in  their  prior   experience.  If  students  cannot  (or  will  not)  give  themselves  an  adequate   example,  an  instructional  presentation  must  provide  it  for  them.  It  is  likely  that   many  different  types  of  examples,  with  many  different  attributes  presented  by   many  different  media  would  serve  similar  cognitive  functions  for  any  given   student.  Instructional  technology  attempts  to  specify  the  need  for  and  type  of   instructional  methods  required  for  the  essential  psychological  support  of   students  as  they  learn.  Delivery  technology  formats  and  packages  essential   instructional  methods  based  on  available  resources  and  the  cost-­‐effectiveness   qualities  of  media  attributes  for  specific  learners  and  learning  contexts.     A  Confusion  of  Technologies     In  a  presentation  for  the  Association  for  Educational  Communications  and   Technology  at  their  1987  Atlanta  convention  I  attributed  our  media  research  and   practice  problem  to  a  “confusion  of  technologies”  (Clark,  1987).  Instructional  or   training  design  technologies  draw  on  psychological  and  social-­‐psychological   research  to  select  necessary  information  and  objectives  (as  a  result  of  task   analysis)  and  design  instructional  methods  and  environments  that  enhance   achievement.  A  very  different  technology  -­‐-­‐  delivery  technology  -­‐-­‐  is  necessary  to   provide  efficient  and  timely  access  to  those  methods  and  environments.  Both   technologies  make  vital  but  very  different  contributions  to  education.  Delivery   technologies  influence  the  cost  and  access  of  instruction  and  information.  Design   technologies  make  it  possible  to  influence  student  achievement.  In  my  view,   there  is  a  long  history  of  a  basic  confusion  between  these  two  technologies  that   strangles  our  study  of  the  contributions  of  media.     Motivation  With  Media     I  also  claimed  that  media  not  only  fail  to  influence  learning,  they  are  also  not   directly  responsible  for  motivating  learning.  Here  I  agreed  wholeheartedly  with   the  views  of  Salomon  (1984)  and  others  who  draw  on  the  new  cognitive  theories   which  attribute  motivation  to  learners’  beliefs  and  expectations  about  their   reactions  to  external  events  -­‐-­‐  not  to  external  events  alone.  There  is  compelling   research  evidence  that  students’  beliefs  about  their  chances  to  learn  from  any   given  media  are  different  for  different  students  and  for  the  same  students  at   different  times.

WHAT  ARE  THE  COUNTER-­‐ARGUMENTS?     While  there  have  been  a  great  variety  of  counter-­‐arguments,  I  categorize  them   into  four  types  of  rebuttals  to  the  basic  argument;  1)  reasoning  based  on  the   usual  uses  of  a  medium:  2)  the  meta-­‐analysis  evidence;  3)  problems  with   empiricism  and  logical  positivism;  and  4)  a  lingering  hope  for  media  attributes.     Usual  Uses     The  majority  of  informal  letters  which  I  received  took  Marshall  McLuhan’s  view   that  media  and  method  were  identical  and  inseparable.  I  think  of  it  as  the  usual   uses  argument.  It  seems  to  develop  because  media  specialists  generate  beliefs   about  the  “best”  contents  and  methods  for  each  medium.  So,  for  example,   television  is  usually  thought  to  convey  “realistic,”  visual,  real  time,  documentary   information.  Computers  most  often  give  semantically  dense  simulations  of   complex  phenomena  as  well  as  drill  and  practice.  Textbooks  have  tended  to  focus   on  the  development  of  encyclopedic  knowledge  with  illustrated  examples  and   heavy  verbal  content.  Many  writers  seemed  to  suggest  that  these  methods  were   somehow  intrinsic  to  a  given  medium.  My  argument  is  that  the  usual  uses  of  a   medium  do  not  limit  the  methods  or  content  it  is  capable  of  presenting.   Computers  can  present  realistic  visual,  real-­‐time  documentary  information,  and   television  can  present  semantically  dense  simulations.  The  method  is  the   simulation  or  the  real-­‐time  depiction.  A  good  example  of  this  point  was   uncovered  in  one  of  the  earliest  and  largest  (and  best  designed)  studies  of   computers  by  Suppes  (in  Clark,  1983)  during  the  1960s.  In  a  study  of  computers   versus  teachers  using  drill  and  practice  in  mathematics,  Suppes  found  that  one  of   his  control  school  districts  had  messed  up  the  data  collection  by  delivering  more   drill  and  practice  in  mathematics  than  was  permitted  by  the  study-­‐using   teachers  and  not  computers.  The  result  was  that  in  that  school  district,   mathematics  achievement  increased  at  exactly  the  same  rate  as  it  did  in  districts   where  computers  were  giving  drill  and  practice.  Suppes  concluded  then  that  it   was  not  the  medium  but  the  drill  and  practice  method  that  influenced   achievement  but  he  noted  that  the  cost  of  the  intervention  might  have  been  less   with  computers.     Meta-­‐analytic  Evidence     Meta-­‐analytic  reviews  of  media  research  have  produced  evidence  for  the  positive   learning  benefits  of  research  with  various  media,  particularly  computers  (see   reviews  in  Clark,  1983,  1985a,b).  These  analyses  report  an  approximate  20   percent  increase  in  final  exam  scores  following  computer-­‐based  instruction   (CBI)  when  it  is  compared  to  traditional  forms  of  instruction  (generally  live   instruction).  After  a  number  of  discussions,  Kulik  (1985),  one  of  the  primary   authors  of  many  of  the  meta-­‐analytic  surveys,  agreed  that  it  is  not  the  computer   but  the  teaching  method  built  into  CBI  that  accounts  for  the  learning  gains  in   those  studies.  More  important,  Kulik  agreed  that  the  methods  used  in  CBI  can  be

and  are  used  by  teachers  in  live  instruction  (Kulik,  1985).  In  fact,  I  reanalyzed  a   30  percent  sample  of  the  studies  he  used  and  found  that  when  the  same   instructional  design  group  produces  CBT  and  presents  the  live  instruction  with   which  it  is  compared  in  many  studies,  there  is  no  achievement  difference   between  the  CBT  and  live  conditions  (Clark,  1985c).  To  characterize  the  fact  that   these  powerful  methods  can  be  and  are  used  in  a  variety  of  media,  Kulik   employed  the  catchy  phrase  .  .  .  diffusion  of  the  innovative  treatment  to  the   control  condition”  (Kulik,  1985,  p.  386).  This  statement  more  or  less   acknowledges  that  most  of  the  studies  which  are  grist  for  the  meta-­‐analytic  mill,   are  confounded  because  the  teaching  method  is  not  controlled  (if  it  were   controlled  it  could  not  “diffuse”  anywhere).     Empiricism  Envy     Cunningham  (1986)  did  not  dispute  my  argument  that  media  made  no  difference   to  learning  or  motivation  but  argued  against  my  empirically-­‐based  claims  that   instructional  methods  were  responsible  for  achievement  gains.  Cunningham  is   well  trained  as  a  quantitative  researcher  but  is  increasingly  attracted  to   qualitative  research  and  not  to  empirical  method  or  logical  positivism.  I  think  his   argument  was  with  the  unreconstructed  empiricism  of  my  argument  rather  than   with  the  theoretical  claims.  I  agreed  with  him  that  my  claim  that  it  is   instructional  methods  which  account  for  learning  gains  is  a  hypothesis,  not  a   conclusion  (Clark,  1986).     Necessary  Media  Attributes     A  number  of  researchers  have  argued  with  my  claim  about  the  unique   contribution  of  what  Gavriel  Salomon  calls  “media  attributes.”  Remember  that   the  capacity  of  movies  to  zoom  into  detail  or  to  unwrap  three  dimensional   objects  has  led  some  to  claim  that  new  media  have  attributes  that  make  unique   cognitive  representations  available  (Salomon,  1979).  A  few  go  so  far  as  to  claim   that  new  “intelligence”  might  be  possible  as  a  result  of  exposure  to  these   attributes  (for  example,  Salomon,  Perkins  &  Globerson,  1991).  I  presented   evidence  (Clark,  1985a,b)  that  many  very  different  media  attributes  could   accomplish  the  same  learning  goal  (i.e.  there  were  a  variety  of  equally  effective   ways  to  highlight  details  other  than  zooming)  and  so  no  one  media  attribute  has   a  unique  cognitive  effect.  Petkovitch  and  Tennyson  (1984)  took  me  to  task  with   an  argument  which  I  still  do  not  completely  understand  but  which  seems  to  be   related  to  the  attributes  argument.  They  seemed  to  agree  that  media  comparison   studies  are  useless  but  claimed  that  certain  media  attributes  make  necessary   contributions  to  learning.  The  evidence  they  offered  was  a  study  where  a   computer  simulation  was  used  to  teach  students  some  skills  required  to  fly  a   plane.  I  responded  that  people  learned  to  fly  planes  before  computers  were   developed  and  therefore  the  media  attributes  required  to  learn  were  obviously   neither  exclusive  to  computers  nor  necessary  for  learning  to  fly.  A  similar  and   more  extensive  argument  has  been  made  by  Kozma  (1991).  The  next  section  of   this  paper  addresses  Kozma’s  (1994)  points  in  this  debate  and  in  his  earlier   work.

Kozma’s  Reframed  Argument  about  the  Influence  of  Media  on  Learning     First,  it  is  important  to  notice  that  Kozma  (1994)  agrees  with  me  that  there  is  no   compelling  evidence  in  the  past  70  years  of  published  and  unpublished  research   that  media  cause  learning  increases  under  any  conditions.  Like  all  other   researchers  who  have  made  a  careful  study  of  the  arguments  and  research   studies  (e.g.,  Winn,  1990),  he  reaches  a  conclusion  that  is  compatible  with  my   claims  (Clark,  1983).  Kozma  then  asks  that  we  reframe  the  argument  about  the   future  possibilities  of  media  as  causal  agents  in  learning.  In  his  discussion  (this   issue)  Kozma  interprets  my  claim  that  media  attributes  are  not  “necessary”   variables  in  learning  studies  by  quoting  scholars  from  the  philosophy  of  science   who  suggest  that  “sufficient  conditions”  are  important  to  a  design  science.   Kozma  states  that  “.  .  .  scientists  concerned  with  necessary  conditions  are  those   interested  in  eliminating  something  undesirable,  such  as  disease  .  .  .  On  the  other   hand,  scientists  interested  in  the  production  of  something  desirable,  such  as   learning,  are  concerned  with  establishing  conditions  that  are  sufficient  to  bring  it   about.  .  .  Necessary  conditions  are  those  in  whose  absence  an  event  cannot  occur,   while  sufficient  conditions  are  those  in  whose  presence  an  event  must  occur”   (1994,  p.  14).  Kozma  offers  those  studies  where  media  attribute  treatments  are   sufficient  for  learning  as  evidence  for  the  value  of  attribute  research.     This  argument  contains  some  of  the  most  important  elements  of  our   disagreement.  My  reply  is  relatively  simple.  When  a  study  demonstrates  that   media  attributes  are  sufficient  to  cause  learning,  the  study  has  failed  to  control   for  instructional  method  and  is  therefore  confounded.  It  is  true  that  in  some   cases  instructional  treatments  containing  media  attributes  are  sufficient  to  cause   learning.  When  this  happens,  the  necessary  condition  to  cause  learning  is   embedded  in  the  sufficient  treatment.  We  know  that  the  active  ingredient  in   successful  media  treatments  is  not  the  media  attributes  because  in  all  known   attempts  to  replicate  these  studies,  different  attributes  produce  similar  learning   results-­‐-­‐-­‐-­‐provided  that  the  required  instructional  method  is  present  in  the   compared  versions  of  the  media  attributes.  That  necessary  condition  or  “active   ingredient”  of  the  treatment  which  was  sufficient  to  cause  learning  from   instruction  is  best  characterized  as  an  instructional  method  which  activates,   compensates  or  supplants  the  cognitive  processes  necessary  for  learning  to   occur  (Salomon,  1979).  In  other  words,  any  treatment  that  is  sufficient  for   learning  must  embody  whatever  is  necessary  to  cause  learning.     Structural  and  Surface  Features  of  Research  Constructs     The  concepts  of  necessary  and  sufficient  are  similar  to  the  concepts  of  structural   and  surface  features  in  research  on  the  role  of  analogies  in  transfer  during   problem  solving  (for  example,  Gick  &  Holyoak,  1987).  Surface  features  of   analogies  are  those  whose  characteristics  are  of  only  limited  and  domain-­‐specific   importance.  For  example,  in  science  and  mathematics  instruction,  irrelevant   features  of  analogies  often  cause  misconceptions  in  learning.  When  told  that  an   atom  is  like  the  solar  system,  students  often  believe  that  electrons  must  attract   each  other  and  be  attracted  to  the  nucleus  of  the  atom  because  planets  are   attracted  to  each  other  and  to  the  sun  by  gravity.  Gravity  is  a  surface  feature  that

is  important  to  understanding  the  solar  system  but  not  the  atom.  The  structural   (necessary)  features  that  underlie  both  systems  are  central  bodies  (nucleus,  sun)   that  are  encircled  by  rotating  spheres  (electrons,  planets).  The  point  that  I  had   hoped  to  make  in  my  earlier  reviews  is  that  media  attributes  are  surface  features   of  learning  systems.  Those  surface  features  may  affect  the  economics  but  not  the   learning  effectiveness  of  instruction.  Instructional  methods  are  structural   (necessary)  features  of  media  attribute  studies.  On  the  other  hand,  instructional   methods  may  be  surface  features  of  treatments  concerned  with  the  economics  of   learning.     I  accept  the  point  that  whenever  learning  occurs,  some  medium  or  mix  of  media   must  be  present  to  deliver  instruction.  However,  if  learning  occurs  as  a  result  of   exposure  to  any  media,  the  learning  is  caused  by  the  instructional  method   embedded  in  the  media  presentation.  Method  is  the  inclusion  of  one  of  a  number   of  possible  representations  of  a  cognitive  process  or  strategy  that  is  necessary   for  learning  but  which  students  cannot  or  will  not  provide  for  themselves.  Kozma   (1994)  accuses  me  of  creating  an  “unnecessary  and  undesirable  schism”  (p.  16)   between  method  and  medium.  My  claim  is  that  Kozma  has  confounded  the  two   constructs.  He  is  asking  you  to  consider  media  as  an  integral  aspect  of  method.  I   am  suggesting  that  if  we  take  his  advice,  we  will  continue  to  misinterpret  the   research  on  instructional  media  and  learning  and  continue  to  fail  in  our  efforts  to   construct  powerful  learning  environments  for  all  students.     All  methods  required  for  learning  can  be  delivered  by  a  variety  of  media  and   media  attributes.  It  is  method  which  is  the  “active  ingredient”  or  active   independent  variable  that  may  or  may  not  be  delivered  by  the  medium  to   influence  learning.  The  derivation  and  delivery  of  a  method  to  support  learning   is  always  necessary.  A  great  variety  of  media  “translations”  of  any  given  method   are  sufficient  to  cause  learning.  Therefore,  aside  from  the  identification  of   necessary  methods  for  learners  and  tasks,  it  is  important  to  derive  media  that   are  capable  of  delivering  the  method  at  the  least  expensive  rate  and  in  the   speediest  fashion.  Media  influence  cost  or  speed  (efficiency)  of  learning  but   methods  are  causal  in  learning.     Let  me  try  to  illustrate  my  point  one  more  time  with  a  medical  analogy.  People   often  have  preferences  for  one  or  another  way  to  use  a  chemical  medicine   prescribed  by  a  physician  to  improve  health.  Some  people  will  argue  for  tablets   and  others  for  liquid  or  injected  forms  of  treatment.  Is  it  sufficient  that  one  take  a   tablet  medicine?  Only  if  the  tablet  contains  the  active  ingredient  required  to  help   us.  Different  forms  of  a  medicine  might  help  us  provided  that  they  all  contain  the   same  method  or  active  ingredient.  The  different  forms  of  medicine  are  similar  to   different  media.  The  media  include  a  variety  of  tablets,  liquid  suspensions,   suppositories  or  injections.  All  of  these  different  media  are  often  capable  of   delivering  a  necessary  active  chemical  ingredient  with  different  levels  of   efficiency,  but  with  more  or  less  equal  effects  on  our  physical  symptoms.  The   active  chemical  ingredient  of  these  medical  media  is  analogous  to  the  necessary   method  in  instruction.  We  could  not  construct  an  adequate  medical  design   science  using  different  (sufficient)  forms  of  delivery  media  alone  and  it  would  be   irrelevant  to  measure  whether  these  delivery  forms  reduce  our  symptoms

(unless  we  were  concerned  with  the  effects  of  belief  on  health).  Scientific   arguments  about  the  necessary  or  sufficient  nature  of  oral  ingestion  of  tablets   versus  an  injection  of  the  liquid  form  of  a  medicine  would  be  largely  irrelevant.   Yet  the  discussion  of  delivery  forms  for  medicine  is  very  important.  Each  of  these   delivery  forms  has  different  efficiency  characteristics.  Some  forms  of  delivery  get   the  active  ingredient  to  the  patient  much  faster  (or  slower)  in  quantities  which   are  more  “pure”  or  more  “diluted”  at  greater  or  less  cost  to  the  patient.  For  this   reason  I  disagree  with  Kozma’s  suggestion  that  we  not  separate  medium  and   method  in  instructional  research.  Instead  I  claim  that  our  failure  to  Separate   medium  from  method  has  caused  enormous  confounding  and  waste  in  a  very   important  and  expensive  research  area.     We  continue  to  invest  heavily  in  expensive  media  in  the  hope  that  they  will   produce  gains  in  learning.  When  learning  gains  are  found,  we  attribute  them  to   the  delivery  medium,  not  to  the  active  ingredient  in  instruction.  When  learning   gains  are  absent,  we  assume  we  have  chosen  the  wrong  mix  of  media.  In  any   event,  many  educators  and  business  trainers  are  convinced  that  they  must  invest   scarce  resources  in  newer  media  in  order  to  insure  learning,  performance  or   motivational  gains.     Evidence  for  Kozma’s  View     Finally,  Kozma’s  evidence  for  his  view  is  to  describe  the  latest  round  of  studies   that  utilize  the  currently  fashionable  media  -­‐-­‐  ThinkerTools  to  teach  force  and   motion  problem  solutions  and  the  Jasper  Woodbury  Series  intended  to  help   students  solve  mathematics  problems  (see  Kozma,  this  issue).  These  studies   were  not  designed  so  that  their  results  would  provide  evidence  about  the  claims   being  made  in  this  dispute.  The  research  conducted  to  validate  these  very   creative  instructional  programs  did  not  control  for  the  sources  of  confounding   that  lie  at  the  root  of  the  argument.  The  computer-­‐  based  ThinkerTools  program   was  compared  with  a  standard  curriculum  for  teaching  force  and  motion.  It  is   not  clear  whether  the  standard  curriculum  used  similar  instructional  methods   but  it  is  very  doubtful.  The  videodisk-­‐based  Jasper  program  group  was  compared   with  a  control  group  that  did  not  receive  instruction  in  “decomposition  and   solution  strategies.”  One  must  question  whether  this  missing  instruction  could   have  been  delivered  with  a  very  different  medium  or  set  of  media  attributes.  One   must  assume  that  these  comparisons  confound  method  and  content  in  the  same   way  that  many  previous  studies  in  this  area  fail  to  control  for  important   alternative  hypotheses.  One  way  to  begin  to  answer  questions  about  the   structural  necessity  of  media  attributes  is  to  ask  whether  other  learners  have   achieved  similar  learning  results  with  different  instructional  treatments.  Have   learners  acquired  problem-­‐solving  techniques  similar  to  those  presented  in   ThinkerTools  or  Jasper  in  the  past?  If  so,  the  media  attributes  available  from   expensive  computers  and  video  disks  are  not  structurally  important  in  learning   problem-­‐  solving  skills.  Yet  in  making  this  point,  I  do  not  want  to  appear  to  be   critical  of  the  developers  of  these  two  excellent  programs.  The  substantive  point   of  both  design  activities  was  to  explore  the  utility  of  different  combinations  of   instructional  method.

CONCLUSlON     Kozma  agrees  with  me  that  evidence  does  not  yet  support  the  claim  that  media   or  media  attributes  influence  learning.  This  has  been  the  conclusion  of  all  media   researchers  who  have  entered  into  a  dialogue  about  this  issue  (e.g.,  Winn,  1990).   However,  Kozma  hopes  that  future  media  research  will  be  more  positive.  He   accepts  the  claim  that  in  thousands  of  media  research  studies  conducted  over  a   period  of  70  years,  we  have  failed  to  find  compelling  causal  evidence  that  media   or  media  attributes  influence  learning  in  any  essential  and  structural  way.   However,  Kozma  remains  optimistic  that  with  careful  consideration  of  cognitive   processes,  we  will  find  a  critical  connection  between  media  attributes  and   learning.  He  suggests  that  my  insistence  that  educational  researchers  separate   these  two  classes  of  variables  will  retard  a  very  promising  area  of  research.     In  brief,  my  claim  is  that  media  research  is  a  triumph  of  enthusiasm  over   substantive  examination  of  structural  processes  in  learning  and  instruction.   Media  and  their  attributes  haven  important  influences  on  the  cost  or  speed  of   learning  but  only  the  use  of  adequate  instructional  methods  will  influence   learning.  I  define  methods  as  the  provision  of  cognitive  processes  or  strategies   that  are  necessary  for  learning  but  which  students  can  not  or  will  not  provide  for   themselves.  I  claim  that  absolutely  any  necessary  teaching  method  can  be   delivered  to  students  by  many  media  or  a  variety  of  mixtures  of  media  attributes   -­‐  with  similar  learning  results.     The  media  research  question  is  only  one  of  a  number  of  similarly  confounded   questions  in  educational  research.  It  is  difficult  for  alternative  questions  to  gain   acceptance,  even  though  adequate  research  exists  to  refute  invalid  but  intuitively   appealing  beliefs.  The  development  of  an  instructional  design  science  is   necessary  but  very  complex.  Part  of  the  difficulty,  in  my  view,  is  that  we  tend  to   encourage  students  (and  faculty)  to  begin  with  educational  and  instructional   solutions  and  search  for  problems  that  can  be  solved  by  those  solutions.  Thus  we   begin  with  an  enthusiasm  for  some  medium,  or  individualized  instruction,  or   deschooling  -­‐  and  search  for  a  sufficient  and  visible  context  in  which  to  establish   evidence  for  our  solution.  Negative  evidence  is  suspect  and  we  are  predisposed   to  believe  that  it  is  flawed.  In  the  case  of  media  research,  70  years  of  largely   negative  evidence  has  been  and  continues  to  be  ignored  by  many  researchers.   Positive  evidence  is  accepted  easily  because  it  confirms  our  expectations  and   helps  to  attract  research  support.  We  need  a  greater  appreciation  for  negative   evidence  and  to  begin  with  a  focus  on  the  problem  (for  example,  the  need  to   increase  achievement,  or  access  to  instruction,  or  to  address  the  labor   intensiveness  of  instruction)  and  then  search  relevant  research  literatures  for   robust,  research-­‐based  theories  that  can  support  the  development  of  a  variety  of   solutions  to  those  problems.  If  we  begin  by  implicitly  and  explicitly  attempting  to   validate  a  belief  about  the  solutions  to  largely  unexamined  problems,  we  are  less   open  to  evidence  that  our  intuitions  might  be  very  far  off  the  mark.

If  the  arguments  advanced  here  have  failed  to  convince  you,  I  ask  you  to  consider   one  or  two  questions  as  you  reason  about  media  research.  Whenever  you  have   found  a  medium  or  set  of  media  attributes  which  you  believe  will  cause  learning   for  some  learners  on  a  given  task,  ask  yourself  if  another  (similar)  set  of   attributes  would  lead  to  the  same  learning  result.  If  you  suspect  that  there  may   be  an  alternative  set  or  mix  of  media  that  would  give  similar  results,  ask  yourself   what  is  causing  these  similar  results.  It  is  likely  that  when  different  media   treatments  of  the  same  informational  content  to  the  same  students  yield  similar   learning  results,  the  cause  of  the  results  can  be  found  in  a  method  which  the  two   treatments  share  in  common.  Design  science  (and  a  world  with  limited  resources   and  many  competing  problems)  requires  that  you  choose  the  least  expensive   solution  and  give  up  your  enthusiasm  for  the  belief  that  media  attributes  cause   learning.