INFORMATION TIPS THE SCALES


  2490 Lee Boulevard #205
Cleveland Heights, OH 44118-1255
Phone (216) 321-5018
Fax (216) 321-3731
Email:
johnmiller@nasw.org

    



  

EVALUATING SCIENTIFIC RESEARCH

Most people believe that if a research study is published in a peer-reviewed journal, its conclusions are almost certainly true. If they weren't, the reviewers--other professionals in the field--would certainly catch any errors.

Unfortunately, that's false.

One quarter of all research studies published in prominent medical journals contain an error in their research design or statistics so important, that error draws the study's main conclusions into question.

The conclusions might still be right. Or they might be wrong. Sometimes the data analysis or even the entire study must be run again to find out for sure.

Startling as that sounds, forty years of studies using expert statisticians as judges of errors all come to exactly the same conclusion. And the journals in question are among the very highest in quality and prestige--New England Journal of Medicine, Journal of the American Medical Association, etc.

Why do so many error-plagued research studies get published? Sadly, many researchers--and reviewers--are poorly trained in statistics and research design.

Sometimes bright researchers draw illogical conclusions. In 1985 200,000 taste tests across America proved that people overwhelmingly preferred New Coke both to Pepsi and to original Coke.

But Coca Cola never asked what anyone thought about the plan it was keeping top-secret: To stop producing original Coke altogether and to replace it with fruitier-tasting New Coke.

So with no relevant research supporting the change, Coca Cola made both announcements, and America howled. Virtually nobody would buy New Coke. They saw it as the killer of old Coke, an American tradition. They wouldn't even try it! Coca Cola might as well have discontinued motherhood and apple pie.

Even if researchers never made errors, their results still wouldn't agree all the time. Again and again well-trained, honest researchers occasionally get results that completely disagree with the long-term thrust of the evidence in their field.

Their one contrary study does not disprove all the others. It must be viewed in perspective. Truth in science is established not by any one study, but by the preponderance of research results completed over many years by different researchers living in different places with different motivations and biases.

Below is a partial list of subjects which I have either investigated as a professional science journalist, studied at the graduate-school level, or taught at the university level. To save space, I include only subjects beginning with the first three letters of the alphabet:

    A

  • Acetylcholine Neurotransmitter
  • Achievement Motive in Personality Development
  • Acid Rain
  • Acne
  • Action Potential (Firing) of a Nerve
  • Acupuncture
  • Addiction
  • Adlerian Psychodynamic Theory
  • Adolescent Psychology
  • Adoption
  • Adult Day Care
  • Aerobics
  • Age of the Universe
  • Agent Orange
  • Aggression
  • Agoraphobia
  • Agriculture
  • AIDS
  • Airplane Fires Fueled by Plastics
  • Airplane Accidents
  • Airplane, X-29 Rocket-powered
  • Airport Radars
  • Air Pollution
  • Alcohol Effects on Humans
  • Allergies
  • Alpha Error in Statistics
  • Alpha Rhythms
  • Alzheimer's Disease
  • Amnesia
  • Amphetamines
  • Amygdala--the brain organ responsible for anger
  • Analgesia, Patient-Controlled
  • Analysis of Variance (ANOVA)
  • Anemia
  • Anesthesia
  • Anger
  • Angiography
  • Animal Rights
  • Anorexia
  • Anti-Ballistic Missiles
  • Anthropology--Earliest Human Skull Remains
  • Antibiotics in Animal Feed
  • Antibiotics Overuse
  • Anxiety Disorder
  • Antihistamines
  • Ape Language and Cognition
  • Approach/Approach Conflicts
  • Approach/Avoidance Conflicts
  • Archetypes of the Collective Unconscious (Carl Jung)
  • Armadillos
  • Artery Bypass Surgery
  • Arterial Diseases
  • Arthritis
  • Artificial Intelligence
  • Asbestos Toxicity
  • Aspirin
  • Associative Cortex in the Human Brain
  • Asthma
  • Astronomy
  • Astrophysics
  • Atherosclerosis
  • Atomic Energy
  • Attention Deficit Hyperactivity Disorder (ADHD)
  • Attitudes As Predictors of Behaviors
  • Attribution Theory
  • Audition (hearing)
  • Authoritarian Parenting Style
  • Automobile Fuel Efficiency
  • Autonomic Nervous System
  • Availability Bias in Human Decision-Making
  • Aversion Therapy

Top of Page

    B

  • Babies--Learning, Low Birth-weight, Premature, RH-Negative, Test-tube
  • Back Diseases
  • Bacterial Infection
  • Baldness
  • Barbiturates
  • Basal Ganglia
  • Batteries
  • Beach Erosion and Movement
  • Bees
  • Behavioral Psychology
  • Behavioral Therapies
  • Benzene as a Powerful Carcinogen
  • Big Bang
  • Binocular Cues in Human Vision
  • Biofeedback to Combat Migraine Headaches
  • Biological Warfare
  • Biology of Radiation Health Effects
  • Biotechnology
  • Bioterrorism
  • Birth
  • Birth Control
  • Birth Defects
  • Birthmarks
  • Birth Order and Intelligence
  • Blood--Artificial
  • Blood Diseases
  • Blood Pressure
  • Blood-Pressure Medications
  • Blood Tests
  • Bone Marrow
  • Brain Asymmetry
  • Brain-Immune System Connections
  • Brain Lateralization
  • Brain Mapping
  • Brain Structures
  • Brain Surgery
  • Brain Tumors
  • Breast Cancer
  • Breast Mammography
  • Broca's Aphasia
  • Bulimia Nervosa
  • Burns
  • Busing
  • Butylated Hydroxytoluene Food Preservative

Top of Page

    C

  • Caffeine Health Effects
  • Campaign Expenditures and Election Outcomes
  • Cancer and Nutrition
  • Cancer and Other Neoplasms--Leukemia, Lymphoma
  • Candida
  • Cannon-Bard Theory of Emotion
  • Capacitors as High-Energy Batteries
  • Carcinogens--Natural and Man-Made
  • Cardiology
  • Cataracts
  • Catatonic Schizophrenia
  • Cathode Ray Tube Health Effects
  • Cellulose Gene
  • Census Errors
  • Central Nervous System
  • Cerebellum
  • Certainty--the Need For As a Motivation to Discount New Information
  • Chemicals in the Household Can Causes Fires and Explosions
  • Chemistry of Buffers in Water
  • Chicken Diseases
  • Chicken Pox
  • Child Abuse
  • Child Psychology
  • Chiropractic
  • Chlamydia
  • Cholesterol--Testing, Types
  • Chorionic Villi Sampling
  • Chunking as a Memory-Aid Tactic
  • Classical Conditioning
  • Classical Physics
  • Client-Centered Therapy
  • Climate Change
  • Clinical Psychology
  • Cocaine--Addiction, Neural Mechanisms, Withdrawal Biochemistry
  • Cochlea (Inner Ear)
  • Coal--Gasification, Research on Cleaner-Burning Technologies
  • Cogeneration of Electricity from Turbines
  • Cognitive Dissonance
  • Cognitive Maps in the Brain
  • Cognitive Psychology
  • Colds
  • Colon Diseases
  • Color Blindness
  • Comets--Halley's Comet
  • Community Mental Health
  • Computers--Mice Causing Muscle Strain, Viruses
  • Computer Programming
  • Concrete Operational Stage (in Piaget's Theory of Child Cognitive Development)
  • Confirmation Bias
  • Confounding Variable
  • Consciousness
  • Consumer Confidence
  • Continuously Variable Transmissions
  • Control and Predictability
  • Cormorants
  • Coronary Artery Disease
  • Coronary Bypass Surgery
  • Correlational Research
  • Correlation Versus Causation
  • Corrosion
  • Cosmology
  • Creationism
  • Creativity
  • Credit-Card Computerization
  • Criminals--Treatment, Behavior
  • Cuban Medical Clinic System Is the Model for American Health Maintenance Organizations
  • Culture-Fair Tests
  • Cystic Fibrosis

Top of Page


To see examples of Dr. John Miller's work evaluating scientific research, click here.