Notes on Scientific Method

The textbook version of scientific method is familiar: observe, hypothesize, experiment, analyze, conclude. Repeat. This clean process is taught in every science class.

But real science is messier, more human, and more interesting.

How Science Actually Works

The Role of Intuition

Scientists don’t start from pure observation. They start with hunches, aesthetic preferences, and theoretical commitments. Einstein famously said, “I want to know God’s thoughts; the rest are details.”

Many breakthroughs came from noticing something odd—an anomaly that didn’t fit existing theory. But you can only notice anomalies if you have strong expectations about what should happen.

Social Epistemology

Science is a social process. Ideas spread through:

• Conferences and conversations
• Peer review (with all its flaws)
• Funding decisions
• Scientific prestige and reputation

The “invisible colleges” of scientists who trust each other’s work shape what gets investigated and what gets ignored.

The Importance of Instruments

New instruments often precede new discoveries. The telescope, microscope, particle accelerator, DNA sequencer—each opened new domains for investigation.

Advances in measurement precision often matter more than theoretical breakthroughs. You can’t test a theory if you can’t measure precisely enough.

Problems in Modern Science

Publication Bias

Journals prefer novel, positive results. This creates incentives to:

• Not publish null results
• “p-hack” until something reaches significance
• Exaggerate effect sizes
• Split papers for more publications

The literature becomes systematically biased toward false positives.

Replication Crisis

Many published findings fail to replicate. This is true across psychology, medicine, economics, and other fields. The problem isn’t necessarily fraud—it’s methodological flexibility, small sample sizes, and publication bias.

Hyperspecialization

Modern science rewards depth over breadth. Scientists become experts in increasingly narrow domains. This makes it harder to:

• Synthesize knowledge across fields
• Question foundational assumptions
• Apply insights from one domain to another

Reforming Science

Several movements aim to improve scientific practice:

• Pre-registration: Specifying hypotheses before collecting data
• Open data and code: Making research reproducible
• Registered reports: Peer review before results are known
• Meta-science: Studying science itself

Progress is possible. Science, like any human institution, can improve through conscious effort.

---

See also: Replication and Reproducibility, On Long-term Thinking