How we come
to know.

01 / The ideaAbout 3 minutes

Science can
study itself.

A discovery is never just a moment of insight. Someone needed time to ask the question, tools to investigate it, and a community able to check the answer.

Science lets us explain an eclipse, trace the spread of a disease, and read the history written in a rock. These achievements depend on ways of working together: recording observations, making useful models, testing explanations, and exposing claims to criticism. Instruments and institutions make that work possible.

Metascience is the study of science itself. It investigates how research is done, funded, shared, evaluated, and rewarded. Its subject might be a statistical method, a grant programme, a journal, a collaboration, or an entire field. Its tools include experiments, historical comparisons, interviews, and the analysis of publications and careers.¹

More than checking for mistakes

Some metascientists investigate unreliable findings. Others study how teams form, why promising ideas are overlooked, or whether longer grants enable more ambitious work. A healthy research system needs both dependable answers and room for discoveries nobody knew to request.

The boundaries are porous. Scientometrics measures patterns in research, often using publications and citations. The science of science examines discovery, collaboration, and scientific careers. History, philosophy, sociology, and economics of science offer complementary ways to understand the enterprise. Open science is a set of practices and commitments whose effects can themselves be studied.²

Trust is something we build

There is no single recipe followed by every science. An astronomer cannot randomise galaxies; a geologist cannot rerun Earth’s history. Different fields combine observation, experiment, mathematics, and comparison in different ways. What matters is whether a claim faces meaningful tests and whether its evidence can be examined.

Correction takes effort. Someone must preserve the data, question an assumption, repeat a measurement, or fund an unfashionable check. Metascience asks whether the arrangements around researchers actually make that work possible.

02 / The inheritanceAbout 5 minutes

How we
got here.

Science has a history. So do the rules, resources, and institutions that make it possible.

No person or civilisation invented inquiry once and for all. Knowledge travelled through trade, translation, migration, and conquest; it was made in workshops and observatories as well as universities. The episodes below are a selective route through that history, not a complete timeline or an inevitable march toward the present.

1. 1st millennium BCE

   Observations outlive their observers

   Babylonian astronomical diaries recorded celestial events alongside weather and other earthly observations. A durable record allowed patterns to become visible across many observers and many years. The British Museum’s surviving tablets make this accumulated labour tangible.³

   An enduring question Who keeps the record, and can the next generation use it?

2. 11th century

   Arrange the world to test an idea

   Working in the Arabic-language scholarly world, Ibn al-Haytham brought mathematical reasoning and experiments to the study of light and vision. His optics belongs to a longer history of ideas being translated, challenged, and developed across cultures.⁴

   An enduring question What observation would distinguish competing explanations?

3. 1665

   A conversation becomes a journal

   Henry Oldenburg launched Philosophical Transactions, drawing on correspondence among natural philosophers. Publication could circulate observations beyond a room or a private letter, establish priority, and invite a response. The Royal Society took institutional responsibility in 1752; more systematic expert refereeing developed later.⁵

   An enduring question How do we share knowledge and assign credit at the same time?

4. 19th–20th centuries

   Research becomes an occupation

   Universities, industrial laboratories, scientific societies, and public institutions expanded the places where research could be a career. Specialisation made deeper inquiry possible, while equipment, salaries, and training made discovery increasingly dependent on sustained support. Access to those careers was sharply unequal.

   An enduring question Who gets the time, training, and permission to investigate?

5. 1945–1950

   A public bargain with science

   In the United States, Vannevar Bush’s Science—the Endless Frontier argued that public support for basic research would serve health, security, and prosperity. Senator Harley Kilgore’s competing vision placed more emphasis on political accountability and the geographic distribution of support. The National Science Foundation emerged in 1950 after years of debate.^{6, 7}

   An enduring question How should scientific independence and public priorities fit together?

6. 1950s–1960s

   Science becomes an object of measurement

   Theodore Sterling documented the striking prevalence of statistically significant findings in journals. Derek de Solla Price studied the growth and organisation of research. These were early examples of a powerful move: treating the scientific record as evidence about science itself.^{8, 9}

   An enduring question What can the pattern of published work tell us—and what is missing?

7. 2010s onward

   Checking becomes a collective project

   Large collaborations began repeating published studies, journals adopted review before results were known, and researchers built shared repositories and protocols. These efforts made problems more visible and created ways to test remedies.^{10, 11}

   An enduring question Can improving research become ordinary, supported scientific work?

These institutions enlarged what people could know, but their benefits and authority were not equally shared. A serious history also asks whose observations were credited, whose labour was hidden, and whose interests determined the questions. Better methods and fairer participation both deserve attention.

The way we organise science is an inheritance. It is also something we can change.

03 / The conditionsAbout 5 minutes

Who pays
for discovery?

Before an experiment can produce an answer, someone has to decide that its question is worth supporting.

Money buys much more than equipment. It buys a researcher's time, a technician’s expertise, access to a collection, the upkeep of a database, and the possibility of trying again. The amount of support matters. So do its duration, conditions, and distribution.

Why fund something whose outcome is unknown?

Some research pursues a practical goal: a better crop, a cleaner industrial process, a treatment. Other work explores a phenomenon without a clear application. The distinction between basic and applied research describes motivations; it does not draw a clean boundary around how discovery happens. Practical problems can open fundamental questions, and basic findings can eventually become useful in unexpected ways.

A central economic argument for public support is that knowledge can benefit people far beyond whoever paid to produce it. A firm may struggle to capture enough of that benefit to justify the original investment. Public funding can support long horizons, shared facilities, and questions whose value is hard to turn into revenue. Bush’s 1945 report made an influential case for that role, alongside national security and public health.⁶

These motives often overlap. A public grant can support a private collaboration; a foundation can sustain an open database. A funding source is a reason to examine study design and disclosure, not a shortcut for deciding whether a finding is true.

The funding rules are part of the experiment

Imagine supporting a researcher with a promising but uncertain idea. Different arrangements make different kinds of work easier. Open a model to explore its purpose and its trade-off.

Good funding policy cannot be reduced to a single score. A portfolio may need patient exploration, urgent problem-solving, careful replication, and unglamorous maintenance. Deciding the balance involves values as well as evidence. Metascience can clarify the consequences; it cannot choose society’s priorities on its behalf.

04 / The cultureAbout 3 minutes

What science
rewards.

Researchers respond to the opportunities around them. What makes a career possible helps shape what gets studied.

A careful replication, a well-maintained instrument, or a dataset others can reuse may be enormously valuable. Yet if hiring and promotion focus on papers in prestigious journals, these contributions can become hard to justify. The problem is a mismatch between work that benefits knowledge and work that earns visible credit.

When a useful measure becomes a target

Citation counts describe attention. They do not directly measure truth, social value, or the quality of an individual’s contribution. Citations differ across fields and accumulate with time; criticism can also generate citations. A journal’s average citation rate says little about any particular paper in it. The Declaration on Research Assessment calls for judging work on its own merits.¹³

Recognition can reinforce itself. A visible paper is easier to encounter and cite, so it becomes more visible still. This resembles the Matthew effect, sociologist Robert Merton’s term for the unequal allocation of scientific credit.¹⁴

Who can afford to take a risk?

A short contract, an uncertain grant renewal, or an overloaded teaching schedule changes which projects are feasible. A researcher with secure support may be able to wait years for an answer. Another may need a publishable result before their contract ends. An invitation to “be bold” means little without conditions that make failure survivable.

Participation matters here too. If training, equipment, networks, and publication fees exclude capable people, science loses both talent and questions. Improving a research system includes crediting technical and collaborative work, widening access, and examining who carries the costs of each reform.¹⁵

05 / The evidenceAbout 5 minutes

When evidence
misleads.

A published finding is part of an investigation. To know what it establishes, we need to understand how it was produced—and what we are not seeing.

A result can be surprising by chance

A p-value describes how unusual data at least as extreme as those observed would be under a specified statistical model, often one with no effect. It is not the probability that a hypothesis is true, the probability that a result happened “by chance,” or a measure of how important an effect is.¹⁶

In the simplified example below, a treatment has no effect on any outcome. Each test has a 5% false-positive rate. Run enough tests and a “significant” result becomes quite unsurprising.

The problem is not measuring many things or exploring data. It is reporting the most favourable result as if it were the only test planned. Preregistration, appropriate statistical adjustments, and clear labels for exploratory findings help readers interpret the evidence.

The studies we never get to read

If striking results are more likely to be written up, submitted, or accepted, the published literature becomes a selected sample of the work actually done. This is publication bias. Even a careful synthesis can mislead if important results are missing.

What happened when researchers checked?

In 2015, the Open Science Collaboration reported replications of 100 studies from three psychology journals. While 97% of the original studies had statistically significant results, 36% of the replications did. The authors also evaluated effect sizes and other measures of agreement.¹⁰

This was evidence of a serious problem in the studied literature, not a measurement of “how much science is false.” The sample was limited, and a significance threshold is only one way to assess agreement. A failed replication can reflect an exaggerated original effect, differences in methods or context, uncertainty in the new study, or an incorrect explanation.

Fraud exists, but unreliable findings do not require dishonesty. Noisy measurements, selective reporting, and reasonable analytical choices can combine to create overconfidence. Diagnosing the mechanism is more useful than treating every failed result as evidence of bad character.

06 / The possibilitiesAbout 3 minutes

Better science
is work we can do.

The strongest case for metascience is constructive: we can study the conditions of discovery, then test how to improve them.

A reform should name a problem, explain how it might help, and make its costs and effects observable. A policy that works in one field may be a poor fit elsewhere. Improving science requires the same willingness to revise an idea that we expect of any researcher.

1. 01

   Make the plan visible

   Preregistration records questions and planned analyses before the relevant outcomes are known. Researchers can still explore and change course; the distinction between planned and exploratory work becomes easier to see. A vague plan or a poorly designed study remains a problem.

2. 02

   Review the question before the answer

   Registered Reports evaluate the rationale and methods before results exist. Journals offer in-principle acceptance, conditional on following the approved protocol and meeting quality requirements. This reduces the role of a surprising outcome in deciding what is published.¹¹

3. 03

   Make checking feasible

   Well-documented data, code, materials, and protocols let others inspect and build on a result. Openness needs stewardship and funding. Privacy, consent, and legitimate restrictions can require controlled access rather than public release.¹⁵

4. 04

   Reward the work that sustains knowledge

   Give replication, mentoring, technical expertise, peer review, and shared infrastructure meaningful recognition. Assessment should examine contributions in context instead of substituting journal reputation for judgment.¹³

5. 05

   Experiment with institutions

   Compare grant durations, review procedures, and support for early-career researchers. Where feasible, randomised pilots can help distinguish a policy’s effects from differences between the people who receive it. Follow outcomes long enough to learn about more than next year’s paper count.

We should ask who benefits from a reform and who has to do extra work. Sharing data without support can shift costs onto the least-resourced teams. New assessment forms can become another bureaucracy. Metascience earns its place by evaluating these consequences, including the inconvenient ones.

07 / Take it with youAbout 2 minutes

A reader’s
field kit.

You do not need to become a statistician to ask better questions about research.

Use these as prompts for understanding, not a scoring system. A study can lack one feature and still be valuable; a checklist cannot replace judgment about the question and its evidence.

1. What is the actual claim?

   Separate observation from explanation, correlation from causation, and the paper’s conclusion from the headline. Results in one population or setting may not transfer to another.

2. How was the comparison made?

   Look at measurement, sampling, controls, and uncertainty. A large sample does not rescue a biased design; a small study can still be informative about a well-chosen question.

3. What was decided in advance?

   Distinguish a planned test from an interesting pattern discovered along the way. Both can contribute, but an exploratory result often needs independent confirmation.

4. How large and how uncertain is the effect?

   A statistically significant difference can be practically tiny. An inconclusive result can still be compatible with an important effect. Ask for estimates and their uncertainty, not only a threshold.

5. What might be missing?

   Look for unpublished work, excluded observations, alternative analyses, and limits to access. A systematic review is useful only to the extent that its evidence and methods are sound.

6. Who supported the work—and can others check it?

   Examine funding, conflicts of interest, methods, and access to materials. Disclosure provides context; it does not by itself prove or disprove a finding.

7. How does it fit the wider evidence?

   Independent studies, different methods, and converging explanations matter more than one striking result. Ask what would change the current level of confidence.