Method

Each paper is scored 0–4 on eight criteria of research method, using the anchored descriptions below. A criterion that genuinely does not apply (for example, research design in a paper making no causal claims) is marked N/A and the remaining weights are rescaled. The paper score is the weighted mean, scaled 0–100.

How a paper is scored

Collected politely. We follow each site's robots.txt, identify ourselves, and make at most one request every five seconds per site. A site that blocks automated access is never worked around; its papers are added by hand.
Read by code first. Text is extracted page by page. Code re-does the paper's own arithmetic where it states its inputs, and checks intervals and units.
Identifying details reduced. Institute and author names, forewords, endorsements and author biographies are removed from the text the models assess. Subject and style can still identify a source, so we claim reduced identifying information, not blindness.
Evidence gathered, then scored. One model assembles page-referenced evidence for each criterion; a stronger model scores from that evidence and reads the full paper when the evidence is uncertain, when code checks found problems, and for a random 10% audit sample.
Every quote verified. Code checks every cited quote against the cited page. A paper whose evidence does not verify is not published.
Signed off by a person. Nothing is published until a reviewer approves it.

Limits we know about

Splitting the reading and scoring across models is a cost saving whose accuracy is not yet measured against a single strong model reading every page. That comparison is planned; until it is run, treat it as an assumption.
Scores are stable to within one point per criterion in our repeat tests only once those tests have run; the method page will say when they have.
Models make mistakes. The evidence and page references are there so anyone can check.

Rubric v1

C1 Claim calibration weight 20

Do the headline and executive-summary claims match the strength of evidence the body and appendices actually provide?

4	Headline claims match the body's strength of evidence exactly.
3	Headline broadly matches the evidence; minor overstatement or missing qualifiers.
2	Headline overstates in places; key qualifications appear only in the body.
1	Headline materially overstates the evidence (e.g. certainty or causation the body does not establish), though without explicit contradiction.
0	Headline asserts causal or certain effects that the body or appendix explicitly disclaims.

C2 Uncertainty carried through weight 15

Are intervals, significance and uncertainty reported and propagated into derived figures?

4	Intervals and significance reported and propagated consistently into every derived figure.
3	Uncertainty reported and mostly propagated; minor gaps.
2	Intervals reported for primary estimates but not carried into derived figures, or inconsistently computed.
1	Uncertainty mentioned for some estimates but absent from the figures the paper leads with.
0	No intervals, or intervals dropped or misreported in downstream figures.

C3 Research design weight 15N/A only if the paper makes no causal claims.

Does the design support the causal claims made?

4	Credible design; threats named and tested.
3	Credible design; main threats named, some tested.
2	Plausible design with significant threats named but not tested.
1	Causal language on a weak design; threats acknowledged only in passing.
0	Correlation presented as causation; threats to validity unaddressed.

C4 Engagement with counter-evidence weight 15

Is the strongest opposing work cited, summarised fairly and answered?

4	Strongest opposing work cited, summarised fairly and answered.
3	Opposing work summarised fairly and answered in part.
2	Opposing work summarised but not substantively answered.
1	Opposing work mentioned but dismissed without argument, or used inconsistently.
0	Strongest opposing work omitted, or listed in the bibliography but ignored.

C5 Cost-benefit symmetry weight 10N/A only if the paper makes no policy recommendation.

Are costs and benefits of the recommended policy treated with the same rigour?

4	Both quantified with the same rigour, or both explicitly unquantified.
3	Both quantified with comparable rigour; minor omissions.
2	Both quantified, but with visibly different rigour or omitted major cost categories.
1	Costs acknowledged qualitatively while benefits are quantified.
0	Benefits quantified, costs 'out of scope'.

C6 Reproducibility weight 10

Could a reader re-derive the headline figures from what is published?

4	Data, code and assumptions published; headline figures re-derivable.
3	Data and assumptions available; code or some steps missing.
2	Assumptions stated; data or code unavailable.
1	Data sources named but assumptions and method too thin to re-derive.
0	Data, code and assumptions unavailable.

C7 Internal consistency weight 10

Are numbers, units, dates and terms consistent across sections, and is the arithmetic right?

4	Fully consistent; code checks pass.
3	Minor slips only.
2	Some inconsistencies or terminology slips, none changing a headline figure.
1	Several inconsistencies, at least one affecting a headline figure.
0	Numbers, units or dates conflict across sections; arithmetic errors.

C8 Assumption sensitivity weight 5

Are key assumptions varied and the sensitivity of results reported?

4	Key assumptions varied and the sensitivity of results reported.
3	Key assumptions varied; reporting incomplete.
2	Some assumptions varied, but not the ones driving the headline.
1	Sensitivity mentioned but not shown.
0	Single-point assumptions, never varied.

Context (shown, not scored)

Funding and conflicts disclosure; review status; whether data and code are published.