# Codebook for the validity citation study (from SCOPE.md rev 7; locked before any page is read for claims). 2 September 2026 Unit of coding: one claim occurrence on one checked page. Unit of reporting: the product (fixed denominator 12 for cohort 1). ## Claim row (data/claims-.json, one object per occurrence) - `product`, `domain`, `url` (final URL), `saved`, `sha256`, `fetched_utc`, `discovery_route` (1 header/footer, 2 sitemap, 3 one hop), `evidence_role` (primary claims and sources / source and lineage evidence only: off-domain destinations never supply primary claims). - `quotation`: the claim verbatim, contiguous page text, with its position (heading or paragraph) noted. - `included_primary` (true/false): true only when the quotation contains a number and explicitly links an assessment, score, test family or selection construct to a named job-related criterion. Reliability, completion rates, pass rates, adverse-impact figures, customer satisfaction, and time or cost reductions are never primary claims. - `commercial_outcome` (true/false): a percentage or multiplier without a stated criterion and predictive meaning or comparator; stored in the separate dataset outside N, M and K. - `metric_type`: validity coefficient or correlation / variance explained / predictive-accuracy percentage / relative multiplier / other. Never converted or compared across types. - `claim_target`: product or named instrument / test family / general construct. A construct claim is eligible but is never described as evidence for the product. - `criterion`: the job-related outcome named (verbatim). - `value`, `comparator` (verbatim), used with target, criterion and metric type to group identical within-product claims; every occurrence is preserved, a group counts once. - `sources[]`: every source the page maps to this claim. A source is identifiable only when an inline citation, footnote or linked reference uniquely names an author or organisation plus a title, year or stable document, and maps to the claim. A general bibliography does not qualify. Each source: `text` (verbatim), `link`, `identifiable` (true/false), `gated` (behind a form: identifiable and gated, lineage unresolved unless the accessible page supplies the bibliography; never submit personal data), `source_type` booleans (`academic`, `vendor_technical_report`, `customer_or_case_study`, `other_third_party`), `lineage` (1998 / 2022 / other / unresolved), `lineage_basis` (primary text named; explicit attribution of the same number to the primary text; one documented citation hop with the secondary document and passage saved; otherwise unresolved). Never infer lineage from a familiar coefficient alone. ## Product rollups (data/products-.json; non-exclusive except the last two) `has_included_claim`, `has_identifiable_source`, `has_1998_lineage`, `has_2022_lineage`, `has_vendor_report`, `has_other_academic_source`, `has_customer_source`, `has_other_third_party_source`, `has_claim_without_mapped_source`, `route_2_unavailable`, `partially_unresolved` (some qualifying pages unreadable), `no_claim_found`, `unresolved` (all qualifying pages unreadable). A product with `route_2_unavailable` or unreadable pages is never coded `no_claim_found`. ## Counts N = products with at least one included primary claim on a checked page; M = products for which at least one checked page mapped a claim to an identifiable source; K = products for which at least one checked page mapped a claim to the Schmidt and Hunter (1998) lineage; all of 12. Source categories are non-exclusive; their arithmetic sum may exceed M. Commercial-outcome claims get a separate secondary count and never enter the headline. ## Wording on the page The page is the actor: "On , the page titled stated <quotation> and named <source>." Absence: "We found no identifiable source mapped to this claim on the pages checked." No temporal or evaluative implication about any source or product. Rows alphabetical. Source year is a bibliographic observation only. ## Before drafting Full primary texts read: Schmidt and Hunter 1998 (saved), Sackett and colleagues 2022 (saved), Sackett and colleagues reply 2023 (saved); Oh, Le and Roth 2023 still abstract-only: drafting waits for AK's manual download from SSRN (abstract_id 4308528). ## Additions after the gate 2 third run (2 September 2026, SCOPE rev 10) - **Context rule.** A numeric row of a PDF figure or an HTML table is a candidate when any row of the same figure or table carries a validity word (`patterns` value "coefficient in a figure or table row whose figure or table carries validity words"). Figure rows are one candidate per OCR line, labelled `PDF FIGURE (<file>, page <n>) row <k>:`; units over 900 characters are chunked and marked "(part k of n)". - **Ledger marker.** Every claim row carries `ledger`: `explicit` (a hand decision in data/decisions-ledger-<date>.json) or `rule` (R1 or R2 applied by code.py). Rule rows that carry an assessment term with an outcome term, a validity word or a figure label must receive an explicit decision before `--strict` passes. - **Reason-code usage settled in this pass.** MS for a site-wide banner statistic (a survey share repeated in a page header); NN for navigation, title and list chunks; OP for survey shares and satisfaction ratings in industry-report charts; TR for a figure's axis scale row; DA for detection or scoring accuracy (plagiarism detection precision); RC for reliability and inter-rater consistency figures. - **Image occurrences** carry `id`, `url`, `page_sha256`, `page_saved`, `page_fetched_utc`, `image_src`, `image_saved`, `image_sha256`, `image_fetched_utc`, `position` and `quotation`, like text occurrences. ## Additions after the gate 2 fourth run (2 September 2026, SCOPE rev 11) - **Images are read, then decided.** A record counts as read only with non-empty OCR (after conversion of WebP and AVIF to PNG and the psm 11 and 2x retries), an SVG placeholder without text nodes, or a manual visual reading in data/image-readings-<date>.json. Every record with a number gets a decision in data/image-decisions-<date>.json: `included` (a transcribed figure with its own group prefix), `duplicate-rendering` (the same figure on another page), `customer-outcome` or `commercial-outcome` (image renderings of such claims, with the text group where one exists), or `excluded` with a reason code from the same list as text rows. A chart that shows a relationship without printing a coefficient or percentage is EX. - **One claim per row, enforced.** A ledger entry may carry `claims: [{group, span}, ...]`; code.py materialises child rows `<id>-1`, `<id>-2` with `claim_span` and `parent_id`, one per group, and asserts in strict mode that included rows and primary text-group occurrences coincide one to one (and customer-outcome rows with customer-outcome groups). - **Locale rule at collection.** A first path segment that is a two-letter language code other than en (optionally with a region) excludes the page from the checked set, whatever the sitemap manifest concluded; the URLs are listed per product as `excluded_by_locale_rule`. ## Additions after the gate 2 fifth run (2 September 2026, SCOPE rev 12) - **Language rule.** The page's own `<html lang>` declaration decides: a primary subtag other than en excludes the page after fetch (recorded per product as `excluded_by_language_declaration` with the declaration, file and hash); pages without a declaration are kept and the per-product distribution is recorded. Path prefixes are not used (an application context path such as HR Avatar's /ta/ is not a locale). - **Splits with their own decision.** A `claims` entry may carry `decision` (default included primary claim), `group`, `reason_code` or `commercial_label`; a card block that mixes one validity claim with case-study outcomes becomes one primary child row and commercial-outcome child rows. - **A table or figure with one value per method is one group per method**, never one group for the table; text sentences that state the same value for the same method join that method's group. Range statements ("between 0.2 and 0.45", "above 0.45") are their own groups. - **Completion and pass rates** (training completion, licensing-exam pass) are customer outcomes when they come from a case study and are never primary claims.