We stand with people living with cystic fibrosis, their families, care teams, and researchers. This independent lab contributes donated time, research notes, and software experiments, not clinical care or medical advice. Support the wider work through the Cystic Fibrosis Foundation.

Donate
Sensei Ewok Research Lab Cystic Fibrosis AI Lab

How we check, and how we check ourselves

Evidence

We would rather show a number we can trace than a claim we cannot. This page is where the lab's checking is laid out so you can check it too.

What our tools may use

The catalog is the lab's list of outside sources. A tool may only use a source the catalog allows, and the catalog records what each publisher says about automated use. The numbers below are counted from the catalog by a script, not typed.

113sources in the catalog
21a tool may download, within each site's rules
24APIs with a recorded usage policy
64a person must download
3refused by the publisher
1need a written request

These counts were taken from the lab's research repository as it stood on October 24, 2026 (version 5e0e412).

“Fetch” sources are files, pages and datasets a tool may download, after reading the site's robots.txt where it has one. A robots.txt file is where a site says which of its pages automated visitors may read. “API” sources (an API is a publisher's interface for programs) publish a usage policy and a rate we respect. “Manual” sources are ones a person downloads in a browser, for example when a publisher refuses automated clients. “Forbidden” sources said no, and we do not ask twice. One source needs a written request.

As the research pipeline expanded, the catalog grew from 67 to 113 sources (67 + 46 = 113). That expansion added medicinal chemistry, genetic variation, and citation-context APIs (chembl, open-targets, scite-mcp, scite-api, uniprot-rest, ncbi-variation, clinvar-bulk, isrctn, and nice-guidance). It also cataloged 37 national cystic fibrosis patient registries and associations across more than 25 countries.

Proof Receipt · Evidence Snapshot Lock & Two-Session Ledger (SHA-256 verified)

Lockfile: data/evidence.lock.json · Snapshot SHA-256: d88f17eb3328ca4491116cde4a5a28149719f36f9c49724c161d8b84448a534c

Input hashes: sources/catalog.yaml (63eab6bb789bfb80…), ledger/2026-10-04-first-session.md (4ee7fcc1c0021590…), and ledger/2026-10-07-second-session.md (185aaabb84687040…).

Verify on GitHub: data/evidence.lock.json · tools/make_evidence.py · cf-research/sources/catalog.yaml

One evidence record

When our history page says Trikafta was approved in the United States in 2019, this is the record behind that year. It names the source, the date it was read, the exact fields, and what the answer does not tell you.

{
  "accessed": "2026-10-05T05:37:48Z",
  "excerpt": null,
  "fields": {
    "active_ingredients": [
      "ELEXACAFTOR, IVACAFTOR, TEZACAFTOR",
      "IVACAFTOR"
    ],
    "application_number": "NDA212273",
    "brand": "TRIKAFTA",
    "dosage_forms": [
      "TABLET"
    ],
    "latest_submission_action": "2026-03-27",
    "original_approval": "2019-10-21",
    "sponsor": "VERTEX PHARMS INC",
    "submissions": 12
  },
  "limitations": "US FDA only. 'original_approval' is the earliest approved ORIG submission in Drugs@FDA; supplements may have changed the indication or age range since. Says nothing about EU or other regulators, or about any individual's eligibility.",
  "provider": "openfda/0.1",
  "publisher_date": "2019-10-21",
  "question": "When did FDA first approve the application for brand 'TRIKAFTA', and under what number?",
  "source_id": "openfda-drugsfda",
  "status": "found",
  "url": "https://api.fda.gov/drug/drugsfda.json?search=openfda.brand_name%3A%22TRIKAFTA%22&limit=1"
}

Run on 2026-10-05 with approval TRIKAFTA. The record is shown as the tool wrote it, formatted for reading.

We checked all three approval years on the history page the same way against the first approved application in openFDA: Pulmozyme on 1993-12-30, Trikafta on 2019-10-21 and Alyftrek on 2024-12-20. Those are United States dates only. The Foundation's statement that Pulmozyme was the first drug developed specifically for CF is the Foundation's, and openFDA does not speak to it.

When one source is not enough: multi-source triangulation (AH-8). A single database record answers a bounded question, such as when a regulator approved an application. For pharmacology, adverse events, or contested trial claims, one database read in isolation can mislead. Our eighth anti-hallucination guardrail (AH-8) pairs primary literature with independent bioactivity, pharmacovigilance, and citation databases:

  • Assay context in ChEMBL (CHEMBL2010601)

    In ChEMBL (DOI: 10.1021/jm400731v), ivacaftor shows an EC50 of 22.0 nM (95% CI 10–46 nM) in primary human bronchial epithelial (HBE) Ussing chambers. In recombinant NIH-3T3 optical membrane-potential assays, its EC50 is 3.0 nM. Quoting 3.0 nM without naming the cell line overstates potency in primary airway tissue by 7.3-fold (22.0 / 3.0 = 7.33).

  • Confounding by indication in Open Targets

    FDA post-marketing (FAERS) signals in Open Targets list infective pulmonary exacerbation (LLR 4265.28, n = 3,489) and Pseudomonas test positive (LLR 819.3, n = 481) under ivacaftor. Cross-checking against controlled clinical trials confirms these reflect the baseline disease of the treated population, not drug-induced toxicity.

  • Citation polarity in Scite

    Counting how often a paper is cited hides whether later experiments confirmed or challenged it. For the Phase 3 elexacaftor/tezacaftor/ivacaftor trial (DOI: 10.1056/NEJMoa1908639), Scite records 109 supporting, 5 contrasting, and 1,134 mentioning citation statements across 1,089 citing publications.

Checks on our own writing

An AI model that writes a note can invent a number, change a digit or add a reason the source never gave, and the result reads well. So the lab checks its own notes with scripts that fail when they find a problem:

Deterministic T0 → T3 Gates
Evidence Verification & Answer-Tier Pipeline Four deterministic verification gates from primary source catalog gating to deterministic claim scripts, blind cross-model lead checks, and cryptographic SHA-256 receipts. PRIMARY EVIDENCE VERIFICATION PIPELINE & ANSWER-TIER GRADUATION (T0 → T3) STAGE 1 · GATEWAY Source Catalog Single source of truth Robots.txt & APIs Access permissions Tier T0 → Cataloged STAGE 2 · VERIFIERS Deterministic Claims check_claims.py check_numbers.py Zero invented digits Tier T1 → Script-Checked STAGE 3 · CHALLENGER Blind Cross-Check Second model lineage Independent review Leads, never votes Tier T2 → Dual-Checked STAGE 4 · ATTESTATION Receipt & Lock SHA-256 fingerprint Cryptographic proof Human sign-off Tier T3 → Verified Lock sources/catalog.yaml · check_claims.py · check_anti_hallucination.py · Independent research, not medical advice
The four-stage evidence verification pipeline: from primary source catalog gating to cryptographic receipt issuance.

Eight machine-enforced guardrails (AH-1 to AH-8) and the 212-entry citation ledger. Across the research and website repositories, two deterministic checkers (check_claims.py and check_anti_hallucination.py) enforce eight rules before any pull request can merge:

What we do not know yet. No registry scientist has read our guides to patient-registry reports, and a review is on our task board. Until then they are reading aids, and we compare no figure from one registry with another.

What we caught, and how

138 mistakes recorded as of October 24, 2026. This is a running total across all of the lab's work, not an error rate per run. Three entries follow, exactly as recorded.

  1. E71 · C inference beyond the evidence

    What happened. The genetics skill said adding unclassified variants to the benign-leaning ones would "roughly quintuple" the count; 127 to 564 is 4.4 times. It also said a 1,047 and 167 split agreed between a page and a workbook when the split is only on the page, and gave the CFF genotype base as the whole Registry when the footnote says genotyped individuals

    How it was found. a blind reader of the final text (Fable). Cost: 3 edits.

    What changed. recompute every ratio and every "agrees" claim from the printed numbers in a script before writing it; the final wording needs its own blind review, not only the extracted facts

  2. E77 · C unchecked claim

    What happened. I told a council that the cloned skills repo had no plugin manifest and labelled it verified; my listing hid dot-folders and the repo has one (a marketplace of bundles)

    How it was found. a council member (Fable) doubting the fact, then `ls -a`. Cost: 1 correction.

    What changed. list hidden files before saying a folder lacks something; a fact marked verified names the command that showed it

  3. E78 · B unchecked state

    What happened. The downloader of registry PDFs had always used the library's default user agent with no run-time robots check, against the lab's own network rules

    How it was found. me, reading the code while building the robots fix. Cost: 1 rewrite with 23 tests.

    What changed. a tool that fetches follows the one maintained network rule set or refuses; audit the older tools when a rule is tightened

These counts were taken from the lab's research repository as it stood on October 24, 2026 (version 5e0e412).

Every caught mistake that exposes a repeatable gap becomes a deterministic check. Across the workspace, check-all.ps1 now runs 134 offline verification suites, including 44 AST and security linters in cf-lab/tasks/. Once a class of error is caught, code blocks it on future runs.

Research notes in the open

The lab's working notes are public, in the research repository. Each note has a status line that says what it is, and each records what was read and checked on its date. None is a finding that stays true by itself. These ten are about how the lab uses AI models, checks its own work, and reads published clinical trial and literature records. Eight more below cover our multi-source verification and biophysical modeling dossiers (10 + 8 = 18 notes).

Mexican Spanish views (en español mexicano): Inicio, La historia de la fibrosis quística and La proteína CFTR en 3D.

Found an error or unverified claim?

We hold every number, citation, and clinical record to deterministic public standards. If you spot a factual error, retracted citation, or calculation defect, please report a correction so we can log it into our public error ledger.

Report a Correction Open an Issue on GitHub