I built a full AI content generation pipeline for KISHO. Three language models. A fact-checking layer. A quality review layer. A circuit breaker that quarantined bad output. Validation rules. Telemetry. An admin dashboard to monitor it all.

Then I deleted the whole thing.

Not because AI is bad. Because for rare disease data that clinicians, researchers, and families depend on, the architecture was wrong.

What we built

KISHO covers 10,888+ rare diseases. Each disease page needs a structured summary: what the disease is, how it presents, the genetic basis, diagnosis, treatment landscape, prognosis, research activity. That content has to be accurate. Not "mostly right." Not "plausible." Accurate.

The original pipeline worked like this. A knowledge packet assembled 40+ structured fields from authoritative sources: ClinicalTrials.gov, FDA, PubMed, Orphanet, MONDO, HPO, GeneReviews. No AI involvement in that step. Pure data assembly with provenance tracking.

Then a large language model took that packet and generated narrative prose. The output went through a second model for fact-checking, comparing every claim against the same knowledge packet. A third model ran quality review, checking treatment accuracy across 8 issue types and 5 severity levels. If a report failed three validation cycles, a circuit breaker quarantined it.

It was thorough. It was expensive. And it had a fundamental problem.

The bug that changed my mind

The therapy regression bug. A disease page for Condition A would occasionally include a treatment that belongs to Condition B. Not because the knowledge packet was wrong. The packet was clean. The AI model, drawing on its training data, would blend information from similar diseases into the output. It would confidently state that a treatment was available when the structured data said otherwise.

The fact-checker would sometimes catch it. Sometimes not. Because the fact-checker is also a language model, also drawing on training data, also capable of the same class of error.

I was building increasingly complex machinery to catch mistakes that should never have been introduced. A fact-checking pipeline is a symptom of unreliable generation, not a solution to it.

What replaced it

Code. Deterministic, testable code.

Every disease page section now runs through a rendering function that reads structured data from a materialized profile and returns formatted content. The function for treatment landscape reads the treatment fields. If there are FDA-approved therapies, it lists them with approval dates. If there are orphan drug designations, it includes them with exclusivity status. If there's nothing, the section doesn't appear.

No generation. No prompt. No temperature setting. No fact-checking pipeline because there are no generated facts to check.

A sentence like "Fabry disease is an X-linked lysosomal storage disorder caused by mutations in the GLA gene, resulting in deficiency of alpha-galactosidase A" is assembled by a sentence builder from structured fields: inheritance pattern, disease mechanism, gene name, protein product. Every word traces to a specific data point. The output reads like a clinician wrote it because it's composed from the same structured knowledge a clinician would reference.

Yes, the prose is formulaic. It follows predictable patterns. For clinical reference content, that's the point. A genetic counselor doesn't need a disease summary that reads like a magazine feature. They need one where the structure is consistent, the facts are traceable, and nothing was invented to make a paragraph flow better.

It also means every claim on a disease page can be traced to its source. An approved therapy links back to the FDA record. A clinical trial count resolves to specific NCT numbers on ClinicalTrials.gov. A gene association traces to HGNC and ClinGen validity classifications. There's no black box between the source data and what appears on the page. A clinician can audit any statement and find exactly where it came from.

What got deleted

The 330-line generation prompt. The 490-line user template with safety checklists. The fact-checking pipeline. The quality review pipeline. The circuit breaker with quarantine logic. 15+ validation rules. A telemetry system tracking error rates, severity distributions, and 90-day rolling windows. An admin dashboard for monitoring generation quality. Three external API dependencies removed from the critical path of every disease page.

The therapy regression bug didn't get fixed. It became impossible.

What AI still does at KISHO

This isn't an anti-AI story. AI still plays specific roles where it's the right tool.

Classification is one. When a new research article comes in, a model classifies it into research types (gene therapy, natural history, case report, etc.) and maps it to relevant diseases. That's a bounded input, bounded output task where the failure mode is a misclassification, not a fabricated fact. Different risk profile entirely.

The other is interactive editing tools. When a researcher or PAG science officer is working on a report in their workspace, they can use inline AI tools to validate claims against source data, simplify language, or add citations. The key difference: the person initiates each action, sees the result, and decides whether to keep it. The AI suggests. The expert with domain knowledge accepts or rejects. Nothing goes live without a human who understands the disease making that call.

The distinction matters. AI for classification and interactive assistance is a different architecture than AI for unsupervised content generation across 10,000+ medical conditions.

Why this matters beyond KISHO

Every health tech company right now is racing to add AI-generated content to their platform. I get it. The demos are impressive. But there's a question that doesn't get asked enough: what happens when the AI is wrong about a treatment for a disease that affects 200 people worldwide?

For common diseases, the training data is dense. The models have seen thousands of papers, guidelines, treatment protocols. The error rate is low, and when errors do appear, clinicians catch them because the correct information is widely known. For a disease with 12 published papers and no approved treatment, the model is interpolating. It's generating plausible prose from sparse signal. And the people reading that content, a parent who just got a diagnosis, a genetic counselor preparing for a family meeting, often have no way to spot the error.

An AI hallucinating a treatment option for breast cancer gets corrected quickly. An AI hallucinating a clinical trial for a disease that affects 200 people offers a family false hope. That family may spend weeks chasing something that doesn't exist. In rare disease, where every signal matters, a fabricated signal does real damage.

The answer isn't better prompts. It isn't more fact-checking layers. It's recognizing that some content should be rendered from verified data, not generated from statistical patterns.

KISHO still uses AI. It just doesn't use AI to write the content that families and clinicians are depending on to be correct. That's a line I'm not willing to blur.

Every KISHO disease page renders from 12+ authoritative sources, no AI generation in the loop. Explore any of the 10,888+ diseases at kishomed.io.