PharmAgent

Pharmacovigilance questions take two to four hours to answer properly because the evidence is scattered across FDA labels, PubMed and clinical guidelines. The obvious automation is also the dangerous one: a drug safety answer that sounds authoritative and is wrong is worse than no answer. So the interesting design problem is not retrieval speed — it is building a loop that can tell when its own retrieval failed.

A six-node LangGraph loop with a genuine cycle in it. Analyse-and-route classifies the query and picks knowledge bases. Hybrid BM25-plus-dense retrieval runs across three corpora. A grading node then scores each chunk for relevance and discards the irrelevant ones — and if too little survives, a rewrite node reformulates the query and retrieval runs again, up to two retries. Only then does generation run, followed by a hallucination check that verifies every claim against source documents.

The published numbers are targets, not a completed benchmark — Precision@10 >0.75, faithfulness >0.90, hallucination <6% are thresholds the system is designed to hold, and the evaluation covers five scenarios, which is a smoke test rather than a study. The retry loop also has a failure mode worth naming: when a query genuinely has no supporting evidence, the rewrite path burns both retries before the system concludes it cannot answer, which is the slowest possible path to “I don't know”.

Cross-references roughly 150,000 FDA DailyMed package inserts, 23.9M PubMed snippets and 9,330 StatPearls articles in under a minute. The most satisfying result was negative: on a false-premise test the system correctly rejected a non-existent drug interaction rather than confabulating a plausible one.