fred — two real test cases
Demonstration of fred, an experimental scientific research agent, on two real research-analyst tasks. All content shown is verbatim output from the eval run on 2026-04-30.
What is fred? A multi-tenant scientific runtime that uses an LLM with a ReAct loop to call real research tools (PubMed, PubChem, UniProt, Reactome, OpenTargets, Brave Search) and synthesize findings into structured research memos. Each test below was run end-to-end with no human intervention beyond the initial prompt.
What this page shows. For each test case: the verbatim prompt fred received, a dataflow diagram of the ReAct loop fred executed, the actual queries fred fired against each external data source, and the full memo fred wrote — including a candid verification of which claims hold up against ground truth.
1The prompt fred received
2Dataflow — what fred actually did
The ReAct loop is fred's reasoning cycle: think, call tools, observe, repeat. This diagram is reconstructed from timestamps in the live log.
3What fred queried (verbatim from log)
Each line below is an actual API query fred fired. fred decided these queries iteratively based on what previous results returned. Grouped by data source.
TDP-43 proteostasis autophagy stress granule ALS drug repurposingTDP-43 inhibitor approved drug late stage clinicalautophagy activator ALS clinical trial repurposingstress granule ALS drug compound inhibitorDNL343 ISRIB ALS stress granule clinical trialtrehalose autophagy ALS amyotrophic lateral sclerosis clinicalnilotinib ALS autophagy c-Abl clinical trialarimoclomol ALS heat shock protein TDP-43AMX0035 sodium phenylbutyrate taurursodiol ALS proteostasis autophagyrapamycin sirolimus ALS autophagy TDP-43 clinicaledaravone ALS oxidative stress autophagy TDP-43guanabenz salubrinal eIF2alpha ALS stress granulenilotinib amyotrophic lateral sclerosis phase 2 clinical trialarimoclomol amyotrophic lateral sclerosis phase 2 3 clinical trialcolchicine ALS autophagy HSPB8 TDP-43 clinical trial resultsTDP-43 antisense oligonucleotide ASO amyotrophic lateral sclerosis- ... 16 more progressively narrower queries
arimoclomol ALS phase 3 clinical trial results 2024colchicine ALS clinical trial results Co-ALSAMX0035 sodium phenylbutyrate taurursodiol ALS phase 3 PHOENIX trial resultstrehalose ALS HEALEY platform trial results 2025 negativeDNL343 fosigotifator ALS Abbvie Calico phase 2 3 trial failure 2025nilotinib FDA approved chronic myeloid leukemia c-Abl inhibitor blood brain barrierguanabenz ALS clinical trial stress granule approved drug- ... 13 more, mostly fact-checking phase 2/3 trial outcomes
{name: "trehalose"}→ MW, logP, H-bond donors/acceptors{name: "nilotinib"}{name: "colchicine"}{name: "edaravone"}{name: "guanabenz"}
4What fred wrote
3494 words. The full memo is below. fred wrote this in a single 4-minute synthesis pause after gathering evidence, then verified by reading the file back.
↓ Download compound_dive_tdp43.mdOpen full memo (compound_dive_tdp43.md)
5Verification status
Specific factual claims spot-checked against PubMed and PubChem. Honest mix of pass and fail.
Caveats for the reader
- Memo is best treated as a "starting draft" — substance is reliable, fine details require verification.
- Citations are author-year format, not full PMIDs. Manual lookup needed to verify each.
- Fred's prompt asked for 1 of 6 prompts in this eval; full eval has wider sampling.
- This run was one shot — no human iteration or feedback loop. Quality reflects fred's first attempt.
1The prompt fred received
2Dataflow — what fred actually did
More tool diversity than TDP-43 (8 of 9 tools used) — fred reached for compound, target, pathway, and Lipinski analysis tools given the focus on a single named drug.
3What fred queried (verbatim from log)
Highest tool diversity of any prompt in the eval. Every category of tool used at least once.
search_compounds({query: "riluzole", max_results: 3})get_compound_properties({name: "riluzole"})→ MW=234.2, XLogP=3.6, HBD=1, HBA=7check_lipinski({hba: 7, hbd: 1, molecular_weight: 234.2, xlogp: 3.6})
get_target_info({identifier: "SLC1A2"})— astrocytic glutamate transporter (EAAT2/GLT-1)get_target_info({identifier: "SCN1A"})— voltage-gated sodium channel
search_pathways({query: "riluzole", max_results: 3})
riluzole mechanism of action ALS binding partners GLT-1riluzole drug interactions co-prescribed agents pharmacokineticsriluzole dosing strategy efficacy tolerability clinical trialriluzole CNS penetration blood brain barrierriluzole glutamate release sodium channelriluzole dose response adverse effects liverriluzole survival benefit ALS mortality meta-analysis randomized controlled trialriluzole interactions quetiapine oxybutynin anticholinergic neurology- ... 26 more progressively narrower queries
riluzole drug interactions pharmacokinetics CYP1A1 CYP1A2 fluvoxamine smokingriluzole dosing 50mg BID efficacy tolerability ALS clinical trialriluzole mechanism of action glutamate release voltage gated sodium channels potassium channelsriluzole P-glycoprotein BBB penetration CNS pharmacoresistance ALSriluzole hepatotoxicity liver enzyme monitoring dose dependent side effectsriluzole transdermal suspension liquid formulation dosing "Tiglutik" "Exservan" oral film- ... 4 more focused on co-prescribed agents and formulation alternatives
4What fred wrote
2093 words. Includes a structured molecular targets table, evidence hierarchy, and 14 named peer-reviewed citations including Bensimon NEJM 1994, Lacomblez Lancet 1996, and the Cochrane 2012 meta-analysis.
↓ Download compound_dive_riluzole.mdOpen full memo (compound_dive_riluzole.md)
5Verification status
For riluzole specifically, the foundational citations are well-known historical RCTs — easily verifiable.
Caveats for the reader
- Riluzole is one of the most-studied ALS drugs ever — the foundational facts (RCTs, mechanism, dosing) are unambiguous in the literature. fred's job here was easier than for novel/contested topics.
- Newer 2024-2025 references (Hershey, Liu transdermal, Bharti PBPK) not individually verified.
- Fred used 8 of 9 registered tools — broadest tool diversity in the eval. This is closer to OpenClaw-equivalent multi-source synthesis.
- Memo length (2093 words) appropriate for the prompt's scope. Format compliance better than TDP-43.