Open-source, self-hosted research harnesses

This is one part of a series on research-agent systems. The landscape piece states where the field stands and sets all twenty-seven systems side by side: Agentic science harnesses: the landscape.

Deep dives: open-source, self-hosted

Eight harnesses you install and run yourself. Costs are your own LLM API usage. All GitHub stats verified via the GitHub API on 03 September 2026 and re-verified on 14, 21 and 28 September and 5, 12, 19 and 29 October and 2 November and 9 November 2026 (star/fork/issue deltas shown in the weekly update log at top). Feynman is covered in its own section above; the Claude Science equivalents (Runcell Science, OpenScience, AIPOCH Open-Science, wisp-science, Biomni, Eigent) are in the section above that.

Literature & retrieval agents

PaperQA2 · OpenScholar · STORM · GPT Researcher

PaperQA2

FutureHouse (nonprofit; Sam Rodriques CEO, Andrew White Head of Science, Jessica Wu Head of Ops)

9,568★

Apache-2.0

Active

Team / origin / funding

Announced 11 Sep 2024 (arXiv 2409.13740; authors Skarlinski, Cox, Laurent, Braza, Hinks, Hammerling, Ponnapati, Rodriques, White). FutureHouse funded by philanthropy/grants: Eric & Wendy Schmidt (first two years), OpenPhilanthropy, NSF, AI Safety Institute. Commercial spinout Edison Scientific raised a $70M seed at ~$250M valuation, co-led by Spark Capital and Triatomic Capital, angels incl. Jeff Dean and Dmitri Alperovitch (techfundingnews.com). Note: Signalbase headline says "FutureHouse secures $70M" but the funding entity is the Edison spinout.

Architecture

High-accuracy agentic RAG for answering questions from scientific documents with citations. Agent tools: find papers, extract useful information, explore the citation graph, formulate answers. LLM driver via LiteLLM (any provider, incl. local models). Built-on applications: WikiCrow (Wikipedia-style summaries) and ContraCrow (contradiction detection). Python package, pip install paper-qa.

Lit review capacity

Search: agentic search over full text; explores the citation graph; searches a paper directory or live sources. Evaluated on LitQA2 (LAB-Bench, arXiv 2407.10362); first agent reported superhuman on lit-search (73.8% human precision vs agent, n=9)
Read/parse: extracts useful information from papers via tools; RAG over paper corpora; PDF parsing (PyMuPDF, NemotronParse)
Synthesis: cited answers; WikiCrow summaries rated more accurate than human Wikipedia by blinded PhD/postdoc reviewers; ContraCrow finds 2.34 ± 1.99 contradictions/paper, 70% expert-validated
Citations: context-cited answers; citation graph exploration; output includes DOIs and citation counts

Pricing / hosting / provider tie

Open-source Apache-2.0, self-hosted (pip install paper-qa); costs = your LLM/embedding API. Hosted WikiCrow demo at paper.wikicrow.ai; hosted platform from Edison Scientific (platform.edisonscientific.com). Any LLM via API (LiteLLM); local models supported. Original paper pipeline used GPT-4o (per AI2 blog).

GitHub: 9,568★ · 953 forks · last push 26 Oct 2026 (default branch last release v2026.08.12). X: @FutureHouseSF. Docs: docs.paperqa.com: TLS broken (confirmed 2026-09-03); GitHub README is the primary maintained doc; futurehouse.gitbook.io redirects to Edison docs.

Positive

Open release celebrated: 296 upvotes, 96% upvoted on r/singularityhttps://safereddit.com/r/singularity/comments/1feh9t3/
Well-cited Alzheimer's example with DOIs on HN launchhttps://news.ycombinator.com/item?id=41513031
WikiCrow: generated articles for all 19.2k human genes weekly, more precise than human Wikipedia claimshttps://news.ycombinator.com/item?id=41926771
Real-world regulatory use: "using PaperQA2 for some regulatory use cases and it's gangster"https://safereddit.com/r/LocalLLM/comments/1pxvlh0/

Negative / caveats

Hallucination concern even with grounding: "extremely dangerous for that one detail you will not expect"https://news.ycombinator.com/item?id=42140356
"Superhuman" claim debated (n=9 humans; MCQ benchmark gaming acknowledged in LAB-Bench paper)https://arxiv.org/abs/2407.10362
AI2 claims OS-8B is 100x more cost-efficient than PaperQA2https://allenai.org/blog/openscholar
Beaten by OpenScholar-8B by 5.5% on ScholarQABench multi-paper synthesishttps://www.nature.com/articles/s41586-025-10072-4

Neutral

Benchmark recency confound: hard to separate model capability from training-data cutoff on LitQA2https://www.lennijusten.com/blog/biology-benchmarks/

OpenScholar

AI2 (Allen Institute for AI) + University of Washington; lead author Akari Asai, senior author Hannaneh Hajishirzi

1,683★

Apache-2.0

Nature 2026

Team / origin / funding

arXiv 2411.14199 (21 Nov 2024); AI2 blog 19 Nov 2024; accepted to Nature, published 4 Feb 2026 (Nature 650, 857-863). 25 authors incl. Zettlemoyer, Weld, Neubig, Downey, Koh, Yih (Meta); collaborators at Meta, CMU, UIUC, Stanford, UNC. Funding: Asai supported by Meta AI Mentorship program; Singapore National Research Foundation / National AI Group AI Visiting Professorship Programme (blog acknowledgments). Models, datastore and code open-sourced.

Architecture

Specialized retrieval-augmented LM: OSDS datastore of 45M open-access papers / 236M passage embeddings (peS2o v2/v3); trained retriever + reranker; instruction-tuned 8B generator (Llama 3.1 fine-tune on ~13k synthetic instruction data, 8xA100); iterative self-feedback loop (each iteration retrieves more papers and refines factuality/coverage/citations); post-hoc citation attribution. Pipeline also runs on off-the-shelf LMs (OS-GPT4o).

Lit review capacity

Search: passage-level retrieval over 45M open-access papers (up to Oct 2024); trained retriever + reranker; Semantic Scholar Paper API + optional you.com web search API
Read/parse: abstracts + full text; 200M+ passage embeddings index (needs large CPU memory)
Synthesis: instruction-tuned generator produces citation-backed responses; OS-8B beats GPT-4o by 6.1% and PaperQA2 by 5.5% on ScholarQABench; OS-GPT4o improves GPT-4o correctness by 12%
Citations: post-hoc citation attribution on par with human experts; GPT-4o hallucinated citations 78-90% of the time vs OpenScholar (Nature abstract)

Pricing / hosting / provider tie

Fully open-source (Apache-2.0); models, datastore, data free on HuggingFace. Public demo (originally openscilm.allen.ai / open-scholar.allen.ai): NOTE: now redirects to Ai2 Asta (108M+ abstracts, 12M+ full-text); the OpenScholar demo has been superseded by Asta. No provider tie: open 8B model; optional proprietary base LMs (e.g. GPT-4o) via API; needs Semantic Scholar API key.

GitHub: 1,683★ · 179 forks · last push 13 Aug 2025 (still stalled; research-prototype cadence; 12 open issues). X: no dedicated handle; launch via @AkariAsai + @allen_ai. Docs: GitHub README (primary); allenai.org/blog/openscholar-nature (Feb 2026). Repo is AkariAsai/OpenScholar (not akariasai/openscholar).

Positive

Nature publication: "first fully open, retrieval-augmented LM specifically designed for scientific research tasks"https://www.nature.com/articles/s41586-025-10072-4
UW News: 16 scientists preferred OpenScholar answers to human-written 51% of the time (70% with GPT-4o)https://www.washington.edu/news/2026/02/04/
r/academia (84% upvoted): editors see reviews changing; systematic reviews seen as mechanicalhttps://safereddit.com/r/academia/comments/1qwed3p/

Negative / caveats

Comparison baseline stale: "The authors are comparing their LLM to an older version of ChatGPT. Does that matter?" (top comment, 69 pts)https://safereddit.com/r/academia/comments/1qwed3p/
Academic user: "I found it severely lacking. It missed some obvious literature and misrepresented the studies that it did offer."https://safereddit.com/r/academia/comments/1qwed3p/
May cite papers that merely mention a method; may cite from parametric knowledge (hallucinated, shown as non-links in demo): acknowledged in AI2 bloghttps://allenai.org/blog/openscholar
Open-access-only datastore degrades performance in paywalled-heavy fields; nuance limits acknowledged
Repo maintenance stalled after Aug 2025; demo superseded by Ai2 Asta

Neutral

r/Professors (91% upvoted): humanities prof acknowledges it's the one PhD task LLMs could do well if hallucination is controlledhttps://safereddit.com/r/Professors/comments/1qvuj21/

STORM / Co-STORM

Stanford OVAL (Open Virtual Assistant Lab), Monica Lam's group

31,840★

MIT

Stalled Sep 2025

Team / origin / funding

STORM: Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, Monica S. Lam (NAACL 2024, arXiv 2402.14207, 22 Feb 2024). Co-STORM: Yucheng Jiang, Yijia Shao, Dekun Ma, Sina Semnani, Monica Lam (EMNLP 2024, arXiv 2408.15232, 27 Aug 2024). Repo created 24 Mar 2024; demo storm.genie.stanford.edu live since ~mid-2024. Funding: thanks Microsoft Azure for cloud credits (demo); Vercel supports open-source hosting; no commercial funding verified.

Architecture

Two-stage LLM knowledge-curation pipeline: (1) pre-writing: Internet-based research to collect references and generate an outline, via perspective-guided question asking (discovers perspectives from similar articles) and simulated conversations between a writer and a topic expert grounded in sources; (2) writing: full-length article with citations. Co-STORM adds a collaborative discourse protocol: LLM experts, a moderator, and a human user; dynamic mind map; implemented modularly in DSPy. Retrievers: YouRM, BingSearch, VectorRM, SerperRM, BraveRM, SearXNG, DuckDuckGoSearchRM, TavilySearchRM, GoogleSearch, AzureAISearch. LLM/embedding via LiteLLM since v1.1.0 (Jan 2025).

Lit review capacity

Search: Internet search engines (no dedicated scientific corpus); perspective discovery surveys similar Wikipedia articles; simulated expert Q&A grounded on trusted Internet sources
Read/parse: retrieves and curates information from sources into an outline; VectorRM allows grounding on user-provided documents
Synthesis: models the pre-writing stage: perspective discovery, simulated Q&A, outline curation, then full report generation. 70,000+ people used the live preview (README)
Citations: generated reports carry citations to retrieved sources; but hallucinated citations and off-topic sources documented in reception

Pricing / hosting / provider tie

Open-source MIT (pip install knowledge-storm); free hosted demo at storm.genie.stanford.edu; self-host costs only your LLM API. No SaaS tier. Any LLM via API: OpenAI API default, other providers via LiteLLM.

GitHub: 31,840★ · 3,007 forks · last push 30 Sep 2025 (still stalled at the 9 Nov check; ~13 months quiet). X: @StanfordOVAL (no dedicated STORM handle). Docs: GitHub README (primary), storm-project.stanford.edu, demo storm.genie.stanford.edu.

Positive

HN 123 pts (repo launch): "I asked it about a niche topic and I got a very useful encyclopedia-type report that included real links to published research"https://news.ycombinator.com/item?id=40004887
r/perplexity_ai 489 pts, 93% upvoted: "Stanford's STORM AI outperforms Perplexity & Google Deep Research - and it's completely FREE"https://safereddit.com/r/perplexity_ai/comments/1htadmb/
r/LocalLLaMA: "storm was cool but co-storm really helps learn things and drill down the report"; "on par with Google deep research"https://safereddit.com/r/LocalLLaMA/comments/1ih0te1/

Negative / caveats

Documented hallucination: "it straight up hallucinated... claimed that google cartographer was written in rust... the citation... did not mention cartographer anywhere"https://news.ycombinator.com/item?id=41553875
"The LLM is not doing research - it is aggregating data"; publication skepticismhttps://news.ycombinator.com/item?id=40004887
Repetitive/verbose output; odd source quality ("short on reputable sources, long on obscure blogs")
Friction: 20-word input limit + mandatory "why do you want this" gate
Losing edge: "with Google and OpenAI Deep Research it's not quite cutting edge"; repo stalled; demo sometimes 500s

Neutral

HN 145 pts (demo): 72 comments, largely engaged discussion of capabilities and limitshttps://news.ycombinator.com/item?id=41553875

GPT Researcher

Assaf Elovic (also founder/CTO of Tavily) + large open-source community

30,594★

Apache-2.0

Active

Team / origin / funding

Repo created 12 May 2023; launched via Medium post "How I Built an Autonomous AI Agent for Online Research" (24 Jul 2023); inspired by AutoGPT's failures + Plan-and-Solve (2305.04091) + RAG (2005.11401). No formal paper (DeepResearchGym paper is a third-party evaluation).

Architecture

Two-stage planner + parallel execution: planner agent generates research questions forming an objective opinion; crawler/execution agents scrape and summarize per resource in parallel (asyncio), tracking sources; publisher filters and aggregates into final report. Extras: local document research (PDF/txt/CSV/Excel/MD/PPTX/DOCX), MCP client + server (gptr-mcp), multi-agent mode via LangGraph and AG2, Deep Research recursive tree exploration, Gemini inline image generation, LangSmith observability. ~3 min average task, ~$0.1-0.4 per research (README/docs).

Lit review capacity

Search: general web search engines: Tavily default, Google/Bing/DuckDuckGo, arXiv retriever option, local docs, MCP data sources; aggregates 20+ sources per research
Read/parse: crawler agent scrapes each source (JS-enabled), summarizes per-resource while keeping source tracking
Synthesis: planner filters + aggregates all summarized sources into a final report (2k+ words, exports PDF/DOCX/MD)
Citations: per-resource source tracking with citations in output; ranked #1 in citation quality on DeepResearchGym (85.36% precision, 90.82% recall, only system >85% both). Honest gap: general web not science-specific; not built for peer-reviewed corpus retrieval

Pricing / hosting / provider tie

Free open source (Apache-2.0), self-hosted (pip install gpt-researcher, Docker, frontend); also Claude Skill install; costs are LLM/search API usage (~$0.1-0.4/run). Any LLM: 100+ supported (OpenAI, Anthropic, Groq, Llama, HuggingFace, local via OpenAI-compatible endpoints); any search engine (Tavily, Google, Bing, DuckDuckGo).

GitHub: 30,594★ · 4,202 forks · last push 9 Nov 2026 (latest release v3.7.0, 26 Sep, the "Jev update": passage scoring via TypeSafe's Jev with a BM25 fallback, no embeddings required; Python 3.12+; 33 open issues). X: no official product handle; founder @assaf_elovic (verified, posts product releases). Docs: gptr.dev (marketing site) + docs.gptr.dev (Docusaurus).

Positive

Ranked #1 on CMU DeepResearchGym benchmark (May 2025) vs Perplexity, OpenAI, OpenDeepSearch, HuggingFace: citation quality, report quality, coveragehttps://arxiv.org/abs/2505.19253
Harrison Chase (LangChain CEO): "one of the more advanced OSS research projects out there"https://gptr.dev/
HN commenter prefers it for source/search-engine controlhttps://news.ycombinator.com/item?id=43752262
Cited in peer-reviewed/arXiv work (Chain of Ideas, LiveRAG, BioKGBench)

Negative / caveats

Launch thread criticism: "why are the titles in the Bibliography wrong hallucinated?" (Reddit r/singularity)https://www.reddit.com/r/singularity/comments/14vt3wj/
TrilogyAI: requires technical setup, not plug-and-play; needs a powerful model for quality; resource-intensive; debugging on userhttps://trilogyai.substack.com/p/comparative-analysis-of-deep-research
Community notes quality drops without strong model; self-hosted setup complexityhttps://www.reddit.com/r/LocalLLaMA/comments/1ilqn04/

Neutral

Tom Dörr: "usually avoid projects with GPT in their name, but GPT Researcher looks pretty solid" (arXiv retriever)https://gptr.dev/

Research frameworks (autonomous pipelines)

Agent Laboratory · ResearchAgent · The AI Scientist

Agent Laboratory

Samuel Schmidgall (JHU/AMD) with Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu (AMD), Michael Moor (ETH Zurich), Zicheng Liu, Emad Barsoum (AMD)

5,926★

MIT

EMNLP 2025

Team / origin

arXiv 2501.04227 (submitted 8 Jan 2025, v2 17 Jun 2025); Findings of EMNLP 2025, Suzhou, pages 5977-6043 (aclanthology.org/2025.findings-emnlp.320). Funding not stated (AMD affiliation implied, unverified). Extends to AgentRxiv (arXiv 2503.18102) for agents building on each other's research.

Architecture

3-stage pipeline: (1) Literature Review, (2) Experimentation, (3) Report Writing. PhD/Postdoc/ML Engineer/SW Engineer/Professor agents; mle-solver for iterative code generation with REPLACE/EDIT, LLM reward scoring, top-program pool, self-reflection; paper-solver for LaTeX report generation with scaffold + arXiv access + NeurIPS-style automated reviewer; human feedback at each stage (co-pilot mode).

Lit review capacity

Search: arXiv API only; PhD agent queries, retrieves abstracts of top-20 relevant papers, extracts full text, iteratively adds papers to curated review (N=max)
Read/parse: abstract via "summary" action; full text via "full text" action; relevance evaluated by the agent per query
Synthesis: curated review finalized once add-paper quota reached; feeds plan formulation and paper-solver scaffold
Citations: references come from the arXiv-based lit review; citation quality not independently verified, honest gap: lit-review stage is shallow (single-source arXiv API, top-20 abstracts)

Pricing / hosting / provider tie

Free open source (MIT), self-hosted (Python 3.12, pip install); run costs are LLM API usage: paper reports $2.33 per paper on gpt-4o, $13.10 on o1-preview; 84% cost reduction vs prior autonomous methods (self-reported). Provider tie: OpenAI (o1, o1-preview, o1-mini, gpt-4o, o3-mini) and DeepSeek (deepseek-chat); README invites PRs for more backends; no local-model path documented.

GitHub: 5,926★ · 805 forks · last push 20 Aug 2025. X: no official handle; lead author Samuel Schmidgall @SRSchmidgall (personal). Docs: agentlaboratory.github.io + GitHub README.

Positive

Accepted to Findings of EMNLP 2025 (verified via ACL Anthology)https://aclanthology.org/2025.findings-emnlp.320/
mle-solver won 4 MLE-bench medals (2 gold) vs OpenHands/AIDE/MLAB (project site, self-reported)https://agentlaboratory.github.io/
Quoted in Science Magazine re AI scientists reliability (May 2026)https://x.com/SRSchmidgall/status/2052911248480845908

Negative / caveats

Hidden pitfalls paper (Luo, Kasirzadeh, Shah: NeurIPS 2025 AI4Science Spotlight) flags 4 failure modes in AI scientist systems incl. Agent Laboratory: benchmark selection, data leakage, metric misuse, post-hoc selection biashttps://arxiv.org/abs/2509.08713
Authors' own NeurIPS-style human review: all backends scored below accepted-paper average (o1-preview 4.0/10 vs 5.9 average accepted); automated reviews overestimated quality (6.1 vs 3.8)https://agentlaboratory.github.io/
HN launch threads low engagement (4 pts, 1 pt)https://hn.algolia.com/api/v1/search?query=%22Agent%20Laboratory%22&tags=story

Neutral

Cited as related work in academic papers (e.g. PETSc knowledge base, Training AI Scientists to Replicate Research)https://dl.acm.org/doi/abs/10.1145/3750720.3757281

ResearchAgent

Jinheon Baek (KAIST), Sujay Kumar Jauhar (Microsoft Research), Silviu Cucerzan (Microsoft Research), Sung Ju Hwang (KAIST / DeepAuto.ai)

74★

No license

NAACL 2025

Team / origin

arXiv 2404.07738 (submitted 11 Apr 2024, v2 9 Feb 2025); NAACL 2025 Long Papers, Albuquerque, pages 6709-6738 (aclanthology.org/2025.naacl-long.342); Microsoft Research publication page; no funding statement. 242 citations / 28 influential (Semantic Scholar, retrieved 3 Sep 2026).

Architecture

Iterated research idea generation: starts from a core paper, augments via academic graph (references/citations) + entity-centric knowledge store mined from co-occurring concepts across papers; generates problem → method → experiment design in fixed sequence; multiple LLM ReviewingAgents provide structured reviews/feedback across iterative revisions, instantiated with human preference-aligned criteria elicited via LLM prompting. Idea-generation focus: does NOT execute experiments or write final papers.

Lit review capacity

Search: academic graph traversal (references + citation relationships) from a seed paper; entity-centric knowledge store of co-occurring concepts across many papers
Read/parse: full-text paper understanding via LLM; academic graph provides related publications
Synthesis: synthesizes into problem/method/experiment proposals; iteratively refined by ReviewingAgents
Citations: grounded in seed paper + graph neighbors + knowledge entities; no citation-handling for output documents (output is an idea proposal, not a report). Honest gap: ideation-focused; no experimental execution; fixed generation sequence criticized by IdeaSynth

Pricing / hosting / provider tie

n/a: research prototype; GitHub repo exists (JinheonBaek/ResearchAgent, 62 stars, no license, last push 24 Aug 2025) but no packaged install; official code minimal; no docs site. LLM-API based (GPT-4 family in the paper); no supported-model list or local path.

GitHub: 74★ · 10 forks · no license · last push 24 Aug 2025. X: no official ResearchAgent handle found. Docs: arXiv abstract + full text; Microsoft Research publication page; Emergent Mind paper summary page.

Positive

HN thread: 124 points, 63 comments (Apr 2024): substantive discussion of LLM ideationhttps://news.ycombinator.com/item?id=40047152
242 citations / 28 influential (Semantic Scholar)https://api.semanticscholar.org/graph/v1/paper/arXiv:2404.07738
Positioned as foundational related work by Agent Laboratory and The AI Scientist (both cite it)https://arxiv.org/html/2501.04227v2

Negative / caveats

IdeaSynth (ACM CHI/ICMR 2025) notes ResearchAgent generates idea facets in a fixed sequence, unlike interactive systems giving users freedomhttps://dl.acm.org/doi/full/10.1145/3706598.3714057
"Can LLMs Generate Novel Research Ideas?" (2409.04109) highlights LLM ideation limitations on feasibility/implementationhttps://www.alphaxiv.org/abs/2409.04109
Small repo footprint (62 stars, no license): no maintained implementation for reusehttps://api.github.com/repos/JinheonBaek/ResearchAgent

Neutral

Downstream systems build on it (e.g. Deep Ideation, AI-Researcher listed on Emergent Mind)https://www.emergentmind.com/papers/2404.07738

The AI Scientist (Sakana AI)

Sakana AI, with the Foerster Lab for AI Research (Oxford) and Jeff Clune & Cong Lu (UBC / Vector Institute)

v1 14,966★ · v2 7,479★ · v3 587★

Custom license (v1/v2); Apache-2.0 (v3)

Nature 2026

Team / origin / launch

v1 announced 12 Aug 2024 (arXiv 2408.06292, v3 1 Sep 2024); v2 technical report 10 Apr 2025 (arXiv 2504.08066); Nature paper published 25 Mar 2026 (Nature 651, 914-919, DOI 10.1038/s41586-026-10265-5, open access). Collaboration with Oxford Foerster Lab and UBC/Vector per the Nature paper. v3, "The Agentic Scientist" (arXiv 2610.18933, 13 Oct 2026), is a separate Apache-2.0 scaffold released under the same lab, with an AutoIllustrate companion repo.

Architecture

v1: four automated processes: (1) Idea Generation (brainstorms diverse ideas from a template codebase, checks novelty via Semantic Scholar); (2) Experimental Iteration (writes and runs code, produces plots with notes); (3) Paper Write-up (full LaTeX manuscript, autonomously finding papers to cite via Semantic Scholar); (4) Automated Paper Reviewing (LLM reviewer with near-human agreement, feeding back into open-ended loops). v2: progressive agentic tree search with a dedicated experiment-manager agent, removes reliance on human-authored templates, generalizes across ML domains, adds a VLM feedback loop to refine figures; produced the first fully AI-generated paper to pass peer review (ICLR 2025 workshop; avg score 6.33 vs acceptance threshold; withdrawn per pre-agreed protocol). Model-agnostic: v1 used GPT-4o and Claude Sonnet as top performers, also DeepSeek and Llama-3.

v3: The Agentic Scientist (added 29 Oct 2026)

v3 abandons the single-pipeline design for orchestration of existing coding agents. It drives Claude, Codex, Cursor, OpenCode, Gemini or Pi through six phases: read a plain task.md brief; reproduce the baseline; run the experiments; run ablations; a sealed final evaluation on held-out data; then write the paper. Environment setup, code changes and long runs go through TODO.md and a CLI so a human can interrupt or take over. A development version co-produced an Error Diffusion paper that was accepted as an ALIFE 2026 oral presentation. 549 stars, Apache-2.0, active (last push 26 Oct 2026).

Lit review capacity

Search: Semantic Scholar API used for novelty checking in idea generation and for autonomously finding papers to cite in the write-up phase
Read/parse: reads retrieved literature during ideation; text/LLM-based with no vision in v1 (cannot read its own plots or fix figure layout)
Synthesis: generates complete scientific manuscripts in LaTeX including figures, tables and references; v2 adds VLM feedback on figure aesthetics/content
Citations: written into LaTeX automatically via Semantic Scholar. KNOWN v1 citation-hallucination/weakness: independent evaluation (arXiv 2502.14297) found a median of only 5 citations per generated paper, most outdated (5 of 34 from 2020 or later), and reviewers did not catch deep flaws. Treat citation quality with caution

Pricing / hosting / provider tie

Open source, free to self-host; cost is your LLM API usage (v1 reported ~$15 per generated paper). No SaaS. Self-hosted; requires LLM API access (any supported provider); GitHub warns to run in a sandbox (Docker) because it executes LLM-written code. LICENSE: NOT Apache-2.0: custom "AI Scientist Source Code License v1.0 (December 2025)" based on Responsible AI Source Code License v1.1; GitHub API reports Other/NOASSERTION; LICENSE file verified by direct fetch.

GitHub: v1 14,966★ / 2,089 forks / last commit 19 Dec 2025; v2 7,479★ / 1,015 forks / last commit 19 Dec 2025; v3 587★ / 80 forks / last push 18 Oct 2026. X: @SakanaAILabs (verified). Docs: sakana.ai/ai-scientist, sakana.ai/ai-scientist-nature, sakana.ai/ai-scientist-first-publication.

Positive

Paper published in Nature (651, 914-919); AI-generated manuscript passed first-round peer review at a top-tier ML workshop (70% acceptance rate)https://www.nature.com/articles/s41586-026-10265-5
v2 produced first fully AI-generated paper to pass human peer review (ICLR 2025 workshop, scores 6/7/6, withdrawn by protocol)https://sakana.ai/ai-scientist-first-publication/
HN launch thread: 203 points, 132 commentshttps://news.ycombinator.com/item?id=41231490

Negative / caveats

HN academic (JBorrow): "the papers that the model seems to have generated are garbage... extremely limited citation to associated works. As an editor of a journal, I would likely desk-reject them"https://news.ycombinator.com/item?id=41231490
Independent evaluation (arXiv 2502.14297): generated ideas often not novel; manuscripts poorly substantiated with median 5 citations per paper; reviewer agent missed serious methodological flaws; automation-bias risk for junior researchershttps://arxiv.org/abs/2502.14297
Sakana's own limitations list: no vision, incorrect implementations/unfair baselines; "bloopers" include the agent modifying and relaunching its own execution script (safety/sandboxing concern)https://sakana.ai/ai-scientist/

Neutral

Nature news coverage: "Researchers built an AI Scientist - what can it do?" (Aug 2024) and "How to build an AI scientist" (Mar 2026)https://www.nature.com/articles/d41586-024-02842-3
HN (visarga): AI may be genuinely useful at the literature-review/writing/formatting stage even if full autonomy is unproven

Computational science harnesses

BootLoops (added 5 Oct 2026)

BootLoops

Prof. Matthew D. Schwartz (Harvard University; visiting researcher at Anthropic): an independent project funded by Anthropic

429★

MIT

New · 1 Oct 2026

What it is

A "living harness" for interdisciplinary quantitative science: certified computational tools and "house engines" for exact, high-precision calculation, ported to a common framework and built to be driven by LLM agents. Model-agnostic, callable with Claude, Gemini or ChatGPT, so the harness improves independently of any frontier lab. Grew out of scattering-amplitude physics; published as BootLoops 1.0 on 1 Oct 2026 and featured in Anthropic's "Claude-shaped science" guest post.

How it works

Verification-first: the "BootLoops Standard": a diagram is solved only when its full functional form is known and a Python script evaluates it to arbitrary precision on a laptop (checks run at 30-100 digits; the AI "can't cheat")
Portfolio: tools ported from mathematics, physics and computer science (AMFlow, Eichler, GPLEval and more), indexed so the model can select per project; docs CC BY 4.0
Results at launch: 30 Feynman integrals computed end-to-end (15 never computed before); applications across ecology (Barro Colorado Island tree-mix turnover 4.5x faster than neutral theory allows), population genetics, cosmology, single-cell biology, authorship attribution (Federalist Papers, Shakespeare, Isaiah, the Voynich Manuscript), linguistics, clinical-trial design and economics

Pricing / hosting / provider tie

Free, open source (MIT; docs CC BY 4.0), self-hosted; costs are your own LLM usage. Provider tie: none: works with any LLM. GitHub: BootLoops-ai/bootloops: 429 stars, 68 forks, 2 open issues (API, 9 Nov 2026; created 1 Oct 2026); the org now carries six repositories (skills, blade, amflow-cpp, kira, jackandjill, feedback). Site: bootloops.ai. Anthropic provided funding; not an Anthropic project (owned and maintained by Schwartz).

Positive

Launched via the Anthropic Science blog (1 Oct): the week's most notable new open harness; 338 stars within eleven days of the repo going public
Concrete example of "Claude-shaped science": problems chosen to fit LLM strengths, with humans supplying the conceptual direction and credit

Neutral

Independent reception still thin (no HN/Reddit threads found in this pass); Schwartz's own caveat: early results were "technically correct but not interesting" until domain experts steered
The site's summaries keep widening: a health-policy analysis of CMS Medicare star-ratings clustering is now posted (the published algorithm approximates a criterion a 1958 method solves exactly; code and data released), seen at the 12 Oct check