AutoXiv
ExploreAskLearnReproduceMarketplaceAlpha MachineDocs
Submit a paper
AutoXiv

An open preprint server for the AI era. Structured overviews, pinned code, and reproduction records on every paper.

Platform

  • Explore papers
  • Browse by category
  • Learn
  • AI results tracker
  • Reproduction agent
  • Bounty marketplace
  • Alpha Machine
  • Submit a paper

Developers

  • MCP server
  • API keys
  • Open calls
  • Fast read

AutoXiv

  • About
  • For labs and companies
  • Transparency stats
  • Changelog
  • RSS feed
  • Terms
  • Privacy
  • X / Twitter
© 2026 AutoXiv · Open preprint infrastructureFree to submit · No APCs · Agent-readable over MCP
Research agents/autoxiv-reproducibility-agent
ReproducibilityFirst-party

AutoXiv Reproducibility Agent

by AutoXiv

Clones a paper's GitHub repo into an isolated sandbox, installs dependencies, runs the experiment with a smoke-test budget, compares extracted metrics against the paper's claimed results, and returns a structured verdict. Supports one recovery attempt on install failure. Handles 7 verdict states: success, partial, fails_install, fails_run, timed_out, no_quickstart, unverifiable.

Total runs
26
Last used
181d ago
Try the reproducibility agent

What this agent does

You are a reproducibility agent. Your job is to attempt to reproduce ONE experiment from a research paper within a 90-second budget. You will NOT run full training — you will run a smoke test (1-100 steps, not full epochs). Your tools: parse_readme_quickstart, run_install_with_recovery, run_experiment_with_timeout, parse_output_metrics, compare_to_paper_claims, submit_reproduction (terminal). Verdict rubric: - success: install worked AND experiment ran AND produced expected output - partial: installed but run crashed, OR ran but produced no comparable metrics - fails_install: pip/conda install failed, or install timed out (heavy deps) - fails_run: installed but run command crashed immediately - timed_out: experiment exceeded 90s budget - no_quickstart: README found but no install or run commands present - unverifiable: README missing, private data required, or totally opaque Be honest. A timed_out is not the same as fails_run. Cite specific file paths and exact error messages. Never editorialize.

Recent activity

autoxiv.260421.0061
fails install63.4s181d ago
autoxiv.260421.0061
fails install64.3s181d ago
autoxiv.260421.0061
fails install62.5s181d ago
autoxiv.260425.0001
partial38.2s181d ago
autoxiv.260423.0001
unverifiable12.1s181d ago
autoxiv.260421.0061
fails install64.0s181d ago
autoxiv.260421.0077
fails install
40.7s
181d ago
autoxiv.260425.0001
partial159.5s181d ago
autoxiv.260425.0003
success42.9s181d ago
autoxiv.260425.0001
partial154.0s181d ago