Click the water to file a bug.

reproduced this session:0

Evidence for every bug report.

Comment /repro on an issue. Pilotfish runs the reporter's exact snippet on a fresh cloud machine and replies in about a minute with a verdict and a proof bundle anyone can re-run.

Install the GitHub App then comment /repro on any issue
See real runs on open-source issues

From a comment to a verdict in about a minute.

Pilotfish only runs when a maintainer asks. Everything happens on a throwaway machine, so nothing touches your CI, your laptop, or the reporter's code.

  1. 0s

    You comment /repro

    On any issue in a repo where the app is installed. Only people with write access can trigger a run.

  2. 3s

    A machine starts from your repo's snapshot

    A disposable VM on Boat, pre-built with your dependencies, so nothing installs from scratch.

  3. 40s

    The reporter's snippet runs as written

    With the versions they listed: runtime, OS, and package. Pilotfish doesn't tidy up or reinterpret their code.

  4. 58s

    A verdict lands on the issue

    With a repro script, logs, before and after output, and one command to run it again. Then the machine is deleted.

run rp_7Hq2xKbrineworks/when #2291

If it can't reproduce, it asks one question.

Pilotfish never closes an issue or calls it "not a bug". It says what it ran and what it saw, asks the reporter for the one detail that's missing, and runs again when they answer.

pilotfish bot commented on #2307
Not reproduced yet

Your snippet prints "5 minutes ago" as expected on Python 3.13.1, Ubuntu 24.04, with TZ=UTC.

One question: what is TZ set to on your machine? Reply here and I'll run it again with that value.

A reply you can check, not just trust.

Short enough to read in a notification email. Complete enough to settle the thread.

  • The verdict comes firstReproduced, or one question for the reporter.
  • Exact versionsWhat the reporter said, and what was actually installed.
  • A proof bundleScript, logs and output, attached to the issue.
  • One command to re-run itOn your machine or on a fresh VM.

humanize() says "in 0 seconds" for past datetimes on Python 3.13 #2291

Open ana-okafor opened this issue 6 minutes ago bug reproduced
ana-okafor commented 6 minutes ago

Since upgrading to 4.2.0, anything in the past comes back as the future:

from datetime import datetime, timedelta, timezone
from when import humanize

past = datetime.now(timezone.utc) - timedelta(minutes=5)
print(humanize(past))   # "in 0 seconds", expected "5 minutes ago"

when 4.2.0, Python 3.13.1, Ubuntu 24.04

mkell commented 2 minutes ago

/repro

pilotfish bot commented 1 minute ago
Reproduced

The snippet fails exactly as reported. It passes on 4.1.3, so this looks like a regression in 4.2.0.

when4.2.0 (from the report), also tried 4.1.3
RuntimeCPython 3.13.1, Ubuntu 24.04, x86_64, TZ=UTC
SnippetRun unchanged, 1 file, 5 lines
4.1.35 minutes ago
4.2.0in 0 seconds
repro.pythe snippet, pinned
run.log212 lines
output.diff4.1.3 vs 4.2.0
env.jsonversions, OS, locale

Run it yourself:

npx pilotfish rerun rp_7Hq2xK

Ran in 58s on a disposable VM, now deleted.

Open Fix sign handling for past datetimes in humanize()#2304
pilotfish / repro #2291 Fails on main, passes on this branch 58s
main @ 3e9c1a0 "in 0 seconds" bug present
fix/humanize-sign @ b72d04f "5 minutes ago" fixed
ci / test (3.13)2m 41s
ci / lint19s

When a fix arrives, the repro becomes the check.

Any pull request that says it fixes the issue gets the same repro as a status check. It has to fail on main and pass on the branch.

Reviewers see the fix work before they read the diff. If the branch still fails, the check says so and links the logs.

Review is moving from reading to evidence.

Maintainers get more issues and pull requests than anyone can read, and more of them are written by AI: fluent, confident, and never run. A well-written report proves nothing. A run that fails before the fix and passes after it does.

65%

of issues labeled "needs reproduction" in popular projects were closed without a fix. Nobody had the time to run them.

13/13

real, closed issues reproduced in a feasibility test across Go, Python, Rust, Django and JavaScript projects.

~1 min

from /repro to a verdict on the issue, because every run starts from a warm snapshot of the repo.

0.1¢

of compute per issue, roughly. The machine exists only for the run.

Let the next bug report prove itself.

Install the GitHub App Pick the repos, then comment /repro on an issue.