The model failed twice. Don't retry blind.
failbook is an open library of real failure cases. Two failures in a row usually mean a hidden wrong assumption in the request or the workflow — not in the model. Search cases of similar questions, confirm with the user, retry with the rephrased request. Only human-error cases are admitted, so they never expire with the next model release.
failbook search "<your situation>"
Get the skill / CLI
The flow: rule of two
-
1
After 2 failures on the same task — before attempt 3 — search failbook with the current situation.
-
2
Each case shows how others phrased it, the human-side root cause, and the rephrased request — confirm with the user.
-
3
Retry with the confirmed phrasing. Hit a case that isn't in the library? Contribute it via PR.
The case library
17 human-error cases and growing via PRs
coding
Editing template structure with bare closing tags as anchors silently eats the wrong block
After an AI edits a template/component, build is green but the new UI element never renders at runtime.
Symptom The edit applied without errors; compiler/type-check/build all green. At runtime the added element is missing or lands in the wrong branch.
The question that led here “the build passes but the dialog/component I asked for doesn't show up, why is the render broken?”
Root cause (human side) Human accepted 'build green' as proof of 'structure right'. Template closers are non-unique text and must never anchor a structural edit; compiler tolerance turns structural damage into a silent pass.
For structural template edits: anchor on unique long context including a content line (never bare closing tags), and after the edit verify at runtime — open the page and confirm the element actually renders — not just the build.
- Re-do the edit with an anchor that includes unique content lines around the target position
- Search the file for orphaned/duplicated closers to confirm structure is intact
- Runtime-verify: actually open the affected view and check the element renders where expected
How to verify The element is visible in a real browser/render at the intended position — not merely a passing build.
ops
Production asset directories overwritten without a ledger check — style drift found only by a human, weeks late
An AI-generated asset batch replaced live files directly; no manifest, no comparison, drift discovered late and the whole batch was scrapped.
Symptom Live assets silently replaced by a new batch with no record of prompt source or approval. Ledger says the old version is current. The drift (overdetailed style, inconsistent palette) is invisible when reviewing single images and surfaces only when a human compares categories side by side.
The question that led here “an agent updated our image assets directly in the production folder and they look off — how do we prevent untracked overwrites?”
Root cause (human side) Human allowed direct writes to the production asset directory with no 'reconcile against the ledger' step, and reviewed replacements one-by-one instead of comparing same-category assets before/after.
Institute the gate: anything appearing in a production asset directory must reconcile with the ledger (run manifest: prompt, source, approver) — unrecorded files get archived first, never overwritten. Before replacing a category, produce a before/after contact sheet for human approval. Always keep the previous batch archived so rollback is one copy.
- On discovery of unrecorded assets: archive them as-is with a run manifest before any disposition
- Require a before/after contact sheet of the same category for approval — never single-image review
- Keep every superseded batch archived (restorable in one step)
- Restrict write access to production asset dirs to the ledger-updating pipeline
How to verify Ledger, manifest, and live directory agree one-to-one; the contact sheet for the last replacement exists and is approved; rollback of the previous batch succeeds.
automation
Helper steps that navigate away silently invalidate every check that runs afterwards
Script reports success but nothing was actually submitted; points/money not consumed, no new records appear.
Symptom Automation logs a success toast/submission, but the platform shows no consumption and no new record. All intermediate checks passed.
The question that led here “my browser automation says the task was submitted successfully but nothing happened on the site, can you make the selector more reliable?”
Root cause (human side) Human assumed intermediate helper steps may freely navigate; the page the checks run on was never asserted. Success was judged by a single weak UI signal ('input box cleared') that the platform's new UI no longer exhibits.
Restructure the flow: do all navigating helper steps first, return to the work page, assert the URL/frame before every critical step, and verify submission by a server-side effect (API response with status code, or new record added after the pre-captured baseline) — never by a UI cosmetic signal.
- Move all navigation-heavy helper steps (baseline capture) before the input/submit phase; return to the work page afterwards
- Assert current URL/page identity before input, before submit, and before each verification
- Replace cosmetic success signals (input cleared, toast) with a server-side effect: listen for the submit API response, or diff records against the pre-captured baseline
- After any platform UI change, re-run the whole chain once with a cheap sample before batch runs
How to verify One paid/consuming sample run: the server-side effect (API ret code or new baseline-diff record) must be observable, not just the script's own success log.
data
Bulk UPDATE with WHERE describing the target state matches zero rows, silently
A migration/backfill 'runs successfully' but changes nothing; zero affected rows looks like success.
Symptom UPDATE completes with no error; verification shows the column unchanged for the intended rows. Command output showed nothing alarming.
The question that led here “the update query executed without errors but when I check, none of the rows are marked — did it even run?”
Root cause (human side) Human wrote the WHERE clause from the goal ('set flag=1') instead of from the selection ('where flag is not yet 1'). Zero-row success is indistinguishable from success unless affected-row counts are checked.
Write the WHERE to describe the to-be-changed set (e.g. `WHERE COALESCE(flag,0)=0` when setting flag=1), run it, then verify with a count query: affected rows match expectation and a recount of the target state returns zero remaining.
- Restate the WHERE as 'rows still needing the change', not 'rows with the goal state'
- Run inside a transaction; check affected row count before committing
- After commit, run a count of rows still matching the old state — must be 0
How to verify Affected-rows number equals the pre-counted expectation, and the post-count of unconverted rows is zero.
data
Sequential string replacements corrupt each other: the previous step's output matches the next step's search
Batch version bumps via chained replace produce mixed versions pointing at non-existent artifacts.
Symptom After a chain of replacements (v1→v2, then v2→v3, ...), some entries that should have become the final version were rewritten twice — referencing a version that never existed; other data files that also embed versions were missed entirely.
The question that led here “I replaced the version strings in several passes and now some entries point to a version that was never released — how did that happen?”
Root cause (human side) Human applied multi-pattern edits as a chain of naive replaces without realizing the intermediate outputs join the match set of subsequent passes, and verified by spot-reading instead of structurally parsing the result.
Apply all substitutions in ONE pass with a single regex alternation (or a function that maps each match exactly once), then verify by parsing the result as data (JSON parse + compare every version field against the expected value) — not by eyeballing a few lines. Keep in mind which files embed the same value; update and check all of them.
- Replace the chain with one regex alternation pass (match-once semantics)
- Structurally verify: parse the file and assert every field equals the expected value
- Enumerate every file carrying the value (data copies exist in more than one place) and repeat there
- Assert no reference points to an artifact version that was never published
How to verify Structural check over the whole corpus: every version field equals the target, zero occurrences of intermediate versions, all referenced artifacts exist.
debugging
'The click does nothing' is two different silent failures — capture the network to tell them apart
An automation click on a web app shows zero reaction; selector fixes never help because the click wasn't the problem.
Symptom Click via automation: no toast, no navigation, no visible request. Switching to a stealth browser driver makes the click 'work' — but still no result appears.
The question that led here “clicking the button via automation shows absolutely no reaction — no toast, no navigation, nothing. Is my selector wrong?”
Root cause (human side) Human judged the failure from UI appearance only. Without capturing the network layer, 'request never sent' (bot detection) and 'request sent but flow silently stopped' (business gate) are indistinguishable — and they have completely different fixes.
Capture network activity around the click and compare: if the expected request appears but the flow stops afterward, it is a business gate — check the balance/eligibility API. If no request appears at all, it is the automation layer being detected — switch to a stealth driver setup. Do not touch selectors based on this symptom alone.
- Record requests before/after the click; diff for the expected POST
- Request present + downstream stops → call the business/eligibility API directly and act on its answer
- No request → change the browser-control layer (stealth profile), not the page interaction
- Verify success by the business result (record created), never by absence of error
How to verify The expected request is observed AND the business result (e.g. new generation record) appears; the diagnosis names the exact blocking layer.
coding
Copied 'working' parsing code fails silently because the input shape was never tested
Login/callback or URL handling copied from another project passes silently — the feature is dead with zero errors.
Symptom The external side works (redirect arrives), but the app behaves as if the callback never happened. No exception, no log line.
The question that led here “I reused the same auth callback code from our other app but the login never completes and there are no errors, what's different?”
Root cause (human side) Human treated 'reference implementation' as 'verified for my input shape'. Parsing rules for custom schemes were assumed identical to http URLs, and the failure branch had no logging, making the dead path invisible.
Before reusing parsing code, write one test with the exact input shape it will see in this project (e.g. `new URL('myapp://auth/callback?code=x')` — inspect host and pathname). Add a log line to every early-return branch so dead paths are observable.
- Print/inspect how the URL actually parses (host vs pathname) for the custom scheme
- Normalize before comparing: reconstruct the path as host + pathname, then assert
- Log every silent return in callback handling with the raw input
- Keep the shape test as a permanent regression test
How to verify The shape test passes and an end-to-end login round-trip lands the user in a logged-in state; killing the handler deliberately shows up in logs.
deploy
Deploy script dies on a non-critical step before the actual deploy runs
Release script aborts mid-way on an auxiliary step (backup push); production never got the update and nobody noticed.
Symptom Script exits with an error after the build; the deploy command never executed. Output looks like a normal failure of the auxiliary step.
The question that led here “our publish script sometimes just stops halfway and the site isn't updated, how should we handle the failing git step?”
Root cause (human side) Human ordered the pipeline as build → backup → deploy and let set -e treat every step as fatal. The deploy is the only step that must run; backup is optional by nature.
Reorder the release script so the deploy command runs first (or is guaranteed regardless of auxiliary failures); make the backup push non-fatal with `|| warn`, and track a pending-backup note when it fails.
- Put the deploy command at the top of the critical path
- Demote auxiliary steps (git backup push) to best-effort: `git push ... || echo WARN backup pending`
- Log a follow-up item when a backup is skipped so it can be pushed later manually
How to verify Simulate the auxiliary failure (disconnect network for git) and confirm the site still deploys and the warning is logged.
deploy
Deploy succeeded but the site still shows old content — data changes were never built
Editing data files and running the publish script ships the old build; the script's success message hides it.
Symptom Publish output says success; production serves previous content. Data edits exist locally but never reached the deployed output.
The question that led here “we updated the data files and published successfully, why does the live site still show the old content?”
Root cause (human side) Human assumed publish implies build. The pipeline had a gap: content generation was a manual, unlisted step, and nothing verified the live result after upload.
Treat 'publish' as build + upload + verify: always run the build step before the publish script, and after deploying, fetch the live URL and check the new content is actually present (following redirects).
- Chain the commands: build step && publish script (never publish alone after data edits)
- After deploy, curl the live page (follow redirects) and grep for the new content
- Optionally move the build invocation into the publish script so the gap closes permanently
How to verify The live-site fetch shows the new content string; a deliberate no-build run is caught by the verify step.
testing
E2E suite reads the real user config and fights for machine-global resources
End-to-end tests pass locally for one person, fail for everyone else: language-dependent selectors, hotkeys taken, timing off.
Symptom A large share of e2e tests fail on machines other than the author's: role names in the wrong language, hotkey registration conflicts, clicks swallowed by visible overlays or focus changes.
The question that led here “half of the e2e tests fail with selectors not found, but they passed yesterday on my machine — is it flaky?”
Root cause (human side) Human ran e2e against the real user environment instead of an isolated one: no temp config, no reserved test-only key range, no handling of physically changing desktop focus.
Make e2e isolated by default: launch the app with a temp config (fixed language, defaults), choose test-only key codes outside both the production default and OS-reserved ranges, and neutralize decorative overlays (pointer-events none). Explicit config paths passed by a test must be respected, not overwritten.
- Default to an isolated temp config: fixed language and known settings
- Pick test hotkeys from a range verified free on target machines; never reuse production defaults
- Disable/neutralize decorative overlays that can swallow pointer events
- Re-run the full suite on a second machine to confirm machine-independence
How to verify Full e2e suite passes identically on two different machines/configs, including a non-default language.
automation
Interactive CLI inside an agent's non-interactive pipe hangs forever, invisibly
Running an interactive program (REPL/chat CLI) from an automation session floods logs with prompts and never exits; piping through tail hides all feedback.
Symptom The tool loads, then spews an interactive prompt thousands of times (stdin is an empty pipe), never exits. Output piped to `tail` shows nothing until EOF, so the session looks frozen. Timeout auto-backgrounding orphans the process, which keeps holding GPU/memory.
The question that led here “I asked the agent to run this model CLI to test something and now the session is stuck and the process won't die — what happened?”
Root cause (human side) Human ran an interactive program in a non-interactive harness and assumed flags would force non-interactive mode. Long output was piped through a buffering viewer, removing all live feedback; there was no log file and no plan for terminating a stuck heavyweight process.
For automated runs, prefer tools that terminate naturally (benchmark/batch modes). If the interactive CLI is unavoidable: close stdin explicitly (`</dev/null`), first probe whether the non-interactive flags are accepted with a tiny run, redirect all output to a log file (no pipes), and keep a kill plan (record the PID) for heavyweight processes.
- Replace interactive verification with a naturally-terminating alternative if one exists
- If not: `interactive-cli ... </dev/null > run.log 2>&1` and a minimal first run to validate flags
- Record the PID and define the kill command before starting heavyweight runs
- Read progress by grepping the log file, never by piping live output
How to verify The run terminates by itself within the expected time, the log file shows the real result, and no orphan process remains.
debugging
Wrapping a command with '&& echo OK || echo FAIL' makes every exit code zero
A failed build/task is reported as success (exit code 0) because the last command in the chain is an echo.
Symptom Automation/notification reports success (exit code 0) while the actual work failed; failure is discovered late, after downstream steps already trusted the code.
The question that led here “the task notification says exit code 0 but the build actually failed, why is the status wrong?”
Root cause (human side) Human wrapped the command for nicer logging and accidentally destroyed the exit code channel. Exit code of a chain is the last command's, not the interesting command's.
Preserve the real exit code: redirect output to a log file and echo the captured status separately, e.g. `cmd > log 2>&1; echo exit=$?`, and gate downstream steps on that captured value.
- Replace `cmd && echo OK || echo FAIL` with `cmd > log 2>&1; echo exit=$?` (or use set -e without masking wrappers)
- For background jobs, judge success by produced artifacts and the tail of the log file — never by a wrapped notification's exit code alone
How to verify Intentionally break the command; the captured/propagated exit code must be non-zero and downstream must not run.
automation
Paid automation fails halfway because balance was never checked and failure was judged by the wrong signal
Automation clicks 'generate/submit' on a credit-based service; the click appears dead, and the team concludes the whole channel is blocked.
Symptom Clicking submit produces no toast, no navigation, no visible request. Later, the team declares the channel fully blocked and falls back to a worse alternative — while candidates from an earlier 'failed' attempt were actually being generated successfully in the background.
The question that led here “the submit button click does nothing on this credit-based site — is the site blocking my automation?”
Root cause (human side) Human skipped a cheap pre-check (balance) and had no final arbiter for success (the service's history/result records), relying on volatile UI signals (URL changes, input clearing) that both false-negative. Conclusions about 'the whole channel' were drawn from a single unprechecked attempt plus outdated memory.
Before any credit-consuming submission: query the balance endpoint and refuse early if below threshold. Capture a baseline of the results/history before submitting; after submitting, judge success ONLY by new records appearing after the baseline — click/UI signals are hints, not verdicts. If the channel truly blocks, report the blocking point and required unlock action; never silently degrade the deliverable.
- Add a balance pre-check against the service's API before every paid run
- Capture a results-baseline before submit; poll the baseline diff as the sole success arbiter
- On apparent failure, compare network captures: request sent + flow stops = business gate; no request at all = control/fingerprint layer
- Re-read the current balance/state before declaring a channel dead — never reuse hours-old memory
How to verify A paid sample run completes with a new record after baseline and the expected balance decrement; the false-'blocked' scenario is disproven by finding the earlier results.
debugging
Packaged app exits instantly with code 0 — check for leftover dev processes first
A packaged desktop app quits silently on launch while the dev build works fine; no log, no error.
Symptom The packaged binary launches and exits immediately (exit 0, no output). Dev mode works. Process lists appear clean at a glance.
The question that led here “the packaged app starts and immediately exits with no error, but running from source works — what's wrong with the packaging?”
Root cause (human side) Human trusted a narrow process-name search and jumped to mechanism-level suspects before ruling out environment leftovers. Debugging order was wrong: cheap environment checks come before mechanism theories.
Before touching packaging config: list all related processes broadly (any dev/runtime binary of this stack, not just the app name), kill leftovers, and re-test the packaged build. Only then investigate the lock mechanism.
- Search processes broadly (e.g. the framework's runtime binary name) and kill orphaned dev instances
- Re-launch the packaged app and confirm it stays up
- If still failing, check error output in full — truncated views (e.g. piping through tail) have hidden real errors before
How to verify Packaged app stays running with no related orphan processes alive; reproducibility confirmed by launching twice.
deploy
Piping build output through 'tail' eats the exit code and hides the failure
A build that never produced artifacts reports success because `| tail` was the last command in the pipeline.
Symptom Build task 'completes' with a green status; the expected artifact is missing or stale. Sometimes the real error (e.g. missing npm script) is nowhere in the captured output.
The question that led here “the background build said it finished fine but there is no output artifact, where did it go wrong?”
Root cause (human side) Human piped long build output for readability, trading away both the exit code and live feedback. Documentation references were also stale (script names changed), so the wrong command was being 'fixed'.
Run builds with output redirected to a log file and report the captured exit code: `npm run build > build.log 2>&1; echo exit=$?`. Check the artifact and the log tail afterwards. Verify command names against the project's package.json, not memory or docs.
- Replace `... | tail -N` with `... > build.log 2>&1; echo exit=$?`
- After the run, check both the artifact exists and grep the log tail for errors
- Confirm the build command/script name from the repo (package.json scripts) before running
How to verify Break the build on purpose once: the redirected run must report a non-zero exit and the log must contain the real error.
automation
Hardcoded screen coordinates fail for self-repositioning, focus-hiding windows
UI automation clicks empty space round after round because the window moves on every show and hides when it loses focus.
Symptom Screenshot shows the button; the click lands on nothing or another app. Between the screenshot and the click, the window repositioned (it spawns near the cursor) or hid itself (it auto-hides on blur — e.g. another automation session stole focus). Mis-clicks have even opened unrelated dialogs.
The question that led here “my click automation worked in testing but keeps clicking the wrong place on this app — the coordinates are right, why?”
Root cause (human side) Human assumed window position and visibility are constant between observation and action. For repositioning/self-hiding windows, position must be queried and visibility asserted inside the same script that clicks.
Make each interaction atomic: in one script, query the window rect at action time, compute the control position relative to that rect, assert visible+foreground immediately before clicking. When multiple automation sessions share the machine, compress the whole observe-decide-act cycle into the single script instead of round-tripping through an agent. For web-view UIs where accessibility enumeration finds nothing, locate the control by scanning pixels for its known color signature as a fallback.
- GetWindowRect at action time; derive control offset relative to the rect
- Assert IsWindowVisible + GetForegroundWindow right before the click; bail out otherwise
- Single-script atomic cycle for focus-stealing environments
- Pixel-signature fallback when UIA/enumeration returns nothing
- Leave true mouse-drag/multi-click human-verification to manual acceptance, automation only asserts command/screenshot level
How to verify N consecutive automated rounds all land on the control while a second session steals focus in between; zero mis-clicks into other apps.
ops
Validation tooling crashes on legitimate placeholder values instead of reporting them
A daily audit script throws an exception because a config contains a documented 'TODO' placeholder — the whole check pipeline goes down.
Symptom The check command dies with an exception (e.g. invalid URL) on the first incomplete entry; everything after it is unaudited. The placeholder was produced by the project's own official template, so it is a legitimate state.
The question that led here “our audit script crashes with a URL parse error ever since someone added a new entry with a placeholder, how do we fix it?”
Root cause (human side) Human wrote validators that treat 'incomplete' as 'invalid input' — an unhandled exception path outside the try scope — instead of a first-class 'pending' outcome.
Every probe/validator must map unusable input (placeholder, empty, malformed) to a pending/skipped result with a reason, never throw. After adding any new field to the templates, immediately run the full check once to prove the chain stays self-consistent — don't wait until fields are filled.
- Wrap each probe so placeholder/empty/malformed values yield a pending result carrying the reason
- Keep exceptions inside the caught set (including argument-construction errors)
- Add a regression test: a record full of TODO placeholders must produce pending, not a crash
- Run the whole check right after template/schema changes
How to verify The check completes over a dataset containing placeholders, reporting them as pending with reasons, exit code 0.
Contribute a case
One case per PR. Root cause must be human-side (survives model upgrades), real and reproducible, desensitized. Run failbook validate before submitting. Admission rules in docs/CONVENTIONS.md.