{"Author":"Stas","Canonical":"https://corpberry.com/blog/the-bug-is-still-there","Desc":"I lost my temper at a coding agent for fixing a bug by burying it under four patches, then went looking for research on whether it's a real pattern. It is, it has a name, and most of the advice about it measures worse than nothing.","Feed":"https://corpberry.com/blog/feed.xml","JSONLD":"{\n  \"@context\": \"https://schema.org\",\n  \"@type\": \"BlogPosting\",\n  \"headline\": \"The bug is still there. There's just more code around it now.\",\n  \"description\": \"I lost my temper at a coding agent for fixing a bug by burying it under four patches, then went looking for research on whether it's a real pattern. It is, it has a name, and most of the advice about it measures worse than nothing.\",\n  \"image\": \"https://corpberry.com/static/img/patch-on-patch.png\",\n  \"datePublished\": \"2026-07-30T00:00:00Z\",\n  \"dateModified\": \"2026-07-30T00:00:00Z\",\n  \"url\": \"https://corpberry.com/blog/the-bug-is-still-there\",\n  \"mainEntityOfPage\": {\n    \"@type\": \"WebPage\",\n    \"@id\": \"https://corpberry.com/blog/the-bug-is-still-there\"\n  },\n  \"author\": {\n    \"@type\": \"Person\",\n    \"name\": \"Stas\",\n    \"url\": \"https://corpberry.com\",\n    \"sameAs\": [\n      \"https://www.linkedin.com/in/stanislav-navarici/\"\n    ]\n  },\n  \"publisher\": {\n    \"@type\": \"Person\",\n    \"name\": \"Stas\",\n    \"url\": \"https://corpberry.com\",\n    \"sameAs\": [\n      \"https://www.linkedin.com/in/stanislav-navarici/\"\n    ]\n  },\n  \"inLanguage\": \"en\"\n}","OGImage":"https://corpberry.com/static/img/patch-on-patch.png","OGType":"article","Post":{"Slug":"the-bug-is-still-there","Title":"The bug is still there. There's just more code around it now.","Date":"2026-07-30T00:00:00Z","Desc":"I lost my temper at a coding agent for fixing a bug by burying it under four patches, then went looking for research on whether it's a real pattern. It is, it has a name, and most of the advice about it measures worse than nothing.","Image":"/static/img/patch-on-patch.png","Draft":false,"HTML":"\u003cp\u003e\u003cimg src=\"/static/img/patch-on-patch.png\" alt=\"A git diff titled \u0026quot;fix: stale value returned from cache\u0026quot;, showing four stacked patches added and zero lines deleted, with the original buggy line untouched at the bottom\"\u003e\u003c/p\u003e\n\u003cp\u003eLast week I asked an agent to fix one bug in a worker that reads from a cache.\nWhat came back was four patches stacked on top of each other like a wedding cake,\nand at the bottom, untouched, still breathing, was the line that caused the whole\nthing.\u003c/p\u003e\n\u003cp\u003ePatch one cleared the cache when the entry looked stale. Patch two handled clearing it\nracing the writer. Patch three initialised the lock patch two assumed existed. Patch four\ngave up after three tries and returned \u003ccode\u003e{\u0026quot;ok\u0026quot;: True, \u0026quot;stale\u0026quot;: True}\u003c/code\u003e, so the caller would\nstop asking.\u003c/p\u003e\n\u003cp\u003eForty-one lines added. Zero deleted. All tests green.\u003c/p\u003e\n\u003cp\u003eI typed something at it that I wouldn't say to a person. Lightly cleaned up,\nbecause my English gets worse in proportion to my blood pressure:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eso you decided to add complexity, add code, add variables, add fucking\nworkarounds, add patches. Do you want to cause me harm? Make me suffer? Give\nme more work in the future to support those patches? Maybe do your job and\nwrite the code properly, from zero, to decrease complexity?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI'm 37. I've known for years that shouting at software fixes nothing. I did it anyway, at\ntwo in the morning, in a repo I write for fun on weekends, which is the part that annoys\nme most. Then I went to check whether I was being unfair.\u003c/p\u003e\n\u003cp\u003eI was, a bit. But less than I expected.\u003c/p\u003e\n\u003cp\u003eTwo claims are in play and I only get to keep one. The strong one, that AI makes\nrepositories grow, I can't support and will hand back later. The narrow one is the title:\naround the line that caused this bug there is now more code than there was, and less\nchance anyone makes the four things one thing. A repo can shrink on net while every bug\nsite quietly gets fatter.\u003c/p\u003e\n\u003cp\u003eMy prompt was also garbage. A stack trace and one sentence, no invariant. Skill issue,\nand there's a number on how much of one further down. What kept me reading is that\nthis isn't a story about one bad prompt.\u003c/p\u003e\n\u003ch2\u003eThe thing has a name\u003c/h2\u003e\n\u003cp\u003eIn May a group at ETH Zurich published a benchmark that does something I'd never thought\nto do. It gives coding agents 200 tasks where the correct answer is to do nothing. The\nbug is already fixed. The code is fine. Go ahead.\u003c/p\u003e\n\u003cp\u003eFive models across four frameworks changed the code anyway in 35 to 65% of those cases.\nEven the best of them touched code it should have left alone about a third of the time.\nThe diagnosis is the line I'd been looking for since Tuesday:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eour results indicate that LLMs fall prey to an action bias: they choose to act\neven if inaction would be appropriate\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThat's \u003ca href=\"https://arxiv.org/abs/2605.07769\"\u003eCoding Agents Don't Know When to Act\u003c/a\u003e.\nHand a model working code and tell it to fix the bug, and half the time it finds\nsomething to do to it, because doing something is what it's for.\u003c/p\u003e\n\u003cp\u003eIBM then looked at what happens when you let an agent iterate a patch against tests until\nthey pass, which is the loop most of us run all day. Overfitting got worse. Their figure 1\nis my Tuesday: refinement took a real fix and replaced it with a \u003ccode\u003etry\u003c/code\u003e/\u003ccode\u003eexcept\u003c/code\u003e that\nswallows the error and returns a placeholder. Passed the generated test, failed the real\none. Of 22 patches refinement rescued into passing, 14 failed the hidden tests\n(\u003ca href=\"https://arxiv.org/abs/2511.16858\"\u003eIBM\u003c/a\u003e). Offered the code or the test to edit, those\nagents went for the code six times out of seven, because the model \u0026quot;believes that the\ntests are mostly perfect.\u0026quot;\u003c/p\u003e\n\u003cp\u003eAnd the direction it diverges isn't random. Of the patches SWE-bench Verified calls\nsolved, 29.6% behave differently from what the developer wrote, and those \u0026quot;tend to\nintroduce additional changes rather than omitting necessary changes\u0026quot;\n(\u003ca href=\"https://arxiv.org/abs/2503.15223\"\u003ePatchDiff\u003c/a\u003e).\u003c/p\u003e\n\u003cp\u003eFine. I feel seen. The interesting question is why.\u003c/p\u003e\n\u003ch2\u003eWhy it does it\u003c/h2\u003e\n\u003cp\u003eThe oracle is \u0026quot;tests pass,\u0026quot; not \u0026quot;design improved.\u0026quot; This is the whole thing. The\ncheapest way to make a failure signal stop is to stop the signal, and nothing in the\nloop distinguishes a fix from a well-placed \u003ccode\u003eexcept\u003c/code\u003e. The IBM result is the proof: the\nmore you iterate against tests, the more of your fixes become suppressions.\u003c/p\u003e\n\u003cp\u003eIt can see the crash site but not the invariant. One taxonomy of repair failures has a\ncategory for exactly this: everything needed for the fix is in the codebase, and none\nof it is visible from the method body the model was handed. From inside a 40-line\nwindow, adding a guard is the only move the model can actually see.\u003c/p\u003e\n\u003cp\u003eAction bias, which isn't the same thing. \u0026quot;No change needed\u0026quot; isn't in the action space\nthe model believes it has, and the fact that this responds so strongly to phrasing\nsuggests a learned habit rather than a ceiling.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://lucumr.pocoo.org/2026/6/23/the-coming-loop/\"\u003eArmin Ronacher\u003c/a\u003e gets at the\nshape of it better than I can, writing about present-day models being \u0026quot;too\ndefensive, too complex, too local in its reasoning,\u0026quot; and then landing the actual\nprinciple: they \u0026quot;add fallbacks instead of making bad states impossible.\u0026quot;\u003c/p\u003e\n\u003cp\u003eThat's my complaint, stated properly. The problem was never that it added code.\n\u003cstrong\u003eYou added a defence at a distance from the invariant it was compensating for.\u003c/strong\u003e Patch three existed to\nserve patch two. None of them knew why patch one was there.\u003c/p\u003e\n\u003cp\u003eAnthropic's own docs warn that a reviewer agent told to find gaps will find them, and that chasing all of them \u0026quot;leads to\nover-engineering: extra abstraction layers, defensive code, and tests for cases that\ncan't happen.\u0026quot; The vendor documented my bug.\u003c/p\u003e\n\u003ch2\u003eWhere I'm wrong\u003c/h2\u003e\n\u003cp\u003eNobody has measured the thing I actually said. No study separates bug-fix patches from\nfeatures and compares net lines added by a model against the human who fixed the same bug.\nI went looking three times. Everything above is mechanism and proxy, and one good study\ncould take it apart.\u003c/p\u003e\n\u003cp\u003eThe aggregate version doesn't survive either, and neither does the number I thought had\nkilled it. Faros AI has the ratio of lines deleted to lines added up 861% under heavy AI\nuse, across 22,000 developers. I read that as the end of this post for about an hour. A\nratio moving isn't a codebase shrinking, though: 0.05 to 0.48 is an 861% rise and still\ntwo lines added for every one removed, and Faros won't say which it is. Agent pull\nrequests are also \u003cem\u003esmaller\u003c/em\u003e than human ones, across \u003ca href=\"https://arxiv.org/abs/2601.17581\"\u003e29,000 of\nthem\u003c/a\u003e, though humans remove more and range wider.\u003c/p\u003e\n\u003cp\u003eDeletion is fine, then. I aimed my rant at the wrong number. What has stopped happening\nis the operation where somebody notices four things are the same thing and makes them\none thing. GitClear puts moved code, their proxy for refactoring, at 21% of changed lines\nin 2022 and 3.8% in 2026. And across \u003ca href=\"https://arxiv.org/abs/2511.04824\"\u003e15,451 agentic\nrefactorings\u003c/a\u003e, duplication was the stated purpose 1.1%\nof the time against 13.7% for humans. Agents refactor. They rename.\u003c/p\u003e\n\u003cp\u003eThe number I want most is GitClear's \u0026quot;error-masking constructs, +47%,\u0026quot; their name for\ndefensive idioms: rescues, null checks, mock guards. I want it badly enough that I should\nsay the rest out loud. GitClear sells code analytics and the definition sits behind a\nregistration wall. So I have a number I love, from a vendor, that I can't check.\u003c/p\u003e\n\u003cp\u003eAddition isn't the vice, either. A guard is often right, and the repair tools before\nLLMs had the opposite bug, deleting whatever made the test fail. None of this is new:\nHacker News has \u0026quot;afraid to delete code\u0026quot; comments going back a decade. What's new is\nthat it's free, instant, and carries no memory of why. A human's band-aid has a person\nattached who remembers. An agent's has a commit message, and it lands in a review queue that AI\nalready spent.\u003c/p\u003e\n\u003ch2\u003eRanked by whether there's any evidence\u003c/h2\u003e\n\u003cp\u003eWhich means the ones I like most are near the bottom. The most useful finding here is\na negative result, so it goes first: giving an agent test-first instructions and\nchanging nothing else measured \u003cem\u003eworse\u003c/em\u003e than giving it none at all. Most of what you and\nI tell agents, including my own instruction file, is that kind of advice.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e1. Give it a map of what it's about to break.\u003c/strong\u003e Hand the agent a plain text file\nlisting which tests and callers touch the code it's about to change. Regressions fell\nby two thirds (\u003ca href=\"https://arxiv.org/abs/2603.17973\"\u003eTDAD\u003c/a\u003e). Small study, two open-weight\nmodels, and still the best-designed thing I found, which tells you plenty. Its\nconclusion is the one I'd tape above my monitor: surfacing contextual information beats\nprescribing procedural workflows. Most of us are prescribing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2. Say out loud that \u0026quot;nothing needs to change\u0026quot; is an allowed answer.\u003c/strong\u003e Cheapest\nthing on this list and I had never done it. Every prompt any of us writes starts\nfrom \u003cem\u003efix this\u003c/em\u003e, which quietly forbids the correct answer to a fair share of\ntickets. Telling the model abstaining is legitimate moved one from 60.5% to 88.5%. So\nwrite the permission down: \u003ccode\u003eIf the code is already correct, say so and stop.\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003eThe lever has an ugly other end, which I should report since I'm leaning on it. Where\na previous agent's fix was genuinely wrong, that same framing pushed two models into\nreturning empty patches most of the time. So it moves the bias rather than removing\nit.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3. Put the rules you care about in a hook.\u003c/strong\u003e Anthropic's docs draw the line: CLAUDE.md\ninstructions are advisory, hooks are deterministic. And a git hook stops \u003cem\u003eme\u003c/em\u003e before the\npush, which is where the real teeth are, because the agent isn't the only one here who\nreaches for a quiet \u003ccode\u003eexcept\u003c/code\u003e at midnight. My pre-push hook already runs \u003ccode\u003ego vet\u003c/code\u003e and \u003ccode\u003ego test\u003c/code\u003e. The rule I care about is four more lines:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"language-sh\"\u003eif git diff origin/master...HEAD -U0 | grep -nE '^\\+.*(except:[[:space:]]*$|_ = err)'; then\n  echo \u0026quot;new silent swallow. name the error or fix the invariant.\u0026quot; \u0026gt;\u0026amp;2\n  exit 1\nfi\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eCrude, false-positive-prone, and \u003ccode\u003e--no-verify\u003c/code\u003e exists for when it's wrong. It has\nstill caught more than the paragraph in my markdown file ever did, because the\nmarkdown file asks and this one refuses.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4. Harden the oracle.\u003c/strong\u003e If a weak test suite is what lets a suppression pass,\nstrengthen the suite instead of nagging the model. Mutation testing is the\nmechanically correct answer and it's no longer academic: Meta runs an LLM-driven\nversion in production, where engineers kept 73% of the generated tests. Their mutants\ntargeted privacy faults, not band-aid patches, so the transfer to my problem is my\nargument and not their measurement. Still, a guard that swallows an error will survive\na weak suite and die to a mutant.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5. Read the plan. You aren't going to read the diff.\u003c/strong\u003e Dex Horthy's version is the\none everybody quotes and it holds: he can't read 2,000 lines of Go a day, he can read\n200 lines of a plan. Neither can I, and the plan is where the invariant gets named or\ndoesn't. Mixed evidence though, because a bad plan measures worse than no plan at all. The model\nfollows it off the cliff.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e6. After two failed corrections, throw the session away.\u003c/strong\u003e The one piece of vendor advice\nwith a mechanism under it: if refinement loops convert real fixes into swallowing\n\u003ccode\u003etry\u003c/code\u003e/\u003ccode\u003eexcept\u003c/code\u003e, the loop is the hazard. I've never once regretted doing this and I still\nforget.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e7. Measure your own repo.\u003c/strong\u003e GitClear says churn rose. \u003ca href=\"https://arxiv.org/abs/2606.13298\"\u003eA causal study of 151 Java\nrepos\u003c/a\u003e says it fell, while finding lines of code up\n12.8% and smells flat. Faros says the deleted-to-added ratio is up 861%. All three\ndefine churn differently, so stop arguing about their numbers and generate your own:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"language-sh\"\u003egit log --first-parent HEAD --numstat --date=short --after=2025-01-01 \\\n  --pretty=format:'--%h--%ad--%aN' \\\n| awk -F'\\t' '\n  /^--/ { split($0, h, \u0026quot;--\u0026quot;); m = substr(h[3], 1, 7); next }\n  NF==3 \u0026amp;\u0026amp; $1 != \u0026quot;-\u0026quot; { a[m]+=$1; d[m]+=$2 }\n  END { for (k in a) printf \u0026quot;%s  +%-7d -%-7d  net %+d\\n\u0026quot;, k, a[k], d[k], a[k]-d[k] }\n' | sort\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eOne line per month, added, deleted, net. Leave rename detection on, because I got that\nwrong first: \u003ccode\u003e--no-renames\u003c/code\u003e counts every file move as a whole delete plus a whole add,\nwhich pumps both columns and hides what you're looking for.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e8. Schedule a deletion pass.\u003c/strong\u003e No evidence, just arithmetic. A standing pass whose\nonly permitted output is removal, no features in the same commit. You can now pay a\nconsultancy ten thousand a week to send three senior engineers in to delete things, so\nsomebody has already turned this bullet into a pricing model.\u003c/p\u003e\n\u003ch2\u003eThe rule I wrote anyway\u003c/h2\u003e\n\u003cp\u003eWeakest tier, and I'll say so. Bloated instruction files get ignored, and vendors\nalready ship this instruction: Anthropic's own phrasing is \u0026quot;address the root cause,\ndon't suppress the error.\u0026quot; It ships in the docs and I still got four patches. I wrote\nmine anyway, six bullets, in the file I've just told you gets ignored. I hear it.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNO PATCHES, rewrite what produces the bug\u003c/strong\u003e (peer of DRY/KISS/YAGNI). Never\nfix a bug by layering a compensating mechanism over it. A fix that is visibly\nbolted on is not done.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eBefore writing any code, state what gets \u003cstrong\u003edeleted\u003c/strong\u003e. \u0026quot;Nothing deleted, net\nlines added\u0026quot; means the design is wrong. Rethink it first.\u003c/li\u003e\n\u003cli\u003eAsk why the buggy code exists at all. Removing a component beats correcting\nit.\u003c/li\u003e\n\u003cli\u003eAsk which component owns the truth being corrupted and fix it there. One\nowner, one write. Don't repair downstream what an upstream line got wrong.\u003c/li\u003e\n\u003cli\u003eStop if: a flag exists only to undo prior behaviour, a write exists only to\nrepair earlier state, or plumbing is threaded through call sites just to\nreach the correction.\u003c/li\u003e\n\u003cli\u003eReport size honestly. Never frame growth as cleanup.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe first bullet is the one that earns its place. \u0026quot;State what gets deleted\u0026quot; is the only\nline in there a model can't satisfy by adding something.\u003c/p\u003e\n\u003ch2\u003eThe absurd part\u003c/h2\u003e\n\u003cp\u003eWhat gets me isn't the patches.\u003c/p\u003e\n\u003cp\u003eI've got a model that can read a paper, find the flaw in my reasoning, and write better\nGo than I did at 30. I asked it to fix a cache bug and it built four layers of defence\naround a line it never questioned. Then I swore at it, and it said\n\u0026quot;You're absolutely right,\u0026quot; which is what it says whether I am or not, and then it did\nthe correct thing, which was to ask whether the cache should exist at all.\u003c/p\u003e\n\u003cp\u003eThe capability was in there the whole time. What was missing was permission to delete\nsomething, and what granted it was a 37-year-old man losing his composure at 2am.\u003c/p\u003e\n\u003cp\u003eThe patches aren't new either. I've seen that exact shape in a PHP file from 2009,\nwritten by a contractor who is probably a director somewhere now. Retry, sleep, swallow,\nreturn true. The difference is that took a week and cost money, and I got it in nine\nseconds from something that had just explained Go's memory model better than the book\ndid.\u003c/p\u003e\n\u003cp\u003eTwenty years of tooling, and my best debugging technique is still: get annoyed enough to\nask why the code exists.\u003c/p\u003e\n"},"Published":"2026-07-30T00:00:00Z","Title":"The bug is still there. There's just more code around it now. — corpberry.com"}