Every scraper eventually breaks because a site redesigns. AI can genuinely repair some of those breaks automatically — and pretending it fixes all of them is how you end up with silently wrong data.
Not because they were written badly. Because the site changed, and it was always going to.
A scraper is a contract with a page structure that the other party never agreed to and can rewrite whenever they like. A class name goes from product-price to price-display. A value that was in the HTML moves into a JavaScript payload. A single-column layout becomes a grid. None of that is malicious and none of it is unusual — it is just what shipping a website looks like from the other side.
The question is not whether your scraper breaks. It is how long it takes you to find out, and how much wrong data you acted on in the meantime.
Loud failures are fine. The selector matches nothing, the parser throws, the run exits non-zero, an alert fires. You know within minutes.
Silent failures are the expensive ones. The selector still matches something — just the wrong thing. Now your price column contains the shipping cost. The run succeeds. The row count looks normal. The alert never fires, and you make decisions on that data for three weeks before somebody notices the margins look strange.
Everything below is really about catching the second kind.
Give a model the old page structure, the field definitions, and the new page HTML, and ask it to locate the same fields again. This works well, and it works because it is a recognition problem rather than a rule-following one — the price still looks like a price, it has just moved.
A repair loop I have had good results with:
1. The run fails validation — nulls spike, or a value falls outside its expected range
2. The system snapshots the page that failed
3. A model is given that snapshot, the field list, and examples of previously-correct values
4. It proposes new selectors
5. The proposal is tested against a stored set of pages with known-correct answers
6. If it passes, the selectors are updated and the run retries; a human is notified either way
Step five is the entire load-bearing part. Without it you have replaced a broken scraper with a confidently wrong one.
Be clear-eyed about the limits.
That last one is worth sitting with. It is the failure mode that no amount of automation addresses, and it is the reason I do not sell "set and forget".
The repair loop only ever triggers because validation caught something. So the validation is the real system:
A pipeline with these checks and no AI at all is far more trustworthy than one with automatic repair and no checks.
Honest version: most systems I build do not repair themselves. They detect and they shout.
The reason is that on a well-scoped pipeline, breakages are rare enough that a same-morning alert plus a fix from me is faster and safer than an automated repair I have to verify anyway. Automated repair earns its place when you are running against many sites at once and manual response does not scale.
What every delivered system does have: validation on every run, alerting the moment a check fails, a stored snapshot of the page that broke so diagnosis does not start from scratch, and fixes from the person who wrote it.
Sites change and scrapers break. Anyone who tells you otherwise is selling something. The measure of a good build is not that it never breaks — it is that you find out from the system rather than from a bad decision three weeks later.
If you have a scraper that keeps failing quietly, or you are about to build something you need to trust for months, that is the conversation.
Contact me at sam@autosmartcode.com for a free quote within 24 hours.