Skip to content
Data Seekers
Back to Insights
DatosTecnología

When AI makes a mistake and nobody notices

Curro Pavón ·
When AI gets it wrong on poor-quality data

When a system fails in an obvious way, you see it. The dashboard breaks, the number does not add up, someone calls. But there is another kind of error that is far harder to detect: the one that happens silently, looking completely normal. When AI gets it wrong, it gets it wrong with great confidence. The system keeps running, keeps producing outputs, keeps giving answers. And you keep making decisions based on those answers.

This is the problem nobody wants to discuss when talking about artificial intelligence applied to business.

For years the conversation about AI has revolved almost always around models: which is more powerful, which reasons better, which scales further. It is a legitimate conversation, but an incomplete one. Because the model decides nothing on its own. What decides is the combination of the model and the data that feeds it. And that second part gets far less attention.

The data hoarding syndrome

In retail and e-commerce this is especially clear. Companies have spent years accumulating data with the same logic some people use to accumulate objects at home: just in case. I call it data hoarding syndrome. You have a lot of data, but no capacity to consume it and no idea what each piece is for. And then comes the promise of AI, which is supposedly going to tidy up that chaos. But AI does not tidy up chaos: it amplifies it. A powerful model on low-quality data does not produce better decisions; it produces wrong decisions faster and with more conviction.

The question we should ask before any investment in AI is not “which model do we use?” but “which data are we going to use it on?”. And the realistic answer, in most of the companies we work with, is that the data is not in good shape. Not because nobody cared, but because keeping data reliable at scale is much harder than it looks. Capturing a hundred products from a marketplace at one point in time is trivial. Capturing a million products, three times a day, for a year, with a minimal error rate, is a completely different game. And that difference is systematically underestimated.

Maintenance is part of the product

There is another conversation that is also avoided: maintenance. There is a fairly widespread idea that maintenance is what you do when something breaks: a patch, a fix, support. I see it differently. Maintenance is part of the product, not support. Data systems live in a constantly changing environment: websites change, formats change, platforms update their structures. A system not designed to anticipate those changes is not reliable: it is a system that works until it stops working. And when it fails, the customer notices before you do.

Designing with that mindset changes how you build. You do not look for the perfect solution for today; you look for one that can be rebuilt correctly when the environment changes. Being adaptable is not having an answer for everything today. It is the ability to build the right answer tomorrow.

If the models disappeared tomorrow

In a recent conversation I was asked a question I found very well put: if all AI models disappeared tomorrow, which capabilities would still be essential? The answer is the same as always: knowing what to measure, building systems to measure it reliably, and making decisions based on that measurement. What changes with AI is the speed at which bad data can spread through your entire decision chain.

The challenge is not really technological. It is one of judgement. Knowing which data you need, how often, at what level of quality, and for which specific decisions. Data that does not lead to an action is a waste of time. And that clarity, at a time when everyone is rushing to implement agents and automations, is scarcer and more valuable than any model.

I am writing this after a conversation on the Ecommerce News podcast where we tried to talk about all of this with the honesty the subject deserves. If you want to go deeper, you can listen here (in Spanish). But the takeaway is simpler: before asking what your AI does, ask which data it is working on.

If you want to know how we capture and maintain that data at scale, we explain it in how we work.

Want to see how this applies to your business?

Request my analysis