qwri

Automation6 min read

How a R1.50 error survived two years of automated reporting

The number was wrong every single day for over two years, in a report that twenty people opened every morning. Nobody questioned it, because it always arrived and it always looked right.

The director did not normally go this deep. That was the unusual part.

His group had been running the same daily sales report for years. It landed every morning, it went to about twenty people, and it was the number the business used. Then a second system started producing the same figure and got a different answer, and rather than assuming the new system was wrong, he sat down with his IT team and asked why the two did not agree.

It took a deep dive to find, and the answer was a carrier bag.

A R1.50 levy, ringing up as its own stock item on every transaction where a customer took one, was included in one system's daily sales total and excluded from the other's. Once someone found it, the two figures tied to the cent, every day, going back as far as anyone cared to check.

Which was the problem. Going back as far as anyone cared to check meant more than two years. For over two years, twenty people had opened a report every morning that was wrong, in the same direction, by a small and variable amount. Nobody raised it. Not once.

The report was not the failure. The silence was.

It is tempting to file this under sloppy configuration and move on. That misses what actually happened, which is that a known-wrong number sat in front of twenty numerate people for two years and none of them questioned it.

They did not question it because there was nothing to question it against.

A figure someone assembles by hand carries an implicit asterisk. Everyone knows who built it, everyone knows roughly how, and when it looks strange somebody walks over and asks. That asterisk is not a weakness of manual reporting. It is the only quality control most manual reporting has.

Automate the same figure and the asterisk disappears. It arrives at 07:00 from nobody in particular, formatted identically to yesterday, and it reads as fact. The report gained reliability and lost its only reviewer in the same change, and the second half of that trade is invisible until something forces the question.

This is worth being precise about, because the usual lesson drawn from a story like this is "check your data before you automate", and that lesson would not have helped here. Somebody almost certainly did check it, once, at the beginning. A single check at setup catches a bug that is present at setup. It does nothing about the two years afterwards.

What actually caught it

Not a process. Not an audit. A disagreement.

The error surfaced the moment a second system computed the same number from a different starting point and produced a different answer. That is the entire mechanism, and it is worth stating plainly because it is cheap to reproduce and almost nobody does it: a number you cannot compare to anything is a number nobody is checking.

For what it is worth, the second system in this case was ours, which is how I came to be in the room. I would rather that detail were less flattering, because the useful part of the story is not which software found it. It is that the finding required two independent answers and a person willing to take the disagreement seriously. The director could have concluded the new system was wrong. Most people do. The whole thing turned on him not doing that.

  • A report that arrives on time and looks official stops being reviewed. It gains reliability and silently loses its only reader who was checking.
  • Checking your data once at setup catches setup bugs and nothing else.
  • Errors surface when two independently derived answers disagree, not when someone re-reads a report.
  • If both of your numbers come from the same summary table, they will agree with each other and both be wrong.

Six reasons two systems disagree, all of them legitimate

Before assuming a difference is a fault, know that most differences are not. There are at least six defensible reasons two systems report different totals for the same day, and each is correct behaviour in the right context.

Difference Usually because
Levies and surcharges Bag levies, recycling fees and deposits ring as stock items in one system and as adjustments in another
VAT treatment One total is inclusive, the other exclusive
Returns and voids Netted off the day of the return, or the day of the original sale
Trading day boundary A store trading past midnight, or a till closing after the report runs
Store scope A branch that stopped trading but stayed in one system's grouping
Tender vs sale Cash-up totals include float movements that were never a sale

Reconciling does not mean the two numbers match. It means you can explain why they do not. The goal is to get every difference named, not to get every difference to zero, and a business that can name its variances is in far better shape than one whose systems happen to agree by luck.

The work is unglamorous and it is finite. Pick a busy day, pull the same figure from both systems, find the difference to the cent, and keep going until you can name it. Then repeat on a day with returns and on a day with a public holiday. Three days usually surfaces everything structural.

Three things worth building into the report itself

Once the numbers tie, the reconciliation should not live in one person's head or in a document nobody opens.

Recompute the headline figure by a second route. If the report totals a day from transaction lines, have something independently total the same day from the daily summary and compare. When they disagree the report should say so on its face rather than quietly print the first answer. This is the single highest-value check available and almost nobody builds it, because it feels like doing the work twice. It is doing the work twice. That is the point.

The catch, and it is the important one: the second route has to be genuinely independent. Two calculations reading the same summary table will agree with each other and both be wrong, which is worse than one calculation, because now you have corroboration.

Show the scope on the face of it. Not in a footnote. In the header. Which stores, which dates, VAT inclusive or exclusive, whether returns are netted. Most arguments about a report are actually arguments about its scope, and they end in about nine seconds when the scope is printed at the top.

Stamp the freshness of every input. A report comparing actuals to budget is only as current as the budget behind it, and budget files go stale silently. If the target was last updated in March, the report should say March on it.

Where this does not apply

Two honest limits.

The first is that a second opinion costs something. Building an independent recomputation for every figure in every report is not a good use of anyone's time. Do it for the numbers people make decisions on, which in most businesses is a shortlist of about five, and leave the rest alone.

The second is that most disagreements are boring. You will spend a morning chasing a R12 gap and find a rounding rule. That is a successful outcome, not a wasted morning, but it does not feel like one at the time, and a team that expects every variance to be a two-year scandal will quietly stop looking.

The order that works

Reconcile, name every difference, agree the definition, then automate, then keep an independent check running inside the report forever.

That last word is the one that matters. The failure in this story was not setup. It was the twenty-four months afterwards, when a number arrived on time every single morning and nobody had any reason to doubt it.