When the computer says no
Someone applies for a loan and is declined within seconds. A job application is filtered out before a person sees it. A payment is flagged and an account frozen. In each case the applicant asks the same question: why? And in too many organisations, nobody in the building can answer.
Automated decisions are not new, but they are spreading into places where the stakes are high and the reasoning is opaque. The question that follows is practical, not philosophical: when such a system gets it wrong, who is answerable?
The temptation to blame the model
It is easy to treat an algorithm as an independent actor. The system decided. The model scored. This framing is convenient because it spreads responsibility so thinly that nobody holds it.
It does not survive scrutiny. People chose to deploy the system, chose its training data, chose the threshold at which a score becomes a refusal, and chose whether anyone reviews the outcome. Each of those is a human decision with an owner, even if the owner has not been named.
A useful rule is that an organisation cannot outsource accountability to software. If a firm uses an automated tool to make decisions about people, the firm answers for those decisions, whoever built the tool.
What the law already says
Regulators have started to codify this, and the direction is consistent even where details differ. The GDPR, in force since May 2018, gives people certain rights concerning decisions based solely on automated processing that significantly affect them, including safeguards such as the ability to obtain human intervention and to contest the decision. Nigeria's Data Protection Act 2023 also contains provisions on automated decision making. The EU AI Act, which entered into force in August 2024 with obligations phasing in over time, takes a risk based approach and places requirements such as human oversight and documentation on systems classed as high risk.
Anyone operating in these markets should read the actual texts and take legal advice on how they apply, since scope and exceptions matter. The design lesson, though, is stable: regulators expect explanation, oversight and a route to challenge.
Building accountability into the system
Accountability is not a policy document. It is a set of features and habits.
- A named owner. Every automated decision process should have a person or role responsible for its outcomes, reachable by affected people.
- Records. Log the inputs, the version of the model or rules, the result and any human intervention. Without this, an explanation after the fact is guesswork.
- Explanations people can use. Not the maths, but the main reasons in plain language, and what the person could do differently.
- Meaningful human review. A person who rubber stamps every machine output is not oversight. Reviewers need time, authority and enough information to disagree.
- A route to challenge. A clear way to contest a decision, with a human answer and a deadline.
- Monitoring. Track outcomes across groups and over time. Systems drift, and unfair patterns often show up in aggregates before anyone complains.
Take a hypothetical lender that uses an automated score to approve small business loans. If the cut off is set too high, deserving applicants are refused in large numbers. Someone chose that cut off. Someone decided whether declined applicants could ask for a second look. Naming those people in advance turns a vague worry into a manageable responsibility.
The honest limits
None of this makes automation safe by itself. Some models are hard to explain faithfully. Some decisions involve trade offs where reasonable people disagree about what is fair. Human reviewers can be biased too, and sometimes a consistent algorithm is better than inconsistent people. The goal is not to prove machines worse, but to make sure that whichever approach is used, someone can be asked why.
A question to ask before deploying
Imagine the worst case: a person harmed by a wrong decision arrives at your door. Can you tell them what happened, who is responsible, and how to put it right? If the answer is no, the system is not ready, however accurate its test results look.