High-Risk AI & Algorithmic Accountability

When software decides something about a person — approves them, flags them, scores their risk — it must be fair, explainable, and overseen. Under the EU AI Act, an automated decision that controls access to financial services is high-risk by definition. Build these systems so a human can understand them, challenge them, and answer for them.

Algorithmic accountability means every automated decision can be explained, evidenced, contested, and overridden by a human. It also means the system was built, tested, and monitored to be fair and reliable. The EU AI Act sets clear obligations for high-risk AI: risk management, data governance, transparency, human oversight, accuracy, robustness, and record-keeping. These are engineering requirements, not legal footnotes.

Our risk-scoring and AML decision-making are clearly in scope. The Finperiti audit found that the agentic risk-scoring component was missing from shipping code, with AI Act obligations unevidenced. This is a warning: when these systems do ship, the controls and the evidence must ship with them. A system that decides someone's access to financial services but cannot be explained is a risk we cannot carry.

Keep a human in charge

• AlwaysGive real human oversight to any decision that has a serious effect on a person. A human can review it, override it, and is accountable for the outcome.
• DoMake decisions explainable. Record the inputs, the factors, and the reasoning behind each automated outcome, in terms a human can understand and a regulator can audit.
• DoGive affected people a way to contest a decision. Treat overrides as important: record them, link them to the person who made them, and use them to improve the system.
• DoFail closed. On missing, errored, timed-out, or unrecognised model output, block and escalate to a human. Never auto-approve.
• ConsiderConfidence thresholds that send low-confidence or edge cases to human review instead of letting the model decide alone.
• NeverShip an automated decision that controls access to financial services without the required human-oversight and explainability controls in place.
• NeverTreat an unknown or unmapped input (country, document type, risk band) as low or medium risk. Unknown means escalate.

Govern the model & its data

• DoControl training and input data for quality, fair representation, and bias. A model is only as fair as the data behind it.
• DoTest for accuracy, robustness, and unfair impact before deployment, and document the results as evidence.
• DoMonitor models in production for drift, degradation, and unexpected outcomes, with alerts and a way to roll back.
• DoVersion models, data, and decision logic. Keep records that let you rebuild why a past decision was made the way it was.
• ConsiderA documented risk assessment and an intended-purpose statement for each high-risk model, kept up to date as it changes.
• NeverDisable, bypass, or weaken a screening/monitoring or AI-decision control without authorisation and a permanent audit record.

Self-review checklist

• AskIf this system decided something about a person, could a human explain why, both to that person and to a regulator?
• AskCan a human review the decision, override it, and be accountable for the outcome?
• AskWhat happens on uncertain, missing, or unrecognised model output? Does it fail closed to human review?
• AskHave we tested and documented this model for accuracy and bias, and do we monitor it in production?