AI ethics in project management: Who is responsible when the algorithm gets it wrong?

Consider the scenario: an Artificial Intelligence (AI) tool flags a supplier as high risk, and the project drops them. Later, it turns out the model was wrong. So, who was responsible for that call? The challenge we have is that in most projects right now, nobody has a good answer.

AI in project management no longer just means a chatbot that drafts your risk register. It ranges from simple rule-based automation right through to generative tools, large language models and increasingly "agentic" systems that can act with a fair degree of independence. That range matters because greater autonomy makes it harder to distinguish human judgement from machine output.

As AI gets embedded deeper into delivery decisions, from prioritising tasks to assessing risk to shaping resourcing allocations, it's quietly reshaping who, or what, is actually in charge.

A lot of the unease around AI in delivery comes down to one thing; we can see what goes in and what comes out, but not what happens in between. That's the "black box" problem, and it's not a new observation, but it's becoming a live governance issue rather than an academic one.

When a decision-making system operates without a visible internal process, several recognised issues arise. Trust is eroded. If people can't see how a decision was made, they're less likely to believe it was made fairly. We must acknowledge that bias hides in plain sight, machine learning models are trained on data, and data carries the fingerprints of past decisions, including the flawed ones. Bias baked into training data doesn't announce itself; it just quietly reproduces itself.

Finally, accountability becomes blurred. For example, if humans are removed entirely from the loop, the result is not simply faster decision-making. It creates a genuine ethical grey zone in which nobody quite owns the outcome. It is this last issue that is really the crux of the accountability question. Some researchers describe this as an "invisible cage": a system opaque enough that the people working within it can't meaningfully challenge or understand it, even when it's shaping their working lives.

It's worth pausing on what we even mean by "governance," because the word gets used loosely. There's a useful distinction between the structural side of governance, the rules, controls and processes, and the normative side, meaning transparency, accountability and integrity. Most organisations are reasonably good at the first. Far fewer have properly grappled with the second. That distinction matters for AI; you can have a perfectly compliant process for deploying an algorithmic tool and still end up with a decision nobody can ethically defend. Ticking the governance box on structure without addressing how things ought to be leads to "the algorithm did it" as an actual excuse in a lessons-learned session.

So, whose fault is it? Accountability doesn’t disappear when decisions are automated; it becomes easier to obscure. If a project manager accepts an AI-generated recommendation without question, that's still their decision. If a sponsor greenlights a tool without asking how it reaches its conclusions, that's a governance failure, not an AI issue. And if an organisation deploys a system it doesn't fully understand, we need to question was the model was wrong or is there a lack of effective governance? Rather than treating AI as a replacement for human judgement, AI is something that has the potential to enhance it. If AI is amplifying human judgement rather than substituting for it, then the human in the loop doesn't get to step back from the decision. They remain the accountable party, whether the recommendation came from a spreadsheet, a colleague, or an algorithm.

There's also a pace problem. AI capability is moving fast and governance frameworks built for a slower, more predictable world simply weren't designed to keep up. Rigid, static rulebooks age quickly when the technology they're governing changes month to month. What's needed instead is governance that can flex without losing its backbone, iterative, responsive to change, but still anchored to fixed principles of transparency and accountability. Agile in its posture, not just its label.

Next time an AI tool has a hand in a delivery decision on your project, it's worth asking a genuinely awkward question: if this goes wrong, who actually owns it?

If nobody in the room can answer that clearly, that's not an AI problem. It's a governance gap, and it existed before the algorithm ever got involved.

As project professionals, we don't get to outsource ethics to our tools. The algorithm might get it wrong. Whether we're responsible for catching that, and answering for it, is entirely up to us.