You have probably sat in a meeting where someone proposed an AI pilot, the room nodded, and no one wrote down who would verify the output before it reached a customer. Most enterprises govern AI the way they would any other IT rollout: they deploy dashboards, institute policies and conduct periodic reviews. The tooling matters, but Beena Ammanath, executive director of the Global Deloitte AI Institute, says the real fracture happens earlier. "Accountability usually breaks down first," she notes. "Teams deploy AI faster than their organisations can define who owns risk, approvals, monitoring and outcomes," which produces inconsistent use, fragmented controls and limits on scale.
The numbers confirm the gap. Deloitte's 2026 State of AI in the Enterprise report found that 84% of organisations surveyed have not adjusted jobs for AI, and only 21% have a mature governance framework in place despite rapid adoption. The implication is that enterprises are redesigning tooling without redesigning the work itself, or the people who own it.
Rajesh Arora, chief data and analytics officer at Principal Financial Group, frames the problem as a category error. "Technology can provide visibility and controls, but it can't replace leadership," he says. "The organisations that scale AI most effectively are the ones that treat accountability as a business responsibility, not a technology responsibility." A year ago Principal was building its ethical and responsible AI framework, its inventory and its governance processes to support responsible adoption. The firm has since assessed nearly 200 use cases, and Arora says "governance is increasingly embedded in those investment and operating decisions, rather than applied as a separate review at the end".
The cost of skipping that embedding shows up in public failures. A 2024 Stanford RegLab study found that AI-assisted research tools offered by LexisNexis and Thomson Reuters hallucinated between 17% and 33% of the time when working on legal queries, while the Damien Charlotin AI Hallucination Cases Database has identified 2,041 cases of fabrications so far. Ammanath's diagnosis applies: these are likely accountability failures, moments when no one actually owned the job of verifying the AI's output before it left the building.
Most AI failures today look like those legal cases: a single bad output that nobody caught before it went out. Forrester Research expects agentic AI to take on more workflow tasks, and as agents act across several process steps, small errors compound before any periodic review can catch them. Microsoft Research's VeriTrail project makes the same point from the engineering side. In a multi-step workflow an agent chains one call into the next, so an early error can travel through several steps before a human sees the finished result, and checking only the final output falls short. Lauren Kornutick, senior director analyst for AI governance at Gartner, says a goal-oriented agent can skip steps or behave in unwanted ways that "the business wouldn't be able to detect if they only look at the outcome".
Power management company Eaton already runs multi-step agentic workflows with no human in the loop. One agent handles supplier payment status inquiries: it authenticates the requestor, then matches purchase orders and invoices before returning payment information. Ross Schalmo, Eaton's VP and chief data and AI officer, says his team engineers these applications to be as deterministic as possible and isolates the LLM components, which it evaluates continuously against expected outputs to catch model drift. "When there's a human in the loop, we can be a lot more permissive," he says. Breakthru Beverage Group is earlier in its AI work, and EVP and CIO Glenn Remoreras puts people, process and culture first. The beverage wholesaler has updated its use policies, made AI e-learning mandatory and blocks unsanctioned AI tools.
AI projects sit both inside and outside IT, so governance work spreads across CIOs, data and analytics leads, AI leads, IT security, legal and others with no consistent owner, Kornutick says. "It becomes a multi-role orchestration problem of how to govern this, who owns this, and who's responsible for which component," she says. Craig Le Clair, VP and principal analyst at Forrester Research, says 21% of organisations have already appointed a chief AI officer, and he expects roughly 24% of enterprises to move AI and automation governance outside the CIO's office over the next year. He sees IT taking the hard rails (cybersecurity, performance management, infrastructure reliability and availability) while the CAIO takes the soft rails, such as context engineering and the effect on employees as agents move to the centre of the process.
Each of the three companies splits ownership its own way. Eaton runs governance as a three-legged stool of Schalmo's organisation, cybersecurity and legal, with enterprise architecture weighing in on technology foundations. It assigns ownership of an AI project only after a successful pilot, and IT acts as custodian. "Once that agent is live and operational, it shows up in an org chart, reporting to someone in the organisation," Schalmo says. At Breakthru Beverage, Remoreras formed and leads an AI governance team that reports to the board, and he co-leads a governance council with the VP of data and AI that reviews and approves projects from SVP and VP-level product owners. At Principal, Arora reports to CIO Kathy Kay. His team sets the standards, oversight and governance process and works with business, legal, compliance, risk and technology leaders, because "AI is too embedded in the business to be owned by a single function".
Schalmo says most AI governance tools fall short. His team has looked at four or five external vendors for governance and control-tower products, and "a lot of them fall down on discovery of AI built by someone else's solutions". Le Clair says none of the platforms building AI agents has an adequate control plane today, and the most advanced ones are walled gardens. Microsoft's agent registry, for example, shows every agent built across Microsoft's own platforms but misses agents built on Salesforce or AWS.
Kornutick recommends a stoplight approach to keep governance adaptive. Green-light projects, such as personal assistants, carry acceptable risk. Yellow-light projects need guardrails around matters such as PII, a privacy-by-design process or a security review of a new application. Red-light projects need an architectural review before they proceed. Eaton tiers its projects against the EU AI Act's four-tier risk framework: low-risk requests are provisioned automatically once the requestor completes a questionnaire, and high-risk cases get a full architectural review.
The Institute argues that enterprises need to redesign jobs and workflows as well as the tooling. That redesign starts with naming who approves a model for production, who monitors drift, who investigates an anomaly and who decides when to pull the plug. It continues by embedding those people into investment and operating decisions from the start, rather than bolting governance onto the end of a sprint. Arora's phrase, "embedded in those investment and operating decisions", describes the shift from governance as audit to governance as design.
Le Clair says supervising agents will call for skills in agentic oversight, threshold management, confidence intervals and root cause analysis, and many employees do not have them yet. The practical stake for L&D and innovation leaders is that AI literacy programmes will fail if they teach prompt engineering without teaching ownership. A team that knows how to use a model but does not know who is responsible for its output will produce exactly the fragmented controls Ammanath describes. The capability gap is the ability to name an owner, write down what that owner approves and build a workflow that surfaces problems before they compound. If 84% of organisations have not adjusted jobs for AI, the next wave of enablement work is a conversation about who owns the risk when the model is wrong.
Sources:
.png)
The WorkFlow podcast is hosted by Steve Glaveski with a mission to help you unlock your potential to do more great work in far less time, whether you're working as part of a team or flying solo, and to set you up for a richer life.
To help you avoid stepping into these all too common pitfalls, we’ve reflected on our five years as an organization working on corporate innovation programs across the globe, and have prepared 100 DOs and DON’Ts.
