Autonomous artificial intelligence (AI) agents have quickly captured attention, from tools that help people schedule their day and respond to emails to systems that let organisations automate complex workflows.

But as many companies are learning, these agents need strong safeguards and solid data foundations to work well. AI systems can make serious errors. If not governed properly, an AI agent could take damaging actions, such as changing or deleting information it should not, or accessing data it is not authorised to use.

As more AI agents are deployed, organisations must pay closer attention to digital resilience, experts say. Safeguards should not be treated as add-ons to a new deployment but built into the digital infrastructure that supports it. “Every AI agent is a digital worker with tools, access and authority,” said Mr Edward Chen, chief AI officer at NCS. “Its safety depends on the foundations beneath it: trusted data, strong controls and a mature operating model. When those foundations are sound, agents multiply capability. When they are weak, agents multiply risk faster than people can react.”

In the rush to adopt AI, some organisations may give agents access or authority before they have properly governed them. “Safe AI is not a technical specification you bolt on,” Mr Chen said. “It is the result of disciplined design – knowing what the agent can access, what it can act on, when it must escalate and how every action can be traced back to human accountability.”

WHERE AI ADOPTION CAN GO WRONG

Many failures in AI adoption follow similar patterns across industries, according to NCS. Organisations may move quickly because the technology appears accessible, but the foundations needed to support it are not yet in place.

If data is fragmented, AI systems inherit those inconsistencies. If cybersecurity controls were not designed with autonomous agents in mind, AI agents can create new points of vulnerability. Operating models may also be built around deterministic systems, where the same input generally produces the same result. When AI behaves in less predictable ways, accountability for the resulting risks can become unclear.

Mr Chen said organisations often underestimate the shift from automation to AI agents. “Automation executes a workflow. An AI agent can interpret a task, choose a path and act across systems,” he said. For that reason, he added, organisations need to define their roles, access limits, supervision and accountability before deploying them.

BUILDING SAFETY THE RIGHT WAY

Mr Chen highlighted three areas organisations should strengthen before deploying AI: data foundation, identity and access, and governance.

If an organisation cannot explain where a piece of information came from, when it was last verified and who is accountable for it, it may not be ready to use AI in decisions or actions with real-world consequences.

Every AI agent should also have a distinct, auditable identity within a system, so organisations know what each agent is allowed to access and do. On governance, organisations need a way to classify risk, assign accountability for AI decisions and determine which use cases require which level of assurance.

“These foundations may seem unglamorous,” Mr Chen noted, “but they are what give organisations the confidence to deploy AI agents safely – so they can move beyond experimentation and translate AI into real-world impact.”

When NCS worked with a government agency’s contact centre to deploy an AI assistant that could transcribe calls and summarise interactions in real time, it first put key safeguards in place.

The AI system was integrated with government cybersecurity requirements, such as audit logging, which records system activity for review. Voice data was processed within strict compliance frameworks. The infrastructure was stress-tested to handle multilingual inputs, while staff could step in quickly when the AI encountered unusual cases.

The deployment led to a reduction of more than 50 per cent in after-call administrative work, allowing staff to spend more time engaging with callers.

PUTTING AI SAFETY INTO PRACTICE

AI safety is more than a slogan. It requires ongoing testing, starting with the question: What could go wrong, and who could be affected?

NCS recently worked with a client to apply Singapore’s AI Verify Foundation test to a career guidance chatbot designed to advise graduates, career switchers and retrenched workers. If the chatbot generated false information about SkillsFuture eligibility or invented a scheme that did not exist, users could be misled into making poor decisions.

To reduce that risk, NCS used its Sunshine.guardian technology, which it likens to a flight simulator for AI agents. Before the chatbot is exposed to real users, Sunshine.guardian puts it through thousands of realistic, multi-turn conversations with synthetic users modelled on the people it is intended to serve.

The outputs are then scored against ratings provided by subject-matter experts. The results are reviewed by business owners and regulators before being used to improve the next iteration of the system.

The value compounds with each use. Every new weakness Sunshine.guardian uncovers is added to a growing library of failure cases, tests and evaluators. This helps NCS test subsequent systems faster and strengthen reliability over time.

Testing AI against the right benchmarks is critical, said Dr Sunil Sivadas, principal AI architect for AI safety at NCS. A career guidance chatbot may perform well on a general knowledge benchmark, yet still give users inaccurate information on specific topics, such as SkillsFuture eligibility. “It is like grading a medical graduate on their general literacy test and concluding they can diagnose a stroke,” he said.

Dr Sivadas said the NCS framework starts from the opposite end. “We begin with the specific use case and user population, define what good looks like with subject-matter experts and build evaluators calibrated against their judgement,” he explained.

Organisations should also assess whether AI is taking the right action, using the right data and operating with the right authority, rather than focusing only on whether it produces the right answer. Testing should involve different user groups, such as older workers and women, because aggregate metrics may not reflect their experiences or needs.

For organisations, this means making safety part of their AI foundations rather than rushing into deployment, said Dr Sivadas. “Building the evaluation environment before the model goes into production is one of the cheapest investments a team can make.”