The Labyrinth Problem: How RAG Can Help Public Sector Teams Get Answers From Their Own Data

Government organisations are sitting on vast stores of internal knowledge — policy documents, operational guidance, and procedural notes. The challenge isn’t having the data. It’s being able to find and trust the right answer quickly. Retrieval Augmented Generation (RAG) is changing that.

The hidden knowledge problem in government

Think about how your team finds answers to policy questions. Someone searches a shared drive. Someone else emails the person who’s been in the role longest. Another person pulls up a PDF that might be the right version — or might be two updates out of date.

This is not a technology failure. It is an information architecture problem, and it affects almost every large public sector organisation we work with. The guidance exists. The knowledge is there. But it is scattered across repositories, intranets, legacy systems and the heads of experienced colleagues who have learned the hard way where things actually live.

When AI is introduced into that environment without proper grounding, the results are predictable: responses that sound plausible but cite the wrong version of a policy, or miss a more recent update entirely. The model is not lying, but it simply does not have access to the right information at the right time.

What is Retrieval Augmented Generation (RAG)?

Retrieval Augmented Generation — RAG, for short — is a technique that gives AI models access to your own internal documents when answering a question, rather than relying purely on what the model learnt during training.

Here is a simple way to think about it. Imagine an AI assistant that, before answering your question, first searches your organisation’s document library, finds the most relevant pages, reads them, and then gives you an answer — with a footnote pointing to exactly where that answer came from. That is broadly what RAG does.

For public sector teams, this matters enormously. It means:

●      Responses are grounded in your most current internal guidance, not general training data

●      Every answer can be traced back to a source document — which supports accountability and audit requirements

●      Knowledge that previously lived only in long documents or experienced colleagues becomes accessible to everyone

What we saw at SAS Innovate on Tour

As a SAS Gold Partner, our team attends SAS Innovate on Tour each year to stay close to what is coming next across the platform. Last month, at the conference, one announcement stood out for its relevance to the public sector knowledge access problem: SAS Retrieval Agent Manager (RAM), part of a wider set of agentic AI advances SAS unveiled for its Viya platform. 

SAS RAM is a no-code RAG platform. That means teams can build and manage AI workflows that pull from internal document repositories — without needing a data engineering team to set it up. It ingests unstructured documents like PDFs, policy notes, and operational guidance and enables AI agents or chatbots to return accurate, citation-backed answers from that content.

A few things make it particularly well suited to regulated environments like UK government:

●      On-premises or hybrid deployment — your documents do not have to leave your environment

●      Citation-backed responses — every answer links to the source material it used

●      Human-in-the-loop oversight — people stay in the decision chain

●      Built-in evaluation tooling — so you can monitor and improve response quality over time

●      No vendor lock-in — it works with multiple large language models and vector databases

Why governance-conscious organisations should pay attention

We often hear from public sector teams that AI governance concerns are slowing down adoption. That is understandable. The questions are real: Is this the right version of the guidance? Who approved it? Is it still a live policy?

But here is the thing: those questions are exactly what a well-designed RAG architecture is built to answer. Version-controlled document ingestion means the model works from current material. Citation-backed outputs mean decisions are traceable. Human oversight means a caseworker or policy analyst remains in the loop.

Good AI governance – as defined in the UK Government's AI Playbook – and good RAG implementation are pointing at the same requirements. You do not have to choose between them.

Departments with the most complex policy environments — where guidance updates frequently, where inconsistent answers carry real consequences, and where experienced staff carry knowledge that has never been formally documented — often have the most to gain from getting this right.

Beyond live queries: requirements gathering and standards alignment

It is not just live caseworker queries that benefit. The same retrieval and citation mechanics are just as useful earlier in the project lifecycle — during requirements gathering.

When teams are scoping a new system, drafting a business case or writing a set of non-functional requirements, they are often working from a patchwork of organisational standards – security policy, accessibility requirements, data retention rules, interoperability standards, and department-specific technical guidance – much of it scattered across the same repositories, causing the labyrinth problem in the first place.

A RAG-backed tool can pull the relevant policy and standards directly into that process, rather than leaving it to whoever happens to know where the latest version sits. In practice, that means:

●  Automatically retrieving current standards — pulling the organisational policy that applies to a given system or service, rather than relying on a project team’s memory of what was agreed last time

●  Populating generic NFR templates with department-specific detail — security classification, accessibility level, retention period — pulled straight from current policy and citation-backed, rather than copied from an older project

●  Keeping requirements aligned as policy changes — so documentation does not quietly drift out of step with the standards it is supposed to reflect

This matters for consistency as much as for speed. Where requirements are gathered project by project, with each team interpreting the same policies slightly differently, the result is exactly the kind of inconsistency that governance teams worry about. Grounding requirements gathering in the same retrieval layer used elsewhere in the organisation means every team is working from the same version of the same standards and can show where each requirement came from.

What good implementation looks like in practice

Getting RAG working well in a government context is not just a technology question. It also requires thinking carefully about which documents go into the system and how they are maintained, how retrieval quality is evaluated on an ongoing basis (not just at launch), what happens when the system returns a low-confidence answer, and how the AI layer fits into existing workflows rather than sitting alongside them.

No-code platforms like SAS RAM lower the technical barrier significantly. But the best outcomes come when data management, governance, and technology expertise work together from the start — which is exactly the kind of cross-functional approach Butterfly Data brings to these projects.

If you are working on AI adoption in a public sector setting and want to talk through how RAG could work in your environment, get in touch. We’d love to help you find the right path through the labyrinth.

___ 

Frequently asked questions

What is Retrieval Augmented Generation (RAG)?

Retrieval Augmented Generation (RAG) is a technique that improves AI accuracy by grounding model responses in retrieved documents from a specific knowledge base, rather than relying solely on training data. The model retrieves relevant source material at query time, uses it as context, and returns a response with citations traceable to that source material.

Why is RAG particularly suited to public sector AI deployments?

Public sector organisations hold large volumes of policy documentation, operational guidance, and procedural knowledge that changes frequently and must be applied consistently. RAG ensures AI responses are grounded in the most current internal documents, supports auditability through cited sources, and can be deployed on-premises to meet government data sovereignty requirements.

What is SAS Retrieval Agent Manager?

SAS Retrieval Agent Manager (RAM) is a no-code RAG platform built on the SAS Viya platform. It enables organisations to deploy AI that retrieves and responds from unstructured enterprise documents — policies, procedures and guidance notes — with citation-backed, auditable outputs. It supports on-premises or hybrid deployment and integrates with multiple large language models and vector databases without vendor lock-in.

What was showcased at SAS Innovate on Tour?

SAS Innovate on Tour showcased advances in SAS Viya’s agentic AI capabilities, including SAS Retrieval Agent Manager as a no-code tool for enterprise RAG deployment. The event highlighted use cases in regulated industries including public sector, where knowledge access and auditability are primary concerns.

How does RAG support AI governance requirements in government?

RAG supports government AI governance by grounding responses in specific, identifiable source documents (enabling auditability), supporting version-controlled document ingestion (ensuring currency of guidance), and enabling human-in-the-loop oversight of outputs. These characteristics align with the transparency and accountability principles in the UK government’s AI governance framework.

Can RAG be deployed without a large in-house AI team?

Yes. No-code RAG platforms like SAS Retrieval Agent Manager are specifically designed to allow non-technical users to build and manage retrieval workflows without deep AI engineering resources. Implementation typically still benefits from expertise in data management, governance design, and document curation to ensure retrieval quality is maintained over time.