Introduction

Artificial Intelligence has shifted from an emerging capability to a core foundation of organisations. Across both private sectors and the public sector,  generative models and agentic workflows are starting to shape how policy or objectives are implemented and how services are delivered.

However, as government and enterprise bodies accelerate their AI adoption, senior leaders face a critical dilemma: 

How do we leverage AI capabilities without surrendering digital control, exposing data, or locking services into proprietary, black-box vendor ecosystems?

We should not think about sovereign AI as each country independently, spending billions on training foundational LLMs from scratch or building national semiconductor labs. While these create a certain amount of cross-border independence it is a false binary. Immediate concerns for sovereign AI should be about architectural control, data governance and strategic autonomy, enabled by building service on top of existing cloud platforms such as AWS Bedrock, Azure AI foundry and Google Cloud Vertex AI.

Why this rationale? For 99% of organisations  building foundation models from scratch is out of the question, for budget, skills and maintainability reasons. While building these models creates true sovereignty, the real threat to government departments isn’t the lack of a custom built base model, it is the uncontrolled architectural dependency the current foundation SaaS model provides.

In the traditional SaaS model, organisations connecting critical workflows directly to SaaS AI foundation providers expose themselves to several key risks: 

• Citizen data and telemetry risk escaping sovereign jurisdictional boundaries.
• Operational logic becomes vulnerable to silent model updates (architectural drift) or sudden API deprecation.
• Vendor lock-in concentrates technical debt into a small handful of commercial providers.

For some of the risks there are straight forward mitigations, for example, model pinning can help with architectural drift, however, most risks do not have a viable or sustainable mitigation above signing a contract. This penalises the provider if any of the risks materialise, but by that point the damage is already done and this is only a compensatory approach.

A pragmatic approach to sovereignty in organisations therefore is achieved by deploying open-weight models (or closed weight models with agreements), within dedicated, legally compliant and likely already established cloud environments. AWS Bedrock, Azure Foundry and GCP Vertex AI are all, with the appropriate configuration, protected by zero-data-retention agreements, localised encryption key management and model gateway functionality, all of which form a solid basis for the deployment and management of models within existing cloud infrastructure. Given the premise of existing infrastructure this means the organisation will likely therefore have the base policies, controls and skillsets to start managing these models securely. 

Foundations of Sovereign AI

Organisational readiness of sovereign AI can be evaluated via 3 key pillars

Data Sovereignty

All data within the organisation must remain under the organisation’s established governance model:

• Local Data Residency – Enforcement of strict boundaries for data, to comply with data protection regulation and security protocols
• Customer Managed Keys – Ensuring data in transit and at rest is encrypted with keys managed directly by the customer.
• Local Telemetry – Application of the same data residency rules and boundaries for prompts and context as traditional data. 

Algorithmic Control

SaaS provided frontier models default to vendor defined safety guardrails and cultural aspects. While acceptable for general use, most organisations require a stricter set of guardrails to align with local and company driven mandates.

• Open Weight Models: Utilising open-weight models (such as Meta’s Llama series, Mistral, or Google’s Gemma) hosted on managed services like AWS Bedrock or Azure AI Foundry allows departments to retain full operational control over model behavior.
• Localised Guardrails: Implementing middleware safety layers (e.g., NeMo Guardrails, Azure AI Content Safety) that programmatically enforce government policy, prevent hallucinated regulatory guidance, and block prompt injection.
• Parameter-Efficient Adaptation (PEFT): Using techniques like PEFT (Parameter-Efficient Fine-Tuning) and LoRA to adapt models to domain-specific language without full model retraining.

Infrastructure Control

Large proportions of organisations already host their infrastructure within sovereign investments made by hyperscalers within national borders. For AI we utilise these existing sovereign clouds to build and host the appropriate abstractions to enable application independence from specific model API’s.

• Model-Agnostic Gateways: Decoupling government applications from specific provider APIs using an internal API proxy.
• Confidential Computing: Utilising Trusted Execution Environments (TEEs) on GCP, Azure, or AWS to encrypt model weights and context in memory during inference execution.
• Switching Resilience: Ensuring that if an enterprise cloud provider changes pricing, alters service terms, or suffers regional degradation, traffic can be seamlessly rerouted to another provider or an on-premise fallback.

Unlocking Sovereign AI 

As a trusted delivery partner we always aim to bridge the gap between policy, requirements and hands-on development. The process of developing a Sovereign AI solution, follows the same, discovery, build, refine mantra, however as outlined above there are a couple of key architectural concerns that are required to be in place to unlock sovereign AI. Establishing the following will be key to the success of Sovereign AI initiatives:

Tiered data governance architecture

Leaning on existing data governance frameworks, sovereign AI requires the appropriate architecture pattern mapped to the established security tiers in the organisation, an example might be:

Model-Agnostic Proxy layer

Arguably the core technical deliverable this enables a single point of control for LLM interactions. A standardised interface across departments, providing at the minimum:

• A single unified API, across all model providers.
• Management of access keys for teams and individuals
• Cost tracking and control for departments, teams and individuals in an organisation
• Unified audit trails, providing complete observability of prompt lineage, session traces, tool calls and outputs. Enabling compliance and efficient response for FOI requests, for example.

Grounded RAG and Custom Guardrails/Evaluation frameworks 

Rather than attempting to train bespoke LLMs, organisations should at first implement Retrieval-Augmented Generation (RAG).

• Data Stays in Place: Sensitive departmental documents remain inside the government body’s secure vector database (e.g., AWS OpenSearch, Azure AI Search, or GCP Vector Search).
• Verifiable Citation: Models generate responses strictly grounded in internal documentation, providing direct citations for caseworkers and policy analysts.
• Deterministic Output Control: Custom guardrail middleware verifies that outputs align with statutory policies before displaying results to end-users.
• Evaluation Frameworks: Evaluation frameworks run on prompt and model changes, ensuring outputs are fully aligned to core policies and guardrails.

An Implementation Roadmap

Successfully implementing sovereign AI requires a scalable roadmap to manage dependencies and ensure that all requirements and policies are considered from discovery to scaling. 

• Phase 1 – Discovery and Risk (Weeks 1 – 6): 

• • • Audit and discover shadow AI initiatives and usage across the organisation
    • Establish the data security tiers and categorise pipelines and data at rest according to the security tiers
    • Define sovereign requirements and ensure the correct data residency practices

• Phase 2 – Establish a secure foundation (Months 2 -4):

• Create and deploy a model agnostic proxy

• Provision model hosting platforms within the selected cloud provider and region aligned with the security and data residency requirements, ensuring customer managed keys and zero data retention policies are in place.

• Deploy the initial models on the hosting platform

• • • Implement observability and cost controls

• Phase 3 – Knowledge Alignment (Months 4-9):

• Implement secure storage to support RAG (Document, Graph, Traditional) and knowledge extraction

• Enable secure data federation across the organization with the appropriate access control, lineage and discovery functionality

• • • Optionally create adapted (PEFT/LoRA) versions of the models and deploy these
    • Build deterministic policy guardrails and automated audit pipelines

• Phase 4 – Scaling and Federation (Months 9+):

• • Scale successful architectures across departments
  • Enable secure data federation
  • Automated CI/CD for model deployment, guardrail updates and evaluations

Conclusion

Sovereign AI is not about isolating government or organisational tech stacks from global innovation. It is about building architectural abstractions that allow organisations to harness the best tools available while maintaining absolute authority over data, system integrity, and expenditure. The key is recognising what is already established and filling in the gaps with robust architectural patterns that abstract details away that would cause vendor locking, while creating absolutely traceability and control. The techniques outlined in this document are the foundation of this. 

If you’re leading AI adoption in a government, healthcare or private sector organisation and want to ensure your strategy ensures you maintain authority over your data and infrastructure, we’d love to help. You can reach out to me on LinkedIn to discuss how this framework can be applied to your specific departmental needs.