Sovereign AI and the Inference Layer: Hosting Models, Owning Strategy
The AI Sovereignty Debate Is Asking the Wrong Question
For the past year, the loudest argument in Indian technology has been about which country builds the best AI model. I think that’s the wrong argument. The question that actually determines who controls AI is much quieter – where does the inference happen, and who governs what the model does once it’s already running inside your workflow?
Two stories this week illustrate the point sharply.
Two stories, one principle. Sarvam AI’s newly launched Token Factory is now serving Chinese open-weight models – DeepSeek, Moonshot AI, Z.ai – on Indian infrastructure, alongside its own models and Google’s Gemma. Critics argue that a company backed by public resources should prioritise original Indian research. Meanwhile, Apollo Hospitals is running a Microsoft-built agent that reviews patient falls, medication errors and readmissions – assembling timelines, flagging gaps, prompting clinicians for missing details, and suggesting assessments for a doctor to check and amend.
Notice the pattern: in both cases, the value – and the politics – have migrated from the training layer down to the inference and orchestration layer.
Hosting is not the same as developing. But nor is hosting trivial. The model is the engine; the inference plane is the road, the toll booth, and the traffic rules. Whoever operates inference sees the prompts, the latencies, the usage patterns and the unit economics. That is the real locus of sovereignty – not the weights, but the serving stack, the data residency, the audit log, and the ability to switch models without rewriting the business. A national AI strategy that only counts the training runs is measuring the wrong KPI. A defensible position looks like this: the data never leaves the jurisdiction, the serving stack is owned locally, the logs are yours, and the switching cost between models approaches zero.
The more architecturally interesting story is Apollo’s. The agent does not decide. It compresses the review cycle – and a clinician amends. The chain of accountability stays intact. And note the careful language in the report: the system “coincided with” fewer recorded incidents, not “caused” them. That epistemic humility is exactly right, and it’s rarer than it should be.
Enterprise architects should treat this as a template. Agents are not features. They are new first-class citizens in the system – they need identity, scoped permissions, immutable audit trails, explainability at the point of decision, and always a human escalation path. The trade-off is speed of review versus accountability. Get it right, and you buy throughput without buying risk. Get it wrong, and you have simply automated a mistake at scale. Which is why I keep pushing for one unglamorous discipline: a thin abstraction layer between your agents and your model providers. We spent a decade learning the cost of locking workflows into a single database or cloud. We are about to repeat that mistake with foundation models unless we design for portability now.
Why this matters closer to home. For those of us building in the Northeast, the implication is concrete. If inference is where control lives, then regional inference capacity – low-latency, jurisdiction-local, and genuinely language-aware for Assamese, Bodo, Khasi and Manipuri – becomes a piece of digital public infrastructure, not a commercial luxury. The DPI we built for payments taught us that interoperable, shared infrastructure lowers costs for everyone and invites innovation at the edges. The inference layer deserves the same treatment.
And to the young founders I mentor: your moat is not the model. Everyone has the same models. Your moat is the workflow you understand better than anyone else in the room – plus the governance wrapper that makes a conservative enterprise trust you with it.
What I’d carry into your next architecture review:
- Sovereignty lives at the inference plane, not the training run.
- Agents need identity, permissions, audit and a human escalation path before they get a login.
- Model portability is cheap to design now and brutally expensive to retrofit later.
- “Coincided with” is not “caused by.” Ship the AI; keep the epistemics honest.
- Build shared inference capacity the way we built DPI – as a commons.
The models will keep changing hands, and the headlines will keep chasing them. The workflows, the audit logs and the trust are what endure. Build the layer that lasts.
About the Author: Sanjeev Sarma is the Founder Director and Chief Software Architect at Webx Technologies. With a core focus on Generative AI integration, Cloud-Native Scalability, and Enterprise Software Architecture, he has spent over two decades driving digital transformation across Northeast India and beyond. Beyond his corporate leadership, Sanjeev is deeply invested in shaping the future of the IT industry. He serves as an Industry Expert on the Board of Studies for Assam Don Bosco University’s School of Technology, advises state technology committees, and actively mentors emerging tech startups at STPI. He brings a unique, dual perspective of high-level enterprise execution and future-ready academic curriculum development.