Architecting Public IT for AI, Cloud and Cyber Resilience
We worship speed: faster models, faster deployments, faster time-to-value. Yet the real limiter for public-sector IT today is not how fast you can spin up an LLM – it’s how quickly an organisation can absorb the change.
Context: a recent GovTech roundtable highlighted a trend many of us are seeing – city and county IT organisations are shifting from classic infrastructure teams toward enterprise-scale digital services, cybersecurity, and newly formed AI teams. One CIO described hiring a chief AI officer and creating a small, dedicated AI team while continuing to scale cybersecurity and operations.
Why this matters for architects and CTOs
The headline is organisational, but the consequence is architectural. Cloud and SaaS have already flattened much of the traditional stack – mainframes became virtual, on-prem services moved to providers, and with that came new expectations: deliver services faster, secure billions of logs, and build analytics and AI capabilities that are trustworthy and sustainable.
Three architectural tensions to resolve
- Centralised AI CoE vs. Distributed AI Capability
- Centralising AI into a centre of excellence buys governance, consistency, and reuse. It reduces early chaos but risks becoming a bottleneck.
- Distributing AI capabilities across product teams accelerates innovation but multiplies governance and compliance surface area.
Action for leaders: adopt a hybrid approach – a platform team that exposes secure, governed APIs, tooling and data contracts, plus embedded ML/product engineers in consuming teams.
- Speed vs. Stability (and vendor lock‑in)
- Rapid adoption of hosted LLMs and SaaS accelerates pilot-to-production cycles. But unmanaged adoption creates tight coupling to vendor APIs and proprietary data handling.
Action for architects: architect for portability – containerised inference, clear data ingress/egress policies, and abstraction layers so you can swap models/providers without reworking business logic.
- Observability and Cyber Resilience at Scale
- Security teams already drown in logs. AI can amplify detection but also introduces new attack surfaces (model poisoning, prompt injection, data exposure).
Action items: invest in end-to-end observability (traces, metrics, labelled datasets), apply Zero Trust principles to AI pipelines, and instrument models with provenance and versioning (MLOps + SIEM integration).
People, not just seats: reshaping roles
Hiring for “willingness to work alongside agentic AI” is more than a buzz phrase. Roles will change:
- Platform engineers and SREs will own reliable model serving and cost-efficiency.
- ML engineers and data engineers will own pipelines, data quality and model governance.
- Product managers will own AI outcomes, not just features.
Create learning paths that blend domain expertise with AI literacy; pair experienced domain teams with AI-savvy engineers to accelerate knowledge transfer.
Sustainability and ethics as architectural constraints
Cloud compute and large models have real energy footprints. Sustainability needs to be part of capacity planning and procurement: measure cost per inference, prefer efficient models for routine tasks, and evaluate on-prem or edge inference where latency and data sovereignty demand it. Build ethics and explainability into the acceptance criteria for production models.
A practical bridge for India (and Northeast) deployments
For governments and public institutions in India, the same dynamics apply – but with additional constraints around DPI, last-mile connectivity, and data sovereignty. Pragmatic approaches work best:
- Start with lightweight, high-impact pilots that run on efficient models.
- Prioritise data minimisation and local inference for sensitive workloads.
- Leverage academic and STPI partnerships to build skills and shared infrastructure rather than duplicating small teams across every department.
Takeaways
- Treat AI adoption as an organisational transformation, not a point-solution.
- Build a platform-first architecture: governed APIs, MLOps, observability, and portability.
- Balance central governance with distributed delivery to keep pace without losing control.
- Factor sustainability and ethics into model choice, procurement and KPIs.
- Invest in reskilling and pairing – human-AI workflows win when teams learn to work together.
Closing thought
Technology scales quickly; institutions scale slowly – the art of modern enterprise architecture is making those two cadences compatible.
About the Author: Sanjeev Sarma is the Founder Director and Chief Software Architect at Webx Technologies. With a core focus on Generative AI integration, Cloud-Native Scalability, and Enterprise Software Architecture, he has spent over two decades driving digital transformation across Northeast India and beyond. Beyond his corporate leadership, Sanjeev is deeply invested in shaping the future of the IT industry. He serves as an Industry Expert on the Board of Studies for Assam Don Bosco University’s School of Technology, advises state technology committees, and actively mentors emerging tech startups at STPI. He brings a unique, dual perspective of high-level enterprise execution and future-ready academic curriculum development.
