Architecting Inclusive On-Device AI: Open RISC-V Platforms for Mass Access
We still default to “cloud-first” thinking for AI – so much so that we forget there are billions of users for whom reliable connectivity and expensive hardware are non-starters. A recent state-led initiative that places AI-capable compute into a compact, open-source desktop is worth more than a press release: it forces us to rethink where intelligence should live and what it costs to deliver it.
The signal
I recently came across a government-backed initiative that brings an AI-ready desktop built on open RISC-V architecture, open hardware/software, and an on-device AI accelerator to classrooms and institutions. Rather than focus on the product, what matters is the principle: edge-first, open architectures aiming to democratize access to modern compute where connectivity or budgets are constrained.
What this implies for architecture and strategy
-
Edge-first AI is not merely a performance play – it’s a sovereignty and inclusion play. Moving inference (and light training) to the device reduces dependency on persistent connectivity, lowers latency, and keeps sensitive data local. For public-sector services, education and healthcare, that changes governance: data residency becomes operationally enforceable rather than an abstract policy.
-
Open hardware + open software is a strategic lever. RISC-V and open-source silicon tooling lower vendor lock-in and create an on-ramp for domestic semiconductor ecosystems. For enterprises and governments designing long-lived systems, the modularity of open stacks enables clearer upgrade paths and competitive supplier markets. But openness also demands stronger standards for interoperability, testing, and certification – otherwise we trade one form of fragmentation for another.
-
The trade-offs are real. On-device AI reduces network costs and improves availability, but it raises questions:
- Security & patch management: distributed devices mean distributed update surfaces. Secure boot, signed updates, and central monitoring must be baked into procurement.
- Model lifecycle and size: practical on-device models require quantization, distillation and pruning – skills many organisations don’t yet have in production.
- Maintenance & total cost of ownership: cheaper per-unit hardware can still be expensive when lifecycle support, teacher training, and local maintenance are considered.
-
Hybrid cloud-edge is the pragmatic architecture. Not every model or workload should run locally. The most resilient designs partition responsibilities: on-device models for latency-sensitive and privacy-sensitive tasks; cloud for heavy retraining, analytics and aggregated learning. Architects must specify clear APIs, model versioning, and telemetry to reconcile these environments.
Opportunities for research, skill-building and industry
- Invest in model compression and federated learning toolchains that are production-ready for low-power devices.
- Build certification labs for open-hardware platforms so software vendors can validate across variants.
- Prioritise curricula that teach system-level thinking: semiconductor basics, embedded AI, secure device management – not just high-level ML.
Relevance to Bharat (and Northeast India)
The initiative’s core problem – last-mile access and intermittent connectivity – is one we encounter across India, including the Northeast. Deploying affordable, locally serviceable AI-capable devices can accelerate inclusion: classrooms that can run local tutoring models, agricultural kiosks that use on-device inference for crop advice, and municipal services that operate even during network outages. But success requires pairing hardware programs with local skilling, reliable maintenance channels, and alignment with Digital Public Infrastructure (DPI) standards so systems can plug into national identity, payments and data services securely.
Practical takeaways for CTOs and policy leaders
- Design for hybrid operation: define which functionality must work offline and test it in network-starved conditions.
- Treat device fleets as software systems: invest in secure OTA, telemetry and rollback mechanisms from day one.
- Prioritise openness, but demand compliance: open architectures need formal conformance suites and security baselines.
- Invest in people: device ecosystems only scale when local technicians and educators can operate and adapt them.
Closing thought
The more we insist that intelligence only belongs in distant data-centres, the more we risk leaving vast populations outside the benefits of AI. Placing capable, open devices closer to citizens reframes inclusion as an architectural requirement – and that should be our starting point for the next decade of digital transformation.
About the Author: Sanjeev Sarma is the Founder Director and Chief Software Architect at Webx Technologies. With a core focus on Generative AI integration, Cloud-Native Scalability, and Enterprise Software Architecture, he has spent over two decades driving digital transformation across Northeast India and beyond. Beyond his corporate leadership, Sanjeev is deeply invested in shaping the future of the IT industry. He serves as an Industry Expert on the Board of Studies for Assam Don Bosco University’s School of Technology, advises state technology committees, and actively mentors emerging tech startups at STPI. He brings a unique, dual perspective of high-level enterprise execution and future-ready academic curriculum development.