Human-in-the-Loop by Design: Governing AI in Capital Markets
In the tech ecosystem, we often glorify autonomous systems and frictionless velocity. Yet, when you move from consumer apps to the foundational plumbing of a nation’s economy, “move fast and break things” ceases to be a strategy-it becomes a systemic liability. In high-stakes environments, friction isn’t a bug; when designed intentionally, it is a vital architectural feature.
Recently, SEBI Chairman Tuhin Kanta Pandey outlined a forward-looking regulatory roadmap for India’s capital markets, announcing imminent guidelines for responsible AI anchored by strict accountability and a ‘human-in-the-loop’ mandate. Alongside deploying its own supervisory AI tools, the regulator is already inventorying cryptographic systems for quantum computing threats projected for 2032–35, while simultaneously re-evaluating the Innovators Growth Platform (IGP) to help long-gestation deep-tech startups access public markets earlier.
Architecting for Probabilistic Systems and Post-Quantum Realities
From an enterprise architecture standpoint, SEBI’s announcements highlight two massive shifts that CTOs and system architects across all regulated sectors-not just fintech-must internalize today.
First is the structural tension between probabilistic AI and deterministic accountability. Modern machine learning and Generative AI models operate on probabilities, whereas financial markets and regulatory compliance demand deterministic audit trails. Mandating a “human-in-the-loop” (HITL) is easy to write into a policy document, but remarkably complex to engineer at scale. If an algorithmic system processes thousands of micro-signals per second, inserting human oversight without creating catastrophic latency requires sophisticated system design. Architects must now build tiered confidence thresholds, explainable AI (XAI) telemetry, and automated circuit breakers that seamlessly route edge-case anomalies to human operators. Governance can no longer be an afterthought; it must be baked into the API layer.
Second, SEBI’s proactive inventory of cryptographic vulnerabilities nearly a decade ahead of the 2032–35 quantum horizon is a masterclass in managing long-term technical debt. In enterprise software, encryption is too often hardcoded deep within legacy dependencies. Transitioning to Post-Quantum Cryptography (PQC) won’t be a simple patch; it will require “crypto-agility”-architecting modular security layers today so encryption algorithms can be swapped out tomorrow without dismantling core infrastructure. Seeing an Indian regulator champion this timeline should serve as a wake-up call for every enterprise CIO still postponing legacy modernization.
Patient Capital for Bharat’s Deep-Tech Ambitions
Equally encouraging is the regulator’s candid acknowledgment that the current listing framework hasn’t gained enough traction for deep-tech and biotech ventures. While mentoring emerging startups at STPI and engaging with young innovators across Northeast India, I have frequently observed a fundamental mismatch between traditional venture metrics and true R&D.
Unlike wrapper-based SaaS apps that scale rapidly on cloud credits, genuine deep-tech, biotech, and hardware-led innovations require years of lab-to-market gestation before generating predictable revenue. By working to make public markets more accessible to these high-risk, high-gestation companies-paired with transparent risk disclosures for investors-India is laying the financial rails for indigenous IP creation. However, young founders must also read the room: as regulators deploy advanced supervisory tech of their own, startups building for Bharat’s Digital Public Infrastructure must treat regulatory compliance and cyber resilience as day-one engineering requirements, not post-Series A checkboxes.
Strategic Imperatives for Tech Leaders
- Engineer Explainability, Not Just Accuracy: Move away from black-box deployments. Build observability pipelines that log why an AI model made a decision to satisfy future regulatory audits.
- Adopt Crypto-Agility Now: Begin decoupling cryptographic protocols from core business logic. Conduct an enterprise-wide cryptographic inventory before quantum readiness becomes an emergency compliance mandate.
- Align R&D with Governance: For deep-tech founders, institutional capital and public market pathways are opening up, provided your data governance and incident-response frameworks match enterprise standards.
Ultimately, sustainable digital transformation isn’t driven by how fast we deploy new algorithms, business models, or compute power-it is defined by how resiliently we engineer trust into the systems that govern them.
About the Author: Sanjeev Sarma is the Founder Director and Chief Software Architect at Webx Technologies. With a core focus on Generative AI integration, Cloud-Native Scalability, and Enterprise Software Architecture, he has spent over two decades driving digital transformation across Northeast India and beyond. Beyond his corporate leadership, Sanjeev is deeply invested in shaping the future of the IT industry. He serves as an Industry Expert on the Board of Studies for Assam Don Bosco University’s School of Technology, advises state technology committees, and actively mentors emerging tech startups at STPI. He brings a unique, dual perspective of high-level enterprise execution and future-ready academic curriculum development.