AWS Machine Learningllm1h agoenergy 130

How Indegene is modernizing and scaling enterprise data quality in life sciences

image: AWS Machine Learning

This post is co-written with Shravan K S from Indegene Limited. Enterprise AI is creating new opportunities for life sciences organizations to improve decision-making across commercial, medical, market access, and other business functions. But the quality of every AI model, dashboard, and business insight depends on one thing: trusted data. As enterprise data grows across multiple platforms, pipelines, and business domains, maintaining consistent data quality has become increasingly difficult. Manual data quality rule creation, fragmented governance, and evolving business definitions make it challenging to scale data quality across the enterprise. This post explores how Indegene built a production-grade, large language model (LLM) powered Data Quality Agent within its NEXT Unified Data Platform (NEXT UDP) on AWS. The agent generates context-aware data quality rules, validates them, and produces ready-to-run SQL and Python code under centralized, human-in-the-loop governance. Together, these capabilities help organizations improve data quality at enterprise scale while reducing the effort required to build and maintain data quality programs. About Indegene Indegene Limited (BSE: 544172, NSE: INDGN) is the strategic operating partner to the life sciences industry, orchestrating commercial, medical, R&D, and safety operations across the product lifecycle. Trusted by the world’s top 20 biopharma companies, Indegene helps run complex, highly regulated workflows at global scale. Indegene combines deep life sciences expertise with AI-embedded platforms and governed operating models to improve speed, consistency, and compliance across mission-critical operations. By integrating technology into how work is executed and scaled, Indegene helps life sciences organizations deliver measurable outcomes in regulated environments. Why conventional data quality programs break down Most organizations have established data quality processes but keeping them effective is becoming increasingly difficult as enterprise data ecosystems evolve. One of the biggest challenges is the divide between technical and business rule ownership. Data engineers typically define technical rules that confirm whether data is structurally correct. For example, verifying that an HCP ID (Healthcare Professional ID) isn’t null, follows the required format, is unique, and maps to a valid reference record. Business teams define rules that determine whether the data makes sense in its operational context. For example, confirming that product market share totals 100 percent within a territory or that an HCP’s prescription activity aligns with their specialty and eligibility criteria. Because technical rules are often maintained in code while business rules reside in glossaries, SOPs, and subject-matter expertise, keeping them aligned and consistently governed across datasets and platforms becomes difficult. Traditional validation techniques also have inherent limitations. Checks for null values, data types, uniqueness, or referential integrity remain essential, but they can’t capture business-specific context. Validating product hierarchies, customer segmentation, territory alignment, or commercial business rules requires domain knowledge that static validation logic can’t easily represent. Creating and maintaining data quality rules is equally demanding. Every new dataset requires data profiling, rule definition, SQL development, validation, and ongoing maintenance. These rules are often spread across SQL scripts, pipeline code, spreadsheets, data catalogs, and tool-specific configurations. As schemas, business definitions, or validation thresholds change, the effort required to update and govern these rules grows significantly. Additionally, the operational impact is significant. Onboarding new datasets becomes slower because many activities are repeated manually. Maintaining quality rules across hundreds of datasets consumes valuable engineering effort. Business users lose confidence when validation rules are difficult to understand, trace, or relate to business context. As a result, data quality often becomes a periodic governance activity rather than a continuous, scalable capability. Rethinking data quality with generative AI Traditional data quality tools often operate as standalone capabilities, disconnected from the broader enterprise data ecosystem. As organizations scale AI and analytics, data quality needs to be embedded within the data platform itself, where governance, metadata, lineage, and analytics work together. Indegene addresses this through the NEXT Unified Data Platform (NEXT UDP) , a modern, AI-driven data platform built specifically for life sciences. NEXT UDP brings together fragmented commercial, medical, clinical, and enterprise data into a single, governed environment so organizations can manage the complete data lifecycle from ingestion to analytics. NEXT UDP provides several standalone AI agents, such as the Data Ingestion agent, Data Quality agent, Orchestration agent, Data Catalog agent, and SQL Bot Generator agent. You can use them standalone within your data pipelines or customize them further for agent-to-agent communication. By integrating data ingestion, transformation, data quality, governance, metadata management, and analytics within a unified platform, NEXT UDP helps organizations: Consolidate fragmented enterprise data into a trusted source of truth. Standardize data quality and governance across business domains. Improve metadata visibility, lineage, and traceability. Reduce manual effort across data engineering workflows. Deliver trusted, AI-ready data for enterprise analytics and decision-making. Figure 1: NEXT Unified Data Platform dashboard with use cases and AI agents Figure 2: NEXT UDP technical architecture on AWS Indegene’s life sciences clients needed an agent that could run reliably in production and meet strict governance requirements, while giving the team flexibility in how they build it. Within NEXT UDP, Generative AI powers an intelligent Data Quality Agent that augments existing data quality processes. The agent boosts data engineer and data steward productivity by accelerating rule creation, simplifying validation, and helping maintain consistent governance at enterprise scale. Indegene built the Data Quality Agent on a set of AWS services, with Amazon Bedrock, Strands Agents and Amazon Bedrock AgentCore at its core. Amazon Bedrock , provides a choice of foundation models (FMs), built on a foundation of security, privacy, and responsible AI controls . For life sciences organizations, this matters because their data often includes sensitive healthcare professional and patient information. Strands Agents streamlines building and updating the agent, and its evaluation tools help Indegene test rule accuracy, which is critical in regulated life sciences environments. The agent runs on Amazon Bedrock AgentCore , a platform to build, connect, and optimize agents at scale, with any framework or model. The solution also inherits existing AWS security and audit controls, so Indegene can focus on data quality logic instead of agent infrastructure. What makes the Data Quality Agent unique? Two knowledge bases that power context-aware data quality Any data quality tool can check for null values, duplicates, or invalid data types. Services such as AWS Glue Data Quality, a capability of AWS Glue, already recommend rules like these from data statistics, and they remain a strong foundation. The harder problem is determining whether the data makes sense for its business purpose. To address this, the Data Quality Agent uses two complementary knowledge bases that combine curated domain knowledge with experience gained from governed usage. The first is an Internal Enterprise knowledge base: It combines internal business glossaries, data dictionaries, SOPs, regulatory guidance, approved data quality rules, validated scripts, profiling results, historical anoma

read the original at AWS Machine Learning →