Natural language security queries at Orca Security using Amazon Bedrock

Security analysts need fast answers about their cloud infrastructure without deep familiarity with domain-specific query languages. Orca Security ’s tool covers over 900 cloud models (the resource types Sonar tracks, such as an Amazon Elastic Compute Cloud (Amazon EC2) instance or an Amazon Simple Storage Service (Amazon S3) bucket). Querying that data required precise model names, nested relationship navigation, and selection from 18 operators (comparison functions like equals or contains used to filter results). To solve this, Orca Security partnered with AWS to build Discovery AI, a natural language query agent on Amazon Bedrock . Orca provides context-aware security for cloud, code, and AI, and Sonar is its domain-specific query system where users search, filter, and explore across the entire data graph. In this post, we share the architecture and results of Discovery AI, which converts a natural language question into a Sonar query. As a result, Orca reduced query costs by 90 percent, input tokens by 93 percent, and latency by 75 percent compared to the initial baseline. Bridging natural language and structured queries Orca Security’s Sonar query system supports 900+ cloud infrastructure and application models with complex relationships between them. To query this data, users construct JSON objects that specify exact model names, select from 18 operators, and navigate parent-child hierarchies across cloud providers. Doing this well requires deep familiarity with the Sonar schema. To build this capability, the Orca team set out to build a natural language interface that can generate valid Sonar queries from plain text. Their initial proof of concept loaded the full Sonar schema into a single prompt on the largest available model. This included over 900 model definitions, every relationship type, and a generous set of example queries. The output quality was strong (design partners and internal users responded positively) but three significant drawbacks emerged at production scale. Token consumption was inefficient, at roughly 2 million input tokens per workload, which drove up cost. Latency was high, with end-to-end response times of around 60 seconds per workload. Accuracy on complex, multi-model queries was limited because nested relationships and multi-condition queries often required additional refinement. The Orca team validated that the concept worked. The next challenge was making it cost-effective, fast, and accurate enough for production. To address this, the Orca team designed and deployed a solution that handles thousands of queries per day at 15-second response times using Amazon Bedrock. Solution overview Orca designed Discovery AI as a hybrid solution on Amazon Bedrock that combines Retrieval Augmented Generation (RAG) with schema-aware query construction. The system converts natural language questions into valid Discovery filters through a multi-stage pipeline, with each stage matched to the most appropriate Amazon Bedrock model. Orca chose Amazon Bedrock because it provides multi-model access through a single API, letting the team allocate different foundation models to each pipeline stage without managing separate integrations. Amazon Bedrock also inherits AWS governance controls through AWS Identity and Access Management (IAM), virtual private cloud (VPC) endpoints, and AWS CloudTrail, meeting Orca’s security requirements. The model-agnostic architecture meant the team could swap in newer model versions as they became available without rearchitecting the pipeline. The following diagram illustrates the nine-stage pipeline architecture. Figure 1: Multi-stage Discovery AI pipeline architecture showing the flow from natural language input to validated Discovery filter queries using foundation models on Amazon Bedrock The Discovery AI pipeline converts a natural language request into a validated Discovery filter through nine steps. Request Entry and Context Setup: The pipeline starts by combining the user’s natural language request with relevant context, including the user’s organization, connected cloud providers, and request metadata. This context helps the pipeline interpret the request accurately and generate a Discovery filter that matches the customer’s environment. Quick Deterministic Matcher: The Quick Matcher identifies structured inputs such as IP addresses, Common Vulnerabilities and Exposures (CVEs), ARNs, file hashes, Amazon Machine Image IDs, Oracle Cloud Identifiers, EC2 IDs, URLs, and alert IDs. It uses regex and rule-based matching to detect high-confidence patterns.When a match is found, the pipeline can return a predefined Discovery filter through a fast path, without using large language model (LLM) inference. When no match is found, the request continues to the next step. LLM Guardrail Screening: The Guardrail component checks whether the request is safe, relevant, technical, and supported. It blocks prompt injection attempts, off-topic requests, and unsupported inputs before they reach the rest of the pipeline. Requests that fail the guardrail checks are rejected with an appropriate error. Approved requests continue to example retrieval. Embedding-Based Semantic Candidate Recall: The pipeline searches for relevant previously validated query examples using Amazon Titan Text Embeddings. It retrieves the 20 nearest examples from the vector store based on semantic similarity. These candidates provide the pipeline with examples of how similar natural language requests were previously translated into Discovery filters. Neural Relevance Reranking: The retrieved candidates are reranked using Cohere Rerank 3.5 on Amazon Bedrock. It identifies the examples that are most relevant to the current request and selects the five highest-signal examples. When an exact or high-confidence match is found, the pipeline can reuse the known-good filter directly. Otherwise, the reranked examples are passed to the generative stages. LLM-Driven Relevant Model Selection: Model Selection narrows more than 900 Sonar models to the small set relevant to the request. It considers the user’s request, connected providers, and retrieved examples when selecting the appropriate models. The selected models are provided to the Generator using the appropriate schema level, ranging from compact micro schemas to more detailed mini schemas. This reduces unnecessary context and makes generation more efficient. Tool-Augmented Agentic Filter Generation: The Generator uses a tool-augmented agent to construct the final Discovery filter. It reasons over the selected model schemas, retrieved examples, user context, and available tools. The agent can iteratively refine the filter as it works toward a valid representation of the user’s request. Static and API-Backed Validation: The generated filter is checked for syntactic and semantic correctness. The pipeline first applies static validation and then verifies the filter against Orca’s Sonar API. When validation fails, the validation error is returned to the Generator so the agent can correct the filter and try again. If the maximum number of retries is reached without producing a valid filter, the pipeline returns an invalid-filter response. The Generator is constrained to the reranked examples and selected schemas. Static validation resolves every model, field, and operator back to those schema definitions. Each validation error goes back to the agent with the offending element identified, so retries re-ground rather than re-guess. Post-Processing and Response: After a valid filter is produced, whether through the Quick Matcher, reuse of a known-good example, or agentic generation, the pipeline prepares the final response.The team allocated different foundation models on Amazon Bedrock across components based on each stage’s requirements. Anthropic Claude Haiku 4.5 powers the Model Selector and Generator. These components require strong reasoning for accurate query construction. Amazon Nova 2 Lite handles the G