AWS Machine Learning6m agoenergy 128

Bring structured and unstructured data together with Amazon Quick

image: AWS Machine Learning

Organizations collect data in many forms. Structured data lives in databases and spreadsheets, whereas unstructured data sits in documents, reports, and files stored across file systems. Analyzing both together typically requires switching between multiple tools, running separate queries, and manually correlating results. Amazon Quick addresses this with a unified experience. You can connect structured data through enriched datasets and unstructured data through S3-backed knowledge bases . You can then query both through a single conversational AI agent. In this post, you learn how to: Connect Amazon Simple Storage Service (Amazon S3) as a data source and create a knowledge base. Build a Space that combines structured and unstructured data. Create and configure a custom chat agent that queries across both data types. Ask questions that span structured and unstructured sources in a single conversation. Solution architecture The following diagram illustrates how each component connects to deliver a unified data experience through Amazon Quick. Figure 1: Solution architecture for combining structured and unstructured data in Amazon Quick The architecture has three layers: Data layer: Your data exists in two forms. Unstructured content (documents, PDFs, reports) is stored in Amazon S3 and indexed through a knowledge base. Structured data (metrics, sales figures, and operational key performance indicators) is loaded as enriched datasets with semantic descriptions that the agent uses to translate questions into SQL. Workspace layer: A Space brings both data types together into a single workspace. The Space references your knowledge base for unstructured queries and your datasets for structured queries. It can also include dashboards, file uploads, and action connectors. This is the layer where data from different sources converges into a unified context. Agent layer: The custom chat agent links to the Space and uses it as its knowledge source. When a user asks a question, the agent determines whether to query the knowledge base, the dataset, or both, and returns a combined response with citations. Amazon Quick also applies built-in AI guardrails , such as content safety screening, to chat interactions. With this layered approach, you can expand the solution over time. When you add a new data source to the Space, the agent linked to that Space can use it, and responses respect the permissions of the user who asks. Prerequisites To follow along with this walkthrough, you need: An Amazon Quick account with an Enterprise subscription for the user who follows this walkthrough (creating datasets requires Enterprise). For details, see Amazon Quick pricing . An Amazon S3 bucket for your business documents. To create one, see the Amazon S3 User Guide . Amazon Quick access to your S3 bucket , granted by an Amazon Quick administrator. In the admin console, the administrator chooses Permissions, then AWS resources, selects Amazon S3, and chooses Select S3 buckets. Selected buckets have read-only permissions by default. Permissions to create integrations, knowledge bases, datasets, spaces, and chat agents in Amazon Quick. Prepare your sample data This walkthrough uses two types of data: Unstructured documents: Upload these to your Amazon S3 bucket for the knowledge base: AnyCompany_Business_Review.pdf AnyCompany_Market_Analysis_Report.pdf AnyCompany_Product_Strategy_Memo.pdf Structured data: Download this file to your computer. You upload it directly to Amazon Quick in Step 3: Product_Sales.csv Step 1: Create an Amazon S3 integration Start by connecting Amazon Quick to your S3 bucket. From the Amazon Quick console, select Knowledge in the left navigation panel. The knowledge page shows available connector types, including Amazon Q Business, Web Crawler, Amazon S3, and third-party options like Google Drive, OneDrive, and Microsoft SharePoint. Under Amazon S3, choose Add . In the Create S3 knowledge base dialog, on the Connect S3 bucket page, provide the following details: Name: A descriptive name for your integration (for example, “My Data Access Integration”). S3 bucket location: Choose Quick Suite instance account to use a bucket in the same AWS account as Amazon Quick, or Other AWS account for a bucket in a different account. S3 bucket URL: The S3 URI of the bucket containing your data. The bucket must be in the same Region as Amazon Quick. Choose Next . Figure 2: Create integration dialog showing the S3 connection fields Step 2: Create a knowledge base After the S3 integration is created, Amazon Quick prompts you to create a knowledge base. The knowledge base indexes your S3 content for AI-powered search. Configure the knowledge base: Name: “Business Knowledge Base”. Description (optional): Notes about how the knowledge base will be used. Content: Select “Add all content” to index everything in the connected S3 bucket, or “Add specific content” to specify the S3 prefixes for the folders and files to include (prefix filters are case-sensitive). Next: Additional settings opens optional settings for document-level access control lists (ACLs) and a metadata files folder. Decide whether to use ACLs before you create the knowledge base, because you can’t change that choice later. This walkthrough uses sample data, so the default settings are sufficient. For details, see Set up and manage Amazon S3 integration . Choose Create . A success banner confirms the S3 integration, and the knowledge base begins indexing in the background. Figure 3: Create knowledge base dialog with the Business Knowledge Base name and content options Step 3: Create and enrich a dataset A dataset is your structured data file loaded into Amazon Quick. By enriching the dataset with semantic descriptions, custom instructions, and correct data types, the AI agent can interpret your data more accurately. It can then translate natural language questions into SQL queries. Create the dataset From the Amazon Quick home page, navigate to Data in the left navigation. Choose Create dataset , then select Upload file . Select the Product_Sales.csv file you downloaded earlier and choose Open . On the confirmation screen, review the column names and data types. Choose Next to create the dataset. Amazon Quick ingests the file into SPICE (the in-memory calculation engine) and the dataset appears in your dataset list. The dataset takes its name from the uploaded file, so it appears as Product_Sales.csv in your account. Enrich the dataset Enriching your dataset with semantic information helps the AI understand your data in business context. The agent uses this context to translate natural language questions into more accurate SQL queries. From the dataset page, choose Edit dataset to enter the data preparation view. Transform: Fix data types In the Transform tab, update data types for columns that were auto-detected incorrectly. For this dataset: Change Order Date and Date Key from String to Date. Change Customer ID from String to Integer. Change Sales , Discount , and Profit from String to Decimal. Change Quantity and Unit Price from String to Integer. Figure 4: Transform view showing data type changes applied to the Product_Sales.csv dataset Output: Add semantic context Select the Output tab to access the dataset-level enrichment fields. Here you provide context that helps the AI understand what your data represents and how to use it. Description: Add a dataset-level description: “Product sales transaction data for AnyCompany, covering orders from FY2024-FY2025 across North America, EMEA, Asia Pacific, and Latin America. Includes order details, customer information, product lines, and revenue metrics.” Custom instructions: Add guidance for how the AI should query this data: USAGE GUIDELINES: Revenue values are in USD. When comparing performance, group by Region (North America, EMEA, Asia Pacific, Latin America). Order Date should be used for time-based trend analysis. The fiscal year matches the calendar year. Date

read the original at AWS Machine Learning →