Fleet-wide computer vision lifecycle at the edge with SageMaker SDK v3

Fleet-wide computer vision model lifecycle management is a problem that shows up the moment a quality inspection proof of concept succeeds. One camera and one model work fine. Then the plant asks for the same result on 40 lines, each with several cameras, and every camera sees the world differently. Each has a different angle, different glare off the same stainless surface, and different ambient light depending on which bay door is open. In production deployments we have worked on, a single model trained on pooled footage underperforms enough individual cameras that each camera ends up with its own fine-tuned model. Now the real problem is no longer training one model well. It’s training, evaluating, versioning, and shipping thousands of them without a human in the loop for each one. This post updates the pattern introduced in the 2023 series Build an end-to-end MLOps pipeline for visual quality inspection at the edge for the current Amazon SageMaker Python SDK v3. It replaces the since-deprecated Amazon SageMaker Edge Manager with Amazon SageMaker Neo for edge optimization. I walk through a dual-branch Amazon SageMaker Pipeline that trains an anomaly detector and a defect classifier per camera, with quality gates that block underperforming models. An in-pipeline SageMaker Neo compilation step ensures every registered model ships with a device-optimized artifact, and an AWS Step Functions orchestration layer paces fleet training. You deploy everything with the AWS Cloud Development Kit (AWS CDK), and a full pipeline run on synthetic data costs about $0.07 in compute. The complete sample is available in the aws-samples GitHub repository . Solution overview The following diagram shows the sample’s fleet-wide model lifecycle. An orchestration layer queues re-training requests and limits concurrent work. A per-camera training pipeline prepares data, trains and evaluates an anomaly detector and a classifier, and produces versioned model artifacts for edge devices. Figure 1: Solution architecture The training half is a single SageMaker Pipeline, parameterized by camera, that runs 13 steps per execution. Three entry steps run in parallel: Preprocess, Fetch Latest Anomaly, and Fetch Latest Classifier. The pipeline preprocesses that camera’s images and fetches the camera’s most recent approved model for each branch, then trains two models. The first is a small convolutional autoencoder that learns what normal product looks like and flags anomalies by reconstruction error. The second is a ResNet18 classifier fine-tuned to confirm the defect. Each branch exports its model to ONNX, evaluates the exported artifact against held-out data, and passes through a quality gate. Past the gate, the model is compiled with SageMaker Neo for the configured edge target, and only then registered in the SageMaker Model Registry. As a result, every approved package has a device-ready compiled artifact. Packages carry the camera ID as metadata, and each branch’s warm-start fetch filters on it, so re-training always resumes from that camera’s own history. The orchestration half paces the fleet. Re-training requests land on an Amazon Simple Queue Service (Amazon SQS) FIFO queue, and an AWS Lambda function starts a Step Functions execution per request. Each execution acquires a slot through a DynamoDB semaphore before starting its pipeline, so a burst of requests drains at a steady rate instead of all launching at once. Completions publish to Amazon Simple Notification Service (Amazon SNS), and operational failures reach the same topic through an Amazon EventBridge rule with the original error preserved. The output of the system is versioned, quality-gated, per-camera model artifacts. The Model Registry holds the portable ONNX, and S3 holds the Neo-compiled build of that model, which is a device-optimized shared library loaded by the DLR runtime rather than ONNX. Both are ready for whatever delivery mechanism your plant uses. Prerequisites To deploy the sample you need the following: An AWS account with permissions to create AWS Identity and Access Management (IAM) roles. An Amazon Simple Storage Service (Amazon S3) bucket. An Amazon Elastic Container Registry (Amazon ECR) repository. The SageMaker AI, AWS Step Functions, AWS Lambda, Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon EventBridge, and Amazon DynamoDB resources in the stacks. Python 3.10, 3.11, or 3.12, Docker. The AWS CDK CLI version 2.1131.0 or later. Previous CLI versions fail with a cloud assembly schema mismatch against recent versions of aws-cdk-lib . No service quota increase is needed. The orchestration layer admits at most five pipeline executions at a time by default, which keeps concurrent SageMaker training jobs inside the standard account quota. If you deploy to an AWS Region other than your CLI default, set all three of AWS_REGION , AWS_DEFAULT_REGION , and CDK_DEFAULT_REGION on the deploy command itself rather than in your shell profile, for example, AWS_REGION=us-east-2 AWS_DEFAULT_REGION=us-east-2 CDK_DEFAULT_REGION=us-east-2 cdk deploy --all . The CDK CLI, Boto3, and the AWS Command Line Interface (AWS CLI) resolve the Region from overlapping but not identical variable sets, and a mismatch produces confusing partial deployments. Prefixing the command sets all three for that invocation only, so your own default Region is left unchanged. Defining the training pipeline with SDK v3 The code in this section is excerpted from the CDK application in the repository rather than a standalone script, and it runs during cdk deploy, when the stack synthesizes the pipeline definition. Environment setup and the deploy commands come later, under Deploying and running the sample. SDK v3 reorganized the SageMaker Python SDK into modular packages, and the imports show the new shape of the library: from sagemaker.train import ModelTrainer from sagemaker.train.configs import InputData, Compute, OutputDataConfig from sagemaker.core.training.configs import MetricDefinition from sagemaker.serve.model_builder import ModelBuilder from sagemaker.mlops.workflow.pipeline import Pipeline from sagemaker.mlops.workflow.steps import ProcessingStep, TrainingStep from sagemaker.mlops.workflow.model_step import ModelStep from sagemaker.mlops.workflow.condition_step import ConditionStep from sagemaker.core.workflow.conditions import ConditionGreaterThanOrEqualTo from sagemaker.core.workflow.functions import JsonGet Installing sagemaker>=3.0.0, pulls in sagemaker-core , sagemaker-train , sagemaker-serve , and sagemaker-mlops as separate packages. Training configuration that v2 spread across Estimator constructor arguments now lives in typed config objects passed to ModelTrainer . This is the anomaly branch’s trainer, verbatim from the sample: anomalib_trainer = ModelTrainer( training_image=custom_training_image_uri, compute=Compute( instance_type=training_instance_type.to_string(), instance_count=1, ), base_job_name="cv-pipeline-anomalib-train", sagemaker_session=pipeline_session, role=role_arn, hyperparameters={ "program": "train_anomalib.py", "model_name": "autoencoder", "image_size": "256", "num_epochs": "3", "batch_size": "16", "learning_rate": "0.001", }, input_data_config=anomalib_input_data, output_data_config=OutputDataConfig( s3_output_path=run_prefix("anomalib-training") ), ).with_metric_definitions( [ MetricDefinition( name="image_auroc", regex=r"validation:image_auroc=([0-9]*\.?[0-9]+)", ), ] ) anomalib_input_data wires four channels ( train , validation , test , pretrained_model ) from upstream step properties, and run_prefix scopes every S3 path to the pipeline execution ID so concurrent fleet runs never collide. Both are in the repository. Two v3 behaviors matter when the pipeline lives inside a CDK application, and neither appears in the changelog as a breaking change. First, SDK v3 validates the pipeline’s execution role against live IAM while building the pipeline definit