Announcing SOCI-indexed images for faster SageMaker AI jobs

Every minute your deep learning container image spends downloading is a minute you pay for and a minute your team waits. Container image download is pure overhead: billable job time that produces no model progress. For modern deep learning images, that overhead is several minutes on every single job. Today, we’re excited to announce Seekable OCI (SOCI) indexing for Amazon SageMaker AI Training and Amazon SageMaker AI processing jobs , which takes full container image download off the critical path of every job. With SOCI indexing, SageMaker AI lazy-loads your container image. It fetches only the files needed to start the container and streams the rest in the background as your code runs, so your job begins in seconds instead of minutes. You adopt it by pointing to a pre-indexed AWS Deep Learning Container (DLC) or adding an index to your own image. Neither approach requires changes to your job definition, entry point, or data channels. The result is faster job starts, lower job cost, and more time spent building models rather than waiting on infrastructure. SOCI-indexed images start Amazon SageMaker AI training containers in seconds, with no impact on training performance. As we show in the What we measured section, this reduced GPU idle time during image pulls by up to 90 percent in our benchmarks. This post shows how to adopt SOCI indexing: what it is, the results we measured, and how to turn it on for your own jobs. The challenge SageMaker AI training and processing jobs run inside container images stored in Amazon Elastic Container Registry (Amazon ECR). Before a job can execute your code, the service downloads the entire image to the instance. It decompresses and writes every layer to disk. For small images this takes seconds. But deep learning images routinely exceed 10 GB or more. That download adds minutes to every job, and you pay for it as job time. The impact is largest exactly where iteration speed matters most: hyperparameter tuning sweeps for hundreds of jobs, quick experiments, and short processing jobs, where image download can dominate total runtime. How SOCI indexing works Seekable OCI (SOCI) is an open source technology that lazy loads container images. Instead of downloading the whole image before starting, the runtime uses a SOCI index to fetch only the files needed to launch the container immediately. It then streams the rest in the background as your code accesses it. A SOCI index is metadata that maps every file inside your image. You store it in Amazon ECR as an OCI-compliant artifact linked to your original image. Because the index is a separate artifact, your image digest and any existing signatures remain valid. SageMaker AI checks for an index automatically When you create a training or processing job, SageMaker AI checks whether the image you specified has a SOCI index and behaves accordingly: Scenario What happens The image includes a SOCI index SageMaker AI lazy-loads the image and starts your container before the full download completes The image has no SOCI index Standard download, no change in behavior The index can’t be read, or lazy loading doesn’t succeed The job falls back to a standard download and continues running Because jobs always fall back to a standard download when needed, adding a SOCI index never prevents your job from running. You only gain faster startup. You never risk breaking an existing workflow. What we measured We pulled the same container images two ways on real SageMaker AI training jobs: standard download and SOCI-indexed lazy loading. We then ran full training jobs from both to measure the effect on training speed. Setup Eight images , 1.8–35.4 GB compressed. GPU and CPU instances ml.g4dn, ml.g5, ml.g6, ml.g6e, ml.g7, ml.g7e, ml.p4d, ml.p5, and ml.m7i, from xlarge to 48xlarge, in the US West (Oregon) Region (us-west-2) and the US East (N. Virginia) Region (us-east-1). Download time measured from the start of the image pull until the container is ready to run. All figures are medians of repeated runs. Result 1: Download time drops from minutes to seconds at every image size Image size (compressed) Standard download Lazy loading Improvement 1.8 GB 48 s 3-11 s ~8x 13.2 GB 3 min 42 s ~6 s ~34x 35.4 GB 7 min 17 s 3-11 s ~38x The following chart compares container image download time for a standard Docker pull against SOCI lazy loading across eight image sizes, from 1.8 GB to 35.4 GB. Each image size shows two bars. The Docker download time bar (blue) grows steadily with image size, from 48 seconds at 1.8 GB to 7 minutes 17 seconds at 35.4 GB. A speedup label above each pair shows the ratio of Docker time to SOCI time. Speedups range from approximately 10x for the smallest image to over 35x for the largest. Figure 1: Container image download time, standard Docker pull compared to SOCI lazy loading, across eight image sizes Standard download time grows with image size because every file is decompressed and written. Lazy loading stayed at 3–11 seconds regardless of size, an 8x–46x improvement. The bigger the image, the more minutes you save. Result 2: The saving holds on every instance type and size Running the 13.2 GB image across 13 instance type/size combinations, standard download ranged from 3 min 16 s to 4 min 20 s, while lazy loading held at 6-8 s. That is about 34x faster on every configuration. You don’t need to change instance type or size to benefit. Figure 2: Download time for a 13.2 GB image across 13 instance type and size combinations Result 3: Training performance is unaffected beyond a first-epoch penalty We ran two kinds of full training jobs from the Docker-pulled and from the SOCI-indexed image and compared every epoch: Image classification on ResNet-50 on Imagenette (9,469 images) Container images: PyTorch (13.2 GB DLC) and TensorFlow (9.9 GB DLC) Instances: ml.p5.48xlarge, ml.p4d.24xlarge, ml.g7e.48xlarge, ml.g7.48xlarge, ml.g6e.12xlarge, ml.g6e.24xlarge, and ml.g6e.48xlarge. Large language model (LLM) fine-tuning: full-parameter supervised fine-tuning (SFT) of Qwen2.5-0.5B on alpaca-cleaned (51,760 examples) with the Hugging Face Trainer. Container image: 9.4 GB Hugging Face PyTorch DLC. Instances: ml.p5.48xlarge, ml.p4d.24xlarge, ml.g6.48xlarge. The following chart breaks job startup into image download time and first training epoch time, comparing a standard Docker pull with a SOCI-indexed image across GPU and CPU configurations for both workloads. Figure 3: Job startup split into image download time and first training epoch time, standard Docker pull compared to a SOCI-indexed image From the second epoch on, time per epoch is the same (within run-to-run variation) on every instance, for both workloads. The first epoch takes 3–16 s longer from the lazy-loaded image, while files the framework touches for the first time are fetched. This first-epoch cost is the same on every instance size. Net: each job finished 1-6 min sooner. That is the download saving minus the one-time first-epoch cost, a fixed saving per job. Test this on your own workload These numbers are from the two workloads described earlier. The first-epoch cost depends on your image and what your code touches at startup: Lazy loading wins when the first training step needs only a fraction of the image, the rest streams in while you train. If your entry point reads most of the image before training starts (for example, scanning, checksumming, or preloading its contents), the first epoch waits for roughly the same bytes a full pull would have fetched. The net improvement would be small. Front-loaded starts still came out ahead in our tests: torch.compile and a just-in-time (JIT) built CUDA extension added 7-15 s to the first epoch (net saving still 3 min 25 s–3 min 56 s) A 6 GB model baked into the image streamed its weights at load time, adding 2-3.5 min, but the overall job still finished 2-2.5 mins sooner on the indexed image. Getting started You can adopt SOCI indexing two ways: use a pre-indexed AWS Deep L