Best practices for Amazon SageMaker HyperPod administration and governance

Amazon SageMaker HyperPod gives machine learning (ML) teams access to large pools of accelerated compute for training and fine-tuning models. When several teams share one cluster, the technical setup is usually straightforward. The challenging part is governance. You must decide which teams can use the cluster, how much capacity each team gets, what happens when one team’s workload competes with another’s, and who is accountable when usage drifts from policy. Amazon SageMaker Unified Studio adds another consideration: You can connect a SageMaker HyperPod cluster to a project so team members can launch workloads from their project workspace. That convenience is valuable, but after multiple teams share visibility into the same cluster, the controls that govern who can do what become even more important. In this post, we show how to administer SageMaker HyperPod through SageMaker Unified Studio while preserving the underlying governance controls. We cover the four layers of control: organization, project, cluster, and workload. We also explain how to design identity, capacity, and observability policies across them. By the end, you will have a repeatable model for offering approved SageMaker HyperPod compute to ML teams in their project context while keeping cluster operations with the infrastructure team. Amazon SageMaker HyperPod is a capability of Amazon SageMaker AI . Amazon SageMaker Unified Studio is the data and AI development environment where teams build with their data and tools. With SageMaker Unified Studio, you can connect a project to an existing SageMaker HyperPod cluster. Members can then launch machine learning workloads, review cluster and task information, and open a JupyterLab workflow. You continue to manage clusters through Amazon SageMaker AI interfaces and APIs. This separation gives infrastructure teams a useful operating model. You can manage cluster infrastructure through established cloud operations processes. At the same time, you can present approved compute to machine learning teams in their project context. In this post, we describe how you can design infrastructure boundaries, govern access, allocate shared capacity, and operate SageMaker HyperPod consistently through SageMaker Unified Studio. Understand the administrative boundaries A well-governed environment separates organizational administration, project access, and cluster operations. Each layer answers a different question and uses a different control. The following table summarizes each boundary, its primary controls, and its administrative purpose. Boundary Primary controls Administrative purpose Organization SageMaker Unified Studio domains, domain units, associated accounts, project profiles, and authorization policies Determines who can create projects, which accounts and Regions projects can use, and which tools are available Project Project membership, project roles, and SageMaker HyperPod connections Defines the collaboration context and the AWS resources that project members can access Cluster SageMaker HyperPod cluster admin roles, Amazon EKS access entries, role-based access control (RBAC), and EKS Pod Identity, or Slurm controls Governs cluster configuration, scheduler access, namespaces, tasks, and infrastructure operations Workload Compute allocations, priority classes, lending and borrowing policies, and task permissions Controls who can submit work and how shared capacity is assigned Treat these controls as layers. A SageMaker HyperPod connection adds an approved cluster to a project. It doesn’t replace the cluster’s AWS Identity and Access Management (IAM), Amazon Elastic Kubernetes Service (Amazon EKS), or Slurm controls. Review the project role, connection access role, EKS access entries and RBAC or Slurm controls, workload identity, data and AWS Key Management Service (AWS KMS) key policies, network policy, task-view restrictions, and scheduler policy together. Make the cluster available only after those controls are aligned. These controls form four layers, shown in the following figure. Figure 1: The four layers of control: organization, project, cluster, and workload. Each layer answers a different question and uses a different control, so review each at its own boundary. SageMaker Unified Studio projects are collaboration boundaries. They aren’t strong runtime security boundaries. Keep the SageMaker HyperPod cluster, scheduler, and scarce accelerator capacity under one designated capacity account. Approved users and datasets can remain in the same account or in separate consumer and data accounts. AWS documents multi-account support for SageMaker HyperPod task governance on Amazon EKS clusters. Although On-Demand Capacity Reservations can be shared across accounts, centralize SageMaker HyperPod cluster ownership and capacity administration in the capacity account. Use approved cross-account access rather than making each consumer account an independent capacity administrator. For Amazon EKS, use a namespace per tenant together with RBAC, per-tenant service accounts and EKS Pod Identity roles, default-deny network policies, and tenant-specific storage and AWS KMS permissions. For Slurm, use Slurm accounting with hierarchical accounts and associations, quality of service (QoS), priority and fair-share policies, and partitions. Also use operating-system identity and file permissions, network controls, and tenant-specific data paths. Use IAM roles and resource policies, rather than project membership alone, to control access to Amazon Simple Storage Service (Amazon S3) buckets, AWS KMS keys, secrets, container registries, and other data services. Use project and domain-unit policies for collaboration and delegation. For Amazon EKS clusters, use SageMaker HyperPod task governance, including quotas, priority classes, lending and borrowing, and preemption, to share the central accelerator pool fairly. For Slurm clusters, use native partitions, quality of service (QoS), priority, fair-share, and preemption controls. Slurm doesn’t provide the same lending-and-borrowing model. These scheduling controls determine when an authorized workload receives compute. They don’t grant namespace or data access. For stronger isolation within a cluster, use dedicated nodes or node groups and admission controls where appropriate. Use a separate cluster or account when legal, regulatory, or security requirements demand hard infrastructure isolation. Cross-account consumer access can preserve account-level ownership without duplicating the central SageMaker HyperPod cluster. Within the capacity account, use project profiles and domain units with project authorization policies to delegate project creation and ownership. The following figure shows centralized capacity with tenant-specific workload, identity, data, network, and visibility controls. Figure 2: Centralized SageMaker HyperPod capacity. The cluster and scheduler remain in one capacity account. Each tenant receives workload, identity, data, network, and task-visibility controls. Hard-isolation requirements use dedicated infrastructure. Figure 3 shows how those boundaries connect when SageMaker Unified Studio exposes approved SageMaker HyperPod compute to project members. Figure 3: Recommended SageMaker HyperPod administration model. SageMaker Unified Studio governs the collaboration context, while cluster identity, workload authorization, scheduler policy, and operational controls remain separate. Decide when SageMaker Unified Studio is the right administrative experience With SageMaker Unified Studio, machine learning teams that already work in a project can follow an approved path to shared SageMaker HyperPod compute. Members can find connected clusters, review status and metadata, inspect supported tasks and metrics, and move into JupyterLab without using a separate infrastructure inventory. This experience doesn’t replace cluster administration. The following table shows how each persona should use Sage