TL;DR
GPU attachment and sharing choices affect consolidation, mobility, maintenance, failure handling, and lifecycle operations. Architects should validate the supported accelerator, server, firmware, driver, hypervisor, and guest combinations before defining a standard placement class. A conceptual atlas does not override the compatibility matrix.
The territories in the image should be treated as reusable workload profiles. Each profile has different optimization goals and different failure modes.
Without automation, the atlas remains a diagram. VCF Automation can expose approved catalogs, templates, infrastructure policies, and lifecycle workflows that translate architecture decisions into repeatable consumption patterns.
Conclusion
There is also a naming guardrail. The image shows VCF Operations, VCF Automation, and Aria Suite together. In a current VCF 9.1 design, use the product and service names from the current documentation and support matrix. Treat the Aria label as lineage and capability context, not as permission to mix legacy and current names without verification.
The design also assumes that AI workloads are not homogeneous. A nightly training job, a regulated retrieval service, an internal coding assistant, and a customer-facing inference API should not inherit the same placement, network, data, and recovery policy simply because they all use machine learning.
Tenant Nations
The image shows a compelling operations center, but real observability cannot stop at green infrastructure tiles. AI services fail in ways that traditional infrastructure dashboards may not detect.
The atlas becomes operational when a team can move from a degraded AI service to the affected model endpoint, Kubernetes workload or virtual machine, GPU, host, datastore, network path, policy change, capacity condition, and recovery state without losing context.
VMware Cloud Foundation provides the common substrate through vSphere, vSAN, NSX, VKS, automation, operations, and protection services. VMware Private AI Foundation with NVIDIA and NVIDIA AI Enterprise can extend that substrate with AI-specific capabilities. The architecture becomes defensible only when those capabilities are connected to workload classification, data and model residency, accelerator policy, service-level telemetry, lifecycle controls, and accountable ownership.
Kubernetes Industrial Zones
apiVersion: dtd.example/v1alpha1
kind: AIPlacementPolicy
metadata:
name: sovereign-inference-gold
spec:
workloadClass: inference
environment: production
ownership:
businessService: claims-assistance
platformTeam: private-ai-platform
modelOwner: applied-ai
dataOwner: regulated-data
operationsOwner: ai-sre
tenancy:
organization: regulated-services
project: claims-ai
costCenter: cc-4820
placement:
region: primary-private-region
failureDomain: dual-zone
runtime: vks
acceleratorProfile: shared-gpu-inference
capacityReservation: required
data:
residency: in-region
allowedClassifications:
– confidential
externalEgress: deny
backupResidency: in-region
model:
registry: approved-enterprise
versionPolicy: pinned
promotionGate: required
rollbackVersion: previous-approved
network:
segment: ai-inference-production
ingress: private
eastWestPolicy: explicit-allow
administration: privileged-access-zone
observability:
serviceSlo: inference-gold
syntheticTest: required
retainAuditDays: 365
capacityForecast: enabled
recovery:
class: regional
destination: secondary-private-region
rpoMinutes: 15
rtoMinutes: 60
testFrequency: quarterly
approvals:
– model-owner
– data-owner
– security-governance
– service-owner
AI disaster recovery must restore the service chain. Recovering a virtual machine while losing the model registry, vector index, prompt configuration, secrets, certificates, network policy, identity mapping, or source data does not restore the AI service.
Default-deny is meaningful only when teams can understand and maintain the allowed flows. Policy ownership, naming, change review, logging, and troubleshooting are as important as the firewall rule itself.
Disaster Recovery Mirror Worlds
The placement decision should consider more than free GPU capacity. Data gravity may make a slightly smaller local accelerator pool more effective than a remote pool connected through a slower or more expensive path. The platform should also distinguish interactive experimentation from scheduled production training. Mixing both without quotas and scheduling policy creates unpredictable contention.
Tenant isolation converts a shared AI platform into a service that multiple teams can trust. Tenancy may align to business units, application portfolios, regulated domains, projects, development stages, or customer environments.
Each tenant needs explicit ownership, identity boundaries, quotas, accelerator entitlements, network segments, namespace or project structure, registry scope, data permissions, cost allocation, and exception handling. Isolation should be strong enough to protect workloads while avoiding a separate snowflake platform for every team.
The Coordinates That Decide Placement
The left panel in the image lists the architectural controls that turn the atlas into a decision system. These are the coordinates that should be captured during intake and enforced through policy.
| Coordinate | Decision question | Failure when ignored |
|---|---|---|
| GPU placement | Which accelerator profile and sharing model does the workload require? | Capacity exists but the workload cannot meet memory, latency, isolation, or support requirements |
| Model residency | Where may model artifacts be stored, loaded, cached, backed up, and replicated? | Approved data stays local while model intellectual property moves outside the boundary |
| Data gravity | Where is the authoritative data, and what movement is technically, legally, and economically acceptable? | Expensive transfers, slow pipelines, duplicate data, and stale retrieval results |
| Network segmentation | Which services, users, models, data stores, and external endpoints may communicate? | Broad trust, uncontrolled egress, lateral movement, and difficult incident containment |
| Latency zone | What end-to-end response or pipeline latency must the service meet? | Infrastructure appears healthy while the application misses its service objective |
| Compliance boundary | Which obligations and evidence requirements apply to the workload? | Controls are assumed rather than mapped to implementation and proof |
| Capacity forecast | What is the expected growth in GPUs, memory, storage, throughput, tokens, users, and models? | Scarce capacity is exhausted by successful adoption |
| Recovery destination | What must be restored, where, in what order, and within which RPO and RTO? | Compute recovers but the AI service remains unusable |
AI environments create multiple storage behaviors at once. Training can generate high-throughput reads, large checkpoint writes, and temporary staging data. Inference services may depend on compact model artifacts but require predictable latency. Retrieval-augmented generation can add vector indexes, source documents, embedding pipelines, and frequent refresh activity.
Inference behaves more like a digital service than a batch job. It must meet request latency, throughput, error-rate, and availability objectives while handling model loading, scaling, version promotion, rollback, and traffic management.
Borders, Routes, and Passports
Define a limited set of supported workload classes, such as batch training, production inference, regulated retrieval, shared development, and platform services. Each class should have a documented infrastructure profile, policy profile, telemetry package, and recovery expectation.
Identity Is the Passport
RPO and RTO values should be assigned to the AI service, not copied blindly from the underlying cluster. Recovery tests should include a representative request and evidence that the correct model and data version are serving after failover.
Kubernetes zones host the repeatable factories that build, deploy, expose, and update AI services. These zones may include model-serving runtimes, retrieval services, data preparation, evaluation pipelines, agent services, API gateways, and supporting observability components.
NSX Policy Is the Border Control
It does not assume that every AI workload must remain on premises. A hybrid design may still be the correct answer for burst capacity, specialized services, global distribution, or lower-risk experimentation. The atlas is useful because it makes those exceptions explicit instead of allowing cloud placement to happen by accident.
A model registry entry without promotion evidence is not enough. Production workflows should connect artifact identity to evaluation results, risk acceptance, deployment history, runtime telemetry, and retirement status.
Provenance Is the Customs Record
Kubernetes does not remove infrastructure concerns. Cluster topology, node placement, storage classes, network policy, registry access, secrets, upgrade sequencing, GPU operators, and workload quotas still have to align with the VCF foundation. VKS is a territory within the atlas, not a separate universe with different governance.
The platform is ready to scale when repeated onboarding becomes easier without weakening governance.
The Operating Model Behind the Map
An inference city should separate service tiers. A customer-facing real-time endpoint may require reserved capacity, multiple failure domains, controlled rollout, and strict recovery objectives. An internal summarization service may tolerate queueing, shared accelerators, and slower restoration. Both use models, but they should not inherit the same architecture.
| Role | Accountable decisions | Required evidence |
|---|---|---|
| Platform owner | Supported service classes, capacity, lifecycle, automation, shared SLOs | Platform health, capacity forecast, compatibility baseline, change records |
| Model owner | Model selection, evaluation, version promotion, rollback, retirement | Evaluation results, model card, approval, runtime performance |
| Data owner | Data classification, residency, access, retention, lineage | Classification, lineage, access review, retention and deletion evidence |
| Security and governance owner | Trust boundaries, policy, exceptions, audit, risk acceptance | Control mapping, policy state, exception record, audit evidence |
| Application owner | Business outcome, request pattern, service SLO, dependency behavior | User-impact metrics, synthetic tests, dependency map, incident history |
| Operations or SRE owner | Monitoring, incident response, recovery, maintenance, capacity action | Alerts, runbooks, recovery tests, error budget, post-incident actions |
The image is intentionally conceptual. Names such as GPU Training Continent or Tenant Nations are not official VMware Cloud Foundation object types. The utilization percentages, model counts, availability figures, and health indicators are also illustrative. They should be replaced with measured values from the actual platform and service telemetry.
Turning the Atlas into Policy
Begin with two or three representative workloads. Capture their data classification, model lifecycle, accelerator needs, request pattern, network flows, service objectives, recovery requirements, and owners. Use these examples to define the first placement coordinates and decision records.
Avoid creating a unique class for every application. The purpose is to standardize the common path while preserving a controlled exception process.
NSX gives the atlas enforceable borders. It can separate management, training, inference, data services, tenant projects, registries, recovery services, and external integration paths. The purpose is not to create as many segments as possible. The purpose is to make allowed communication explicit and observable.
Observability Must Describe the Service
Enterprise AI architecture is often presented as a vertical stack. Hardware sits at the bottom, virtualization and Kubernetes sit above it, model services appear near the top, and governance is added as a side panel. That view is useful for identifying components, but it is weak at explaining how a real platform behaves across teams, regions, data classifications, latency requirements, failure domains, and recovery obligations.
A world map becomes useful when it shows what may cross each border. The same is true for private AI.
| Telemetry layer | What to measure |
|---|---|
| Infrastructure | Host health, CPU, memory, GPU utilization, GPU memory, storage latency, network loss, capacity headroom |
| Platform | VKS cluster health, node pools, operators, registries, automation failures, policy drift, certificate state |
| Model service | Request latency, time to first response, throughput, queue depth, error rate, saturation, model load time |
| Model and data | Active model version, evaluation status, data freshness, retrieval quality, index state, provenance |
| Governance | Denied requests, policy exceptions, privileged access, egress events, unapproved artifacts, audit completeness |
| Economics | Accelerator allocation, idle capacity, cost by tenant, cost by service, cost per successful request or job |
| Recovery | Backup success, replica lag, artifact completeness, last recovery test, measured RPO and RTO |
The key operating principle is lifecycle consistency. A model service is not production-ready because it runs once. It must be reproducibly deployed, upgraded, observed, rolled back, recovered, and retired.
A Practical Build Sequence
The atlas therefore needs a decision record, not only a scheduler. The record should preserve the workload classification, assumptions, selected territory, rejected alternatives, control requirements, owner approvals, and review trigger.
Define the Coordinates and Owners
The practical takeaway is simple: do not begin by asking where to install an AI stack. Begin by defining how workloads are classified, where data and models are allowed to live, which services may communicate, how accelerators are allocated, what telemetry proves health, and who owns each decision. The atlas turns those questions into an operating model that can be automated, observed, audited, and tested.
The Foundation Under Every Territory
The service boundary matters. Infrastructure availability does not prove that the model endpoint is healthy. Synthetic requests, model-response validation, queue depth, time to first response, throughput, saturation, and dependency health must be measured at the service level.
Validate that a request can be fulfilled consistently and that the resulting environment contains the evidence needed for operations and audit.
Codify the Classes in Catalogs and Policies
In this model:
That metaphor is powerful because enterprise AI is fundamentally a placement and governance problem. The challenge is not only providing GPUs or deploying a model endpoint. The harder work is deciding which resources a workload may consume, what data it may reach, where its artifacts may move, which network boundaries apply, what service level it must meet, how it is recovered, and who is accountable when the assumptions change.
Build Service-Level Telemetry
A production training profile should define accelerator type, sharing model, minimum memory, CPU and memory ratios, local or remote data sources, checkpoint destination, maximum run duration, preemption behavior, network class, observability, and cost owner.
Test Lifecycle and Recovery
The following YAML is an illustrative policy contract. It is not a native VMware Cloud Foundation API schema. Its purpose is to show the information a service catalog, policy engine, or orchestration workflow should collect before deployment.
VCF Operations provides the health, capacity, performance, inventory, and operational context needed to manage the shared platform. AI service telemetry should be integrated with this infrastructure view rather than isolated in a separate dashboard that operators only consult after an incident.
Scale Through Onboarding
The atlas assumes an enterprise private AI environment built on VMware Cloud Foundation 9.1, with virtual machine and Kubernetes workload patterns, accelerator-backed compute, model and container artifacts, software-defined networking, policy-driven automation, operations telemetry, and recovery services.
An atlas does not replace a bill of materials or a validated design. It provides the shared mental model that architects, platform engineers, security teams, data owners, AI engineers, and operations teams can use before they argue about individual products or cluster counts.
