Vmware Validated Design 62 SDDC Introduction
Vmware Validated Design 62 SDDC Introduction
Vmware Validated Design 62 SDDC Introduction
Validated Design
Modified on 30 MAR 2021
VMware Validated Design 6.2
VMware Cloud Foundation 4.2
Introducing VMware Validated Design
You can find the most up-to-date technical documentation on the VMware website at:
https://docs.vmware.com/
VMware, Inc.
3401 Hillview Ave.
Palo Alto, CA 94304
www.vmware.com
©
Copyright 2016-2021 VMware, Inc. All rights reserved. Copyright and trademark information.
VMware, Inc. 2
Contents
2 SDDC Architecture 8
7 Post-Deployment Documentation 37
VMware, Inc. 3
About Introducing VMware Validated
Design
The Introducing VMware Validated Design document provides guidance on using the content
of VMware Validated Design™ for Software-Defined Data Center. The guide also contains a
high-level overview of the Software-Defined Data Center (SDDC) design that is supported in this
VMware Validated Design version.
n Design objectives
Intended Audience
Introducing VMware Validated Design is intended for cloud architects, infrastructure
administrators, cloud administrators, and cloud operators who want to become familiar with
VMware Validated Design to deploy and manage an SDDC that meets the requirements for
capacity and scalability.
Update History
This Introducing VMware Validated Design is updated with each release of the product or when
necessary.
VMware, Inc. 4
Introducing VMware Validated Design
Revision Description
30 MAR 2021 This release of VMware Validated Design now includes guidance on cloud operations and automation
solutions for a dual-region SDDC. See Deployment Workflow for a Multi-Region SDDC, Cloud
Operations Layer, and Cloud Automation Layer.
VMware, Inc. 5
Features of VMware Validated
Design 1
Use VMware Validated Design to build a scalable Software-Defined Data Center that is based on
VMware best practices.
After you satisfy the deployment requirements, follow one consistent path to deploy an
SDDC.
VMware Validated Design provides a tested solution path with information about product
versions, networking architecture, capabilities, and limitations.
VMware Validated Design supports an SDDC that has the following features:
n Operational continuity
VMware, Inc. 6
Introducing VMware Validated Design
You can implement a data center without engaging in design work and product research.
After you download all SDDC products, follow the detailed design and step-by-step
instructions.
Every version of a VMware Validated Design accommodates new product releases. If you
have deployed an SDDC according to an earlier version of a VMware Validated Design, you
can directly follow the validated design to upgrade your environment.
VMware, Inc. 7
SDDC Architecture
2
VMware Validated Design supports an SDDC architecture according to the requirements of your
organization and the resource capabilities of your environment.
monitor,
collect and analyze logs
solution
vRealize Log Insight solution vRealize Suite Lifecycle life cycle
life cycle Manager management
management
launch in context,
notification events, life cycle management
UI integration
identity and access
management
vRealize Operations vRealize Automation
workload metrics,
Manager
workload costing
solution load balancing,
life cycle logical switching,
monitor, store product binaries
management logical routing
log collection
NSX-T Data Center life cycle management life cycle management vCenter Server
monitor,
collect and analyze logs
central user management
central user management
life cycle management
vSphere Cluster
Active Directory
central user management
ESXi ESXi ESXi ESXi central management
of virtual infrastructure
monitor,
collect and analyze logs
VMware, Inc. 8
Introducing VMware Validated Design
SDDC Architecture
VMware Validated Design supports the Standard SDDC architecture of VMware Cloud
Foundation. This architecture implements a production-ready SDDC that is dual-region. Each
region includes two workload domains - management and virtual infrastructure.
For information on the workflow for deploying the SDDC, see Chapter 5 SDDC Deployment Flow
of VMware Validated Design. For information on the types and components of the workload
domains in this validated design, see Chapter 4 Workload Domains in VMware Validated Design.
VMware, Inc. 9
Design Objectives of VMware
Validated Design 3
According to the SDDC implementation type, a VMware Validated Design has objectives to
deliver prescriptive content about an SDDC that is fast to deploy and is suitable for use in
production.
Number of regions and disaster recovery Single-region SDDC with multiple availability zones that you can
support potentially use as a best practice for a second VMware Cloud
Foundation instance.
Availability zones are separate low-latency, high-bandwidth connected
sites. Regions have higher latency and lower bandwidth connectivity.
The documentation provides guidance for a deployment that supports
two regions for failover in the following way:
n The design documentation provides guidance for an SDDC whose
management components are designed to operate in the event of
planned migration or disaster recovery.
n The deployment documentation provides guidance for an SDDC that
supports two regions for both management and tenant workloads.
VMware, Inc. 10
Introducing VMware Validated Design
Maximum number of virtual machines and By using the SDDC Manager API in VMware Cloud Foundation, you can
churn rate deploy a VMware vCenter Server™ appliance of a specified deployment
and storage size. As a result, in this VMware Validated Design, you
determine the maximum number of virtual machines in the SDDC
according to a medium-size vCenter Server deployment specification or
larger.
n 4,000 running virtual machines per virtual infrastructure workload
domain
n 56,000 running virtual machines overall distributed across 14 virtual
infrastructure workload domains
n Churn rate of 750 virtual machines per hour
Number of workload domains in a region Minimum two-domain setup, with a minimum of 4 VMware ESXi™ hosts
in a domain
The validated design requires the following workload domains for SDDC
deployment:
n Management domain. Contains the appliances of the SDDC
management components.
n One or more solution-specific workload domains for Infrastructure-
as-a-Service (IaaS) and containers. Up to 14 workload domains per
region.
n Contains the tenant workloads.
n Contains the required SDDC services to enable the solution that
is deployed.
See Chapter 4 Workload Domains in VMware Validated Design.
Shared use of components for This VMware Validated Design uses a dedicated NSX-T Manager cluster
management of workload domains for each workload domain.
Data center virtualization Maximized workload flexibility and limited dependencies on static
data center infrastructure by using compute, storage, and network
virtualization.
VMware, Inc. 11
Introducing VMware Validated Design
Authentication, authorization, and access n Use of Microsoft Active Directory as the identity provider.
control n Use of service accounts with least privilege role-based access
control for solution integration.
Certificate signing Certificates are signed by an external certificate authority (CA) that
consists of a root and intermediate authority layers.
Hardening Tenant workload traffic can be separated from the management traffic.
VMware, Inc. 12
Workload Domains in VMware
Validated Design 4
In VMware Validated Design, a workload domain represents a logical unit that groups ESXi hosts
managed by a vCenter Server instance with specific characteristics according to VMware SDDC
best practices.
A workload domain exists in the boundaries of an SDDC region. A region can contain one or
more domains. A workload domain cannot span multiple regions.
n At least one vSphere cluster with vSphere HA and vSphere DRS enabled. See Cluster Types.
n One vSphere Distributed Switch per cluster for system traffic and segments in VMware NSX-
T Data Center™ for workloads.
n One NSX-T Manager cluster for configuring and implementing software-defined networking.
n One NSX-T Edge cluster that connects the workloads in the domain for logical switching,
logical dynamic routing, and load balancing.
n In either of the two regions in a multi-region SDDC, one NSX-T Global Manager cluster for
configuring software-defined networks that span multiple regions
Management Domain
Contains the SDDC management components.
Feature Description
VMware, Inc. 13
Introducing VMware Validated Design
Feature Description
Virtual switch type n vSphere Distributed Switch for system traffic and
NSX-T network segments
n NSX-T Virtual Distributed Switch (N-VDS) on the NSX-
T Edge nodes
NSX-T Edge cluster for north-south routing, east-west First cluster in the domain
routing, and load balancing
NSX-T Global Manager cluster for global networking First cluster in the domain
across multiple regions
Region-specific Workspace ONE Access for central role- First cluster in the domain
based access control
Feature Description
VMware, Inc. 14
Introducing VMware Validated Design
Feature Description
Virtual switch type n vSphere Distributed Switch for system traffic from
the management domain and for NSX-T network
segments
n N-VDS on the NSX-T Edge nodes in the workload
domain
Shared storage type vSAN, vVols, NFS, or VMFS on FC for principal storage
NSX-T Manager cluster First cluster in the management n ✓ for workload domains where workloads
domain share the same overlay transport
zone cross-domain, including domains
where you use vRealize Automation for
workload provisioning
NSX-T Edge cluster for north- Shared edge and workload n ✓ for workload domains where workloads
south and east-west routing cluster in the workload domain share the same overlay transport
zone cross-domain, including domains
where you use vRealize Automation for
workload provisioning
VMware, Inc. 15
Introducing VMware Validated Design
Feature Description
Virtual switch type n vSphere Distributed Switch for system traffic from
the management domain and for NSX-T network
segments
n N-VDS on the NSX-T Edge nodes in the workload
domain
Shared storage type vSAN, vVols, NFS, or VMFS on FC for principal storage
Deployment method You use SDDC Manager for environment validation and
the vSphere Client for enabling vSphere with Tanzu
Table 4-6. Management Workloads for a vSphere with Tanzu Workload Domain
NSX-T Manager cluster First cluster in the management n ✓for workload domains where workloads
domain share the same overlay transport
zone cross-domain, including domains
where you use vRealize Automation for
workload provisioning
VMware, Inc. 16
Introducing VMware Validated Design
Table 4-6. Management Workloads for a vSphere with Tanzu Workload Domain (continued)
NSX-T Edge cluster for north- Shared edge and workload n ✓ for workload domains where workloads
south and east-west routing cluster share the same overlay transport
zone cross-domain, including domains
where you use vRealize Automation for
workload provisionin
VMware, Inc. 17
SDDC Deployment Flow of
VMware Validated Design 5
Тhe deployment of the SDDC is automated. You use VMware Cloud Builder in VMware
Cloud Foundation to deploy the SDDC management domain, SDDC Manager in VMware Cloud
®
Foundation to deploy workload domains for tenant workloads, and VMware vRealize Suite
Lifecycle Manager™ in VMware Cloud Foundation mode to deploy the vRealize Suite products in
this design. You deploy SDDC management components manually only in a few cases according
to the instructions.
In VMware Validated Design 6.2, you can deploy an SDDC in a single-region or in a dual-region
configuration. To design your SDDC in the second region (Region B), you apply the design
guidance for a single region, modifying configurations for a single region to accommodate a
dual-region setup or introducing configurations specific to a dual-region SDDC.
For more details on the deployment steps, see VMware Validated Design documentation page.
VMware, Inc. 18
Introducing VMware Validated Design
3.2. vRealize Suite Lifecycle Manager vRealize Operations Manager vRealize Operations Manager
deploys the vRealize Suite products
deployment flow
in a workload domain
1.3 User deploys Region-Specific Region-Specific Workspace ONE Access Region-Specific Workspace ONE Access
Workspace ONE Access
SDDC Manager
1.1 User installs ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi 2.1 User installs ESXi
on the domain hosts on the domain hosts
VMware, Inc. 19
Introducing VMware Validated Design
Figure 5-2. SDDC Deployment Workflow with a vSphere with Tanzu Workload Domain
deployment flow
in a workload domain
1.3 User deploys Region-Specific Region-Specific Workspace ONE Access Region-Specific Workspace ONE Access
Workspace ONE Access
SDDC Manager
1.2 Cloud Builder deploys NSX-T Data Center NSX-T Data Center
virtual infrastructure and
SDDC Manager
2.2 SDDC Manager deploys
vCenter Server vCenter Server
virtual infrastructure
1.1 User installs ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi 2.1 User installs ESXi
on the domain hosts on the domain hosts
Stage Steps
Plan and prepare for Prepare the data center and fill in the environment specification.
SDDC deployment Work with the technology team of your organization on configuring the physical servers,
network, and storage in the data center. Collect the environment details and write them
® ®
down in the Planning and Preparation Workbook in Microsoft Excel spreadsheet format
(XLS).
1. Deploy the See VMware Validated Design Deployment of the Management Domain.
management domain of
the SDDC. Prerequisites Prepare the deployment specification of the management
domain.
Download the deployment parameter workbook from My
VMware and fill in the details for the management domain
deployment. You can use the details from the Planning and
Preparation Workbook.
1. Prepare the environment Install and configure ESXi on the physical servers.
for the management
domain
VMware, Inc. 20
Introducing VMware Validated Design
Table 5-1. Deployment Flow for an SDDC with a Single Region (continued)
Stage Steps
3. Deploy manually the Deploy the region-specific Workspace ONE Access instance
region-specific Workspace from an OVA file by using the vSphere Client, connect it to
ONE Access instance the Active Directory domain, and connect the management
domain components to the region-specific Workspace ONE
Access instance.
2. Deploy a virtual See VMware Validated Design Deployment of a Virtual Infrastructure Workload Domain
infrastructure workload and VMware Validated Design Deployment of a vSphere with Tanzu Workload Domain.
domain or vSphere
1. Prepare the environment Install and configure ESXi on the physical servers. Create a
with Tanzu workload
for the workload domain. network pool for the workload domain, and upload product
domain.
license keys.
3. Connect manually the Connect the management components for the workload
region-specific Workspace domain to the region-specific Workspace ONE Access
ONE Access instance to instance.
the workload domain
VMware, Inc. 21
Introducing VMware Validated Design
Table 5-1. Deployment Flow for an SDDC with a Single Region (continued)
Stage Steps
4. For a vSphere with Validate the domain configuration by using SDDC Manager
Tanzu workload domain, and enable vSphere with Tanzu by using the vSphere
enable vSphere with Client. Then, you can deploy applications or provision Tanzu
Tanzu. Kubernetes clusters on the initial Supervisor Cluster.
3. Deploy the solutions See VMware Validated Design Deployment of Cloud Operations and Automation .
for cloud operations
and automation. 1. Deploy VMware vRealize By using SDDC Manager, download the vRealize Suite
Suite Lifecycle Manager in Lifecycle Manager install bundle and deploy vRealize Suite
VMware Cloud Foundation Lifecycle Manager.
mode. SDDC Manager provides inventory information about the
management domain in vRealize Suite Lifecycle Manager.
SDDC Manager also configures the NSX-T Tier 1 gateway to
support the load balancer for the cross-region solutions.
2. Deploy the solutions. Import the product binaries as software install bundles in
SDDC Manager, synchronize them in vRealize Suite Lifecycle
Manager, and deploy the solutions.
VMware, Inc. 22
Introducing VMware Validated Design
Table 5-1. Deployment Flow for an SDDC with a Single Region (continued)
Stage Steps
3. Connect the solutions to As a result from the integration between vRealize Suite
the management domain. Lifecycle Manager and SDDC Manager, vRealize Suite
Lifecycle Manager calls SDDC Manager to perform the
following operations during the automated deployment of the
vRealize Suite products:
n Configures the NSX-T load balancer that is required
for the cross-region Workspace ONE Access instance,
vRealize Operations Manager, and vRealize Automation.
n Connects the vRealize Suite components to each other.
®
n Connects VMware vRealize Operations Manager™ and
®
VMware vRealize Log Insight™ to the management
domain vCenter Server and the principal vSAN datastore.
n Connects vRealize Log Insight to the NSX-T instance for
the management domain.
You connect manually the following components for the
management domain:
n vRealize Suite products to the region-specific Workspace
ONE Access
n vRealize Operations Manager to the NSX-T instance for
the management domain
®
n VMware vRealize Automation™ to vRealize Operations
Manager
n NSX-T Edge nodes for the management and vRealize
Suite Lifecycle Manager to vRealize Log Insight
4. Connect the solutions to After you deploy vRealize Operations Manager and vRealize
the workload domain. Log Insight, use SDDC Manager to integrate them with the
virtual infrastructure of the workload domain.
You connect manually the following components for the
workload domain:
n vRealize Operations Manager to the NSX-T instance for
the workload domain
n vRealize Automation to the workload domain vCenter
Server and NSX-T instance
n NSX-T Edge nodes for the workload domain and vRealize
Suite Lifecycle Manager to vRealize Log Insight
VMware, Inc. 23
Introducing VMware Validated Design
3. Cloud Operations
and Cloud Automation 3.1 vRealize Suite integration for 3.2 vRealize Suite integration for
Solution the first region the second region
2.3. User deploys NSX-T Global Manager instances for the virtual
infrastructure workload domains
2. Virtual
Infrastructure
Workload Domains
2.1. Virtual Infrastructure 2.2. Virtual Infrastructure
Workload Domain Workload Domain
deployment flow
in a region
1.3. User deploys NSX-T Global Manager instances for the management domains
1. Management
Domains
Region A Region B
Stage Steps
Plan and prepare for SDDC Prepare the data center and fill in the environment specification for both
deployment regions in the Planning and Preparation Workbook.
1. Deploy the management domain of See Deployment of the Management Domain in the First Region and
the SDDC in each region. Deployment of the Management Domain in the Second Region.
VMware, Inc. 24
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
VMware, Inc. 25
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
2. Deploy the virtual infrastructure See Deployment of a Virtual Infrastructure Workload Domain in the First
workload domain in each region. Region and Deployment of a Virtual Infrastructure Workload Domain in the
Second Region.
1. Deploy the workload domain in the first region by following the workflow for
a single-region SDDC.
VMware, Inc. 26
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
VMware, Inc. 27
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
3. Deploy the solutions for cloud See Deployment of Cloud Operations and Automation.
operations and automation
VMware, Inc. 28
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
VMware, Inc. 29
Introducing VMware Validated Design
Table 5-2. Deployment Flow for an SDDC with Two Regions (continued)
Stage Steps
VMware, Inc. 30
Documentation Structure and
Audience 6
The structure of the VMware Validated Design documentation reflects the best practices in
designing and deploying a data center that is capable of automated workload provisioning. The
documentation components of the validated design are organized according to the audience and
deployment stage.
For information on the order in which you deploy the SDDC, see Chapter 5 SDDC Deployment
Flow of VMware Validated Design.
VMware, Inc. 31
Introducing VMware Validated Design
For details on the latest available documentation, see VMware Validated Design documentation
page.
Architecture Overview
The first part of a VMware Validated Design is Architecture Overview and it introduces the terms
and components in the design.
Detailed Design
After you learn about the basic modules in the SDDC design, you proceed with detailed design of
the management components and the required infrastructure.
VMware, Inc. 32
Introducing VMware Validated Design
VMware, Inc. 33
Introducing VMware Validated Design
Deployment
After you make sure that your environment has the required structure and configuration, follow
the Deployment in the First Region documentation to start the SDDC implementation in the first
region.
VMware, Inc. 34
Introducing VMware Validated Design
Deployment of Region B
After you make sure that your environment has the required structure and configuration, follow
the Deployment in the Second Region documents to start the SDDC implementation in the
second region.
VMware, Inc. 35
Introducing VMware Validated Design
VMware, Inc. 36
Post-Deployment Documentation
7
VMware Validated Design provides several types of documentation for operating, maintaining,
extending, and modifying a deployed SDDC. This documentation is delivered as a set of add-on
packages that could be asynchronously published.
For details on the latest available documentation, see VMware Validated Design documentation
page.
Operational Guidance
The operational guidance in VMware Validated Design provides a prescriptive guidance on the
common operations that you perform after the SDDC implementation is completed.
VMware, Inc. 37
SDDC Architecture Overview
8
SDDC layers represent aggregations of logically related functionality and operations in your
environment. In a layer, you can interchange components as part of the end solution or outcome.
If a particular component design does not fit the business or technical requirements, you can
replace it with another similar component.
Orchestration
Virtual Hypervisor
Infrastructure Identity and Access
Monitoring Fault Tolerance
Management
Pools of Resources
Virtualization Control
Backup &
Logging Industry Regulations
Restore
Physical Compute
Infrastructure
Storage
Life Cycle Security Policies
Network Management
VMware, Inc. 38
Introducing VMware Validated Design
Physical Consists of the compute, network, and storage components. The compute component contains
Infrastructure the x86-based servers that run the management components, NSX-T Edge nodes, and tenant
Layer workloads. This validated design provides only some guidance about the physical capabilities
that are required to implement this architecture. You select a specific type or brand of hardware
according to VMware Compatibility Guide.
The physical infrastructure layer configuration is part of the implementation of the SDDC
management domain and workload domains.
Virtual Controls the access to the underlying physical infrastructure and allocates resources to the
Infrastructure management and tenant workloads. The management workloads consist of elements in the virtual
Layer infrastructure layer itself, together with elements in the cloud operations, cloud automation, and
security and compliance layers.
The virtual infrastructure layer groups physical infrastructure in pools of resources such as
workload domains and clusters. See Chapter 4 Workload Domains in VMware Validated Design.
The virtual infrastructure layer configuration is part of the implementation of the SDDC
management domain and workload domains.
Cloud Operations Provides operations management for continuous day-to-day service delivery. Cloud operations
Layer management consists of life cycle management, monitoring, logging, and other operation types.
The architecture of the cloud operations layer includes management components that support
the main types of operations in an SDDC. You monitor the underlying physical infrastructure, and
the management and tenant or containerized workloads in real time. Information is collected in
the form of structured data (metrics) and unstructured data (logs). The cloud operations layer
also collects data about the SDDC topology, that is physical and virtual compute, networking, and
storage resources, which are key in intelligent and dynamic operational management.
The cloud operations layer configuration is part of the implementation of the SDDC management
domain and workload domains, and of the solutions for cloud operations and automation.
Cloud Automation Requests resources and orchestrates the actions of the lower layers from a user interface or over
Layer an API.
The cloud automation layer configuration is part of the implementation of the SDDC solutions for
cloud operations and automation.
Security and n Incorporates security guidance from NIST 800-53 across the VMware Validated Design to
Compliance Layer establish a baseline of security.
n Identifies and implements security best practices from setup to operations to secure your
SDDC, and make it more resilient to internal and external threats.
n Provides role-based access control by implementing an identity and access management
solution which integrates with Microsoft Active Directory.
VMware, Inc. 39
Introducing VMware Validated Design
Region-
SDDC Specific vSphere with Tanzu vSphere with Tanzu
Manager Workspace
ONE Access
NSX-T NSX-T
(1:1 or 1:N) (1:1 or 1:N)
NSX-T
vCenter Server vCenter Server
vCenter Server
Principal Storage Principal Storage
Principal Storage (vSAN, vVols, NFS, or VMFS on FC) (vSAN, vVols, NFS, or VMFS on FC)
(vSAN)
Consolidated SDDC
Architecture
The SDDC layers are gradually implemented as you follow the implementation of the SDDC.
1 To provide the physical and virtual infrastructure, and local identity and access management
for the SDDC management components, implement the management domain.
2 To provide the physical and virtual infrastructure for the virtualized or containerized
workloads, implement one or more workload domains.
3 To operate the SDDC and deploy workloads on the workload domains, implement the
solutions for cloud operations and automation including identity and access management
for these solutions.
For information about the design and deployment of each layer at each deployment stage, see
the VMware Validated Design documentation page.
VMware, Inc. 40
Introducing VMware Validated Design
VMware, Inc. 41
Introducing VMware Validated Design
External
connection
Management
cluster
(4 ESXi hosts)
Workload cluster
(19 ESXi host each)
Workload Domains
The compute, storage, and network resources are organized in workload domains. The physical
layer also includes the physical network infrastructure, and storage setup. For information on
workload domains and clusters, see Chapter 4 Workload Domains in VMware Validated Design.
Compute
The physical compute resources are delivered through ESXi, a bare-metal hypervisor that installs
directly onto your physical server. With direct access and control of underlying resources, ESXi
logically partitions hardware to consolidate applications and cut costs. ESXi is the base building
block of the Software-Defined Data Center.
Network
VMware Validated Design can use most physical network architectures. When building an SDDC,
the following considerations exist:
n A Top of Rack (ToR) switch is typically located inside a rack and provides network access
to the servers inside that rack.
VMware, Inc. 42
Introducing VMware Validated Design
n An inter-rack switch at the aggregation layer provides connectivity between racks. Links
between inter-rack switches are typically not required. If a link failure between an inter-
rack switch and a ToR switch occurs, the routing protocol ensures that no traffic is sent to
the inter-rack switch that has lost connectivity.
n Using quality of service tags for prioritized traffic handling on the network devices
VMware vSphere® Distributed Switch supports several NIC teaming options. Load-based
NIC teaming supports an optimal use of available bandwidth and redundancy if a link
failure occurs. Use a minimum of two 10-GbE connections, with two 25-GbE connections
recommended, for each ESXi host in combination with a pair of top of rack switches.
802.1Q network trunks can support as many VLANs as required. For example, management,
storage, overlay, and VMware vSphere® vMotion® traffic.
Because of the considerations for the physical network architecture, providing a robust physical
network to support the physical-to-virtual network abstraction is an important requirement of
network virtualization.
Represent the fault domain of the SDDC. Multiple availability zones can provide continuous
availability of an SDDC. This VMware Validated Design supports one availability zone per
region. See Multiple Availability Zones.
Region
Each region is a separate SDDC instance. You use multiple regions for disaster recovery
across individual SDDC instances.
In this VMware Validated Design, regions have similar physical and virtual infrastructure
design but different naming.
VMware, Inc. 43
Introducing VMware Validated Design
Storage
This VMware Validated Design provides guidance for the storage of the management
components. A shared storage system not only hosts the management and tenant or container
workloads, but also template repositories and backup locations. Storage within an SDDC can
include either or both internal and external storage as either principal or supplemental storage.
For the management domain, this validated design includes internal storage by using vSAN for
principal storage and external NFS storage for supplemental storage.
Internal Storage
vSAN is a software-based distributed storage platform that combines the internal compute
and storage resources of clustered VMware ESXi hosts. By using storage policies on a
cluster, you configure multiple copies of the data. As a result, this data is accessible during
maintenance and host outages.
External Storage
External storage provides non-vSAN storage by using NFS, iSCSI, or Fiber Channel. Different
types of storage can provide different levels of SLA, ranging from just a bunch of disks
(JBODs) using SATA drives with minimal to no redundancy, to fully redundant enterprise-
class storage arrays.
Principal Storage
VMware vSAN™ storage is the default storage type for the SDDC management components.
All design, deployment, and operational guidance are performed on vSAN. Considering block
or file storage technology for principal storage is out of scope of the design. These storage
technologies are referenced only for specific use cases such as backups to supplemental
storage.
The storage devices on vSAN ready servers provide the storage infrastructure. This validated
design uses vSAN in an all-flash configuration.
For workloads in workload domains, you can use vSAN, vVols, NFS, and VMFS on FC.
Supplemental Storage
NFS storage is the supplemental storage for the SDDC management components. It provides
space for archiving log data and application templates.
Supplemental storage provides additional storage for backup of the SDDC. It can use the
NFS, iSCSI, or Fibre Channel technology. Different types of stage can provide different levels
of SLA, ranging from JBODs with minimal to no redundancy, to fully redundant enterprise-
class storage arrays. For bandwidth-intense IP-based storage, the bandwidth of these pods
can scale dynamically.
VMware, Inc. 44
Introducing VMware Validated Design
Cluster Types
This VMware Validated Design uses the following types of clusters:
Management Workloads
vSAN
Management Cluster
VMware, Inc. 45
Introducing VMware Validated Design
Figure 8-5. Shared Edge and Workload Cluster in a Virtual Infrastructure Workload Domain
vSAN
Workload Domain
vCenter Server
Resides in the management domain and runs the virtual machines of the components
that manage the data center, such as vCenter Server, NSX-T Manager, SDDC Manager,
® ®
Workspace ONE Access, VMware vRealize Suite Lifecycle Manager™, VMware vRealize
®
Operations Manager™, VMware vRealize Log Insight™, vRealize Automation, and other
management components.
Represents the first cluster in the virtual infrastructure workload domain and runs the
required NSX-T services for north-south routing between the data center and the external
network, and east-west routing inside the data center. This shared cluster also hosts the
tenant workloads. As you extend your environment, you must add workload-only clusters.
Workload Cluster
VMware, Inc. 46
Introducing VMware Validated Design
Resides in a virtual infrastructure workload domain and runs tenant workloads . Use workload
clusters to support a mix of different types of workloads for different types of Service Level
Agreements (SLAs). You can mix different types of workload clusters and provide separate
compute pools for different types of SLAs.
Management Cluster Shared Edge and Management Cluster Shared Edge and
Workload Cluster Workload Cluster
Management Workload Management Workload
Domain Domain Domain Domain
vCenter vCenter vCenter vCenter
Server Server APP APP APP Server Server APP APP APP
OS OS OS OS OS OS
ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi
vCenter Server instances You deploy vCenter Server instances in the following way:
n One vCenter Server instance for the management
domain.
n One vCenter Server instance for each workload
domain.
Using this model provides the following benefits:
n Isolation of management domain vCenter Server and
workload domain vCenter Server
n Simplified capacity planning
n Separated upgrade
n Separated roles
VMware, Inc. 47
Introducing VMware Validated Design
Resource pools for tenant workloads and dedicated NSX On the shared edge and workload cluster in a workload
components domain, you use resource pools to distribute compute
and storage resources to the tenant or container
workloads, and the NSX-T components carrying their
traffic.
Virtual network segments are created on the vSphere Distributed Switch for the first cluster in
the management domain and for the shared edge and workload cluster in a workload domain.
VMware, Inc. 48
Introducing VMware Validated Design
Region A
ToR
Switches
Region A
BGP ASN
Tier-0
Gateway
SR SR
SDDC
BGP ASN DR DR
SR SR
Tier-1 DR DR DR DR DR DR
Gateway
NSX-T NSX-T ESXi ESXi ESXi ESXi
Edge Edge Host 1 Host 2 Host 3 Host 4
Node 1 Node 2
VM VM VM
VMware, Inc. 49
Introducing VMware Validated Design
n Tier-0 gateway with ECMP enabled for north-south routing across all regions
You apply the no-export BGP community to all routes learned from external neighbors.
Because the NSX-T SDN in the first and second regions does not have an independent path
between those autonomous systems, re-advertising data center networks would give a false
indication of a valid, independent path.
VMware, Inc. 50
Introducing VMware Validated Design
Virtual network segments provide support for limited access to the nodes of the applications
through published access points.
VC SDDC Mgr
OS OS
ToR Switches
ECMP
Tier-1 Gateway
192.168.11.0/24 192.168.31.0/24
xreg-m01-seg01 sfo-m01-seg01
n Cross-region virtual network segment that connects the components that are designed to fail
over to a recovery region.
n Region-specific virtual network segment in Region A for components that are not designed to
fail over.
VMware, Inc. 51
Introducing VMware Validated Design
n Region-specific application virtual network in Region B for components that are not designed
to fail over.
Applications store their data according to the default storage policy for vSAN.
vRealize Log Insight uses NFS exports as supplemental storage for log archiving.
Tenant 1
Software-Defined Storage vSAN, vVols, NFS, or VMFS on FC vVols, NFS, VMFS on iSCSI, and VMFS on FC
vSAN NAS
VMware, Inc. 52
Introducing VMware Validated Design
For information on the account configuration in Active Directory and local accounts, see Planning
and Preparation Workbook.
Figure 8-11. Cross-Region and Region-Specific Workspace ONE Access Deployments in Region A
Region A
Identity Provider
Directory Services
e.g. AD, LDAP
Access Access
Cross-Region Region-Specific
Workspace Workspace
ONE Access ONE Access
Virtual
Primary Secondary Secondary
Appliance
vRealize Operations
vRealize Log Insight
Manager
vRealize Automation
VMware, Inc. 53
Introducing VMware Validated Design
Figure 8-12. Logical Design of the Region-Specific Workspace ONE Access Deployment
Region A Region B
Access Access
Region-Specific Region-Specific
Workspace Workspace
ONE Access ONE Access
Virtual Virtual
Appliance Appliance
NSX-T NSX-T
Data Center Data Center
VMware, Inc. 54
Introducing VMware Validated Design
Identity and access management setup n Integration with the rainpole.io Active Directory
domain.
n Directory Service connection is Active Directory with
Integrated Windows Authentication
Identity and access management setup n Integration with the rainpole.io Active Directory
domain.
n Directory Service connection is Active Directory with
Integrated Windows Authentication
VMware, Inc. 55
Introducing VMware Validated Design
SDDC Manager
You use SDDC Manager in VMware Cloud Foundation to perform the following operations:
n Deploy virtual infrastructure workload domains and extend the virtual infrastructure of the
management domain.
n Manage the life cycle of the virtual infrastructure components in all workload domains, and of
vRealize Suite Lifecycle Manager.
Region A Region B
External Services Access Identity Source Identity Source Access External Services
vCenter vCenter
Server Server
vRealize
Suite Lifecycle
Manager
VMware, Inc. 56
Introducing VMware Validated Design
Setup for workload domain and product deployment n Direct integration with My VMware to access install
and upgrade bundles
n Configuration with an external certificate authority
for replacing the certificates of the management
components in the SDDC
Starting with VMware Cloud Foundation 4.1, vRealize Suite Lifecycle Manager is deployed in
VMware Cloud Foundation mode. In this mode, vRealize Suite Lifecycle Manager and VMware
Cloud Foundation are integrated for inventory synchronization, life cycle management of the
vRealize Suite products, and workload domain integration.
VMware, Inc. 57
Introducing VMware Validated Design
Region A
Identity Management
Cross-Region
Access Workspace
ONE Access
User Interface Integration
SDDC
REST API Manager
vRealize Suite
Lifecycle Manager
in VMware Cloud
Foundation Mode
Life Cycle Management Endpoint
vRealize vCenter
Operations Server
Manager
vRealize
Log Insight
vRealize
Automation Shared
Storage
Cross-Region
Workspace
ONE Access
Region B
Life Cycle Management
vRealize
Operations
Manager
Collectors
vRealize
Log Insight
VMware, Inc. 58
Introducing VMware Validated Design
Deployment model One appliance that deploys and upgrades the vRealize
Suite components on a virtual infrastructure that is
controlled by the management domain vCenter Server.
The appliance is deployed by using SDDC Manager.
VMware, Inc. 59
Introducing VMware Validated Design
vRealize Operations Manager is also sized to accommodate the number of tenant workloads
according to the design objectives.
Region A
Access
vCenter
Remote
Server
Primary Replica Collector 1
Storage
Devices
Remote
Integrations
Data 1 Data n Collector 2
vRealize Workspace
Automation ONE Access
Management Management
Packs Packs
vRealize Additional
Log Insight Solutions
Supporting Supporting
Infrastructure, Infrastructure,
shared Storage, shared Storage,
AD, DNS, NTP AD, DNS, NTP
SMTP SMTP
Management
Packs Workspace
ONE Access
Suite API
Additional
Solutions
VMware, Inc. 60
Introducing VMware Validated Design
VMware, Inc. 61
Introducing VMware Validated Design
Region A Region B
Region-Specific Region-Specific
Workspace Access Access Workspace
ONE Access ONE Access
User Interface User Interface
vRealize
Operations
Manager Integration
vRealize vRealize
Log Insight Log Insight
vSphere vSphere
Integrated Integrated
Load Load
Balancer Balancer
Logging Clients Logging Clients
VMware, Inc. 62
Introducing VMware Validated Design
Deployment model Cluster of primary node and two worker nodes with an
integrated load balancer in each region. The vRealize
Log Insight nodes are deployed by using vRealize Suite
Lifecycle Manager.
VMware, Inc. 63
Introducing VMware Validated Design
Region A
User Interface
Cross-Region
Workspace
ONE Access
API
Integration Accounts
SDDC
Manager
Private Cloud Accounts NSX-T Data Center
Load Balancer
vRealize
Orchestrator
VMware Cloud vRealize
Foundation Automation Cluster
vRealize
Operations
Manager
vCenter Server
My VMware
NSX-T Supporting Components:
Data Center Kubernetes, Docker, Postgres,
FaaS, Traefik, Flannel, Fluentd GitHub, GitLab,
and BitBucket
Supporting Infrastructure:
Shared Storage,
Additional
AD, DNS, NTP, SMTP
Solutions
e.g. IPAM, K8s,
Terraform,
Ansible, Puppet
vCenter Server
NSX-T
Data Center
VMware, Inc. 64
Introducing VMware Validated Design
Service Broker
Administrator Authoring
Administration of Services Authoring
cloud resources Cloud Zones
Deployment model of vRealize Automation A cluster of three vRealize Automation nodes with a load
balancer. The cluster is deployed by using vRealize Suite
Lifecycle Manager.
VMware, Inc. 65
Introducing VMware Validated Design
In a stretched cluster configuration, both availability zones are active. If a failure in either
availability zone occurs, the virtual machines are restarted in the operational availability zone
because virtual machine writes occur to both availability zones synchronously.
Extending the management cluster to a vSAN stretched cluster provides the following
advantages:
Using a vSAN stretched cluster for the management components has the following
disadvantages:
n Increased footprint
VMware, Inc. 66
Introducing VMware Validated Design
n Licensing requirements
Physical Infrastructure
You must use homogenous physical servers between availability zones. You replicate the hosts
for the first cluster in the management domain and shared edge and workload cluster in a
workload domain, and you place them in the same rack.
Region A Region B
VMware, Inc. 67
Introducing VMware Validated Design
You can start deploying the SDDC in a single availability zone configuration, and then extend the
environment with the second availability zone.
Figure 8-20. vSphere Logical Cluster Layout for Multiple Availability Zones for the Management
Domain
Management Cluster Shared Edge and Management Cluster Shared Edge and
Workload Cluster Workload Cluster
Management Workload Management Workload
Domain Domain Domain Domain
vCenter vCenter vCenter vCenter
Server Server APP APP APP Server Server APP APP APP
OS OS OS OS OS OS
ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi
Network Configuration
NSX-T Edge nodes connect to top of rack switches in each data center to support northbound
uplinks and route peering for SDN network advertisement. This connection is specific to the top
of rack switch that you are connected to.
VMware, Inc. 68
Introducing VMware Validated Design
Uplink VLAN 2
Tier-0 SR SR
Gateway
DR DR
SDDC BGP ASN
SR SR
DR DR DR DR DR DR Tier-1 DR DR DR DR
Gateway
ESXi ESXi ESXi ESXi NSX-T NSX-T ESXi ESXi ESXi ESXi
Host 1 Host 2 Host 3 Host 4 Edge Edge Host 1 Host 2 Host 3 Host 4
Node 1 Node 2
VM VM VM
If an outage of an availability zone occurs, vSphere HA fails over the edge appliances to the other
availability zone by using vSphere HA. Availability Zone 2 must provide an analog of the network
infrastructure which the edge node is connected to in Availability Zone 1.
The management network in the primary availability zone, and the Uplink 01, Uplink 02, and Edge
Overlay networks in each availability zone must be stretched to facilitate failover of the NSX-T
Edge appliances between availability zones. The Layer 3 gateway for the management and Edge
Overlay networks must be highly available across the availability zones.
The network between the availability zones should support jumbo frames and its latency must be
less than 5 ms. Use a 25-GbE connection with vSAN for best and predictable performance (IOPS)
of the environment.
Uplink01 x
Uplink02 x
Edge overlay ✓
VMware, Inc. 69
Introducing VMware Validated Design
Witness Appliance
When using two availability zones, deploy a vSAN witness appliance in a location that is not local
to the ESXi hosts in any of the availability zones.
VMware Validated Design uses vSAN witness traffic separation where you can use a VMkernel
adapter for vSAN witness traffic that is different from the adapter for vSAN data traffic. In this
design, you configure vSAN witness traffic in the following way:
n On each management ESXi host in both availability zones, the vSAN witness traffic is placed
on the management VMkernel adapter.
n On the vSAN witness appliance, you use the same VMkernel adapter for both management
and witness traffic.
Witness Site
Physical
Upstream
Router
Witness
Appliance
VLAN: lax-m01-cl01-vds01-pg-mgmt
VLAN: sfo-m01-cl01-vds01-pg-mgmt
VLAN: az2_sfo-m01-cl01-vds01-pg-mgmt
VMware, Inc. 70
Introducing VMware Validated Design
Figure 8-23. vSAN Witness Network Design in a Virtual Infrastructure Workload Domain
Witness Site
Physical
Upstream
Router
Witness
Appliance
VLAN: lax-m01-cl01-vds01-pg-mgmt
VLAN: sfo-m01-cl01-vds01-pg-mgmt
VLAN: sfo-w01-cl01-vds01-pg-mgmt
VLAN: az2_sfo-w01-cl01-vds01-pg-mgmt
VMware, Inc. 71