Nuage 210 WBX Software Installation Guide

210 WBX Software Installation Guide
Release 20.10.R1
3HE16771AAAA
October 31, 2020

CONTENTS
1 About this Document 2

1.1 Validity of this Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Required Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Overview 5
2.1 210 WBX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 210 WBX 32QSFP28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.2 210 WBX 48SFP28 6QSFP28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Nuage Software Installation and Upgrade 8

3.1 Setting up NOS with ONIE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Initializing the System and Downloading Software . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.2 Installing NOS from the SD Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.3 Installing NOS from the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Boot after the NOS Installation is Complete . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 Automated NOS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3.1 Configuring OS From the SD Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3.2 Configuring NOS From the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3.3 Cloud-init File Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
nuage_wbx cloud-init module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3.4 Example of SD Card Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
cloud-init.cfg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
config.cfg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.3.5 Example of cloud-init Static IP SD Card Configuration . . . . . . . . . . . . . . . . . . . . 15
3.3.6 Example of DHCP plus HTTP Server Configuration . . . . . . . . . . . . . . . . . . . . . . 15
3.4 Reinstalling OS and Software Upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4.1 Persistent Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.5 Uninstalling OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.6 Remote Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.6.1 WBX Upgrade for SROS VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
BoF Pointing to the external FTP Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
BoF Pointing to the Local User Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.6.2 210 WBX Upgrade for Hypervisor and SROS VM . . . . . . . . . . . . . . . . . . . . . . 19
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.7 Ensuring Availability of SROS VM and Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8 SROS CLI Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
i
4 Getting Started with the Nuage 210 WBX 23
4.1 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2 Linux Hypervisor on the Nuage 210 WBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.1 SROS VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2.2 VSG/VSA and 210 WBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 Hypervisor Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.1 Monitoring with Linux Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.2 New MIB Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3.3 Example of use of the MIBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.4 README File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5 Nuage SROS for 210 WBX 39

5.1 Nuage SROS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.1.1 VM Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.1.2 Identifying a 210 WBX in a Busy Data Center . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.1.3 Power Supplies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.2 BOF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.3 Watermarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.3.1 Event Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.4 Dynamic ECMP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.5 210 WBX ACL Scale Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.5.1 Reverting to ACL Global Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Workaround . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.6 VRF Scalability Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.6.1 Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.6.2 Configuring VRF Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.7 210 WBX Control Plane Hardware Queues Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.7.1 Verifying and Clearing CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.8 Quality of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.8.1 Forward Error Correction and Auto Negotiation . . . . . . . . . . . . . . . . . . . . . . . . 62
5.9 Tech Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.9.1 Admin tech-support for the Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.9.2 Admin tech-support for the SROS VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6 Configuration of the 210 WBX Models with Nuage Software 66

6.1 210 WBX - 32QSFP28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.1.1 Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.1.2 Cards and MDAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.1.3 Port Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.1.4 LAG-97 and LAG-98 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.2 210 WBX - 48SFP28 6QSFP28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.2.1 Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.2.2 Cards and MDAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.2.3 Port Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
25GE Capable Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
100GE Capable Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.2.4 LAG-97 and LAG-98 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
7 Appendix 82
7.1 Installation Trace Example from the SD card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
ii
210 WBX Software Installation Guide, Release 20.10.R1
Release: 20.10.R1
Issue: 1
Issue Date: October 31, 2020
Document Number: 3HE16771AAAA
NUAGE NETWORKS – PROPRIETARY & CONFIDENTIAL
This document contains proprietary/trade secret information which is the property of Nokia Corporation. Not to be
made available to, or copied or used by anyone who is not an employee of Nokia Corporation except when there is a
valid non-disclosure agreement in place which covers such information and contains appropriate non-disclosure and
limited use obligations.
This document is protected by copyright. Except as specifically permitted herein, no portion of the provided infor-
mation can be reproduced in any form, or by any means, without prior written permission from Nokia Corporation /
Nuage Networks.
Nuage Networks and the Nuage Networks logo are trademarks of the Nokia group of companies. Nokia is a registered
trademark of Nokia Corporation. Other product and company names mentioned herein may be trademarks or trade
names of their respective owners.
The information presented is subject to change without notice.
Nokia Corporation / Nuage Networks assumes no responsibility for inaccuracies contained herein.
Copyright©2018 Nokia Corporation / Nuage Networks. All rights reserved.
Build Number: 63
CONTENTS 1
CHAPTER
ONE
ABOUT THIS DOCUMENT
• Validity of this Document (page 3)

• Audience (page 3)
• Technical Support (page 3)
• Required Documents (page 3)
2
For a complete list of applicable user documentation, see the Technical Publications section of the Release Notes for
your Nuage Networks software version.
Validity of this Document
Printed versions of this document may not be up to date. Only the Web version of this document is current.
Audience
This manual is intended for enterprise system administrators who are responsible for enterprise network configuration
and administrators for the Nuage VSP/VNS software. It is assumed that the reader is familiar with virtualization and
networking technologies. Other assumptions are explicitly called out in the relevant chapters.
Technical Support
If you purchased a service agreement for your Nuage Networks VSP/VNS solution and related products from a dis-
tributor or authorized reseller, contact the technical support staff for that distributor or reseller for assistance. If you
purchased an Alcatel-Lucent or Nokia service agreement, contact your welcome center:
https://networks.nokia.com/support
Nokia Online Services (NOLCS) provides registered customers with access to technical support, software downloads,
training, documentation, literature, and other related assets for our products and solutions. For assistance with NOLCS,
including inability to access, contact us as follows:
• Inside the U.S. and Canada: 1-866-582-3688, prompt 7.
• Outside the U.S.: 1-630-224-9000
• Via email: NOLS.support@nokia.com
Required Documents
This guide complements the 210 WBX Chassis Installation Guide and the Nuage VSP User Guide, providing instruc-
tions about how to set up a 210 WBX once it has been physically installed. It includes information about:
• Software installation
• Hypervisor and SROS management and configuration
• SROS details
You are required to use the following documents in conjunction with this guide:
• Nuage VSP Release Notes
• 210 WBX Chassis Installation Guide
• Nuage VSP User Guide
• Nuage 210 WBX User Guides Zipped Collection
1.1. Validity of this Document 3

Note: For all supported features, see the Nuage VSP User Guide.
1.4. Required Documents 4

CHAPTER
TWO
OVERVIEW
• 210 WBX Overview (page 5)

– 210 WBX 32QSFP28 (page 5)
– 210 WBX 48SFP28 6QSFP28 (page 6)
210 WBX Overview
The 210 WBX 32QSFP28 and 210 WBX 48SFP28 6QSFP28 are the next-generation leaf/spine switch for Data Center
and Enterprise networks deployments. 210 WBX offers high density QSFP28/QSFP+/SFP28/SFP+/SFP supporting
native 100GbE, 50GbE, 40GbE, 10GbE and 1GbE ports, and using splitter cables, the possibility to offer 4x25GbE,
2x50GbE, or 4x10GbE breakouts.
210 WBX 32QSFP28
The 210 WBX is a 1U rack-mount or desktop chassis. The following figure shows the front panel and the features of
the 32QSFP28 unit.
Fig. 2.1: 210 WBX 32QSFP28 Front Panel
5
Table 2.1: 210 WBX 32QSFP28

Front Panel Features
Key Description
A SD card slot
B Ethernet QSFP28 ports
C Console port
D Management port
E System LEDs
F USB port
G ESD grounding jack
H Reset button
The following figure shows the rear panel and the features of the 32QSFP28 unit.
Fig. 2.2: 210 WBX 32QSFP28 Rear Panel
Table 2.2: 210 WBX 32QSFP28 Rear

Panel Features
Key Description
A Chassis ground point
B Power tray for PEM
C PEM LED
D Chassis cooling fan modules
210 WBX 48SFP28 6QSFP28
The 210 WBX is a 1U rack-mount or desktop chassis. The following figure shows the front panel and the features of
the 48SFP28 6QSFP28 unit.
Fig. 2.3: 210 WBX 48SFP28 6QSFP28 Front Panel
2.1. 210 WBX Overview 6

Table 2.3: 210 WBX 48SFP28

6QSFP28 Front Panel Features
Key Description
A System power LED
B System Status LED
C System Fan LED
D Ethernet SFP28 ports
E Ethernet QSFP28 ports
F Management port
G Console port
The following figure shows the rear panel and the features of the 48SFP28 6QSFP28 unit.
Fig. 2.4: 210 WBX 48SFP28 6QSFP28 Rear Panel
Table 2.4: 210 WBX 48SFP28 6QSFP28 Rear

Panel Features
Key Description
A Chassis ground point
B Power tray for PEM
C PEM LED
D ESD grounding jack
E Reset button (reserved for future use)
F SD card slot
G USB port
H Chassis cooling fan modules
Note: You are required to use the 210 WBX Chassis Installation Guide to install the 210 WBX router and follow
the site preparation, installation, and other required sections before you can use this guide. This guide only provides
the Nuage software configuration of the 210 WBX after the router is installed and set up.
You are required to use the following documents in conjunction with this guide:
• Nuage VSP Release Notes
• 210 WBX Chassis Installation Guide
• Nuage VSP User Guide
2.1. 210 WBX Overview 7

CHAPTER
THREE
NUAGE SOFTWARE INSTALLATION AND UPGRADE
• Setting up NOS with ONIE (page 9)

• Initializing the System and Downloading Software (page 9)
– Prerequisites (page 9)
– Installing NOS from the SD Card (page 9)
– Installing NOS from the Network (page 11)
* Boot after the NOS Installation is Complete (page 12)

• Automated NOS Configuration (page 12)
– Configuring OS From the SD Card (page 13)
– Configuring NOS From the Network (page 13)
– Cloud-init File Format (page 14)
* nuage_wbx cloud-init module (page 14)

– Example of SD Card Configuration (page 15)
* cloud-init.cfg (page 15)

* config.cfg (page 15)
– Example of cloud-init Static IP SD Card Configuration (page 15)
– Example of DHCP plus HTTP Server Configuration (page 15)
• Reinstalling OS and Software Upgrades (page 16)
– Persistent Partition (page 17)
• Uninstalling OS (page 17)
• Remote Upgrade (page 18)
– WBX Upgrade for SROS VM (page 18)
* BoF Pointing to the external FTP Link (page 18)

* BoF Pointing to the Local User Disk (page 18)
– 210 WBX Upgrade for Hypervisor and SROS VM (page 19)
* Prerequisites (page 19)

* Example (page 20)
8
• Ensuring Availability of SROS VM and Hypervisor (page 20)

• SROS CLI Enhancements (page 21)
Setting up NOS with ONIE
The Nuage 210 WBX installation and upgrade support applies to both SROS VM and hypervisor using the procedures
described by Open Network Install Environment (ONIE).
ONIE is an open source initiative, part of the Open Compute Project. It enables automatic installation of a Network
Operating System (NOS) such as the Nuage 210 WBX software. It provides the following services:
• Installing and reinstalling an OS
• Booting in rescue mode
• Formatting the system
ONIE provides the infrastructure to install NOS Nuage 210 WBX software on the internal 210 WBX disk.
Initializing the System and Downloading Software
Prerequisites
• Complete installation of the 210 WBX router using the information in the 210 WBX Chassis Installation
Guide including all site and safety requirements.
• 210 WBX software license 3HE12204AA OS (210 WBX Nuage RX.0 License)
Note: Nuage recommends using the latest released software, which might be different from that installed in the SD
card. You must contact your Nuage representative to find out about the latest release and the process to download the
supported software image.
Once your system is ready for the software download, complete one of the following processes to initialize the 210
WBX router:
• Installing NOS from the SD card
• Installing NOS from the network to boot the device
Installing NOS from the SD Card
The 210 WBX comes with ONIE preinstalled. Ask your Nuage representative to recommend which software to install
and then complete the following steps:
Step 1 Download the latest software on the Nuage 210 WBX SD card. Nuage WBX SD cards are shipped
for Windows with vfat file system. The 210 WBX ONIE installation from SD card fails when the
SD card is write-protected.
Step 2 Connect to the console management port of the switch (C) marked by a red circle. Do not connect
to the Ethernet management port (D).
3.1. Setting up NOS with ONIE 9

Fig. 3.1: 210 WBX Console Management Port
Step 3 Insert the SD card and power up the system. The system boots up from the internal disk following
the boot, with GRUB. The following screen is displayed when it boots from ONIE. An USB can
also be used instead of the SD card, provided that is formatted as FAT32. The onie-installer-x86_64
file must be in the root otherwise the installation will fail.
Note: Install OS option is the default.
Fig. 3.2: 210 WBX ONIE Bootup
If the auto boot is interrupted for some reason, then select the highlighted Install OS option explicitly. The auto-
boot results in selecting this option automatically, if there is no interruption. The installation begins after ONIE finds
the onie-installer-x86_64 file that exists on the SD card. ONIE auto-mounts the file system to find this file.
An installation trace example looks like the following:
Booting ÒNIE: Install OS'
Welcome to GRUB!
ONIE: OS Install Mode ...
Version : 2016.02
Build Date: 2017-02-08T17:09+0800
3.2. Initializing the System and Downloading Software 10

Info: Mounting kernel filesystems... done.

Info: Mounting ONIE-BOOT on /mnt/onie-boot ...
Installing for i386-pc platform.
Installation finrandom: mktemp urandom read with 19 bits of entropy available
<snip>
<snip>
Note: For the detailed trace example, go to the Appendix.
After the installation is complete, the SD card is not needed by the system and can be removed.
Installing NOS from the Network
ONIE supports a number of methods for locating a NOS installer image. See the details in the “Installing over the
Network” section of the ONIE user guide: https://opencomputeproject.github.io/onie/user-guide/index.html
Note: An SD card is not required to perform installation from the network.
As the prerequisite for the NOS configuration, ONIE requires a valid DHCP server and an HTTP server. The DHCP
server provides the IP, netmask, default gateway and so on to enable the auto-configuration of the Ethernet management
port.
Note: The 210 WBX has two RJ45 ports at the front panel: the Ethernet management port and the console port.
Step 1 Connect the Ethernet management port of the 210 WBX to the LAN network so that it can reach
the DHCP server.
Note: Connect to the Ethernet management port (D) of the switch marked by a red circle. Do not
connect to the console management port of the switch (C).
Fig. 3.3: 210 WBX Ethernet Management Port
Step 2 Copy the NOS Installer Image to the HTTP server.

Step 3 In the DHCP server configuration, in addition to the IP subnet, default netmask, default gateway,
and so on, add the default-url option that points to the Nuage installer file in the HTTP server.
option default-url "http://<webserver-address>/onie-installer-x86_64"
3.2. Initializing the System and Downloading Software 11

Note: For details, see the ONIE user guide: https://opencomputeproject.github.io/onie/user-guide/

index.html
Boot after the NOS Installation is Complete
After the Nuage 210 WBX NOS is installed, the 210 WBX automatically boots from the NOS software. The NOS
software boots automatically unless the boot menu is interrupted. The following display shows the boot menu from
the console after the NOS is installed.
Note: The sample output is only an example and the Nuage software may not match the output on your terminal.
Fig. 3.4: 210 WBX NOS Bootup
The boot menu content reflects the version of 210 WBX NOS image installed on the disk. The menu displayed
indicates that the NOS is successfully installed.
A separate ONIE option lets you reinstall the software.
Automated NOS Configuration
The 210 WBX supports automated configuration of the system without the need to log in to the switch. This is done
using configuration files when located on the SD card or on the HTTP server serving the ONIE installer.
The configuration files are:
• cloud-init.cfg, a file in YAML format used to configure the Linux hypervisor with the Cloud-Init soft-
ware.
3.3. Automated NOS Configuration 12

• config.cfg, the CLI configuration of the TiMOS virtual machine (VM) controlling the WBX switch.
The SD card configuration takes precedence over the configuration retrieved from the HTTP server. To avoid conflicts
and duplication, it is recommended that these configuration files be located either with the HTTP server or on the SD
card, but not both.
Configuring OS From the SD Card
1. In order to customize the switch from the SD card, manually edit the cloud-init.cfg and config.cfg
files in the root directory of the SD card.
2. For cloud-init to work correctly, confirm to see that the SD card has the correct label. If it does not show the
correct label, complete the steps provided in the For a Fresh Install section.
For a Fresh Install
1. Change the volume label on a Windows system with a SD card reader/writer to VSGX-SD.
2. To change the label in Windows, go to the left pane in Windows explorer under Computer, select the disk
representing the SD card, right-click and rename the file.
Preinstalled 210 WBX OS
Change the label with WBX OS using the following command:
mlabel -i /dev/sdb1 ::VSGX-SD
Note: You may be need to modify /etc/mtools.conf file with the following line because sometimes the mlabel
command may fail if not applied correctly.
mtools_skip_check=1
Configuring NOS From the Network
When ONIE installation is performed using a DHCP server, the default-url option is used to specify the URL of
the ONIE installer.
During NOS installation, the ONIE installer queries the HTTP server for a cloud-init.cfg file with the same
filepath as the ONIE installer.
For example:
If the default-url option is set to the following:
option default-url "http://<webserver>/switch001/onie-installer-x86_64"
The installer queries the URL for a configuration file:
http://<webserver>/switch001/cloud-init.cfg
If the configuration file is found, the configuration is automatically applied at first boot.

Cloud-init File Format
The cloud-init.cfg must start with the line #cloud-config.

At first boot, the Linux system is configured according to the cloud-init configuration. Upon reboot, the initial config-
uration persists and the cloud-init.cfg file is ignored. To apply the configuration again, you must perform an
ONIE reinstall.
The system can be configured with cloud-init modules.
In order to secure the 210 WBX switch, it is common practice to disable password root login and inject public SSH
keys as shown:
#cloud-config
ssh_pwauth: false
disable_root: true
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEA3FSyQwBI6Z+nCSjUU ...
It is also possible to execute custom commands at the time of first boot:
#cloud-config
bootcmd:
- echo "Cloud-init configuration in progress...."
- echo "Installing additional packages"
- yum -y install nfs-utils
Configuration management tools such as Puppet or Ansible can be applied using cloud-init and remote syslog config-
uration. A complete list of modules is documented in the cloud-init modules documentation.
nuage_wbx cloud-init module
A custom nuage_wbx cloud-init module allows the datacenter administrator to configure the boot options of the
TiMOS VM without logging in to the machine with the bof YAML key.
This allows configuration of a remote FTP server for switch configuration. Such a configuration enables the operator
to centralize the configuration of several switches in a single server, while retaining the ability to push configuration
changes with the admin save command:
#cloud-config
nuage_wbx:
bof: #boot option file
primary-config: "ftp://user:pass@<ftp-server>/./switch001/config.cfg"
Note: BoF config options above and static routes below should be in double quotes.
static-route:
- "21:488:a0::/40 next-hop 211:488:a2:405:135:228:0:167"
- "21:499:a0::/40 next-hop 211:488:a2:405:135:228:0::"
By default, the 210 WBX switch is designed to persist the configuration when an ONIE reinstallation is performed,
by storing relevant configuration files in a persistent partition. Such behaviour is not desirable in the context of fully
automated remote configuration. The nuage_wbx cloud-init module provides the wipe_persistent_data key
to ensure the system is provisioned from a clean state.

The config_url option lets you populate a TiMOS configuration from a local or remote (http) location. This
configuration file is applied only if wipe_persistent_data is set to true, and config_url is set to point to
this file.
#cloud-config
nuage_wbx:
wipe_persistent_data: true
config_url: http://<webserver>/switch001/config.cfg
Example of SD Card Configuration
cloud-init.cfg
#cloud-config
ssh_pwauth: false
disable_root: true
- ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEA3FSyQwBI6Z+nCSjUU ...
nuage_wbx:
config_url: file:///vsgx-sd/config.cfg
config.cfg
exit all
configure
#--------------------------------------------------
echo "System Configuration"
#--------------------------------------------------
system
name "switch001"
...
Example of cloud-init Static IP SD Card Configuration
[root@localhost ~]# cat /vsgx-sd/cloud-init.cfg

#cloud-config
runcmd:
- '\sed -i "s/BOOTPROTO=dhcp/BOOTPROTO=none\nDEFROUTE=yes\nIPADDR=135.227.
˓→223.80\nNETMASK=255.255.255.128\nGATEWAY=35.227.223.1/" /etc/sysconfig/
˓→network-scripts/ifcfg-mgmt'
- '\nmcli connection load /etc/sysconfig/network-scripts/ifcfg-mgmt'

- '\nmcli device reapply mgmt'
Example of DHCP plus HTTP Server Configuration
To automatically configure several switches in a datacenter, a recommended approach is to have a unique

default_url per switch in the DHCP server, for example:

host switch001 { hardware ethernet D0:99:D5:AB:CD:EF; fixed-address 10.10.10.

˓→1; option default-url "http://<webserver>/switch001/onie-installer-x86_64";
˓→ }
host switch002 { hardware ethernet D0:99:D5:AB:CD:E1; fixed-address 10.10.10.

˓→2; option default-url "http://<webserver>/switch002/onie-installer-x86_64";
˓→ }
The recommended HTTP server file structure is as follows:
[root@webserver ~]$ tree /var/www/html

.
+-- onie-installer-x86_64 -> onie-installer-x86_64-nuage-5.1.2-40
+-- onie-installer-x86_64-nuage-5.1.2-40
+-- switch001
+-- cloud-init.cfg
+-- config.cfg
+-- onie-installer-x86_64 -> ../onie-installer-x86_64
+-- switch002
+-- cloud-init.cfg
+-- config.cfg
+-- onie-installer-x86_64 -> ../onie-installer-x86_64
Example of cloud-init.cfg :
#cloud-config
ssh_pwauth: false
disable_root: true
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEA3FSyQwBI6Z+nCSjUU ...
nuage_wbx:
config_url: http://<webserver>/switch001/config.cfg
Example of config.cfg :
exit all
configure
#--------------------------------------------------
echo "System Configuration"
#--------------------------------------------------
system
name "switch001"
At scale, a templating engine may be used to quickly generate config.cfg and cloud-init.cfg files for all
datacenter switches.
Reinstalling OS and Software Upgrades
At boot time, selecting the ONIE option in the console launches the ONIE bootloader. It attempts to reinstall the
switch by using the same sources described earlier for installing NOS from the SD card or installing NOS from the
network.
Warning: After selecting the ONIE option from the boot menu, you must proceed to reinstall. The system
partitions are lost after the reinstall.
3.4. Reinstalling OS and Software Upgrades 16

Console Unavailable
When the console access is unavailable or not practical, issue the following command as root on the Linux hypervisor
to automatically trigger ONIE reinstallation at next reboot:
grub2-set-default ONIE
Persistent Partition
By default, during ONIE reinstall, the switch configuration and management network configuration is persistent. That
enables you to upgrade the software version of the switch without losing the network configuration. For reinstalling
ONIE, choose the ONIE: Install OS option from the console screen.
This behaviour can be changed by setting wipe_persistent_data to True in cloud-init.cfg and setting
config_url or primary-config to retrieve the configuration from a central location such as an HTTP or FTP
server.
Any manual customization of the Linux OS is not persistent and is lost after reinstall. For Linux configuration to
persist, it is recommended that it be configured declaratively with cloud-init modules.
Uninstalling OS
To uninstall NOS, interrupt the NOS auto-boot, select the ONIE option followed by ONIE: Uninstall OS as
shown in the figure captioned 210 WBX ONIE Uninstall OS:
Fig. 3.5: 210 WBX ONIE Uninstall OS
3.5. Uninstalling OS 17
Remote Upgrade
Starting with Nuage VSP Release 5.4.1, 210 WBX supports Remote Upgrade using the BOF (Boot File) in a similar
way the Nokia 7X50 Service Routers family upgrades remotely. Upgrades have been enhanced with two new variants:
SROS VM only upgrade, and Hypervisor plus SROS VM upgrade. Upgrading the SROS VM is possible when there
are no hypervisor changes. If a software requires changes to the hypervisor, then the upgrade involves update to both
the hypervisor and the SROS VM.
Note: Enhancements are only available once the node has already been upgraded to release 5.4.1 using current using
ONIE methods. Starting with Nuage VSP Release 5.4.1, ONIE and BOF are supported. Refer to the Installing NOS
from the SD Card (page 9) section or Installing NOS from the Network (page 11) section to upgrade the 210 WBX to
5.4.1.
WBX Upgrade for SROS VM
When there are no upgrades needed in the hypervisor, the SROS VM can be individually upgraded following regular
BoF procedures. The BoF can be pointing to an external server (FTP) or can be downloaded locally (FTP, sFTP, scp)
so that the BoF points to the local user disk. BOF allows to provision a primary-image and a secondary-image. If the
primary-image cannot be used, then it tries with the secondary.
Note: For more information about BOF configuration, please see the 210 WBX Basic System Configuration Guide.
BoF Pointing to the external FTP Link
*A:WBX-X# bof primary-image ftp://user:password*@35.227.180.72/./images/x86-

˓→both.tim
*A:WBX-X# show bof

===============================================================================
BOF (Memory)
===============================================================================
primary-image ftp://*:*@35.227.180.72/./images/x86-both.tim
primary-config ftp://*:*@35.227.180.72/./configs/vsgx_1.cfg
address 169.254.1.2/24 active
*A:WBX-X# admin reboot now
Note: The name of the .tim file may vary with the software release and differ from the ouput command example
BoF Pointing to the Local User Disk
The file can be downloaded to the 210 WBX using FTP, sFTP, or SCP, directly to the SROS VM. The file is available
in the WBX and accessible to the BoF.
A:WBX-B# file scp root@10.31.138.167:/root/wbx_upgrade_images/0.0/BPR2925/

˓→both.tim cf1:/ force router "management"
root@10.31.138.167's password:
3.6. Remote Upgrade 18

both.tim 100% |*****************************| 56 MB 00:15
*A:WBX-X# bof primary-image cf1:\both.tim
*A:WBX-X# show bof

===============================================================================
BOF (Memory)
===============================================================================
primary-image cf1:\both.tim
primary-config cf1:\config.cfg
*Truncated info
*A:WBX-X# admin reboot now
Note:
1. File can be downloaded using out of band management and inband management. Note that download rate via
out of band management is faster as inband is protected (throttled)
2. Bootloader is not upgraded, only the both.tim file gets upgraded. Bootloader gets upgraded during Hypervi-
sor plus SROS VM upgrade using ONIE installation.
210 WBX Upgrade for Hypervisor and SROS VM
Prerequisites
1. Verify BoF configurations in SROS VM: Ensure BoF to point to the primary-image to cf2:/timos/both.tim in
case of localURL. In case it points to the remote URL, make sure the remote FTP server has the new SROS
Image.
2. Verify BoF parameters in cloud-init.cfg file are present in the network/sd-card: Ensure either the primary-image
field is not present under BoF section, or if configured, it should be cf2:/timos/both.tim (in case of local URL)
or points to a Remote location which has the new SROS Image.
3. ONIE image must be downloaded manually into the VM (cf1:/) location and then the upgrade is triggered using
the CLI command.
The file can be downloaded using out of band management and in-band management. However, the download out of
band management rate is faster as in-band is protected (throttled).
MD5 checksum validations are run before triggering the upgrade. If an upgrade is attempted with an errored/corrupted
image, the validations will fail and upgrade will not be triggered. If the upgrade gets triggered and failed in the later
stage, the image needs to be re-installed with any of the existing ONIE methods (External SD card/ONIE from the
network).
There are two new commands available on SROS VM related to reboot and upgrade. Besides the command admin
reboot now that reboots the VM, there is a new command admin reboot vmhost to reboot the hypervisor and the
VM at the same time. This command does not force the software upgrade. For the software upgrade, use the admin
reboot upgrade-vmhost <URL> now command (see the following example).
To Reboot the VM and hypervisor from the SROS CLI, use the admin reboot vmhost command:
admin reboot vmhost
To Trigger upgrade of VM and hypervisor from SROS CLI, use the admin reboot upgrade-vmhost <URL> now
command:
3.6. Remote Upgrade 19

admin reboot upgrade-vmhost <URL> now
Example
The following is an example of Hypervisor plus VM Upgrade:
A:WBX-B# file scp root@10.31.138.167:/root/wbx_upgrade_images/0.0/BPR2925/

˓→onie-installer-x86_64 cf1:/ force router "management"
onie-installer-x86_6 100% |*****************************| 481 MB 01:30
A:WBX-B# file dir cf1:/
Volume in drive cf1 on slot A is SROS VM.
Volume in drive cf1 on slot A is formatted as FAT32
Directory of cf1:\
11/14/2018 03:24p 405 bof.cfg

11/13/2018 03:20a 307 bof.cfg.bkp
11/14/2018 03:26p 2805 bootlog.txt
11/13/2018 03:20a 0 config.cfg
11/14/2018 03:26p 336 nvsys.info
11/14/2018 04:17p 504601644 onie-installer-x86_64
6 File(s) 504605497 bytes.
0 Dir(s) 3928293376 bytes free.
A:WBX-B# admin reboot upgrade-vmhost cf1:/onie-installer-x86_64 now

***********************************
** ---> W A R N I N G VM <--- **
** ALL REQUIRED CHASSIS FIRMWARE **
** UPGRADES WILL BE DONE ON THIS **
** REBOOT AND MAY TAKE SEVERAL **
** MINUTES. THE CHASSIS MUST NOT **
** BE RESET OR POWERED DOWN, NOR **
** CARDS INSERTED OR REMOVED, **
** DURING THIS PROCESS. ANY OF **
** THESE PROHIBITED ACTIONS MAY **
** CAUSE CARDS TO BE RENDERED **
** INOPERABLE! **
***********************************
Executing upgrade command /usr/nuage/vsgx-upgrade.sh /onie-installer-x86_64
*Truncated info
Ensuring Availability of SROS VM and Hypervisor
The health of the SROS VM is monitored by a health check between SROS and the hypervisor using watchdog failure
detection implemented in the hypervisor. In any instance, when the hypervisor believes that the SROS VM is affected,
it gracefully restarts the SROS VM.
A vsgvm-monitor service has been added to Linux. This monitor tracks the status of the vsgvm service (SROS
VM) as given by systemd to see if it failed. It attempts to restart the SROS VM if it fails. The service maintains a
3.7. Ensuring Availability of SROS VM and Hypervisor 20

heartbeat with SROS using TCP with a keep-alive. If the keep-alive does not work with the VM for a determined
period, then the vsgvm-monitor service declares the VM as down and restarts the VM. If VM fails to establish TCP
connection with vsgvm-monitor on five successive restarts, the hypervisor gets rebooted (In this case, tech-support file
with Hypervisor/SROS VM logs gets generated under /vsgx-data). Logs are maintained in /var/log/messages.
Line 6859: Jan 17 02:12:59 WBX-c_a vsgvm-monitor: Connection Not Alive with
˓→VM, trigger recovery... => Triggers VM restart
Line 8262: Jan 17 02:24:05 WBX-c_a vsgvm-monitor: VM does not come up in 10
˓→minutes, listen timed out => Triggers VM restart
Line 13855: Jan 17 03:08:07 WBX-c_a vsgvm-monitor: VM does not come up for 5
˓→consecutive restart attempts,rebooting host =>Triggers Host reboot
Line 13856: Jan 17 03:08:07 WBX-c_a vsgvm-monitor: rebootHost: rebooting
˓→Linux
SROS CLI Enhancements
The following CLI Commands were added in SROS in 5.4.1:

1. To read the content of hypervisor’s data partition (/vsgx-data), enter the tools vmhost dir [<path>] command:
*A:NS1817T0321# tools vmhost dir

total 297488
-rw-r--r-- 1 root root 69632 Nov 2 20:32 1.log
-rw-r--r-- 1 root root 233472 Nov 6 22:22 1ty.log
-rw-r--r-- 1 root root 82480 Nov 4 18:26 4a.log
*Truncated info
*A:NS1817T0321# tools vmhost dir /tmp/

total 20
-rwx------ 1 root root 836 Nov 27 11:09 ks-script-5mwpG5
-rwx------ 1 root root 3852 Nov 27 11:09 ks-script-M12Cpk
-rw------- 1 root root 0 Nov 27 11:00 yum.log
*Truncated info
2. To delete a file in /vsgx-data (deletion is restricted to the files present only under /vsgx-data), enter the tools
vmhost delete <Filename> command:
*A:NS1817T0321# tools vmhost delete /vsgx-data/1.log

File deleted
*A:NS1817T0321# tools vmhost delete /tmp/yum.log

File cannot be deleted
3. To display Hypervisor’s software version, enter the tools vmhost version command:
*A:NS1817T0321# tools vmhost version

Nuage 210-WBX 0.0-BPR3236-34204c
3.8. SROS CLI Enhancements 21

4. To perform SSH from SROS VM into hypervisor, enter the tools vmhost ssh <user-name> command:
*A:NS1817T0321# tools vmhost ssh root

The authenticity of host '169.254.2.1 (169.254.2.1)' can't be established.
RSA key fingerprint is 95:0a:53:13:fa:66:9f:74:4c:25:22:a4:19:bd:ac:90.
Are you sure you want to continue connecting (yes/no)? yes
Authorized uses only. All activity may be monitored and reported.

Authorized uses only. All activity may be monitored and reported.
[root@badcvsgx09 ~]#
[root@badcvsgx09 ~]# exit
logout
Connection to 169.254.2.1 closed.
*A:NS1817T0321#
.. Note:: Using SSH from the VM might not support all linux commands,
˓→recommended use is to ssh the hypervisor using the management IP.
The known limitations are as follows:
- SSH sets the default term as dump terminal. User has to perform ":set
˓→term=ansi" in the VI editor to edit the file via SSH session to Host from
˓→VM.
- To increase the window size of VI: "export TERM=xterm; stty rows 40 cols
˓→170"
- Linux "clear" to perform clear screen is not working
5. To collect hypervisor tech-support logs from SROS VM, enter the tools vmhost tech-support cf1:/ command:
*A:WBX-A# tools vmhost tech-support cf1:/

Excuting command : /usr/nuage/vsgx-tech-support.sh /
Success
*A:WBX-A# file dir
Directory of cf1:\
01/29/2019 05:17a 367 bof.cfg

01/29/2019 05:00a 367 bof.cfg.bkp
01/29/2019 05:18a 2690 bootlog.txt
01/29/2019 05:00a 2687 bootlog_prev.txt
01/25/2019 03:22a 0 config.cfg
01/28/2019 05:38p 524135756 corrupt_onie-installer-x86_64
01/29/2019 05:18a 337 nvsys.info
01/28/2019 06:00p 522942764 onie-installer-x86_64
01/29/2019 07:56a 233383 tech-support-Tue_Jan_29_075613_EST_
˓→2019.tgz
3.8. SROS CLI Enhancements 22

CHAPTER
FOUR
GETTING STARTED WITH THE NUAGE 210 WBX
• Getting Started (page 23)

• Linux Hypervisor on the Nuage 210 WBX (page 25)
– SROS VM (page 26)
– VSG/VSA and 210 WBX (page 26)
• Hypervisor Monitoring (page 29)
– Monitoring with Linux Commands (page 29)
– New MIB Parameters (page 35)
– Example of use of the MIBs (page 36)
• README File (page 38)
Getting Started
After the software is installed from the console, the system shows the login prompt of the Linux hypervisor:
Nuage 210-WBX 6.0.1-1

localhost login:
Step 1 Login as root/UFXCr4733F
Note: If the system was booted and there was a functional DHCP server, hence a dynamic IP
address for the Ethernet management port was configured as part of the process, then skip Step 2;
otherwise complete Step 2 to configure a static IP address for the Ethernet management port.
Step 2 Configure the Ethernet management port using the ifcfg-mgmt file and modify to the desired
configuration. The following is an example:
[root@localhost network-scripts]# more ifcfg-mgmt

TYPE=Bridge
NAME=mgmt
DEVICE=mgmt
> IPADDR=178.227.223.73
23
NETMASK=255.255.255.128
GATEWAY=178.227.223.1
Step 3 Restart the network configuration by entering:
\nmcli connection load /etc/sysconfig/network-scripts/ifcfg-mgmt

\nmcli device reapply mgmt
Step 4 Enter the ifconfig command to display all available interfaces of the hypervisor:
[root@localhost ~]# ifconfig

xbcmh0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:a0:c9:00:00:00 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
xbcmh1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500

ether 34:12:78:56:01:00 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
TX packets 0 bytes 0 (0.0 B)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 23880 bytes 2219206 (2.1 MiB)
TX packets 23880 bytes 2219206 (2.1 MiB)
mgmt: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 178.227.223.73 netmask 255.255.255.128 broadcast 178.227.
˓→223.127
ether d0:99:d5:95:12:41 txqueuelen 1000 (Ethernet)

device memory 0xfbd00000-fbd7ffff
virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 169.254.1.1 netmask 255.255.255.0 broadcast 169.254.1.255
ether 52:54:00:9f:72:e2 txqueuelen 1000 (Ethernet)
RX packets 9062 bytes 982398 (959.3 KiB)
TX packets 10452 bytes 774295 (756.1 KiB)
virbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 169.254.2.1 netmask 255.255.255.0 broadcast 169.254.2.255
ether 52:54:00:a1:35:10 txqueuelen 1000 (Ethernet)
4.1. Getting Started 24

vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet6 fe80::fc54:ff:fee9:b148 prefixlen 64 scopeid 0x20<link>
ether fe:54:00:e9:b1:48 txqueuelen 1000 (Ethernet)
vnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet6 fe80::fc54:ff:fe76:ae80 prefixlen 64 scopeid 0x20<link>
ether fe:54:00:76:ae:80 txqueuelen 1000 (Ethernet)
Step 5 Once the Ethernet management port is configured, access the hypervisor using ssh:
ssh -p 893 root@ip
As part of the installation process, the system automatically launches the SROS virtual machine (VM).
Note: Wait for 5 minutes before accessing the VM console after a power cycle (cold reboot) or new installation.
Linux Hypervisor on the Nuage 210 WBX
After the installation, there is access to the Linux hypervisor as a regular commercial Linux system. The host OS is
running Centos 7.3.
Hypervisor partition:
After the installation, the partitions should look like below
[root@localhost /]# gdisk -l /dev/sda

GPT fdisk (gdisk) version 0.8.6
Partition table scan:

MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.

Disk /dev/sda: 125045424 sectors, 59.6 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): A9E8C47D-0666-420F-A8B8-655D61AA6B72
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 125045390
Partitions will be aligned on 2048-sector boundaries
Total free space is 3407 sectors (1.7 MiB)
Number Start (sector) End (sector) Size Code Name
4.2. Linux Hypervisor on the Nuage 210 WBX 25

1 2048 6143 2.0 MiB EF02 GRUB-BOOT

2 6144 268287 128.0 MiB FFFF ONIE-BOOT
3 268288 57936526 27.5 GiB 8300 VSGX-ROOT
4 57937920 125045390 32.0 GiB 8300 VSGX-DATA
VSGX-DATA is the persistence partition and all data to be preserved in a upgrade/reboot should be in this partition
SROS VM
Step 1 To check that the SROS VM is running, enter the virsh command:
[root@localhost ~]# virsh list --all
Id Name State
----------------------------------------------------
1 vsgvm-wbx210-48s running
Step 2 To access and exit the SROS VM console using the Linux hypervisor, use command console
vsgvm to get to the VM console. This command enhances virsh console Id as the entire console life
output of the VM which is redirected to a specific file as shown in the following example:
[root@localhost ~]# console vsgvm
[Enter `Êc?' for help]
Login:
Password:
A:NS1712T0021#
Note: As in regular SROS, default user and password is admin/admin.
Step 3 To exit the VM console session, type Êc. - Ctrl-E (followed by) c (followed by) . -
Note: The complete console output is redirected to the file /var/log/conserver.vsgvm.log
Once in the Nuage SROS VM, the unit behaves as a regular system.
The SROS process is controlled from the hypervisor and it can be stopped and restarted if needed using the commands
systemctl stop vsgvm and systemctl start vsgvm. You must use these two commands and no other Linux commands.
VSG/VSA and 210 WBX
There are certain differences when comparing a BOF on a VSG/VSA and on a 210 WBX. The system runs on a VM
and therefore, the VM has an internal IP address, netmask, DNS, and static route that points to the hypervisor as the
default gateway. This configuration is common to all 210 WBX VMs. The BOF also shows the Ethernet management
IP information, because that is the external point to access the SROS VM (in the example, previously in this user guide
the ifcfg-mgmt file has been configured with IPADDR=178.227.223.73).
A:# show bof
===============================================================================
BOF (Memory)
===============================================================================

primary-image cf2:\timos\both.tim
primary-dns 169.254.1.1
static-route 0.0.0.0/0 next-hop 169.254.1.1
no auto-boot
autonegotiate
duplex full
speed 100
wait 3
persist off
no li-local-save
no li-separate
console-speed 115200
===============================================================================
Host Information
===============================================================================
linuxhost-ipv4-address 178.227.223.73 netmask 255.255.255.128
linuxhost-ipv6-address-1 Not available
===============================================================================
Note: Although IPv6 can be configured in the Ethernet management port and is displayed as part of the BOF, the
IPv6 address must be configured in the BOF in order for the SROS VM to use IPv6 to communicate with the external
world.
The following example displays an IPv6 management address for the SROS VM:
A:>bof# address cafe:2::d299:d5ff:fe95:2222/64
*A:>bof# exit
*A:# show bof
===============================================================================
BOF (Memory)
===============================================================================
primary-image ftp://*:*@mvdcvsgx03/./images/x86-both.tim
primary-config ftp://*:*@mvdcvsgx03/./images/WBX-c_a.cfg
address cafe:2::d299:d5ff:fe95:2222/64 active
no auto-boot
autonegotiate
duplex full
speed 100
wait 3
persist off
no li-local-save
no li-separate
no fips-140-2
===============================================================================
Host Information
===============================================================================
linuxhost-ipv6-address-1 cafe:2::d299:d5ff:fe95:1111 prefixlen
˓→128 0x0<global>

linuxhost-ipv6-address-2 cafe:2::b0bb:6aff:fefc:36df prefixlen

64 0x0<global>
˓→
===============================================================================
The hypervisor can be accessible with applications using port 893 (See the ssh example earlier), otherwise applications
default to the SROS VM.
The BOF (or files referenced in the BOF like config) does not exist as an individual file, it is contained within the user
disk vsgvm-wbx210-*.usrdisk.raw. The user disk is saved in the following folder:
[root@localhost ~]# cd /vsgx-data
[root@localhost vsgx-data]# ls -la
total 216904
drwxr-xr-x 3 root root 4096 Jul 11 20:51 .
dr-xr-xr-x. 20 root root 4096 Jul 11 21:17 ..
drwx------ 2 root root 16384 Apr 26 2000 lost+found
-rw-r--r-- 1 qemu qemu 222082560 Jul 12 22:33 vsgvm-wbx210-32q.usrdisk.raw
It is possible to list the content of the user disk:

[root@localhost ~]#mdir -i vsgvm-wbx210-32q.usrdisk.raw@@32256 ::/
Volume in drive : is SROS VM
Volume Serial Number is 77BB-4E50
Directory for ::/
bof cfg 307 2017-08-21 19:17 bof.cfg

config cfg 0 2017-08-21 19:17 config.cfg
BOOTLOG TXT 2621 2017-08-23 21:45 bootlog.txt
N~999990 INF 320 2017-08-23 21:46 nvsys.info
B~999988 TXT 2627 2017-08-22 17:58 bootlog_prev.txt
5 files 5 875 bytes 221 798 400 bytes free
It is also possible to save a file from the user disk locally in the persistency partition /vsgx-data (example bof.cfg):
[root@localhost ~]#mcopy -i vsgvm-wbx210-32q.usrdisk.raw@@32256 ::/config.
˓→cfg .
[root@localhost vsgx-data]# ls
config.cfg lost+found vsgvm-wbx210-32q.usrdisk.raw
However for ease of use and whenever possible, it is recommended that you save the VM files externally using an FTP:
*A:210-WBX# show bof

===============================================================================
BOF (Memory)
===============================================================================
primary-config ftp://*:*@172.203.15.48/./configs/tor10.cfg
After the installation, the Nuage software is saved under /usr/nuage/

[root@localhost ~]# cd /usr/nuage/
[root@localhost nuage]# ls -la
total 354604
drwxr-xr-x 4 root root 4096 Jul 11 21:17 .
drwxr-xr-x 14 root root 4096 Jul 6 00:35 ..
-rw-r--r-- 1 root root 2823 Jul 6 00:26 i2c_lock_functions
-rw-r--r-- 1 root root 884 Jul 11 21:16 last-vm-shut-trace.txt
-rwxr-xr-x 1 root root 17237683 Jul 5 23:11 nuagerpcd

-rw-r--r-- 1 root root 98703360 Jul 6 00:25 nuage-vsgx-imgdisk.raw

-rw-r--r-- 1 root root 222082560 Jul 6 00:30 nuage-vsgx-usrdisk.raw
-rwxr-xr-x 1 root root 919240 Jul 6 00:30 onie-syseeprom
-rw-r--r-- 1 root root 809 Jul 6 00:26 parser.awk
-rwxr-xr-x 1 root root 24076200 Jul 6 00:29 qemu-kvm
-rwxr-xr-x 1 root root 2210 Jul 6 00:26 qemu-vsgvm-emulator
lrwxrwxrwx 1 root root 41 Jul 11 21:17 read-env -> /usr/nuage/wbx210-
˓→32q/read-env-wbx210-32q
-rwxr-xr-x 1 root root 279 Jul 6 00:26 read-if.sh

-rw-r--r-- 1 root root 344 Jul 6 00:26 README
lrwxrwxrwx 1 root root 46 Jul 11 21:17 set-fan-speed -> /usr/nuage/
˓→wbx210-32q/set-fan-speed-wbx210-32q
lrwxrwxrwx 1 root root 40 Jul 11 21:17 set-led -> /usr/nuage/wbx210-

˓→32q/set-led-wbx210-32q
lrwxrwxrwx 1 root root 40 Jul 11 21:17 sfp_i2c -> /usr/nuage/wbx210-

˓→32q/sfp_i2c_wbx210-32q
-rw-r--r-- 1 root root 257 Jul 6 00:26 vsg-net-mgmt.xml

-rw-r--r-- 1 root root 256 Jul 6 00:26 vsg-net-rpc.xml
-rwxr-xr-x 1 root root 13593 Jul 6 00:26 vsgvm
-rw-r--r-- 1 root root 12938 Jul 6 00:26 vsgvm_functions
-rwxr-xr-x 1 root root 2326 Jul 6 00:26 vsgx-bootmsg.sh
drwxr-xr-x 2 root root 4096 Jul 6 00:36 wbx210-32q
drwxr-xr-x 3 root root 4096 Jul 6 00:36 wbx210-48s
Hypervisor Monitoring
Starting in release 6.0.1, SROS MIBs (contained in TIMETRA-SYSTEM-MIB) have been enhanced to display the
result of hypervisor monitoring Linux Commands and are available to any SNMP client. These MIBs are used to
display well known Linux commands with information about CPU cores utilization, memory, and top processes.
Monitoring with Linux Commands
The hypervisor is a regular Linux system and among the different commands to monitor the status of the hypervisor,
three commands (mpstat, top, and free) are normally used to check the core utilization, the process utilization, and the
memory used/available. The following commands are examples:
root@e2e-proto-1 ~]# mpstat -P ALL
Linux 3.10.0-693.21.1.el7.x86_64 (e2e-proto-1.mv.nuagenetworks.net) 06/
˓→19/2019 _x86_64_ (12 CPU)
10:44:51 PM CPU %usr %nice %sys %iowait %irq %soft %steal

˓→%guest %gnice %idle
10:44:51 PM all 6.94 0.00 2.43 0.00 0.00 0.01 0.00
˓→10.11 0.00 80.52
10:44:51 PM 0 11.03 0.00 3.03 0.00 0.00 0.05 0.00
˓→0.00 0.00 85.89
10:44:51 PM 1 11.40 0.00 3.40 0.00 0.00 0.02 0.00
˓→0.00 0.00 85.18
10:44:51 PM 2 11.87 0.00 4.41 0.00 0.00 0.01 0.00
˓→0.00 0.00 83.71
10:44:51 PM 3 14.97 0.00 4.88 0.01 0.00 0.00 0.00
˓→0.00 0.00 80.13
10:44:51 PM 4 14.49 0.00 4.60 0.00 0.00 0.00 0.00
˓→0.00 0.00 80.91
4.3. Hypervisor Monitoring 29

10:44:51 PM 5 19.35 0.00 4.58 0.00 0.00 0.00 0.00

˓→0.00 0.00 76.06
10:44:51 PM 6 0.00 0.00 1.21 0.00 0.00 0.00 0.00
˓→2.62 0.00 96.17
10:44:51 PM 7 0.00 0.00 0.62 0.00 0.00 0.00 0.00
˓→1.70 0.00 97.68
10:44:51 PM 8 0.00 0.00 0.59 0.00 0.00 0.00 0.00
˓→2.75 0.00 96.66
10:44:51 PM 9 0.00 0.00 0.67 0.00 0.00 0.00 0.00
˓→4.91 0.00 94.42
10:44:51 PM 10 0.00 0.00 1.09 0.00 0.00 0.00 0.00
˓→9.57 0.00 89.34
10:44:51 PM 11 0.00 0.00 0.07 0.00 0.00 0.00 0.00
˓→99.60 0.00 0.34
[root@e2e-proto-1 ~]#
[root@e2e-proto-1 ~]# top -b -n 1

top - 22:45:10 up 12 days, 18:29, 2 users, load average: 2.30, 2.36, 2.35
Tasks: 178 total, 2 running, 176 sleeping, 0 stopped, 0 zombie
%Cpu(s): 16.1 us, 2.6 sy, 0.0 ni, 81.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
˓→st
KiB Mem : 16219716 total, 10421124 free, 4653312 used, 1145280 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 11135908 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

28488 qemu 20 0 4618428 194736 47932 S 120.0 1.2 14322:05 qemu-kvm
2377 root 20 0 270840 9300 4024 R 106.7 0.1 18295:49 python
1 root 20 0 191628 4672 2508 S 0.0 0.0 4:38.35 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:02.39
˓→ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→0:0H
7 root rt 0 0 0 0 S 0.0 0.0 0:13.57

˓→migration/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh

9 root 20 0 0 0 0 S 0.0 0.0 4:47.35 rcu_sched
10 root rt 0 0 0 0 S 0.0 0.0 0:12.18 watchdog/
˓→0

˓→1
12 root rt 0 0 0 0 S 0.0 0.0 0:22.99

˓→migration/1
13 root 20 0 0 0 0 S 0.0 0.0 0:00.37

˓→ksoftirqd/1
15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→1:0H

˓→2
17 root rt 0 0 0 0 S 0.0 0.0 0:23.12

˓→migration/2
18 root 20 0 0 0 0 S 0.0 0.0 0:00.75

˓→ksoftirqd/2
20 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→2:0H

˓→3
22 root rt 0 0 0 0 S 0.0 0.0 0:49.05

˓→migration/3

23 root 20 0 0 0 0 S 0.0 0.0 0:00.66

˓→ksoftirqd/3
25 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/
˓→3:0H

˓→4
27 root rt 0 0 0 0 S 0.0 0.0 0:40.83

˓→migration/4
28 root 20 0 0 0 0 S 0.0 0.0 0:00.26

˓→ksoftirqd/4
30 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→4:0H

˓→5
32 root rt 0 0 0 0 S 0.0 0.0 0:27.41

˓→migration/5
33 root 20 0 0 0 0 S 0.0 0.0 0:00.36

˓→ksoftirqd/5
35 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→5:0H

˓→6
37 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/6
38 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/6
39 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→6:0
40 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→6:0H

˓→7
42 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/7
43 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/7
44 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→7:0
45 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→7:0H

˓→8
47 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/8
48 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/8
49 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→8:0
50 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→8:0H

˓→9
52 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/9
53 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/9
54 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→9:0

55 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→9:0H
˓→10
57 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/10
58 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/10
59 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→10:0
60 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→10:0H

˓→11
62 root rt 0 0 0 0 S 0.0 0.0 0:00.17

˓→migration/11
63 root 20 0 0 0 0 S 0.0 0.0 0:00.00

˓→ksoftirqd/11
64 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→11:0
65 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→11:0H
67 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs

68 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns
69 root 20 0 0 0 0 S 0.0 0.0 0:00.25
˓→khungtaskd
70 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 writeback

71 root 0 -20 0 0 0 S 0.0 0.0 0:00.00
˓→kintegrityd
72 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset

73 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kblockd
74 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 md
80 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kswapd0
81 root 25 5 0 0 0 S 0.0 0.0 0:00.34 ksmd
82 root 39 19 0 0 0 S 0.0 0.0 0:01.98
˓→khugepaged
83 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 crypto

91 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kthrotld
94 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kmpath_
˓→rdacd
104 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kpsmoused

106 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ipv6_
˓→addrconf
125 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 deferwq

175 root 20 0 0 0 0 S 0.0 0.0 0:00.20 kauditd
318 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 iscsi_eh
329 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 rpciod
330 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xprtiod
447 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ata_sff
449 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0
450 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→0
451 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1

452 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→1
453 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_2

454 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→2

455 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_3

456 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→3
457 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_4

458 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→4
459 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_5

460 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 scsi_tmf_
˓→5
473 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ixgbe

506 root 0 -20 0 0 0 S 0.0 0.0 0:00.22 kworker/
˓→0:1H
524 root 20 0 0 0 0 S 0.0 0.0 0:04.10 jbd2/

˓→sda3-8
525 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ext4-rsv-

˓→conver
626 root 20 0 110476 67480 67156 S 0.0 0.4 0:14.16 systemd-

˓→journal
646 root 20 0 118848 1296 960 S 0.0 0.0 0:00.00 lvmetad

660 root 20 0 43932 1916 1316 S 0.0 0.0 0:00.19 systemd-
˓→udevd
717 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kvm-

˓→irqfd-clean
724 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 edac-

˓→poller
727 root 0 -20 0 0 0 S 0.0 0.0 0:00.41 kworker/

˓→1:1H
733 root 20 0 0 0 0 S 0.0 0.0 0:00.00 jbd2/

˓→sda2-8
734 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ext4-rsv-

˓→conver
736 root 20 0 0 0 0 S 0.0 0.0 0:00.01 jbd2/

˓→sda4-8
737 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ext4-rsv-

˓→conver
757 root 16 -4 55452 880 480 S 0.0 0.0 0:00.89 auditd

781 root 0 -20 0 0 0 S 0.0 0.0 0:00.95 kworker/
˓→2:1H
782 root 0 -20 0 0 0 S 0.0 0.0 0:13.10 kworker/

˓→3:1H
810 root 20 0 24204 1700 1392 S 0.0 0.0 0:03.76 systemd-

˓→logind
813 polkitd 20 0 534900 13116 4880 S 0.0 0.1 0:02.99 polkitd

815 root -2 0 4380 2332 1980 S 0.0 0.0 4:00.81 watchdog
816 root 20 0 219240 5504 3800 S 0.0 0.0 0:00.01 abrtd
820 root 20 0 216912 4720 3312 S 0.0 0.0 0:03.70 abrt-
˓→watch-log
828 root 20 0 293608 51456 48284 S 0.0 0.3 0:31.76 rsyslogd

832 root 20 0 21604 1256 972 S 0.0 0.0 0:50.90
˓→irqbalance
834 dbus 20 0 24628 1908 1308 S 0.0 0.0 2:56.08 dbus-

˓→daemon
842 root 20 0 195040 1240 764 S 0.0 0.0 0:00.00 gssproxy

844 chrony 20 0 118212 2104 1600 S 0.0 0.0 0:00.88 chronyd
877 root 20 0 333984 28736 6688 S 0.0 0.2 0:02.04 firewalld
878 root 20 0 7080 284 132 S 0.0 0.0 0:00.00 mcelog
887 root 20 0 115520 1228 700 S 0.0 0.0 0:21.24 ksmtuned
914 root 20 0 546416 12068 6692 S 0.0 0.1 0:10.66
˓→NetworkManager

1047 root 20 0 113372 15920 3424 S 0.0 0.1 0:00.16 dhclient

1049 root 0 -20 0 0 0 S 0.0 0.0 0:00.23 kworker/
˓→5:1H
1291 root 0 -20 0 0 0 S 0.0 0.0 0:00.51 kworker/

˓→4:1H
1342 root 20 0 562416 16632 5884 S 0.0 0.1 1:04.46 tuned

1348 root 20 0 1204760 16000 10504 S 0.0 0.1 0:00.50 libvirtd
1383 root 20 0 10464 3016 1996 S 0.0 0.0 11:56.74 nuagerpcd
1494 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
1496 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfsalloc
1497 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 xfs_mru_
˓→cache
1524 root 20 0 89544 2156 1112 S 0.0 0.0 0:01.88 master

1526 postfix 20 0 89716 4028 3016 S 0.0 0.0 0:00.41 qmgr
1545 root 20 0 105996 4116 3140 S 0.0 0.0 0:00.01 sshd
1546 root 20 0 126232 1568 948 S 0.0 0.0 0:01.14 crond
1625 root 20 0 198040 2536 1876 S 0.0 0.0 0:00.00 login
1626 root 20 0 110044 804 688 S 0.0 0.0 0:00.00 agetty
1734 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/
˓→0:1
2077 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 vfio-

˓→irqfd-clea
2233 root 20 0 274652 3676 2784 S 0.0 0.0 0:00.00 virtlogd

2246 root 20 0 17788 1320 1112 S 0.0 0.0 0:00.99 systemd-
˓→machine
2367 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→7:1
2368 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→9:1
2369 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→10:1
2370 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→8:1
2371 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→11:1
2372 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→6:1
2392 root 20 0 1104 368 308 S 0.0 0.0 0:05.47 fand

2458 root 20 0 116168 2816 1680 S 0.0 0.0 0:00.04 bash
2528 root 0 -20 0 0 0 S 0.0 0.0 0:01.25 loop0
4602 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/
˓→1:0
6622 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→4:2
7665 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→3:1
9061 root 20 0 0 0 0 S 0.0 0.0 0:28.03 kworker/

˓→1:2
9158 postfix 20 0 89648 4020 2996 S 0.0 0.0 0:00.00 pickup

11185 root 20 0 0 0 0 S 0.0 0.0 0:11.08 kworker/
˓→3:0
15727 root 20 0 0 0 0 S 0.0 0.0 0:00.22 kworker/

˓→u24:1
20048 root 20 0 0 0 0 S 0.0 0.0 0:00.09 kworker/

˓→u24:2
21692 root 20 0 0 0 0 S 0.0 0.0 0:10.36 kworker/

˓→4:1
24979 root 20 0 147880 5476 4184 S 0.0 0.0 0:00.02 sshd

24986 root 20 0 116172 2816 1668 S 0.0 0.0 0:00.02 bash

25545 root 20 0 107904 612 516 S 0.0 0.0 0:00.00 sleep
25553 root 20 0 157732 2116 1480 R 0.0 0.0 0:00.00 top
26429 root 20 0 0 0 0 S 0.0 0.0 0:06.54 kworker/
˓→2:3
27740 root 20 0 0 0 0 S 0.0 0.0 0:07.04 kworker/

˓→0:0
27838 root 20 0 0 0 0 S 0.0 0.0 0:05.10 kworker/

˓→5:1
28101 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/

˓→5:3
28123 nobody 20 0 15604 848 644 S 0.0 0.0 0:00.00 dnsmasq

28255 nobody 20 0 15604 848 644 S 0.0 0.0 0:00.00 dnsmasq
28585 root 20 0 0 0 0 S 0.0 0.0 0:05.78 vhost-
˓→28488
28587 root 20 0 0 0 0 S 0.0 0.0 2:22.12 vhost-

˓→28488
28595 root 20 0 0 0 0 S 0.0 0.0 64:51.92 kvm-pit/

˓→28488
28620 root 20 0 98980 3796 1480 S 0.0 0.0 0:00.01 conserver

28621 root 20 0 98980 3824 1500 S 0.0 0.0 0:00.36 conserver
32051 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/
˓→2:1
[root@e2e-proto-1 ~]#
[root@e2e-proto-1 ~]# free -m

total used free shared buff/cache
˓→available
Mem: 15839 4544 10176 56 1118

˓→10874
Swap: 0 0 0
Note: For more information about these three commands (mpstat, top and free), read the Linux man page.
New MIB Parameters
+--sysHvInfo(26)
|
+-- -R-- Unsigned sysHvInfoMemTotal(1)
| Range: 1..65535
+-- -R-- Unsigned sysHvInfoMemAvail(2)
| Range: 1..65535
|
+--sysHvInfoCpuStatTable(3)
| |
| +--sysHvInfoCpuStatEntry(1)
| | Index: sysHvInfoCpuStatIndex
| |
| +-- ---- Unsigned sysHvInfoCpuStatIndex(1)
| | Range: 1..13
| +-- -R-- String sysHICScpu(2)
| | Size: 0..64
| +-- -R-- String sysHICSusr(3)
| | Size: 0..64

| +-- -R-- String sysHICSnic(4)

| | Size: 0..64
| +-- -R-- String sysHICSsys(5)
| | Size: 0..64
| +-- -R-- String sysHICSiow(6)
| | Size: 0..64
| +-- -R-- String sysHICSirq(7)
| | Size: 0..64
| +-- -R-- String sysHICSsft(8)
| | Size: 0..64
| +-- -R-- String sysHICSstl(9)
| | Size: 0..64
| +-- -R-- String sysHICSgst(10)
| | Size: 0..64
| +-- -R-- String sysHICSgnc(11)
| | Size: 0..64
| +-- -R-- String sysHICSidl(12)
| Size: 0..64
|
+--sysHvInfoProcStatTable(4)
|
+--sysHvInfoProcStatEntry(1)
| Index: sysHvInfoProcStatIndex
|
+-- ---- Unsigned sysHvInfoProcStatIndex(1)
| Range: 1..5
+-- -R-- String sysHIPSpid(2)
| Size: 0..64
+-- -R-- String sysHIPSpri(3)
| Size: 0..64
+-- -R-- String sysHIPSnic(4)
| Size: 0..64
+-- -R-- String sysHIPSvir(5)
| Size: 0..64
+-- -R-- String sysHIPSres(6)
| Size: 0..64
+-- -R-- String sysHIPSshr(7)
| Size: 0..64
+-- -R-- String sysHIPScpu(8)
| Size: 0..64
+-- -R-- String sysHIPSmem(9)
| Size: 0..64
+-- -R-- String sysHIPStim(10)
| Size: 0..64
+-- -R-- String sysHIPScmd(11)
Size: 0..64
Example of use of the MIBs
Memory stats: sysHvInfo
[root@localhost ~]$ snmpget -v2c -c public 135.250.255.47 1.3.6.1.4.1.6527.3.

˓→1.2.1.26.1.0 2>/dev/null
sysHvInfoMemTotal.0 = Gauge32: 15839

[root@localhost ~]$ snmpget -v2c -c public 135.250.255.47 1.3.6.1.4.1.6527.3.
˓→1.2.1.26.2.0 2>/dev/null

sysHvInfoMemAvail.0 = Gauge32: 10872
CPU stats: sysHvInfoCpuStatTable
[root@localhost ~]$ snmptable -v2c -c public 1.250.255.47

˓→sysHvInfoCpuStatTable 2>/dev/null
SNMP table: sysHvInfoCpuStatTable
sysHICScpu sysHICSusr sysHICSnic sysHICSsys sysHICSiow sysHICSirq sysHICSsft

˓→sysHICSstl sysHICSgst sysHICSgnc sysHICSidl
"all" "6.94" "0.00" "2.43" "0.00" "0.00" "0.01"

˓→"0.00" "10.11" "0.00" "80.52"
"0" "11.03" "0.00" "3.03" "0.00" "0.00" "0.05" "0.
˓→00" "0.00" "0.00" "85.88"
"1" "11.40" "0.00" "3.40" "0.00" "0.00" "0.02" "0.
˓→00" "0.00" "0.00" "85.18"
"2" "11.86" "0.00" "4.41" "0.00" "0.00" "0.01" "0.
˓→00" "0.00" "0.00" "83.72"
"3" "14.97" "0.00" "4.88" "0.01" "0.00" "0.00" "0.
˓→00" "0.00" "0.00" "80.13"
"4" "14.49" "0.00" "4.60" "0.00" "0.00" "0.00" "0.
˓→00" "0.00" "0.00" "80.90"
"5" "19.36" "0.00" "4.58" "0.00" "0.00" "0.00" "0.
˓→00" "0.00" "0.00" "76.06"
"6" "0.00" "0.00" "1.21" "0.00" "0.00" "0.00" "0.
˓→00" "2.62" "0.00" "96.17"
"7" "0.00" "0.00" "0.62" "0.00" "0.00" "0.00" "0.
˓→00" "1.70" "0.00" "97.68"
"8" "0.00" "0.00" "0.59" "0.00" "0.00" "0.00" "0.
˓→00" "2.75" "0.00" "96.66"
"9" "0.00" "0.00" "0.67" "0.00" "0.00" "0.00" "0.
˓→00" "4.91" "0.00" "94.42"
"10" "0.00" "0.00" "1.09" "0.00" "0.00" "0.00"
˓→"0.00" "9.57" "0.00" "89.34"
"11" "0.00" "0.00" "0.07" "0.00" "0.00" "0.00"
˓→"0.00" "99.60" "0.00" "0.34"
Top 5 processes: sysHvInfoProcStatTable
[root@localhost ~]$ snmptable -v2c -c public 1.250.255.47

˓→sysHvInfoProcStatTable 2>/dev/null
SNMP table: sysHvInfoProcStatTable
sysHIPSpid sysHIPSpri sysHIPSnic sysHIPSvir sysHIPSres sysHIPSshr sysHIPScpu

˓→sysHIPSmem sysHIPStim sysHIPScmd
"28488" "20" "0" "4618428" "194736" "47932" "126.7"
˓→ "1.2" "14319:22" "qemu-kvm"
"2377" "20" "0" "270840" "9300" "4024" "106.7"
˓→ "0.1" "18293:45" "python"
"1" "20" "0" "191628" "4672" "2508" "0.0"
˓→"0.0" "4:38.31" "systemd"
"2" "20" "0" "0" "0" "0" "0.0"
˓→"0.0" "0:00.02" "kthreadd"
"3" "20" "0" "0" "0" "0" "0.0"
˓→"0.0" "0:02.39" "ksoftirqd/0"

README File
You must follow the README file and not modify or delete the contents (add, remove, change, etc) from this directory.
Note: Any accidental change can break the software installation and the only solution is to reinstall the software
again.
[root@localhost nuage]# more README

NOTICE
======
This directory (and subdirectories within this) accommodates all executables,

configuration and others that are needed for Nuage software to run.
Any changes to the contents of this directory that are made without Nuage being
aware may make the platform inoperable.
DO NOT MODIFY OR DELETE CONTENTS OF THIS DIRECTORY HIERARCHY

============================================================================
4.4. README File 38

CHAPTER
FIVE
NUAGE SROS FOR 210 WBX
• Nuage SROS (page 39)

– VM Information (page 40)
– Identifying a 210 WBX in a Busy Data Center (page 40)
– Power Supplies (page 40)
• BOF (page 41)
• Watermarks (page 43)
– Event Examples (page 44)
• Dynamic ECMP Group (page 46)
• 210 WBX ACL Scale Enhancements (page 47)
– Reverting to ACL Global Mode (page 50)
* Workaround (page 51)

• VRF Scalability Enhancements (page 51)
– Restrictions (page 51)
– Configuring VRF Scalability (page 52)
• 210 WBX Control Plane Hardware Queues Monitoring (page 53)
– Verifying and Clearing CPU (page 53)
• Quality of Service (page 56)
– Forward Error Correction and Auto Negotiation (page 62)
• Tech Support (page 63)
– Admin tech-support for the Hypervisor (page 63)
– Admin tech-support for the SROS VM (page 64)
Nuage SROS
The 210 WBX reuses the same Nuage SROS used on the 7850 VSG. The main difference is the 210 WBX runs the
SROS on a virtual machine and uses an x86 platform running Linux. Also it uses a different silicon 100GE capable
chip set. However, for a user that logs into the system directly in the VM, there are no major differences between the
39
two products, other than the differences about the use of different interfaces, and some related features. The 210 WBX
supports 100GE, 50G and 25G, which are new compared to a 7850 VSG.
VM Information
The VM is accessible using the Host information IPv4 (example, from ssh):
ROL:~ $ ssh admin@178.227.223.74
TiMOS-DC-B-5.0.2-28 both/x86 NUAGE 210 Copyright (c) 2000-2017 Nokia.
All rights reserved. All use subject to applicable license agreements.
Built on Wed Jul 5 20:29:23 PDT 2017 [080cc4] by builder in /rel5.0-DC/
˓→release/panos/main
admin@178.227.223.74's password:
Identifying a 210 WBX in a Busy Data Center
To identify a particular 210 WBX in a busy data center, enter the enable-uid-beacon command to trigger an “all lights
on LED” amber blink. All LEDs flash in amber and a technician on site can find the switch in question.
*A:#configure system [no] enable-uid-beacon
Power Supplies
To verify the state of the power supplies (up or failed), enter the show chassis power-supply command.
Note: By default, the power supplies displayed in the show chassis command shows DC power supplies and appear
as up, irrespective of the number of power supplies fed or if being AC or DC.
*A:# show chassis power-supply
===============================================================================
Chassis Information
===============================================================================
Power Supply Information
Number of power supplies : 2
Power supply number : 1

Defaulted power supply type : dc
Status : up

Status : up
===============================================================================
To configure the Power supplies, enter the configure system power-supply command:
*A: >config>system# power-supply

- power-supply <power-supply-id> <type>
<power-supply-id> : [1..2]. Specifies power supply 1 or 2
5.1. Nuage SROS 40

<type> : keywords - dc|ac {single|multiple}|default|none.

˓→Specifies DC or AC
<single|multiple> : keywords
*A:# configure system power-supply 1 ac single

*A:# configure system power-supply 2 ac single
After the changes, the system shows the real status of the power supplies. Status failed is displayed if the power supply
has any problem or it has not been supplied with power.
*A:# show chassis power-supply
===============================================================================
Chassis Information
===============================================================================

Configured power supply type : ac single
Status : up
AC power : within range
DC power : within range
Over temp : within range

Configured power supply type : ac single
Status : failed
AC power : out of range
DC power : out of range
Over temp : within range
Note: The multiple keyword is not supported in the following output example:
*A:# configure system power-supply 1 ac multiple

MINOR: CHMGR #1001 Invalid - Unsupported power supply type ac multiple
BOF
As shown in the Linux Hypervisor on the Nuage 210 WBX section of this document, BOF has been modified to reflect
the VM, and points to the hypervisor as a default gateway for out of band management. The Ethernet management of
the hypervisor has been added to the BOF display to show the IP address that is actually being used for out-of-band
management.
In the following example, the Host Information is different from the IP address of the VM.
*A:210-WBX-32Q# show bof

===============================================================================
BOF (Memory)
===============================================================================
5.2. BOF 41
no auto-boot
autonegotiate
duplex full
speed 100
wait 3
persist off
no li-local-save
no li-separate
===============================================================================
Host Information
===============================================================================
linuxhost-ipv6-address-1 fe80::d299:d5ff:fe93:e41 prefixlen 64
˓→0x20<link>

===============================================================================
WBX allows to configure the L2 and L3 tables, depending on the use that the system is going to have. If the system is
going to be used predominantly as an L2 switch (using overlay L2 Domains) or as an L3 domains (using overlay L3
Domains), system can be configured to have more L2 tables capacity or more L3 hosts capacity. This is configured
via the BOF with the command lookup-table-profile. There are then two possible values: L2-focused (default) or
host-focused:
*A:WBX-A# bof lookup-table-profile

- lookup-table-profile {host-focused|l2-focused}
- no lookup-table-profile
<host-focused|l2-f*> : keywords
The command tools dump system-resources has been enhanced to show information about the MAC and IP tables:
*A:WBX-A# tools dump system-resources

Resource Manager info at 013 m 03/05/19 03:40:06.590:
Hardware Resource Usage for Slot #1, CardType iom-48-25g+6-100g, Cmplx #0:
| Total | Allocated | Free
-------------------------------+-----------+-----------+------------
Ingress Shared ACL Entries | 0| 0| 0
Ingress MAC ACL Entries | 256| 2| 254
Ingress IPv4 ACL Entries | 256| 2| 254
Egress MAC ACL Entries | 256| 17| 239
Egress IPv4 ACL Entries | 256| 17| 239
Ingress Total QoS Entries | 1024| 1012| 12
Ingress QoS Entries/group(0) | 512| 512| 0
Ingress Total QoS Meters | 512| 506| 6
Ingress QoS Meters/group(0) | 256| 256| 0
HashTable Statistic | Count

-------------------------------+-------------
TlsFdbHashBucketFull | 0
TlsFdbLearntMacOverwritten | 0
5.2. BOF 42
pipIpHashBucketFull | 0
pipIpHashBucketOverwrites | 0
ProfileTable Info | Count

DscpProfileCount | 0
DscpProfileFreeCount | 125
Forwarding Table Statistics | Used / Free / Max

-------------------------------+-----------------------
L2 Fdb Entries | 20 / 40939 / 40959
Ipv4 Host Entries | 80 / 106412 / 106496
Ipv6 Host Entries | 2 / 53206 / 53248
Ipv4Routes | 68 / 13068 / 13136
Ipv6Routes Prefixlength < 65 | 0 / 5710 / 5744
Ipv6Routes Prefixlength > 65 | 0 / 112 / 112
Service manager resources

-------------------------
Total VPLS Svcs: 272/ 3072
Total L3 Svcs: 2/ 401
Total Mirror Svcs: 1/ 4
Total SAPs: 290/ 8191
Total SDP Bindings: 1/ 8191
Watermarks
Starting in VSP Release 20.5.R1, the 210 WBX provides the ability to configure high and low watermarks that define
the thresholds at which a message event is displayed in log 99 for forwarding information base (FIB) (Ipv4Routes and
Ipv6Routes Prefixlength) and MAC (L2 Forwarding Database (FDB) Entries) based on current consumption. Starting
in VSP Release 20.10.R1, the watermark for the Host tables are added. Host table is used for storing IPv4 Host Entries
and IPv6 Host Entries.
High and low watermark values are configured in percentages. Default values for high and low watermark are 85 and
70, respectively. The following output example shows the usage of the configure system watermarks command:
A:# configure system watermarks

[no] fib - Configure system fib watermarks
[no] host - Configure system host watermarks
[no] mac - Configure system MAC watermarks
A:# configure system watermarks fib

- fib high <percent-watermark> low <percent-watermark>
- no fib
<percent-watermark> : [1..99]
A:# configure system watermarks host

- host host <percent-watermark> low <percent-watermark>
- no mac
5.3. Watermarks 43
A:# configure system watermarks mac

- mac high <percent-watermark> low <percent-watermark>
- no mac
The command show system information has been enhanced to display the configured watermarks:
A:# show system information

* Truncated info
Fib High Watermark : 85
Fib Low Watermark : 70
Host High Watermark : 85
Host Low Watermark : 70
MAC High Watermark : 85
MAC Low Watermark : 70
Logs are also generated when the tables hit their maximum capacity:
• When the maximum capacity of the FIB table is reached, and routes are being rejected, an event in log 99 is
logged. The log displays used and free entries for IPv4 and IPv6 (short) and IPv6 (long). These reject events
are rate limited to 1 second for the first fifty entries. The rate limit is only applied when rejects are occurring in
bursts greater than 1 per second. Otherwise all rejects are logged in log 99.
• When the L2 Fdb Table has reached the limit of available MACs and no more MACs can be added (there can be
hash collisions before reaching the total capacity of the table), a log 99 message is displayed indicating than the
MAC table has had a collision while trying to learn a MAC. Information regarding the specific MAC entry such
as the MAC, vlan, port and service is displayed in the log.
• When the Host table has reached the limit of available hosts and no more hosts can be added, a console log is
displayed indicating the host that could not be added.
The command tools dump system-resources has been enhanced to display the rejects. Use the show system infor-
mation command to see the forwarding table statistics:
A:# show system information

/snip/
Forwarding Table Statistics | Used / Free / Max / Reject
---------------------------------+-----------------------------------
L2 Fdb Entries | 0 / 163839 / 163839
Ipv4 Host Entries | 10 / 147446 / 147456
Ipv6 Host Entries | 0 / 73723 / 73728
Ipv4Routes | 1 / 13135 / 13136 / 0
Ipv6Routes Prefixlength < 65 | 0 / 5744 / 5744 / 0
Ipv6Routes Prefixlength > 65 | 0 / 112 / 112 / 0
Event Examples
• Event when the FIB table crosses the LOW watermark value while adding routes.
280 2020/05/14 06:37:45.40 UTC CRITICAL: IOM #2001 Base IOM

"IOM:FIB utilization crossed its LOW watermark level of 15% for IPv4. FIB can
˓→accommodate 11165 IPv4 routes or 5582 IPv6(short) routes "
• Event when the FIB table crosses the HIGH watermark value while adding routes.
5.3. Watermarks 44

"IOM:FIB utilization crossed its HIGH watermark level of 55% for IPv4. FIB can
˓→accommodate 5911 IPv4 routes or 2955 IPv6(short) routes "
• Event when the FIB table crosses the LOW watermark value while releasing routes.
"IOM:FIB utilization crossed its LOW watermark level of 15% for IPv4.
˓→FIB can accommodate 11167 IPv4 routes or 5583 IPv6(short) routes "
• Events for Reject route

"IOM:Route reject for Prefix 0:200.1.76.0/24. Usage (used/free) : IPv4
˓→(13136/0) IPv6 (0/0) IPv6-long (0/112) Reject count 32 "

"IOM:Route reject for Prefix 0:200.1.75.0/24. Usage (used/free) : IPv4
˓→(13136/0) IPv6 (0/0) IPv6-long (0/112) Reject count 31 "
• Event when the MAC table crosses the LOW watermark while learning new MACs.
"IOM:MAC table utilization has crossed its LOW watermark level of 1%.
˓→Total capacity (40959) Consumed (410) Can further accomodate (40549)
˓→entries."
• Event when the MAC table crosses the HIGH watermark while learning new MACs.
"IOM:MAC table utilization has crossed its HIGH watermark level of 2%.
˓→entries."
• Event when the MAC table crosses the LOW watermark while releasing MAC entries.
"IOM:MAC table utilization has crossed its LOW watermark level of 1%.
˓→entries."
• Event for MAC collision.

"IOM:MAC table has hit collision while trying to add MAC (00:11:00:00:9f:
˓→fc) VID (0x1f9) Port (0x44) Svc Id (10) "
• Event when host table cross LOW watermark value while adding host entries.
"IOM:Host table utilization has crossed its LOW watermark level of 50%.
˓→Total capacity (106496) Consumed (53249) Can further accomodate
˓→v4 (53247) and v6 (26623)entries."
• Event when host table cross HIGH watermark value while adding host entries.
"IOM:Host table utilization has crossed its HIGH watermark level of 74%.
˓→Total capacity (106496) Consumed (78808) Can further accomodate v4
˓→(27688) and v6 (13844) entries."
5.3. Watermarks 45
• Event when host table cross LOW watermark value while releasing host entries.
"IOM:Host table utilization has crossed its LOW watermark level of 50%.
˓→Total capacity (106496) Consumed (53247) Can further accomodate v4
˓→(53249) and v6 (26624)entries."
• Console log when host table is full

[002 m 10/27/20 07:26:14.684] 1:iomRadix-1:IOM:program_l3_l3table bcm_l3_
˓→host_add failed for IP 0/3000:1:1:1::752d: Table full egr:100014 hash_
˓→index :0xffffffff egr-flags 0 l3_aflags:0x0
Note: Ipv4Routes and Ipv6Routes Prefixlength <65 share the same table. The consumption of any of them impact
the availability numbers of the other one. The same applies to the host table, for Ipv4 Host Entries and Ipv6 Host
Entries.
Dynamic ECMP Group
WBX supports ECMP groups in hardware, and starting in VSP Release 5.3.2 U13, group sizes are configurable. By
default, the ECMP hardware group size is blocks of 64 (as ECMP support by default is 64).
*A:WBX# show bof

===============================================================================
BOF (Memory)
===============================================================================
no auto-boot
autonegotiate
duplex full
speed 100
wait 3
persist off
hw-max-ecmp 64
To configure this feature, enter the configure bof hw-max-ecmp <max-ecmproutes> command, where <max-ecmp-
routes> is [2-64].
*A:WBX# configure bof hw-max-ecmp <max-ecmproutes>
The configuration of a smaller value might be needed for scenarios that require a large number of ECMP groups that
are not using ECMP 64.
Note: Any modifications to the default value requires a reboot (like any other BOF parameter).
The command “tools dump system-resources” has been enhanced to display the ECMP groups consumption:
5.4. Dynamic ECMP Group 46

A:# tools dump system-resources | match ECMP post-lines 3
ECMP Group Statistics | Used / Free / Max

---------------------------------+-----------------------
Entries | 1 / 255 / 256
210 WBX ACL Scale Enhancements
The 210 WBX chip set allows to configure ACLs globally or assign ACLs per pipelines. Starting GA support in VSP
Release 6.0.2, the 210 WBX system support enables ACL per pipelines, where all ports belong to a specific pipeline,
out of 4 pipelines. Mapping of ports to pipelines is fixed and not configurable.
When the system comes up the ACL is in Pipeline mode: ‘ hw-pipe line mode‘ is “ON” and shows 1. This port to
pipeline mapping is different in the 210 WBX Leaf and the 210 WBX Spine:
*A:WBX-A# /show bof

===============================================================================
BOF (Memory)
===============================================================================
primary-config ftp://*:*@135.250.255.7/./configs/ACL/VSD_EVPN/DUT-
˓→A.cfg

no auto-boot
lookup-table-profile host-focused
autonegotiate
duplex full
speed 100
wait 3
persist off
hw-max-ecmp 64
hw-pipeline-mode 1
===============================================================================
Host Information
===============================================================================
linuxhost-ipv6-address-1 fe80::d299:d5ff:feb8:5441 prefixlen 64
˓→0x20<link>
Note: To enable the ACL global mode, BOF option hw-pipeline-mode must be set to 0 and it is mandatory to reboot
the system.
To set the ACL global mode and to set the hw-pipeline-mode to 0, enter the following commands:
*A:WBX-A# /bof hw-pipeline-mode 0

*A:WBX-A# /bof save
*A:WBX-A# /admin reboot now
After the reboot, the system comes up with Global mode. You can verify using the /show bof command.
In hw-pipeline mode, /tools dump system-resources displays the resources output for all 4 pipelines:
5.5. 210 WBX ACL Scale Enhancements 47

*A:WBX-A>config>service# /tools dump system-resources

Resource Manager info at 011 m 06/21/19 05:17:48.470:
Hardware Resource Usage for Slot #1, CardType iom-48-25g+6-100g, Pipe #0:
-------------------------------+-----------+-----------+------------
-------------------------------+-----------+-----------+------------
-------------------------------+-----------+-----------+------------
-------------------------------+-----------+-----------+------------


HashTable Statistic | Count

---------------------------------+-------------
TlsFdbHashBucketFull | 0
TlsFdbLearntMacOverwritten | 0
pipIpHashBucketFull | 0
pipIpHashBucketOverwrites | 0
ProfileTable Info | Count

DscpProfileCount | 0
DscpProfileFreeCount | 125
Forwarding Table Statistics | Used / Free / Max

---------------------------------+-----------------------
L2 Fdb Entries | 7 / 73720 / 73727
Ipv4 Host Entries | 0 / 73728 / 73728
Ipv6 Host Entries | 0 / 36864 / 36864
Ipv4Routes | 0 / 13136 / 13136
Ipv6Routes Prefixlength < 65 | 0 / 5744 / 5744
Ipv6Routes Prefixlength > 65 | 0 / 112 / 112

-------------------------
Total SAPs: 5/ 8191
=================================================================
To map the port to pipeline, enter the /tools dump front-port-to-timos-port command:
*A:WBX-A>config>service# /tools dump front-port-to-timos-port

--------------------------------------------------------------------
Front panel port | TiMOS port | HW PortNo | HW PipeNo
--------------------------------------------------------------------
1 | 1/1/1 | 68 | 2
2 | 1/1/2 | 69 | 2
3 | 1/1/3 | 70 | 2
4 | 1/1/4 | 71 | 2
5 | 1/1/5 | 72 | 2
6 | 1/1/6 | 73 | 2
7 | 1/1/7 | 74 | 2
8 | 1/1/8 | 75 | 2
9 | 1/1/9 | 84 | 2
10 | 1/1/10 | 85 | 2

11 | 1/1/11 | 86 | 2
12 | 1/1/12 | 87 | 2
13 | 1/1/13 | 88 | 2
14 | 1/1/14 | 89 | 2
15 | 1/1/15 | 90 | 2
16 | 1/1/16 | 91 | 2
17 | 1/1/17 | 34 | 1
18 | 1/1/18 | 35 | 1
19 | 1/1/19 | 36 | 1
20 | 1/1/20 | 37 | 1
21 | 1/1/21 | 38 | 1
22 | 1/1/22 | 39 | 1
23 | 1/1/23 | 40 | 1
24 | 1/1/24 | 41 | 1
25 | 1/1/25 | 42 | 1
26 | 1/1/26 | 43 | 1
27 | 1/1/27 | 44 | 1
28 | 1/1/28 | 45 | 1
29 | 1/1/29 | 50 | 1
30 | 1/1/30 | 51 | 1
31 | 1/1/31 | 52 | 1
32 | 1/1/32 | 53 | 1
33 | 1/1/33 | 102 | 3
34 | 1/1/34 | 103 | 3
35 | 1/1/35 | 104 | 3
36 | 1/1/36 | 105 | 3
37 | 1/1/37 | 106 | 3
38 | 1/1/38 | 107 | 3
39 | 1/1/39 | 108 | 3
40 | 1/1/40 | 109 | 3
41 | 1/1/41 | 110 | 3
42 | 1/1/42 | 111 | 3
43 | 1/1/43 | 112 | 3
44 | 1/1/44 | 113 | 3
45 | 1/1/45 | 122 | 3
46 | 1/1/46 | 123 | 3
47 | 1/1/47 | 124 | 3
48 | 1/1/48 | 125 | 3
49 | 1/2/1 | 5 | 0
50 | 1/2/5 | 1 | 0
51 | 1/2/9 | 13 | 0
52 | 1/2/13 | 9 | 0
53 | 1/2/17 | 21 | 0
54 | 1/2/21 | 17 | 0
==================================================================
Reverting to ACL Global Mode
If a downgrade is needed or for troubleshooting purposes, you must roll back to the global mode.
When reverting from hw-pipeline-mode = 1 to hw-pipeline-mode = 0 you need to ensure that the configuration file
properly executes upon reboot as the system is moving from a higher scale mode to a lower scale mode.
Note: If you want to move back to ACL global mode with “Higher Filter Entries,” all the saved configu-
rations may not get executed.

In this case, the configuration might fail to reload upon boot due to resource exhaustion. This means the configuration
run aborts on the line that caused the exhaustion to be triggered. This could result in partial or incomplete configuration,
in some cases, with critical parts missing (such as a router interface) if enough ACLs are used in previous entries.
If the ACL scale is driven exclusively from VSD driven ACLs, then this problem may not manifest itself. The 210
WBX could still have issues trying to activate the desired number of ACLs from VSD and result in “failed” vPorts but
would not result in a failed CLI configuration.
Workaround
You can either remove the needed number of ACLs using the CLI previous to rebooting or download the configuration
file or edit using the built-in VI editor and remove the references to the ACLs under the SAPs/interfaces/services that
are using them. Then modify the BOF to set the hw-pipeline-mode to global (mode 0) and reboot using the new
configuration file.
In any case, after rebooting a node after changing the ACL mode, check the bootlog messages (using the show boot-
messages command) to ensure the configuration has been fully loaded and no part was aborted. Only then you can
save the configuration and start re-applying the previously removed ACLs.
VRF Scalability Enhancements
Starting in VSP Release 20.10.R1 the 210 WBX VRF Scalability Enhancements feature increases the number of
interfaces and L3 services (L3 domains or VRFs) available on the 210 WBX 48 SFP28 6QSFP28. When enabling the
feature, the additional interfaces come at the cost of consuming additional resources on the forwarding information
base (FIB) table and number of SAPs or vPorts. The feature allows to configure up to 400 VRFs (L3 domains) with
two subnets each.
The MDA3 of the 210 WBX has 14 100GE ports dedicated to LAG-97 by default. When enabling this feature, LAG-
97 is reduced to the usage of only 2 or 3 ports (configurable), rest of the ports of MDA3 cannot be used. To enable
or disable this feature, a cold reboot is needed because it impacts the sizing of the internal ternary content-addressable
memory (TCAM) memories.
Restrictions
• This feature is not supported on the 210 WBX 32QSFP28.

• Release 20.10.R1 only supports 2port-lag97; 3port-lag97 is not yet supported.
• The system cannot support more than 400 network interfaces when using IPv6 longer routes (Ipv6 Routes
Prefixlength > 65). Interfaces are created when defining interfaces using CLI and also subnets in L3 Domains
from VSD. If the limit is achieved, the offending L3 service is brought down.
Important: VRF scalability feature is very demanding in terms of ECMP groups and ECMP hardware entries needed.
The number of ECMP groups and entries consumed depend not only on current remote nodes and subnets but also
on remote nodes being configured with new subnets/domains, new nodes deployments, or during instabilities like
reboots of MCLAG MCS nodes (MCS nodes reboots transition from VXLAN Multichassis VTEP IP to system IP
advertisements). ECMP programming is always make-before-break, so in case of changes in the number of remote
nodes/subnets, to cope with the changes, additional free ECMP groups need to be available in hardware to handle these
transitions. The recommended max value to use this feature is “bof hw-max-ecmp 16”, but network administrators
should configure the max ecmp accordingly based on the network requirements, so less than 16 might be needed in
some cases. For information about how to configure the ECMP groups, please read Dynamic ECMP Group (page 46)
section.
5.6. VRF Scalability Enhancements 51

Configuring VRF Scalability
Step 1 Enter the bof ip-interface-max 2port-lag97 command to configure the maximum IP interfaces
with the chosen LAG-97 ports option:
*A:WBX-X# bof ip-interface-max 2port-lag97
*A:WBX-X# bof ip-interface-max

- ip-interface-max {2port-lag97|3port-lag97}
- no ip-interface-max
<2port-lag97|3port*> : keywords
Step 2 Enter the bof save command.
Note: To enable the ip-interface-max BOF option it is mandatory to reboot the system.
Step 3 Enter the admin reboot now command.

Step 4 After the reboot, enter the show bof command to verify that the feature is enabled:
*A:WBX-A# show bof

===============================================================================
BOF (Memory)
===============================================================================
primary-config ftp://*:*@135.250.255.7/./images/dut-a.cfg
no auto-boot
autonegotiate
duplex full
speed 100
wait 3
persist off
hw-max-ecmp 64
hw-pipeline-mode 1
ip-interface-max 2port-lag97
===============================================================================
Host Information
===============================================================================
linuxhost-ipv6-address-1 fe80::d299:d5ff:feb8:5441
˓→prefixlen 64 0x20<link>
===============================================================================
The command tools dump system-resources has been enhanced to display the number of available RVPLS interfaces.
Also note the capacity for IPv4 routes and IPv6 Routes Prefix length <65 has been reduced to accommodate the
required interfaces to support the new VRF scalability. The ECMP group statistics are also available in the tools
dump system-resources command:
*A:WBX-A# tools dump system-resources
* Truncated info
5.6. VRF Scalability Enhancements 52

Forwarding Table Statistics | Used / Free / Max / Reject

---------------------------------+-----------------------------------
L2 Fdb Entries | 24 / 98279 / 98303
Ipv4 Host Entries | 83 / 206741 / 206848
Ipv6 Host Entries | 12 / 103370 / 103424
Ipv4Routes | 212 / 10724 / 11536 / 0
Ipv6Routes Prefixlength < 65 | 300 / 5044 / 5344 / 0
Ipv6Routes Prefixlength > 65 | 0 / 112 / 112 / 0

-------------------------
Total SAPs: 1025/ 8191
Total RVPLS Interfaces: 513/ 1200
ECMP Group Statistics | Used / Free / Max

---------------------------------+-----------------------
Entries | 500 / 524 / 1024
210 WBX Control Plane Hardware Queues Monitoring
Starting in VSP Release 6.0.1, the visualization only of CPU hardware queues via CLI commands is supported showing
information about control plane protocols mapping in queues and packets processed/dropped.
Note: Hardware queues are not configurable.
Verifying and Clearing CPU
You can enter the following commands to display the status of the CPU statistics or clear the CPU statistics:
• show system cpu-queue-map
• show system cpu-queue-stats
• /clear cpu-queue-stats
To display the system CPU queue map, enter the show system cpu-queue-map command:
*A:WBX-A# show system cpu-queue-map
===============================================================================
CPU Queues to Protocol Mapping
===============================================================================
Protocol-name : bfd pkts
CPU Queue : 43
Protocol-name : bgp pkts

CPU Queue : 39
Protocol-name : lacp pkts
5.7. 210 WBX Control Plane Hardware Queues Monitoring 53

CPU Queue : 39
Protocol-name : ospf pkts

CPU Queue : 39
Protocol-name : isis pkts

CPU Queue : 39
Protocol-name : xmpp pkts

CPU Queue : 39
Protocol-name : mc-lag pkts

CPU Queue : 36
Protocol-name : lldp pkts

CPU Queue : 34
Protocol-name : arp reply pkts

CPU Queue : 33
Protocol-name : arp request pkts

CPU Queue : 32
Protocol-name : icmp broadcast pkts

CPU Queue : 31
Protocol-name : telnet pkts

CPU Queue : 30
Protocol-name : radius pkts

CPU Queue : 30
Protocol-name : tacacs pkts

CPU Queue : 30
Protocol-name : snmp pkts

CPU Queue : 30
Protocol-name : ssh pkts

CPU Queue : 29
Protocol-name : icmp pkts

CPU Queue : 28
Protocol-name : ftp pkts

CPU Queue : 27
Protocol-name : icmp redirect pkts

CPU Queue : 27
Protocol-name : ip data pkts

CPU Queue : 25
Protocol-name : l3 destination failure pkts

CPU Queue : 24
Protocol-name : default l3 pkts

CPU Queue : 24

===============================================================================
Note: The hardware queue to protocol mapping is as shown in the command. There are a few cases where protocols
can use a different queue depending on the entry interface to the system. There are three entry interfaces types: global
routing interface, rVPLS, or VPRN SAP.
To display the system CPU queue statistics, enter the show system cpu-queue-stats command:
*A:WBX-A# show system cpu-queue-stats
======================================
CPU Queue Stats
======================================
CPU Packets Packets
Queue Processed Dropped
--------------------------------------
24 0 0
25 0 0
26 0 0
27 0 0
28 0 0
29 0 0
30 0 0
31 0 0
32 0 0
33 0 0
34 0 0
35 0 0
36 17727 0
37 0 0
38 0 0
39 5489 0
43 0 0
======================================
To clear CPU queue stats, enter the /clear cpu-queue-stats command:
*A:WBX-A# /clear cpu-queue-stats
After clearing the CPU queue stats and to check system CPU queue statistics, enter the show system cpu-queue-stats
command:
*A:WBX-A# show system cpu-queue-stats
======================================
CPU Queue Stats
======================================
CPU Packets Packets
Queue Processed Dropped
--------------------------------------
24 0 0
25 0 0
26 0 0
27 0 0
28 0 0

29 0 0
30 0 0
31 0 0
32 0 0
33 0 0
34 0 0
35 0 0
36 7 0
37 0 0
38 0 0
39 3 0
43 0 0
======================================
Quality of Service
The 7850 VSG and the 210 WBX are slightly different in terms of Egress QoS and Queue hierarchy.
• The 7850 VSG has one scheduler and then 8 queues which are scheduled based on the config (strict and wrr
queues).
• The 210 WBX has one scheduler per forwarding class (FC) per port. So the wrr-weight is configured per FC
and not within the queue.
210 WBX Egress QoS model has one scheduler per port per FC available to be configured. This makes a total of
8 schedulers per port. Each of these schedulers have one multicast and one unicast queue feeding them. Default
configuration sets FC-8 to FC-5 in strict priority, and FC-4 to FC-1 to operate in weighted-round-robin (FC, from
high priority to low priority are NC, H1, EF, H2, L1, AF, and BE). However, the CLI allows for more schedulers
to be brought into weighted-round-robin by appropriately configuring the wrr-queues <from-fc> to <to-fc>. Once
the schedulers are not assigned from the strict priority and are provisioned as wrr, they are available for weight
configuration under fc <> sched-wrr-weight <weight>, where weight ranges from 1 to 100.
The sample output shows the default QOS policy on the 210 WBX:
A:# configure qos egress 1

A:as5712>config>qos>egress# info detail
----------------------------------------------
description "Default egress QoS policy."
no remarking
dot1p-remarking
wrr-queues 4 to 1
no policy-name
queue 1 create
adaptation-rule pir closest cir closest
rate max cir 0
mbs default
exit
queue 2 create
rate max cir 0
mbs default
exit
queue 3 create
rate max cir 0
mbs default
5.8. Quality of Service 56

exit
queue 4 create
rate max cir 0
mbs default
exit
queue 5 create
rate max cir 0
mbs default
exit
queue 6 create
rate max cir 0
mbs default
exit
queue 7 create
rate max cir 0
mbs default
exit
queue 8 create
rate max cir 0
mbs default
exit
queue 11 multipoint create
rate max cir 0
mbs default
exit
rate max cir 0
mbs default
exit
rate max cir 0
mbs default
exit
rate max cir 0
mbs default
exit
rate max cir 0
mbs default
exit
rate max cir 0
mbs default
exit

rate max cir 0

mbs default
exit
rate max cir 0
mbs default
exit
fc af create
queue 3
multipoint-queue 13
dot1p-in-profile 2
dot1p-out-profile 2
no de-mark
dscp-in-profile af11
dscp-out-profile af12
sched-wrr-weight 1
exit
fc be create
queue 1
multipoint-queue 11
dot1p-in-profile 0
dot1p-out-profile 0
no de-mark
dscp-in-profile be
dscp-out-profile be
sched-wrr-weight 1
exit
fc ef create
queue 6
multipoint-queue 16
dot1p-in-profile 5
dot1p-out-profile 5
no de-mark
dscp-in-profile ef
dscp-out-profile ef
no sched-wrr-weight
exit
fc h1 create
queue 7
multipoint-queue 17
dot1p-in-profile 6
dot1p-out-profile 6
no de-mark
dscp-in-profile nc1
dscp-out-profile nc1
no sched-wrr-weight
exit
fc h2 create
queue 5
multipoint-queue 15
dot1p-in-profile 4
dot1p-out-profile 4
no de-mark
no sched-wrr-weight
exit

fc l1 create
queue 4
multipoint-queue 14
dot1p-in-profile 3
dot1p-out-profile 3
no de-mark
sched-wrr-weight 1
exit
fc l2 create
queue 2
multipoint-queue 12
dot1p-in-profile 1
dot1p-out-profile 1
no de-mark
dscp-in-profile cs1
dscp-out-profile cs1
sched-wrr-weight 1
exit
fc nc create
queue 8
multipoint-queue 18
dot1p-in-profile 7
dot1p-out-profile 7
no de-mark
dscp-in-profile nc2
no sched-wrr-weight
exit
----------------------------------------------
For ease of comparison, the following output example shows the default QOS policy for the 7850 VSG:
*A:>config>qos>egress$ info detail

----------------------------------------------
no description
no remarking
no dot1p-remarking
wrr-queues 4 to 1
no policy-name
queue 1 create
rate max cir 0
mbs default
wrr-weight 1
exit
queue 2 create
rate max cir 0
mbs default
wrr-weight 1
exit
queue 3 create
rate max cir 0
mbs default
wrr-weight 1

exit
queue 4 create
rate max cir 0
mbs default
wrr-weight 1
exit
queue 5 create
rate max cir 0
mbs default
no wrr-weight
exit
queue 6 create
rate max cir 0
mbs default
no wrr-weight
exit
queue 7 create
rate max cir 0
mbs default
no wrr-weight
exit
queue 8 create
rate max cir 0
mbs default
no wrr-weight
exit
rate max cir 0
mbs default
wrr-weight 1
exit
rate max cir 0
mbs default
wrr-weight 1
exit
rate max cir 0
mbs default
wrr-weight 1
exit
rate max cir 0
mbs default
wrr-weight 1
exit
rate max cir 0

mbs default
no wrr-weight
exit
rate max cir 0
mbs default
no wrr-weight
exit
rate max cir 0
mbs default
no wrr-weight
exit
rate max cir 0
mbs default
no wrr-weight
exit
fc af create
queue 3
multipoint-queue 13
dot1p-in-profile 2
dot1p-out-profile 2
no de-mark
exit
fc be create
queue 1
multipoint-queue 11
dot1p-in-profile 0
dot1p-out-profile 0
no de-mark
dscp-in-profile be
dscp-out-profile be
exit
fc ef create
queue 6
multipoint-queue 16
dot1p-in-profile 5
dot1p-out-profile 5
no de-mark
dscp-in-profile ef
dscp-out-profile ef
exit
fc h1 create
queue 7
multipoint-queue 17
dot1p-in-profile 6
dot1p-out-profile 6
no de-mark
dscp-in-profile nc1
exit
fc h2 create

queue 5
multipoint-queue 15
dot1p-in-profile 4
dot1p-out-profile 4
no de-mark
exit
fc l1 create
queue 4
multipoint-queue 14
dot1p-in-profile 3
dot1p-out-profile 3
no de-mark
exit
fc l2 create
queue 2
multipoint-queue 12
dot1p-in-profile 1
dot1p-out-profile 1
no de-mark
dscp-in-profile cs1
dscp-out-profile cs1
exit
fc nc create
queue 8
multipoint-queue 18
dot1p-in-profile 7
dot1p-out-profile 7
no de-mark
dscp-in-profile nc2
exit
Forward Error Correction and Auto Negotiation
Forward error correction (FEC) is applicable to 25GE ports (clause 74) and 100GE ports (clause 91). FEC is disabled
for interoperability with Nokia 7X50 portfolio (note that this might not be the case in platforms belonging to other
vendors where FEC might be enabled by default). Recommendation is to configure FEC whenever is possible, as
25GE and 100GE are prone to errors such as CRC and FCS.
Note: Auto negotiation (AN) is not supported for 1G port in WBX.
# configure port ethernet [no] forward-error-correction
Auto Negotiate (AN) is applicable to 25G and 100G ports with copper interfaces. AN is enabled by default and only
is applied when a port is operating at 25G or 100G and a copper SFP has been detected. AN is not supported for 1G
port in the 210 WBX.
# configure port ethernet [no] autonegotiate

Note: 210 WBX does have a limitation for AN and 100G FEC. Although it supports AN and supports 100G FEC,
it does not support negotiating 100GE FEC. If the peer sets its negotiation to support and require FEC, the link does
not come up with the default settings. If the peer sets its negotiation to support FEC, the link comes up, but without
FEC. You must either disable AN on both the 210 WBX and the peer to get the link to come up with FEC, or disable
FEC on the peer to get AN to bring the link up without FEC. Within 210 WBX and 7850 the link comes up with AN
enabled and FEC disabled (default values). In general the link comes up as long as the FEC setting is the same on both
ends and auto negotiate is enabled on both ends. FEC is negotiated correctly on 25G ports.
Tech Support
Similar to the admin tech-support command available on SROS to gather the necessary logs and configuration files
for technical support purposes, the 210 WBX models have a similar utility.
Two admin tech-supports are needed:
1. Admin tech-support for the hypervisor
2. Admin tech-support for the VM (same as a hardware VSG or 7750)
Admin tech-support for the Hypervisor
For the hypervisor, the utility is /usr/nuage/support/tech-support and it executes commands listed in
/usr/nuage/support_commands, and other contents that exist in /usr/nuage/logs and /var/log.
The result is archived and stored in /run/nuage-support/.
The following display shows an example output:
[root@localhost support]# ./tech-support

Running command: cat /etc/issue
Running command: uname -a
Running command: rpm -qa | egrep "kernel|vsg|nuage"
Running command: /usr/nuage/onie-syseeprom
Running command: cat /var/run/oniepfenv.json
Running command: dmidecode
Running command: cat /etc/sysconfig/network-scripts/ifcfg-mgmt
Running command: ifconfig -a
Running command: ip link show
Running command: netstat -r
Running command: cat /etc/resolv.conf
Running command: firewall-cmd --list-all
Running command: iptables -L
Running command: blkid
Running command: lsblk
Running command: df -hT
Running command: mount
Running command: cat /proc/mounts
Running command: virsh list --all
Running command: systemctl status -l vsgvm.service
Running command: systemctl status -l nuage-rpc.service
Running command: dmesg
Running command: mpstat -P ALL 5 1
Running command: top -n 1 -b
Running command: sleep 3
5.9. Tech Support 63

Running command: top -n 1 -b

Running command: lspci -k
Running command: cat /etc/udev/rules.d/80-nuage-net-names.rules
Running command: find /usr/nuage -exec \ls -ls {} \;
Running command: ls -ls /vsgx-data
Running command: ls -ls /vsgx-sd
Running command: /usr/nuage/support/dump-virsh-info
Copying logs...
============================================================================
Tech support file is at:
/run/nuage-support/tech-support-Fri_Aug_11_185112_EDT_2017.tgz
The file is kept with the system for only 2 hours.
Upload this file soon to technical support for analysis.
============================================================================
Note: Although, the Tech support file is marked for deletion after 2 hours, it may not be deleted from the file system
until after 26 hours have passed due to systemd-tmpfiles running every 24 hours. If the Tech Support file is created
4 hours before the systemd-tmpfiles runs, then it is deleted in less time. The maximum time needed is 26 hours.
However, if the Tech Support file is generated 1 hour 59 minutes before systemd-tmpfiles runs, the file does not get
deleted during that pass and gets deleted only after 24 hours later when systemd-tmpfiles runs again.
Admin tech-support for the SROS VM
For the SROS VM, tech-support is executed as normal (saved to a local file, or to an ftp server). For more info about
how to copy a file from the VM to the hypervisor host, see the VSG/VSA and 210 WBX section.
A:# admin tech-support
- tech-support <file-url>
<file-url> : <local-url>|<remote-url>
local-url - [<cflash-id>/][<file-path>]
200 chars max, including cflash-id
directory length 99 chars max each
remote-url - [{ftp://|tftp://}<login>:<pswd>@
<remote-locn>/][<file-path>]
255 chars max
directory length 99 chars max each
remote-locn - [ <hostname> | <ipv4-address> | <ipv6-
˓→address> ]
ipv4-address - a.b.c.d
ipv6-address - x:x:x:x:x:x:x:x[-interface]
x:x:x:x:x:x:d.d.d.d[-interface]
x - [0..FFFF]H
d - [0..255]D
interface - 32 chars max, for link local
˓→addresses
cflash-id - cf1:|cf1-A:|cf1-B:|cf2:|cf2-A:|cf2-B:
A:WBX-LEAF2# admin tech-support cf1:/techlog

Processing CPM...
Second Pass
Processing CPM...

Processing IOM in Slot 1...

Processing MDA in 1/1...
Done...
A:# file dir
Directory of cf1:\
07/05/2017 09:25p 307bof.cfg

08/23/2017 09:56p 2637bootlog.txt
08/11/2017 09:53p 2637bootlog_prev.txt
04/19/2017 09:52a 10413config.cfg
04/19/2017 09:43a 9746config.cfg.1
04/19/2017 08:51a 8888config.cfg.2
07/05/2017 09:25p 0config.cfg.3
08/23/2017 09:56p 319nvsys.info
08/22/2017 11:29p 167836tech
08/24/2017 12:05a 167764techlog
10 File(s) 370547 bytes.
0 Dir(s) 221425664 bytes free.

CHAPTER
SIX
CONFIGURATION OF THE 210 WBX MODELS WITH NUAGE

SOFTWARE
• 210 WBX - 32QSFP28 (page 66)

– Chassis (page 66)
– Cards and MDAs (page 68)
– Port Configuration (page 71)
– LAG-97 and LAG-98 (page 72)
• 210 WBX - 48SFP28 6QSFP28 (page 73)
– Chassis (page 73)
– Cards and MDAs (page 74)
– Port Configuration (page 78)
* 25GE Capable Ports (page 78)

* 100GE Capable Ports (page 79)
– LAG-97 and LAG-98 (page 80)
The 210 WBX models have many specific and common configuration steps when running with the Nuage software.
The details are provided in the individual sections.
• 210 WBX - 32QSFP28
• 210 WBX - 48SFP28 6QSFP28
210 WBX - 32QSFP28
Chassis
210 WBX chassis requires a new personality (personality is also used for management from VSD and VSAP).
To get the details of the chassis,enter the show chassis command. The sample output shows the Type of a 210 WBX
32QSFP28, shows the environment info (five fans) and the power supply info.
*A:210-WBX-32Q# show chassis
===============================================================================
66
Chassis Information
===============================================================================
Name : 210-WBX-32Q
Type : Nuage Networks 210 WBX-32Q
Location :
Coordinates :
CLLI code :
Number of slots : 2
Number of ports : 128
Major LED state : Off
Minor LED state : Off
Over Temperature state : OK
Base MAC address : d0:99:d5:93:0e:41
Virtual Chassis(VC) mode : Disabled
Hardware Data
Part number : Sim Part#
CLEI code : Sim CLEI
Serial number : NS1711T0016
Manufacture date : 01012003
Manufacturing deviations : Sim MfgDeviation wbx210-32q
Manufacturing assembly number : 01-2345-67
Time of last boot : 2017/07/12 01:18:13
Current alarm state : alarm cleared
-------------------------------------------------------------------------------
Environment Information
Number of fan trays : 5

Number of fans : 10
Fan tray number : 1

Speed : half speed
Status : up
Fan tray number : 2

Speed : half speed
Status : up
Fan tray number : 3

Speed : half speed
Status : up
Fan tray number : 4

Speed : half speed
Status : up
Fan tray number : 5

Speed : half speed
Status : up
-------------------------------------------------------------------------------

Status : up

6.1. 210 WBX - 32QSFP28 67

Status : up
===============================================================================
Cards and MDAs
Note: The cards and MDAs in a 210 WBX are different compared to a 7850 VSG.
The 210 WBX 32QSFP28 system has an iom-32-100g (that indicates that it is a 32p 100GE) and an sfm-210-WBX.
*A:210-WBX-32Q# show card
===============================================================================
Card Summary
===============================================================================
Slot Provisioned Type Admin Operational Comments
Equipped Type (if different) State State
-------------------------------------------------------------------------------
1 iom-32-100g up up
A sfm-210-WBX up up/active
===============================================================================
The system shows that there are two MDAs, and each MDA has 16 ports 100GE QSFP28
*A:210-WBX-32Q# show mda
===============================================================================
MDA Summary
===============================================================================
Slot Mda Provisioned Type Admin Operational
-------------------------------------------------------------------------------
1 1 m16-100g-qsfp28 up up
2 m16-100g-qsfp28 up up
===============================================================================
Physically, the numbering of the ports is from left to right, up and down (1 to 32). However, the physical number does
not match the software port numbering. As the Nuage 210 WBX reuses the 7850 VSG Nuage SROS, the software has
been adapted to expand each 100GE physical port into four different ports (a single 100GE can break out into four
different ports), maintaining the concept of IOM cards and MDAs used on VSG.
6.1. 210 WBX - 32QSFP28 68

Fig. 6.1: 210 WBX 32QSFP28 MDAs and Port Numbering
• Physical port number 1 (upper left in the figure), corresponds to port 1/1/1 (card 1/mda1/port number). If
expanded by configuration with a breakout to 4x25G, or 4x10GE, then ports 1/1/2, 1/1/3, and 1/1/4 become
visible. If expanded by configuration with a breakout of 2x50GE, then only port 1/1/3 becomes visible.
• Physical port number 32 (lower right in the figure), corresponds to port 1/2/61 (card 1/mda1/port number) . If
To see the available ports, enter the show port command. The following example output shows the 32 ports available
(not using breakout configuration):
*A:210-WBX-32Q# show port
===============================================================================
Ports on Slot 1
===============================================================================
Port Admin Link Port Cfg Oper LAG/ Port Port Port C/QS/S/XFP/
Id State State MTU MTU Bndl Mode Encp Type MDIMDX
-------------------------------------------------------------------------------
1/1/1 Up Yes Up 9212 9212 - hybr dotq lgige 40GBASE-CR4
1/1/9 Down No Down 9212 9212 - hybr null cgige
6.1. 210 WBX - 32QSFP28 69


===============================================================================
Ports on Slot A
===============================================================================
-------------------------------------------------------------------------------
A/1 Up Yes Up 1514 1514 - netw null faste MDI
===============================================================================
To verify the front port mapping in SROS, enter the tools dump front-port-to-timos-port command:
*A:210-WBX-32Q# tools dump front-port-to-timos-port

-----------------------------------
Front panel port | TiMOS port
-----------------------------------
1 | 1/1/1
2 | 1/1/5
3 | 1/1/9
4 | 1/1/13
5 | 1/1/17
6 | 1/1/21
7 | 1/1/25
8 | 1/1/29
9 | 1/1/33
10 | 1/1/37
11 | 1/1/41
12 | 1/1/45
13 | 1/1/49
14 | 1/1/53
15 | 1/1/57
16 | 1/1/61
17 | 1/2/1
18 | 1/2/5
19 | 1/2/9
20 | 1/2/13
21 | 1/2/17
22 | 1/2/21
23 | 1/2/25
24 | 1/2/29
25 | 1/2/33
26 | 1/2/37
27 | 1/2/41
28 | 1/2/45
29 | 1/2/49
30 | 1/2/53
6.1. 210 WBX - 32QSFP28 70

31 | 1/2/57
32 | 1/2/61
Port Configuration
The 210 WBX 32QSFP28 supports 32 ports QSFP28/QSFP+, and each port is individually configurable as 100GE,
50GE, 40GE, 2x50GE, 4x25GE or 4x10GE. The media-mode keyword defines port-type. The default is 100G.
*A:210-WBX-32Q#configure port ethernet media-mode

- media-mode {4x10G|4x25G|40G|50G|2x50G|100G}
- no media-mode
<4x10G|4x25G|40G|5*> : 4x10G|4x25G|40G|50G|2x50G|100G
When you enter the show port command to check the Port Type, depending on the media-mode, the Port Type
automatically changes to show the following:
• 100 GE = cgige
• 50 GE = lgige
• 40 GE = xlgige
• 25 GE = xxvge
• 10 GE = xgige
Each port has 4 LEDs, one per port and show the port configuration and speed based on the color.
Fig. 6.2: 210 WBX 32QSFP28 LEDs per Port
6.1. 210 WBX - 32QSFP28 71

Table 6.1: 210 WBX 32QSFP28 LED Status Descriptions

LEDs Status Descriptions
LED 1
• Green - 100G operation
• Amber - 40G operation
• White - 50G operation
• Blue - 25G operation
• Purple - 10G operation
• Off - No Link
LED 2 - Working in breakout only

• Blue 25G operation
• Purple 10G operation
• Off 40G/100G operation (Assuming LED 1 is il-
luminated) or No Link

• White - 50G operation
• Blue - 25G operation
• Purple - 10G operation
• Off - 40G/100G operation (Assuming LED 1 is
illuminated) or No Link

• Blue 25G operation
• Purple 10G operation
Note: No link: Off / Linkup: Solid, LED / Traffic: Flashing
LAG-97 and LAG-98
LAG-97 and LAG-98 are configured by default in any VSG or 210 WBX.
• LAG-97 is used for the internal loop used when the system is configured as an L3 HW VTEP
• LAG-98 is used for the inter chassis (IC) connectivity when configuring MC-LAG
Note: For more info about MC-LAG, see the “Multichassis Link Aggregation Group” section in the Nuage VSP User
Guide.
LAG-97 (L3 loop) has no ports configured by default.
*A:210-WBX-32Q# configure lag 97

*A:210-WBX-32Q>config>lag# info
----------------------------------------------
description "System Loopback LAG"
encap-type dot1q
no shutdown
----------------------------------------------
6.1. 210 WBX - 32QSFP28 72

However, any port could be provisioned in the LAG-97. No real loop is needed (no external cables), loop is internal
but port cannot be used for anything else.
By default, LAG-98 (IC) has no ports configured either.
*A:210-WBX-32Q>config# lag 98
*A:210-WBX-32Q>config>lag# info
----------------------------------------------
description "Multichassis interconnect LAG"
encap-type dot1q
lacp active administrative-key 36864
no shutdown
----------------------------------------------
However, any port can be provisioned in the LAG-98. Note that in this case, cables are needed to interconnect both
MC-LAG nodes.
210 WBX - 48SFP28 6QSFP28
Chassis
210 WBX chassis requires a new personality (personality is also used for management from VSD and VSAP).
To get the details of the chassis, enter the show chassis command. The following output shows the Type of a 210
WBX 48SFP28 6QSFP28, shows the environment information (four fans) and the power supply information.
*A:210-WBX-48S# show chassis
===============================================================================
Chassis Information
===============================================================================
Name : 210-WBX-48S
Type : Nuage Networks 210 WBX-48S
Location :
Coordinates :
CLLI code :
Number of slots : 2
Number of ports : 86
Major LED state : Off
Minor LED state : Off
Over Temperature state : OK
Base MAC address : d0:99:d5:95:12:41
Virtual Chassis(VC) mode : Disabled
Hardware Data
Part number : Sim Part#
CLEI code : Sim CLEI
Serial number : NS1712T0021
Manufacture date : 01012003
Manufacturing deviations : Sim MfgDeviation wbx210-48s
Manufacturing assembly number : 01-2345-67
Time of last boot : 2017/02/09 10:50:04
Current alarm state : alarm cleared
-------------------------------------------------------------------------------
Environment Information
Number of fan trays : 4
6.2. 210 WBX - 48SFP28 6QSFP28 73

Number of fans : 8
Fan tray number : 1

Speed : half speed
Status : up
Fan tray number : 2

Speed : half speed
Status : up
Fan tray number : 3

Speed : half speed
Status : up
Fan tray number : 4

Speed : half speed
Status : up
-------------------------------------------------------------------------------

Status : up

Status : up
===============================================================================
Cards and MDAs
Note: The cards and MDAs in a 210 WBX are different compared to a 7850 VSG.
The 210 WBX 48SFP28 6QSFP28 system has an iom-48-25g+6-100g (that indicates that it is a 48 ports 25GE and 6
ports 100GE) and an sfm-210-WBX2.
*A:210-WBX-48S# show card
===============================================================================
Card Summary
===============================================================================
Slot Provisioned Type Admin Operational Comments
-------------------------------------------------------------------------------
1 iom-48-25g+6-100g up up
A sfm-210-WBX2 up up/active
===============================================================================
The system shows there are three MDAs. The first MDA contains the 48 ports 25GE SFP28, the second MDA contains
the 6 ports 100GE QSFP28, and in the third MDA there are the 14 internal 100GE ports (to be used in LAG-97)
*A:210-WBX-48S# show mda
6.2. 210 WBX - 48SFP28 6QSFP28 74

===============================================================================
MDA Summary
===============================================================================
Slot Mda Provisioned Type Admin Operational
-------------------------------------------------------------------------------
1 1 m48-25g-sfp28 up up
2 m6-100g-qsfp28 up up
3 m14-100g-internal up up
================================================================================
Physically, the numbering of the ports is from left to right, up and down (1 to 54). However, the physical number does
not match the software port numbering. As the Nuage 210 WBX reuses the 7850 VSG Nuage SROS, the software has
been adapted to expand each 100GE physical port into four different ports (a single 100GE can break out into four
different ports), maintaining the concept of IOM cards and MDAs used on VSG.
Fig. 6.3: 210 WBX 48SFP28 6QSFP28 MDAs and Port Numbering
• Any SFP port numbers from 1 to 48, corresponding to 1/1/[1..48] (card1, mda1/port number).
• Physical port number 49 (first 100GE capable port), corresponds to port 1/2/1 (card 1/mda2/port number) . If
• Physical port number 54 (lower right in the drawing and last 100GE capable port), corresponds to port 1/2/21
(card 1/mda2/port number) . If expanded by configuration with a breakout to 4x25G, or 4x10GE, then ports
1/2/22, 1/2/23, and 1/2/24 become visible. If expanded by configuration with a breakout of 2x50GE, then only
port 1/2/23 becomes visible.
To see the available ports, enter the show ports command. The following example output shows the 54 available ports
(not using breakout configuration) split in MDA1 and MDA2. MDA3 corresponds to LAG-97. For more details, see
the ‘LAG-97 and Lag 98‘_ section for WBX-48SFP28.
*A:210-WBX-48S# show port
===============================================================================
Ports on Slot 1
===============================================================================
6.2. 210 WBX - 48SFP28 6QSFP28 75


-------------------------------------------------------------------------------
1/1/1 Up Yes Up 9212 9212 - hybr dotq xgige GIGE-CX
1/1/2 Down No Down 9212 9212 - hybr null xgige
1/1/5 Down No Down 9212 9212 - hybr null xxvgige
1/2/5 Up Yes Up 9212 9212 - hybr dotq xlgige 40GBASE-CR4
1/3/1 Up Yes Up 9212 9212 97 hybr dotq cgige
6.2. 210 WBX - 48SFP28 6QSFP28 76


===============================================================================
Ports on Slot A
===============================================================================
-------------------------------------------------------------------------------
A/1 Up Yes Up 1514 1514 - netw null faste MDI
===============================================================================
To verify the front port mapping in SROS, enter the tools dump front-port-to-timos-port command:
*A:210-WBX-48S# tools dump front-port-to-timos-port

-----------------------------------
Front panel port | TiMOS port
-----------------------------------
1 | 1/1/1
2 | 1/1/2
3 | 1/1/3
4 | 1/1/4
5 | 1/1/5
6 | 1/1/6
7 | 1/1/7
8 | 1/1/8
9 | 1/1/9
10 | 1/1/10
11 | 1/1/11
12 | 1/1/12
13 | 1/1/13
14 | 1/1/14
15 | 1/1/15
16 | 1/1/16
17 | 1/1/17
18 | 1/1/18
19 | 1/1/19
20 | 1/1/20
21 | 1/1/21
22 | 1/1/22
23 | 1/1/23
24 | 1/1/24
25 | 1/1/25
26 | 1/1/26
27 | 1/1/27
28 | 1/1/28
29 | 1/1/29
30 | 1/1/30
6.2. 210 WBX - 48SFP28 6QSFP28 77

31 | 1/1/31
32 | 1/1/32
33 | 1/1/33
34 | 1/1/34
35 | 1/1/35
36 | 1/1/36
37 | 1/1/37
38 | 1/1/38
39 | 1/1/39
40 | 1/1/40
41 | 1/1/41
42 | 1/1/42
43 | 1/1/43
44 | 1/1/44
45 | 1/1/45
46 | 1/1/46
47 | 1/1/47
48 | 1/1/48
49 | 1/2/1
50 | 1/2/2
51 | 1/2/3
52 | 1/2/4
53 | 1/2/5
54 | 1/2/6
55 | 1/2/7
56 | 1/2/8
57 | 1/2/9
58 | 1/2/10
59 | 1/2/11
60 | 1/2/12
61 | 1/2/13
62 | 1/2/14
63 | 1/2/15
64 | 1/2/16
Port Configuration
The 210 WBX 48SFP28 6QSFP28 supports:

• 48 ports SFP28, SFP+, or SFP, and each port is individually configurable as 25GE, 10GE, or 1GE.
• 6 ports QSFP28 or QSFP+, and each port is individually configurable as 100GE, 50GE, 40GE, or 2x50GE,
4x25GE and 4x10GE if breakout configuration. The media-mode keyword defines port-type.
25GE Capable Ports
The default is 4x25GE capable ports. The physical ports 1-48 by default are 25GE.
The media-mode defines port-type: default 4x25G The speed defines port-speed: default 25000
*A:210-WBX-48S#configure port ethernet media-mode

- no media-mode
<4x10G|4x25G|40G|5*> : 4x10G|4x25G|40G|50G|2x50G|100G
6.2. 210 WBX - 48SFP28 6QSFP28 78

Ports can be configured as 25GE (default) or 10GE using media-mode. However, ports are gathered by banks of
four ports. To change the port speed per bank of four to 10GE, the first significant port should be configured to be
media-mode 4x10GE:
*A:210-WBX-48S#config port ethernet media-mode 4x10G
Then the corresponding four ports are converted as 10GE ports.

The 10GE ports can be individually provisioned as 1GE configuring speed-media 1000. 1G speed is only supported
when media-mode is 4x10GE. When the first port of a same bank is configured with media-mode 4x25GE and speed
1000, then all ports of the bank will be down.
*A:210-WBX-48S#config port ethernet speed

- speed {1000|10000|25000}
<1000|10000|25000> : keywords
Note: As a hardware limitation, 10G and 25G ports cannot be configured in the same quad.
100GE Capable Ports
The physical ports 49-54 by default are 100GE media-mode defines port-type: default 100G
*A:210-WBX-48S#configure port ethernet media-mode

- no media-mode
<4x10G|4x25G|40G|5*> : 4x10G|4x25G|40G|50G|2x50G|100G
When you enter the show port command to check the Port Type, depending on the media-mode, the Port Type
automatically changes to show the following:
• 100 GE = cgige
• 50 GE = lgige
• 40 GE = xlgige
• 25 GE = xxvge
• 10 GE = xgige
6.2. 210 WBX - 48SFP28 6QSFP28 79

Table 6.2: 210 WBX 48SFP28 6QSFP28 LED Status Descriptions

LEDs Status Descriptions
LED 1
• Off - No Link

• Green 25G operation
• Amber 10G operation

• Off - 40G/100G operation (Assuming LED 1 is
illuminated) or No Link

• Green 25G operation
• Amber 10G operation
Note: No link: Off / Linkup: Solid, LED / Traffic: Flashing
LAG-97 and LAG-98
LAG-97 and LAG-98 are configured by default in any VSG or 210 WBX.
• LAG-97 is used for the internal loop used when the system is configured as an L3 HW VTEP
• LAG-98 is used for the inter chassis (IC) connectivity when configuring MC-LAG
Note: For more info about MC-LAG, see the “Multichassis Link Aggregation Group” section in the Nuage VSP User
Guide.
LAG-97 (L3 loop) has all MDA3 ports configured by default.
*A:210-WBX-48S# configure lag 97

*A:210-WBX-48S>config>lag# info
----------------------------------------------
description "System Loopback LAG"
encap-type dot1q
port 1/3/1
port 1/3/2
port 1/3/3
6.2. 210 WBX - 48SFP28 6QSFP28 80

port 1/3/4
port 1/3/5
port 1/3/6
port 1/3/7
port 1/3/8
port 1/3/9
port 1/3/10
port 1/3/11
port 1/3/12
port 1/3/13
port 1/3/14
no shutdown
----------------------------------------------
No real loop is needed (no external cables), loop is internal but ports cannot be used for anything else.
By default, LAG-98 (IC) has no ports configured either.
*A:210-WBX-48S>config# lag 98
*A:210-WBX-48S>config>lag# info
----------------------------------------------
description "Multichassis interconnect LAG"
encap-type dot1q
lacp active administrative-key 36864
no shutdown
----------------------------------------------
However, any port can be provisioned in the LAG-98. Note that in this case, cables are needed to interconnect both
MC-LAG nodes.
6.2. 210 WBX - 48SFP28 6QSFP28 81

CHAPTER
SEVEN
APPENDIX
• Installation Trace Example from the SD card (page 82)
Installation Trace Example from the SD card
Note: The following output is an example and should be used only as a reference because the output may vary based
on the Nuage 210 WBX software release.
Booting ÒNIE: Install OS'

Welcome to GRUB!
ONIE: OS Install Mode ...
Version : 2016.02
Build Date: 2017-02-08T17:09+0800
Info: Mounting kernel filesystems... done.
Info: Mounting ONIE-BOOT on /mnt/onie-boot ...
Installing for i386-pc platform.
Installation finrandom: mktemp urandom read with 19 bits of entropy available
ished. No error reported.
Info: Using eth0 MAC address: d0:99:d5:95:12:41
Info: eth0: Checking link... scsi 6:0:0:0: Direct-Access Generic STORAGE DEVICE
˓→ 0903 PQ: 0 ANSI: 6
sd 6:0:0:0: [sdb] 15826944 512-byte logical blocks: (8.10 GB/7.54 GiB)
sd 6:0:0:0: [sdb] Write Protect is off
sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or
˓→FUA
sd 6:0:0:0: [sdb] Attached SCSI removable disk

down.
ONIE: eth0: link down. Skipping configuration.
ONIE: Failed to configure eth0 interface
Starting: dropbear ssh daemon... done.
Starting: telnetd... done.
discover: installer mode detected. Running installer.
Starting: discover... done.
Please press Enter to activate this console. Info: eth0: Checking link... down.
ONIE: eth0: link down. Skipping configuration.
ONIE: Failed to configure eth0 interface
82
ONIE: Starting ONIE Service Discovery

EXT2-fs (sdb1): warning: mounting unchecked fs, running e2fsck is recommended
random: nonblocking pool is initialized
EXT2-fs (sdb1): warning: mounting unchecked fs, running e2fsck is recommended
ONIE: Executing installer: file://dev/sdb1/onie-installer-x86_64
*** Nuage 210-WBX Installer script 5.0.2_28 ***
Setting eth0 MAC from syseeprom: D0:99:D5:95:12:41 - Change MAC Done
Found 2 devices: sda sdb
sdb Non-Internal Storage detected
Install Device set to /dev/sda
Install Device /dev/sda is of 125045424 sectors, size 59.6 GiB
Last installed partition detected = 2
Removing any non-ONIE/non-VSGX partitions
Partition ONIE-BOOT is either ONIE or VSGX
Partition GRUB-BOOT is either ONIE or VSGX
No existing Root Partion found for vsgx; Creating ROOT partition
Found last partition: INSTALL_LAST_PART="2"
Last partition set to="2"
Creating new partion VSGX-ROOT on /dev/sda3
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
No exisiting persistant data partition foud, creating new data partition
Creating new partion VSGX-DATA on /dev/sda4
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
mke2fs 1.42.8 (20-Jun-2013)
Discarding device blocks: done
Filesystem label=VSGX-DATA
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
2097152 inodes, 8388433 blocks
419421 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
256 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Allocating group tables: done

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Extracting image. This will take some time...

0+292768 records in
0+292768 records out
2056224768 bytes (1.9GB) copied, 69.974810 seconds, 28.0MB/s
Image extraction complete. Running fsck to prepare for filesystem resizing...
fsck 1.42.8 (20-Jun-2013)
e2fsck 1.42.8 (20-Jun-2013)
Superblock last mount time (Thu Jul 6 04:32:03 2017,
7.1. Installation Trace Example from the SD card 83

now = Thu Feb 9 10:03:53 2017) is in the future.

Fix? yes
Superblock last write time (Thu Jul 6 04:40:48 2017,

now = Thu Feb 9 10:03:53 2017) is in the future.
Fix? yes
Pass 1: Checking inodes, blocks, and sizes
[ 5.135001] i8042: No controller found

[ OK ] Started Show Plymouth Boot Screen.
[ OK ] Reached target Paths.
[ OK ] Reached target Basic System.
[ OK ] Found device M.2__S42__3ME2 VSGX-ROOTFS.
Starting File System Check on /dev/sda3...
[ OK ] Started File System Check on /dev/sda3.
[ OK ] Started dracut initqueue hook.
Starting dracut pre-mount hook...
[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Reached target Remote File Systems.
[ OK ] Started dracut pre-mount hook.
Mounting /sysroot...
[ OK ] Mounted /sysroot.
[ OK ] Reached target Initrd Root File System.
Starting Reload Configuration from the Real Root...
[ OK ] Started Reload Configuration from the Real Root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Reached target Initrd Default Target.
Starting dracut pre-pivot and cleanup hook...
[ OK ] Started dracut pre-pivot and cleanup hook.
Starting Cleaning Up and Shutting Down Daemons...
Starting Plymouth switch root service...
[ OK ] Stopped target Timers.
[ OK ] Stopped Cleaning Up and Shutting Down Daemons.
[ OK ] Stopped dracut pre-pivot and cleanup hook.
Stopping dracut pre-pivot and cleanup hook...
[ OK ] Stopped target Initrd Default Target.
[ OK ] Stopped dracut pre-mount hook.
Stopping dracut pre-mount hook...
[ OK ] Stopped target Remote File Systems.
[ OK ] Stopped target Remote File Systems (Pre).
[ OK ] Stopped target Basic System.
[ OK ] Stopped target Slices.
[ OK ] Stopped target Paths.
[ OK ] Stopped target Sockets.
[ OK ] Closed Open-iSCSI iscsiuio Socket.
[ OK ] Stopped target System Initialization.
[ OK ] Stopped target Local File Systems.
[ OK ] Stopped Apply Kernel Variables.
Stopping Apply Kernel Variables...
[ OK ] Stopped target Swap.
[ OK ] Stopped target Encrypted Volumes.
[ OK ] Stopped dracut initqueue hook.
Stopping dracut initqueue hook...
[ OK ] Stopped udev Coldplug all Devices.
Stopping udev Coldplug all Devices...

[ OK ] Stopped dracut pre-trigger hook.

Stopping dracut pre-trigger hook...
Stopping udev Kernel Device Manager...
Stopping Open-iSCSI...
[ OK ] Stopped udev Kernel Device Manager.
[ OK ] Stopped dracut pre-udev hook.
Stopping dracut pre-udev hook...
[ OK ] Stopped
Welcome to CentOS Linux 7 (Core)!
[ OK ] Stopped Switch Root.

[ OK ] Listening on udev Control Socket.
[ OK ] Reached target Swap.
[ OK ] Created slice User and Session Slice.
[ OK ] Listening on udev Kernel Socket.
[ OK ] Listening on /dev/initctl Compatibility Named Pipe.
[ OK ] Listening on LVM2 poll daemon socket.
[ OK ] Created slice system-serial\x2dgetty.slice.
[ OK ] Listening on Device-mapper event daemon FIFOs.
Mounting Huge Pages File System...
[ OK ] Set up automount Arbitrary Executab...ats File System Automount Point.
[ OK ] Reached target Slices.
[ OK ] Reached target Encrypted Volumes.
Mounting POSIX Message Queue File System...
[ OK ] Listening on LVM2 metadata daemon socket.
Starting Monitoring of LVM2 mirrors... dmeventd or progress polling...
[ OK ] Stopped Journal Service.
Starting Journal Service...
Mounting NFSD configuration filesystem...
Starting Create list of required st... nodes for the current kernel...
[ OK ] Listening on Delayed Shutdown Socket.
Mounting Debug File System...
Starting Collect Read-Ahead Data...
[ OK ] Created slice system-getty.slice.
[ OK ] Stopped target Switch Root.
[ OK ] Stopped target Initrd Root File System.
[ OK ] Stopped target Initrd File Systems.
[ OK ] Started Collect Read-Ahead Data.
Starting Remount Root and Kernel File Systems...
Starting Load Kernel Modules...
[ OK ] Started Create list of required sta...ce nodes for the current kernel.
Starting Create Static Device Nodes in /dev...
[ OK ] Mounted POSIX Message Queue File System.
[ OK ] Mounted Huge Pages File System.
[ OK ] Mounted Debug File System.
[ OK ] Started Remount Root and Kernel File Systems.
[ OK ] Started Journal Service.
Starting Load/Save Random Seed...
Starting Configure read-only root support...
Starting Flush Journal to Persistent Storage...
Starting Rebuild Hardware Database...
[ OK ] Started LVM2 metadata daemon.
Starting LVM2 metadata daemon...
[ OK ] Started Load/Save Random Seed.
[ OK ] Started Load Kernel Modules.
Starting Apply Kernel Variables...
[ OK ] Started Create Static Device Nodes in /dev.
Starting udev Kernel Device Manager...

[ OK ] Started Flush Journal to Persistent Storage.

[ OK ] Mounted NFSD configuration filesystem.
[ OK ] Started Apply Kernel Variables.
[ OK ] Started Configure read-only root support.
[ OK ] Started udev Kernel Device Manager.
[ OK ] Started Rebuild Hardware Database.
Starting udev Coldplug all Devices...
[ OK ] Started udev Coldplug all Devices.
[ OK ] Found device /dev/ttyS0.
[ OK ] Found device M.2__S42__3ME2 VSGX-DATA.
[ OK ] Found device M.2__S42__3ME2 ONIE-BOOT.
[ OK ] Started Monitoring of LVM2 mirrors,...ng dmeventd or progress polling.
[ OK ] Reached target Local File Systems (Pre).
Mounting /vsgx-data...
Mounting /onie-boot...
[ OK ] Mounted /onie-boot.
[ OK ] Mounted /vsgx-data.
[ OK ] Reached target Local File Systems.
Starting Mark the need to relabel after reboot...
Starting Preprocess NFS configuration...
Starting Tell Plymouth To Write Out Runtime Data...
Starting Rebuild Journal Catalog...
Starting Import network configuration from initramfs...
[ OK ] Started Mark the need to relabel after reboot.
[ OK ] Started Preprocess NFS configuration.
[ OK ] Started Rebuild Journal Catalog.
Starting Update is Completed...
[ OK ] Started Update is Completed.
[ OK ] Started Tell Plymouth To Write Out Runtime Data.
[ OK ] Started Import network configuration from initramfs.
Starting Create Volatile Files and Directories...
[ OK ] Started Create Volatile Files and Directories.
Starting Security Auditing Service...
[ OK ] Started Security Auditing Service.
Starting Update UTMP about System Boot/Shutdown...
[ OK ] Started Update UTMP about System Boot/Shutdown.
[ OK ] Reached target System Initialization.
[ OK ] Listening on Virtual machine log manager socket.
[ OK ] Listening on Virtual machine lock manager socket.
[ OK ] Reached target Paths.
[ OK ] Reached target Timers.
[ OK ] Listening on Open-iSCSI iscsiuio Socket.
[ OK ] Listening on RPCbind Server Activation Socket.
[ OK ] Listening on Open-iSCSI iscsid Socket.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting OpenSSH Server Key Generation...
Starting NTP client/server...
Starting Initial cloud-init job (pre-networking)...
[ OK ] Started Nuage VSG-X Boot Message.
Starting Nuage VSG-X Boot Message...
Starting GSSAPI Proxy Daemon...
[ OK ] Started irqbalance daemon.
Starting irqbalance daemon...
Starting Dump dmesg to /var/log/dmesg...
[ OK ] Started ABRT Automated Bug Reporting Tool.
Starting ABRT Automated Bug Reporting Tool...

Starting Install ABRT coredump hook...

[ OK ] Started ABRT kernel log watcher.
Starting ABRT kernel log watcher...
Starting Kernel Samepage Merging...
Starting Load CPU microcode update...
Starting Resets System Activity Logs...
Starting Authorization Manager...
[ OK ] Started D-Bus System Message Bus.
Starting D-Bus System Message Bus...
Starting Login Service...
[ OK ] Started GSSAPI Proxy Daemon.
[ OK ] Started Dump dmesg to /var/log/dmesg.
[ OK ] Started Kernel Samepage Merging.
[ OK ] Started Load CPU microcode update.
[ OK ] Started Resets System Activity Logs.
Starting Kernel Samepage Merging (KSM) Tuning Daemon...
[ OK ] Reached target NFS client services.
[ OK ] Started NTP client/server.
[ OK ] Started Kernel Samepage Merging (KSM) Tuning Daemon.
[ OK ] Started Login Service.
[ OK ] Started Install ABRT coredump hook.
[ OK ] Started Authorization Manager.
Starting firewalld - dynamic firewall daemon...
[ OK ] Started OpenSSH Server Key Generation.
[ 9.057137] cloud-init[800]: Cloud-init v. 0.7.5 running 'init-local' at Thu, 09
˓→Feb 2017 10:04:31 +0000. Up 9.02 seconds.
[ OK ] Started Initial cloud-init job (pre-networking).

[ OK ] Started firewalld - dynamic firewall daemon.
Starting Network Manager...
[ OK ] Started Network Manager.
Starting Network Manager Wait Online...
Starting Network Manager Script Dispatcher Service...
[ OK ] Started Network Manager Script Dispatcher Service.
Starting Hostname Service...
[ OK ] Started Hostname Service.
[ 15.926440] cloud-init[1142]: Cloud-init v. 0.7.5 running 'init' at Thu, 09 Feb
˓→2017 10:04:38 +0000. Up 15.88 seconds.
[ 15.967788] cloud-init[1142]: ci-info: ++++++++++++++++++++++++Net device

˓→info+++++++++++++++++++++++++
[ 15.968092] cloud-init[1142]: ci-info: +-----------+------+-----------+-----------

˓→+-------------------+
[ 15.968301] cloud-init[1142]: ci-info: | Device | Up | Address | Mask

˓→| Hw-Address |
˓→+-------------------+
[ 15.968705] cloud-init[1142]: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0

˓→| . |
[ 15.968896] cloud-init[1142]: ci-info: | enp5s0f0: | True | . | .
˓→| 00:a0:c9:00:00:00 |
[ 15.969094] cloud-init[1142]: ci-info: | ens1: | True | . | .

˓→| d0:99:d5:95:12:41 |
[ 15.969284] cloud-init[1142]: ci-info: | enp5s0f1: | True | . | .

˓→| 34:12:78:56:01:00 |

˓→+-------------------+
[ 15.969685] cloud-init[1142]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info

˓→failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Nuage 210-WBX 5.0.2-28

localhost login: [ 16.971194] cloud-init[1522]: Cloud-init v. 0.7.5 running
˓→'modules:config' at Thu, 09 Feb 2017 10:04:39 +0000. Up 16.91 seconds.
[ 17.287023] cloud-init[1573]: Cloud-init v. 0.7.5 running 'modules:final' at Thu,

˓→09 Feb 2017 10:04:39 +0000. Up 17.22 seconds.
ci-info: no authorized ssh keys fingerprints found for user centos.

[ 17.316134] cloud-init[1573]: ci-info: no authorized ssh keys fingerprints found
˓→for user centos.
ec2:
ec2: #############################################################
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 256 3a:d6:ac:51:b5:95:31:0c:01:f9:9b:05:6d:b2:62:1f (ECDSA)
ec2: 256 85:f2:46:ca:69:94:12:0a:c0:34:9f:7f:b1:73:c8:c6 (ED25519)
ec2: 2048 60:9c:59:50:89:64:56:c2:93:09:70:95:76:df:4e:9f (RSA)
ec2: -----END SSH HOST KEY FINGERPRINTS-----
ec2: #############################################################
-----BEGIN SSH HOST KEY KEYS-----
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD7rN/
˓→FD4PJIqfpnfaMLrL/ToHD35gcz+3Ufsq3PCEQcomuD0wtJwLRXooXykmMt1bffp/R9A+3UCU/Q0RFasdI=
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOqL3mnHx/IkQb36Gv60epPGyy6u7YdwHKqVn9v2VQme
ssh-rsa
˓→AAAAB3NzaC1yc2EAAAADAQABAAABAQC3IMOOe+2vl9FfMYeulY6TpaNSfmimlcZ341cI4HzzY1UZfPIuSkiBEPgCg1MI0yxHMnr
v8/F/
˓→AzjhYCBIcCggHhrn9EkJdFbaQPWu39pmyQTi6BW1RaN9GIbqn4bcqt9WFtLgZdKTDb+BQq99+oE4mG99NGJd/
˓→FcnWCZiuUU79QDvz7LLGgu3jCD+bYnFu2QdCBWgprz7F7ENdkAsH7JZS2A3ANmR0EXBX1GNbCGeHjnRfKNR86xoYx9srfT
kIeeTHVyfQmd
-----END SSH HOST KEY KEYS-----
[ 17.345404] cloud-init[1573]: Cloud-init v. 0.7.5 finished at Thu, 09 Feb 2017 10:
˓→04:39 +0000. Datasource DataSourceNone. Up 17.34 seconds
Nuage 210-WBX 5.0.2-28

localhost login:

Nuage 210 WBX Software Installation Guide

Uploaded by

Copyright:

Available Formats

Nuage 210 WBX Software Installation Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nuage 210 WBX Software Installation Guide

Uploaded by

Copyright:

Available Formats

210 WBX Software Installation Guide

October 31, 2020

1 About this Document 2

3 Nuage Software Installation and Upgrade 8

5 Nuage SROS for 210 WBX 39

6 Configuration of the 210 WBX Models with Nuage Software 66

NUAGE NETWORKS – PROPRIETARY & CONFIDENTIAL

Copyright©2018 Nokia Corporation / Nuage Networks. All rights reserved.

ABOUT THIS DOCUMENT

• Validity of this Document (page 3)

Validity of this Document

1.1. Validity of this Document 3

1.4. Required Documents 4

• 210 WBX Overview (page 5)

210 WBX Overview

210 WBX 32QSFP28

Fig. 2.1: 210 WBX 32QSFP28 Front Panel

Table 2.1: 210 WBX 32QSFP28

Fig. 2.2: 210 WBX 32QSFP28 Rear Panel

Table 2.2: 210 WBX 32QSFP28 Rear

210 WBX 48SFP28 6QSFP28

Fig. 2.3: 210 WBX 48SFP28 6QSFP28 Front Panel

2.1. 210 WBX Overview 6

Table 2.3: 210 WBX 48SFP28

Fig. 2.4: 210 WBX 48SFP28 6QSFP28 Rear Panel

Table 2.4: 210 WBX 48SFP28 6QSFP28 Rear

2.1. 210 WBX Overview 7

NUAGE SOFTWARE INSTALLATION AND UPGRADE

• Setting up NOS with ONIE (page 9)

* Boot after the NOS Installation is Complete (page 12)

* nuage_wbx cloud-init module (page 14)

* cloud-init.cfg (page 15)

* BoF Pointing to the external FTP Link (page 18)

* Prerequisites (page 19)

• Ensuring Availability of SROS VM and Hypervisor (page 20)

Setting up NOS with ONIE

Initializing the System and Downloading Software

Installing NOS from the SD Card

3.1. Setting up NOS with ONIE 9

Fig. 3.1: 210 WBX Console Management Port

Note: Install OS option is the default.

Fig. 3.2: 210 WBX ONIE Bootup

3.2. Initializing the System and Downloading Software 10

Info: Mounting kernel filesystems... done.

Note: For the detailed trace example, go to the Appendix.

Installing NOS from the Network

Note: An SD card is not required to perform installation from the network.

Fig. 3.3: 210 WBX Ethernet Management Port

Step 2 Copy the NOS Installer Image to the HTTP server.

option default-url "http://<webserver-address>/onie-installer-x86_64"

3.2. Initializing the System and Downloading Software 11

Note: For details, see the ONIE user guide: https://opencomputeproject.github.io/onie/user-guide/

Boot after the NOS Installation is Complete

Fig. 3.4: 210 WBX NOS Bootup

Automated NOS Configuration

3.3. Automated NOS Configuration 12

Configuring OS From the SD Card

mlabel -i /dev/sdb1 ::VSGX-SD

Configuring NOS From the Network

option default-url "http://<webserver>/switch001/onie-installer-x86_64"

A:WBX-X# bof primary-image ftp://user:password@35.227.180.72/./images/x86-