Tellabs 8600 Smart Routers System Troubl PDF

Tellabs 8600 Smart Routers
System Troubleshooting
using Tellabs 8000 INM and CLI
Course Introduction
05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 3

Course Introduction

Course Introduction

Course Introduction

Course Introduction

Course Introduction

Course Introduction

Course Introduction

Tellabs 8600
Hardware Troubleshooting 1
Tellabs 8600 Hardware Troubleshooting




Access to the console through direct connection to the CDC card is only available on the active CDC
card. An error message will appear on the console if connected to the inactive CDC card.
With ESW version 2.11.150 and above, it is possible to have up to 17 simultaneous connection to a
Tellabs 86xx node, eight via Telnet IP connections, eight via IP SSH connections and one via direct
connection though the console port. Earlier versions of ESW supported up to nine or less simultaneous
connections.
Opening a console connection to the node gives the user full access to the node and its configurations.
Only by configuring authentication will any connections be prompted for login credentials.

Tellabs 8660/8630 CDC card connector types: DB9 male (CDC) - DB9 female (PC) serial cable
Tellabs 8605/8607/8609/8611 console cable: RJ45 (860X) - DB9 female (PC)
Pinouts for all console cabling can be found in the Hardware Installation Guide for the required Tellabs
86xx model.



Information regarding card type, HW revision, serial number and ESW version are all stored in the node
inventory.
In order to replace a faulty card, it is necessary to ensure certain properties of the new card match those
of the old card. A comparison check is made between the newly inserted cards’ information and those of
the old card which are stored in the inventory. As long as the required parameters match the new card
will be allowed to boot and load configuration parameters. If the parameters do not match the card will
not be allowed to start. Card type and hardware version must match, serial number, etc are not critical.
If the new card being installed has an older version of ESW, then the node will copy the current version
of ESW from another card to it and upgrade the card automatically.

Because of the different slot options for IFCs in a Tellabs 8660/8630 there is no inventory configured
before the node is delivered to site. Installation engineers must populate the node with cards according
to the installation plan and then build the node inventory.
Other Tellabs 86xx nodes which do not have multi slot capability simply create their inventory based on
what is seen in the node. As a result installation engineers do not need to create the inventory on these
nodes and can proceed straight to configuration.
If an inserted card is not up and running in the inventory, no configurations can be made to the
interfaces in its modules. By looking at the interface list in the running configuration, interfaces belonging
to cards not up and running will not be seen.

Automatic upgrade for replacement cards will only have an effect on the ESW, boot and mini-application
software will have to be upgraded manually if necessary.
If the card has a newer version of ESW than that being used on the node, the node will not downgrade
the card ESW, but will disable the card and prevent it from loading its ESW.
In this case IFC or CDC cards can be manually downgraded by the operator and rebooted.
Because the inventory only affects cards when they boot, it is possible to modify the inventory
information of a card while the card is up and running.
If a module change needs to be performed, it is useful to first modify the module information in the
inventory before removing and changing the hardware, this means that when the card is removed and
module changed, when reinserted, the inventory already has the new hardware information and allows
the card to start.

CDC DC-Feed board provides power to the Control board (+3.3V and +30V), Fan modules (+26V) and
other cards in the switch (-48V). CDC Control board contains an integrated power supply, DC/DC
converter. 30V from DC-Feed board is converted to voltages, which are needed.
Needed operational voltages: 1.2V, 1.8V, 2.0V, 2.5V, 5V, 12V and -12V. (3.3V comes directly from DC-
Feed board.)
Current Temperature
The current temperature measurement.
High Timeconditional Threshold

If the temperature of the unit remains over the limit for at least 2 minutes, a high fault is raised. When the
temperature drops below the limit, the high fault is cleared.
High Unconditional Threshold

If the temperature of the unit is over the limit, a high fault is raised immediately.
Critical Timeconditional Threshold

If the temperature of the unit remains over the limit for at least 2 minutes, a critical fault is raised. When
the temperature drops below the limit, the critical fault is cleared.
Critical Unconditional Threshold

If the temperature of the unit is over the limit, a critical fault is raised immediately..
A faulty temperature sensor will show value of 0 Celsius. The output shown in the figure displays a
”Cooling fan local” that is within the specified low and high thresholds.

However ”Cooling fan remote” shows a very low voltage that lies outside the low threshold.
In this case this indicates that there is no CDC present in slot number one, or in the case there is a CDC,
then it would indicate that the DC-DC converter, in that card, has a malfunction.
The equipment will go to reset mode after 15 minutes if the measured temperature is +90° C or higher or
immediately if it is +100° C or higher. This is to protect the equipment from permanent damage caused
by high temperature.

Even though the output refers to slots 15 and 16, they do not actually exist on the node. They do not
have a function nor effect and can be ignored.

Node clock should be set correctly in order to see accurate information in the fault outputs and different
logs of the node. Clock can be set locally in the node or alternatively the time can be polled from the
NTP server.





Wingine is an internal processor inside the Win processor. There are 4-6 Wingines depending on the
module and Win processor type.
Here WINGINE is an average of all of the processors.
MBUSx = memory bus number

There are Mbuses for application (ESW) and data.
QMEM1 = queue buffer


The SFP/XFP Connector Data dialog allows you to monitor Small Form-Factor Pluggable (SFP/XFP)
connector diagnosis data in real time. The data is unique to each optical interface which have an
SFP/XFP connector installed. The data is refreshed either automatically, or you can refresh it manually.
The interface list contains all optical interfaces in the selected module. If an SFP/XFP connector is not
installed in the current interface, all values are set to N/A (not available).
Interface
Name and number of the optical interface.
Temperature
Internally measured temperature of the module. The measurement unit is °C, Celsius.
Transceiver Voltage
Internally measured supply voltage in the transceiver.
Bias Current
Internally measured transmitter laser bias current.
TX out power
Measured TX output power in watts (µW, mW) and dBm. The data is assumed to be based on the
measurement of a laser monitor photodiode current. The data is not valid when the transmitter is
disabled.
RX input power
Measured RX input power in watts (µW, mW) and dBm.

For avoiding problems caused by power shortage in a subrack, Power Balance application provides two
GUI tools for assisting operators not to overbook available electric power in Network Elements. One is
Power Balance Overview Dialog for displaying overview of power balance of NEs, the other is Power
Balance Details Dialog for displaying details of power balance of NEs. The application can be launched
in Network Editor under normal workstation or power workstation configurations of Tellabs 8000
manager with Macro Manager Service configured and running.
Supported NE Types
Currently Power Balance Calculation application only supports 8660 Edge Switch and 8630 Access
Switch.

There are three general sections outputted when the ’show running-config’ command is executed. First
are global settings, these begin with a list of the software versions in each slot, followed by the node
level QoS settings. After this, any pseudo wire circuits, IP VPNS and Ethernet bridge VSIs. Following
these are policers and access control lists.
The next section contains interface settings, showing layer 1 and layer 2 configurations, followed by
logical layer 3 and above settings.
The last section sjows routing and protocol configurations, also management communications proocols
and settings.

Empty lines seen in the output of the command history indicate commands/configurations that have
been done from the network management system using BMI.
Command history can also be viewed on a per user basis, as long as authentication for users has been
configured.


For any combination of CDC and IFC card in a node, they must be either all type A, or all type B. Mixing
of the card types is not supported.
No error is generated when different cards types are placed into a node, so it is the responsibility of
whoever is installing the cards to ensure they are the correct type.
Issues regarding mixing different card types generally occur with protections being implemented. The
main difference between the card types is that a type B card has half the memory of a type A card. A
problem may occur if protection switches from type A card to a type B card and the contents of the
memory in the A card cannot be reproduced in the smaller memory of the B card.

POST = Power On Self Test
All POST and boot messages are seen on console, there can be many messages which might seem to
indicate a fault, but actually are normal message output displaying the normal stages of the startup
sequence. Care needs to be taken to allow the node and its cards to fully boot before checking to see if
any faults are actually present. The ’show faults active’ command will identify faults which are present
after the node is fully booted.

When replacing a faulty card with a new card, simply insert the new card into the same slot. The node
will take care of copying the required configurations from CDC to the new card. If there are any existing
configurations on the new card they will be cleaned away before the previous card’s configurations are
copied to it.
The ’preconfigured’ option which can be used with cards that already have configurations being inserted
into an empty inventory slot. It allows for the merging of certain configuration parameters on the new
card into the node configuration. This is not a normal operator or service procedure and should not be
attempted unless specifically requested and guided by support personnel from Tellabs TAC or R&D.


Layer 1 and layer 2 information and counters can be seen from the output of the ’show interface’
command. The first section indicated by Interface ge13/0/7 refers to layer 1, and the following section
indicated by Logical ge13/0/7 refers to layer 2.
Interface ge13/0/7 section:

• L1 Admin Status – up/down
• L1 Fault Status – ok/not ok
• Loop – Not set/to-line/to-equipment
• L1 negotiation information is shown here and can indicate if auto negotiation has correctly succeeded
for speed and duplex. For example, sometimes negotiation may incorrectly set a gigabit ethernet port
to operate at 100M speed, indicated as 100BASE-X. This can affect other parameters such as
available bandwidth and metric calculations for routing protocols.
Logical ge13/0/7 section:

• MAC address being used by interface
• Negotiated bandwidth of interface
• Allocated bandwidth shows actual bandwidth reported available. The ’bandwidth-if’ command is set
by default in the configuration for 8xGE module types is used to automatically restrict the available
bandwidth per interface to the configured amount, in this case 200M.
• Existing Vlans shows the configured Vlan interfaces on this ethernet interface.
• L2 ge13/0/7 shows the Link up/Link down status of the L2 interface.

VLAN statistics are currently only supported in 8660/8630 in Etherenet IFM R2 and IFC2 cards only.
When creating or modifying the VLAN interface, in the dialog there is a ’Statistics Settings’ tab where the
counters can be enabled.
In the VLAN interface configuration the following command is used to enable VLAN interface statistics
collection. Statistics collection is disabled by default in 8660/8630 nodes, but enabled in
8605/8607/8609.
Router(cfg-if[ge5/0/7.51])# vlan statistics non-qos in-qos

The ’show ip interface’ command shows layer 3 information for the interface.
There are three MTU values specified in the output:

• Link MTU shows the Ethernet MTU default or configured value (default is 1530).
• IP MTU shows the IP layer MTU value used (default is 1500), changes to this can affect transport
and result in problems with routing protocol adjacency establishment.
• MPLS MTU is set by default to 1530, this allows for additional VLAN tags and label headers on top of
the standard IP packet.
Label switching is turned on in the interface, but the output does not show what protocols are being
used, that information would have to be checked in the running-configuration interface parameters.
DSTE Bandwidth Constraint Mode being used is shown here to be Russian Doll Model (RSDL), the
other configurable option being Maximum Allocation Model (MAM).

The ’show ip interface log’ command shows all notification messages for configuration and status
changes which have occured on an IP interface since the last reboot. You can further focus to show only
output for a specific interface by referencing the specific interface in the command.

A quick method of listing all the IP interfaces and showing their addresses on the node is to use the
’show ip interface brief’ command. The output generates a row in for any interface has functions based
on IP forwarding. It is easy to verify that correct IP addresses have been configured to the correct
interfaces.
The Admin state column indicates whether the interface has been configured with the ’shutdown/no
shutdown’ command.
The Line state column indicates the link activity status of the interface. This can be configured using the
’shutdown-if/no shutdown-if’ command, also if the interface is optical it may be necessary to use the
’laser/no laser’ command.


Each IFC and CDC card in Tellabs 8660/8630 has different pieces of software installed per card. Each
Tellabs 8605 and Tellabs 8607 node also has three different pieces of software installed.
BootFLASH memory:
Boot software – used to initialize the card and perform POST.
Mini-application – used as backup SW with limited functionality in the event of the main ESW failing.
ApplicationFLASH memory:
ESW – element software used for operating the card and carrying out all forwarding and traffic related
functions.
The ’show sw-version’ command shows all three pieces of software being used per card. Even if there is
more than one copy of ESW installed into flash memory, only the currently active version is listed by the
command.

Cold reset of the node requires power to be removed from the card before rebooting. Typically this can
be done by the following methods:
• Power off the node
• Remove and reinsert the card/unit
• Use the reset-hw command to cold boot the card/unit
reset-hw
Resets node or selected unit HW (more thoroughly than normal reset does) and restarts selected
executable SWs. This command is working with the CDC versions 4.0 or higher.
reload-sw
Restarts node or selected unit SW.



On older HW versions flash memory was RAW uncompressed. New HW version uses compressed flash
memory management.
Storing backup ESW files simply results in flash memory used unnecessarily. Backup ESW files are only
used if newly activated ESW fails on its first load. If new ESW successfully loads and is identified as
stable, any subsequent failure results in mini-application being loaded on reboot.
Other files are also stored in flash memory (configuration files, logs, etc), however the ’show flash’
command only shows ESW files stored in flash.

If an ESW file is copied to the wrong location, for example a CDC cbz2712 file copied to an IFC flash
memory, the system will simply show a corrupted sw file message. The message does not indicate that
it is simply the wrong type of file for this type of card, and may seem to indicate that the ESW is
somehow faulty. This is not the case, simply delete the incorrect file and copy the correct file to the flash.

Unlocking and deleting active ESW is not a general operator level procedure. The process of unlocking
ESW or attempting to delete active ESW, should only be carried out if specifically requested and guided
by Tellabs technical support.
In rare circumstances, the active ESW may become corrupted. If no backup ESW exists, the node/unit
continually boots to mini-application. New ESW can be copied to flash but cannot be set as active
because the current active ESW is identified as corrupted. A backup ESW file is only used if a new ESW
fails first time after activation and attempted to load. The active ESW is locked to prevent deletion, so it
must be unlocked for deletion so new inactive ESW file can be set to active and loaded.

FP2.9A -> FP2.10 and above
Requires active ESW in flash to be deleted before copying new ESW. This is due to large number of
FPGAs now used and need for extra flash memory space. See Embedded SW notes for FP2.10 for
information on correct upgrade steps.

By default, FTP is not enabled on Tellabs 8600 nodes. Before copying software to a node it needs to be
enabled.
This can be done via CLI using the ’ftp-server enable’ command, or the operator can have the Tellabs
8000 INM enable it and disable it again as part of the ESW upgrade process.



OS logs are generated when the node boots and then recorded during the nodes operation. After
powering on or resetting the node, the log is stored in RAM until the start-up sequence is complete, then
the log is copied to Flash memory.
If the node is reset again before it has completed the start-up sequence, then the log which was being
stored in RAM is lost and a new one is generated for the new start-up.

Node log files are seen as text (txt) files in the log folder in flash memory.
Node log files can be copied from the node using FTP for sending to Tellabs TAC. Remember to turn on
FTP client on node to do this.
Reset information is stored into files in directories:

/flash/debug/slot(x)/log (8660/8630)
/flash/debug/log (8620/8611/8609/8605)
If ftp connection is not available, user can give the following cli command: ‘show debugging sw os-log
slot <slot_number>’. This command will display the os-log information into the console.

Binary log files store more detailed information which can be useful for R&D analysis. For example
greater detail on the reason a node may have reset. These are seen as zipped (zip) files in the log folder
in flash memory.

The ’show tech-support’ command generates output of all information available on the node. Typically
this output should be stored to a file as its being generated for sending to Tellabs TAC.
To prevent having to hit the space key to always show the next page of information, it is advisable to use
the ’no terminal more’ command so all the output is displayed at once. It takes some time to run this
command and should only be run at the specific request of Tellabs TAC.
Warning!
There is a danger that using ’show tech-support’ command will trigger an IFC reset with certain old
software versions.
A software fix for this problem has been issued for the following FP releases:
• FP 2.9 SP10
• FP 2.10A SP3
• FP 2.11A and all newer FPs
For more information refer to PPN 09092 Rev. C

Snapshot files can only be restored to the same version of ESW from which they were taken. ESW
version of the snapshot file is shown in the Node Configuration Backup utility, it is also shown at the
beginning of the running-configuration output stored in the snapshot file itself. Including running-config of
the node to a snapshot-config is optional and is defined separately in the command syntax.
It is good practice to give a .zip ending to snapshot-config file. By doing that you can later on open the
zipped file, it contains some attachment text files with all the relevant information of the node where the
backup was taken (SW information and HW-inventory). The attachment files help you to recognize the
node type you can use when restoring the configuration to another node (it should have exactly the
same HW/SW versions).
It is also a good practice to avoid using spaces in file names.

Tellabs 8000 INM can handle file names with spaces in them.
If there is a need to restore a snapshot-config file from CLI, then double quotation marks need to be
used.
Notice that there may be one or several spaces at the end of the file name, and manipulating (deleting,
copying, restoring) such a file would not be easy.

EmergencyRestore.zip file is created automatically by the system 15 minutes after last reset or any
configuration update made to the system. File is created for each unit slot separately. File could be
taken in to use if the system enters to reset-loop. If even emergency restore does not work,
miniapplication SW is used to get system back online.

The configuration parameters in the running-configuration are not stored in a single file on the node,
therefore it is not possible to copy them from or to the node as a single file.
Copying and pasting the resultant output on screen from the ’show running-config’ command to a text
file is one method of acquiring and rebuilding the configuration. This text file can be modified and the
configurations and commands pasted back into the console.


Temperature critical, 1507330 (dec): Temperature is above critical threshold. This will soon damage to
the unit permanently.
Temperature too high, 1507331 (dec): Temperature is above high threshold. This may cause unit
overheating.
Voltage too high, 1507332 (dec): Voltage is above high threshold. This may soon damage the unit.
Voltage too low, 1507333 (dec): Voltage is below low threshold. This may cause unexpected HW
behavior.
Power consumption exceeded, 1507363 (dec): Node’s estimated power consumption exceeds power
module power output. One or more units need to be removed from the rack.
For more information about fault codes, see Tellabs 8600 Fault Management Configuration Guide from
Tellabs Portal (www.portal.tellabs.com).


The following command is only available from FP 2.10 onwards:

show debugging sw swmgt esw-version-history private-all 14





General Network Building
Troubleshooting 2
General Network Building Troubleshooting




The Tellabs 8000 INM does not write the management IP address to the node when the node is first
added to the database. It is critical that the IP address used is the same in both the node and the
database.

The Tellabs 8000 INM does not care where the management IP address is located on the node, as long
as it is reachable. Loopback interfaces are preferred for inband as they are reliable, and typically easily
reachable when routing is configured properly.



Outband reachability is dependent on the DCN between node and COMM server being set up correctly.
It could be built as switched, routed, or a combination of both.
The IP addresses used need to effectively administered, especially if there are multiple routed hops
between COMM server and nodes.
Static/Default routes can be configured on the nodes. But any later changes in the network may require
these to be modified, resulting in manual configuration changes and possible mistakes.
Using a routing protocol on the node to establish outbound communications has a high risk of the routing
information from the DCN being exchanged with the transport network. This could result in alternative
service paths through the DCN being discovered. Of course, separate routing process instances could
be used for transport network and DCN, however, this will increase the processing and resources load
on the node.

If access control lists or firewalls are implemented in the DCN routers, then it is important to ensure the
management protocols used between COMM server and nodes are not blocked.

For inband reachability, getting from the first hop 8660/8630 node to the outlying nodes through the
transport network is typically done by enabling routing. If routing is used it is important to make sure the
routing of the transport network is kept separate from the DCN.
Static routes can be configured on the outer nodes as an alternative to routing. This means manual
configuration and effective changes in the case of network path changes. If default routes are used there
can be issues for services built later. For example, LDP used to signal pseudo wires does not work with
default routes.

From FP 2.10 and above the command mgmt-traffic qos cs7 was globally set by default to Tellabs 8600
node configuration. It sets all management traffic which originates on the node to have automatic CS7
marking.
For management traffic being sent inband from the Comm Server to other nodes through a first hop by
the gateway node, it does not affect incoming traffic, or traffic passing through the node, thus it is
necessary to still use a access control list on the incoming interface to identify and mark management
traffic as CS7.
Management Traffic Command
Command Description
Configures the QoS used for management traffic (CLI (telnet, SSH), BMP (including ccn),
SNMP, SFTP and syslog).
Use the no option to cancel the command and return to default values.
Command Syntax
[no] mgmt-traffic qos { ef | cs7 | af4 | af3 | af2 | af1 | be }
Default Value: Cs7

Command Release History 8660/8630/8620

2.10 The command was introduced.
Command Release History 8605

1.2 The command was introduced.
Command Release History 8607

1.0A The command was introduced.
Usage Guidelines
This affects only packets sent from local agents towards Manager (Manager must set same
Qos for packets it sends back in order to have same QoS in commands and replies).
Command Examples
router(config)# mgmt-traffic qos be



If only the CLI management is used and not the Tellabs 8000 INM system, Router ID must be manually
configured. In the case that RID is not configured, the NE takes the following selection:
• The highest IP on the node
• If loopback interface exists, NE selects loopback with highest IP address as Router ID

Recommended way to configure Router ID

The recommended way to configure Router ID is to input the Router ID value in the Node Parameters
dialog when adding a new node to the DB using Tellabs 8000 INM. Tellabs 8000 INM automatically
pushes the router ID configured in the Node Parameters window to the hardware during node
Autodiscovery operation.
Note! If the router ID has been manually configured using CLI before Autodiscovery, it will be used
instead and Autodiscovery updates the Node Parameters dialog.
In order to fix a wrong NE Router ID to match the DB value the following tools can be used:
• CLI command
• Tellabs 8000 INM IP Host Parameters window
• Tellabs 8000 INM Consistency Checker









Tellabs 8600 node with ESW for FP2.9 can record all management operations to an audit log. The
management operation can be done either with Tellabs 8000 intelligent network manager or with CLI.
The audit log events are stored in the Tellabs 8000 database and they can be viewed with the Node
Configuration Audit Log tool.

The audit logging can be configured in the node with the following CLI commands:
bmp-server ccn destination primary [ip address] port [port]
where an IP address is the IP address of Communication Server and the port is 50000.
The source of the messages on the node needs to be also identified. This needs to be either the inband
or outband reachable from the Communication Server.
bmp-server ccn source [loopback if / mfe if]
The audit log tool can be launched from Tellabs 8000 Intelligent Network Manager Toolbox by selecting
the Security - Audit Logs - 8600 Node Configuration Audit Log menu option.
The tool is launched and it shows audit log events for all nodes.
The source of updates must correspond to the management address set for the node in the Node
Parameters dialog in Tellabs 8000 INM Network Editor tool.
The use of mfe for outband management and loopback for inband management are recommendations.
However, Audit Log will work with any other reachable ip interface as long as it is correctly configured in
Tellabs 8000 INM.






IP Network Building
Troubleshooting 3
IP Network Building Troubleshooting





CLI
The command show ip interface brief can also be used to quickly show all IP addresses assigned to
interfaces on a node.
Tellabs 8000 INM

In Node Manager, you can view all IP addresses assigned to interfaces by expanding the IP interfaces
section of the Tree View. Also, by right clicking on the node in the Tree View, selecting List View and
then IP List View.


Some useful commands are: show ip route, show ip ospf neighbor, show ip ospf database, etc…
Tellabs 8000 INM

In Node Manager, you can view all IP addresses assigned to interfaces by expanding the IP interfaces
section of the Tree View. Also, by right clicking on the node in the Tree View, selecting List View and
then IP List View.



In BGP the router-id has a different use than most other protocols. Tellabs 8000 INM requires that a
router-id be entered before the BGP process can be enabled.
However in CLI a router-id is not required. This is because we can use any interface as a source of
updates. While most other protocols have a unique source of updates BGP can have multiple sources of
updates. In BGP the source of updates is specified for every neighbour relationship.
The LDP router-id is the LSR ID for the LDP process. The LSR sends hello messages using UDP
packets to other LSRs in its network, identifying itself with the LDP RID. The hello message contains an
optional Transport Address TLV, which can be used to instruct its peers to open the TCP session to the
transport address instead of the source address found in the IP header. By setting the Transport
Address, service flaps can be avoided in case the global RID is not manually configured. In that case,
any changes in the loopback ip address can affect the LDP process.
In Tellabs 8000 INM, the Transport Address can be configured in ”LDP transport address” in the Node
MPLS dialog. In CLI, it can be configured within the router ldp process.














When creating the OSPF process if the Router ID field is left blank in the Create OSPF Router Process
dialog, then the node will be examined and whatever router ID is being used on the node will be taken
and used for the process. No OSPF router-ID will be set.

When creating the OSPF process if a Router ID is set by the operator in the Create OSPF Router
Process dialog, then the node will be configured with this value as the router ID for this process only.
Other processes will use their own value of the global value on this node.










In Node Manager, the IP OSPF dialog for each interface shows default OSPF interface settings. These
settings are not set to the node by default when the node or interface state is raised.
In the dialog it is seen that the default OSPF cost used for all interfaces is 10, regardless of type. This is
a placeholder value which can be changed by the operator. If the operator changes this value it is also
changed in the DB, and then becomes the DB value available for this interface.








Developed in 1987 by Digital Equipment Corporation’s DECnet (for ISO - International Organization for
Standardization) IS-IS is a public standard, published as International Organization for Standardization
(ISO) 9542 and republished as RFC 995. IS-IS offers support for IP and OSI protocols - called
Integrated IS-IS or Dual IS-IS.
The OSI suite uses Connectionless Network Service (CLNS) to provide connectionless delivery of data,
and the actual Layer 3 protocol is Connectionless Network Protocol (CLNP). CLNP is the solution for
unreliable delivery of data, similar to IP. IS-IS uses CLNS addresses to identify the routers and to build
the link-state database (LSDB).
Although Integrated IS-IS can support IP exclusively, IS-IS still uses CLNS to transmit reachability
information and still forms adjacencies using ES-IS. IP packets are not used during IS-IS
communications between nodes. IS-IS uses ConnectionLess Network Protocol (CLNP) packets when
communicating between nodes.


IS-IS Level 0 Routing

OSI routing begins when the hosts discover the nearest IS by listening to ISH (Intermediate System
Hellos) packets.
ES-IS (End System-to-Intermediate System) performs the following tasks:
• Identifies the area prefix to ESs
• Creates adjacencies between ESs and ISs
• Creates data link-to-network address mappings
When an ES needs to send a packet to another ES, it sends the packet to an IS on an attached network.
This process is known as Level 0 routing.

Level 1 routing within an IS-IS area recognizes the location of the end system (ES) and IS then builds a
routing table to reach each system. All devices in a Level 1 routing area have the same area address.

If a destination address is in another area, the Level 1 IS sends the packet to the nearest Level 2 IS
(Level 2 routing). Packet forwarding continues through Level 2 ISs until the packet reaches a Level 2 IS
in the destination area.

Routing between separate domains is called Level 3 routing. Level 3 routing passes traffic between
different autonomous systems.

This is just an example of one type of IS-IS infrastructure.

• When F talks with H, the path will be: F – E – A – B – C – G – H (traffic goes through B because it is
L2 enabled)
• When A talks with C, the path will be A – D - C (traffic goes through D because it is L1 enabled. It
cannot go through B because B is not L1 enabled)
• Since B is L2 only, it cannot route any intra-area traffic to/from A, C, or D.

• AFI (Authority Format Identifier) set to 49, which signifies that the AFI is locally administered and
individual addresses can be assigned
• The area identifier (ID), which must be at least 1 byte. Variable-length field (1 to 13 octets)
• System ID: Unique host
• NSEL, which must always be set to “00” for a router
Note: NSAP address, ending with a selector of 00, in the OSI world, is called a NET (Network Entity
Title) address.
• The NET address is required even if IS-IS is solely used for routing IP.
• To route IP traffic between areas, the Area ID is used.
• To route IP traffic within the same area, the Sys-ID is used.

When Router A sends IP packets to Router D:

• Router A sees that Router D’s Area ID (49.0111) is not the same as its own Area ID (49.0101)
Router A forwards the traffic to the closest L1/L2 router (Router B). Router A uses its own L1 database
to determine path to Router B.
• Router B sees that Router D’s AreaID (49.0111) is not the same as its own Area ID (49.0101).
Router B uses its own L2 database to determine the best path to 49.0111
• Router C sees that Router D’s AreaID (49.0111) is the same as its own.
Router C uses its own L1 database to determine the best path to Sys-ID 4444.4444.4444.




Level 1 and Level 2 Intermediate Systems have adjacency rules that must be followed.
1. Level 1 adjacencies will only form inside the same area and only if both sides are issuing level 1
hellos.
2. Level 2 adjacencies will form anywhere as long as both sides are issuing level 2 hellos

It should be noted that IS-IS can be configured with two network types: Broadcast(default option) and
Point-to-point. In broadcast network type, instead of having each router connected to the LAN advertise
an adjacency with every other router on the LAN, the entire network is considered a router, called the
pseudo-node. Each router just advertises a single adjacency to the pseudo-node. Otherwise, each IS on
a broadcast subnetwork with n connected ISs requires (n) (n – 1) / 2 adjacency advertisements.
Generating LSPs for each adjacency creates considerable overhead in terms of LSDB synchronization.







ISO 10589(IS-IS) mandates the padding of transmitted hello packets, which are used to establish and
maintain adjacencies, to the maximum possible data size a router can receive and process on an
interface. This provides a packet-size negotiation mechanism, which ensures adjacency forms only
between systems that can receive and process the largest possible data size that the other can transmit.



Popularity of IS-IS protocol usage for routing the IP traffic introduced several enhancements both within
IETF and vendor-specific implementations. With recent ESWs, Tellabs 8600 routers require to have
directly connected routers to be under the same IP subnet range in order to form IS-IS adjacency.




Use this command to turn on debugging for specified criteria.

Use the no option to cancel the command and return to default values.
Command Mode
Privilege
Configuration
Command Syntax
[no] debug isis [ ifsm | nfsm | events | pdu | lsp | spf | nsm | all ]
ifsm Debugging for Interface Finite State Machine.
nfsm Debugging for Neighbor Finite State Machine.
events Debugging for internal events.
pdu Debugging for IS-IS PDU.
lsp Debugging for LSP.
spf Debugging for route calculation.
nsm Debugging for NSM messages.
all Enable all debugging.







MPLS Network Building
Troubleshooting 4
MPLS Network Building Troubleshooting




MPLS labels use a platform specific label space and have fixed allocation as follows in
T8660 and T8630:
• 16...1 023 are reserved for static LSPs. These labels are preserved at SW reset.
• 1 024...8 599 are reserved for future use.
• 8 600...85 999 are reserved for static LSPs. These labels are preserved at SW reset.
• 86 000...1 048 575 are reserved for dynamic LSPs (LDP and RSVP-TE protocols). These
labels are re-allocated at SW reset.
MPLS labels use a platform specific label space and have fixed allocation as follows in
T8620, T8607 and T8605:
• 16...1 023 are reserved for static LSPs. These labels are preserved at SW reset.
• 1 024...8 599 are reserved for dynamic LSPs (LDP and RSVP-TE protocols). These
• 8 600...32 767 are reserved for static LSPs. These labels are preserved at SW reset.
• 32 768...1 048 575 are reserved for dynamic LSPs (LDP and RSVP-TE protocols). These

Specifying the downstream-on-demand or downstream -unsolicited mode affects which LSR will initiate
mapping requests and mapping advertisements.
The downstream-unsolicited mode distributes labels to peers without waiting for a label request. This
mode is typically used with the liberal label retention mode.
In the downstream-on-demand mode, a router distributes a label to a peer only if there is a pending label
request from a peer. The reaction of the downstream router to this request depends on the label
advertising mode supported on the next hop. The downstream-on-demand mode is typically used with
the conservative label retention mode.
When an LSR receives a label binding for a particular FEC from another LSR that is not its next hop for
that FEC, it may keep track of such bindings or discard them. Liberal method retains all label bindings to
FEC received from label distribution peers even if the LSR is not the current next hop. Conservative
method maintains only label bindings for valid next hops in a LSP.
Liberal label retention mode allows for quicker adaptation to routing changes, whereas conservative
label retention mode requires an LSR to maintain fewer labels.





































Tellabs 8600
Hands-on Exercises 5
TELLABS 8600 SMART ROUTERS
HARDWARE TROUBLESHOOTING- HANDS-ON LAB EXERCISE
HANDS-ON LAB EXERCISE 1
1. Validate the operational state of the node.
2. Verify the correct inventory and software configuration.
Slot:
Card Type Software Version
Module 0 Module 1
Slot:
Module 0 Module 1
Slot:
Module 0 Module 1
Slot:
Card Type: Software Version:
Module 0 Module 1
1
3. Verify correct working voltages.
Slot:
Card Type
Voltages
Slot:
Card Type
Voltages
Slot:
Card Type
Voltages
Slot:
Card Type:
Voltages
2
4. Verify correct temperatures.
Slot:
Card Type
Temperatures
Slot:
Card Type
Temperatures
Slot:
Card Type
Temperatures
Slot:
Card Type:
Temperatures
3
5. Verify backplane operation.
Slot Link A Status Link B Status
6. Produce a snapshot config of a node containing the HW inventory and the running-config.
7. Verify the file actually contains the inventory and the configuration.
8. Produce a text file containing command history, fault history, ESW versions, inventory and
running-config.
9. Select a Fast Ethernet or Gigabit Ethernet interface on one node. Verify the admin and
operational status of the selected interface.
4
10. Check the counters and log for selected interfaces.
5
6
GENERAL NETWORK BUILDING TROUBLESHOOTING
HANDS-ON LAB EXERCISE
HANDS-ON EXERCISE 2
1. Turn on Node configuration auditing.
2. Validate node communications from NMS, repair if necessary.
3. Identify the source of Router ID selected for nodes.
Node Router-ID Source
4. Verify the MPLS protocol configuration in interfaces, repair with Consistency Checker Tool if
necessary.
5. Identify fixes made in Node Configuration Audit Tool.
1
GENERAL NETWORK BUILDING TROUBLESHOOTING
2
IP NETWORK BUILDING TROUBLESHOOTING
 Exercise for OSPF enabled network

1. Verify IP connectivity and reach ability in the network.
2. Identify a reason for any issues.
3. Generate a text file record of the steps used to fix any problems.
1
4. List the following information about the OSPF process for each Tellabs 8630 node:
 Router ID – how it is selected
 List the neighbors
 List the type and metric for all interfaces used
Node: Router ID Source
Neighbor ID IF Used Cost
Node: Router ID
Node: Router ID
2
5. Select another node in the network.

 Validate the path to this node
 Show how its path cost is calculated
Source Node address
Destination Node address
Out IF IP Address Cost
Total cost
Explanation:
3
 Exercise for IS-IS enabled network

1. Verify IP connectivity and reach ability in network
2. Identify a reason for any issues
3. Generate and text file record of the steps used to fix any problems
4
4. For each group 8630 node list the following information about the IS-IS process
 List the neighbors
 List the circuit-type for interfaces
Node:
Neighbor System ID IF Used Circuit-type
Node:
Node:
5. Select another node in the network.

 Validate the path to this node by traceroute.
5
6
MPLS NETWORK BUILDING TROUBLESHOOTING
1. Verify LDP LSPs have been discovered correctly.
1
2. Select a single FEC on 8605 node, map the path, label numbers and outgoing IFs to the egress
node for LSP.
Source Node LSP FEC
Label in Label out Out IF
LSR node
LSR node
LSR node
LSR node
3. Verify the path is the optimal one.
2
4. Prove labelled packets are used from start to finish of LSP.
5. Fix any issues found.
Issues Discovered/Fixed:
6. Generate a debug output showing LDP adjacency establishment.
3
4
Tellabs 8600 Lab Networks 6
Tellabs 8600 Exercise Network
Link Interfaces and IP Addresses
1
Use number in node

name for last octet in
link interface address! 10.17.18.0 /24 nxE1
0/0.10
N18 N17
nxE1 GE
N15 10.14.18.0 /24 6/0/0.10

3/0.10 10.14.15.0 /24
9/0/7
GE 9/0/6
N14 IFC2
9/0/7 10.18.22.0 /24
6/0/3.10
10.13.14.0 /24 IFC2

6/0/1.10 13/0/6 N22
GE 9/0/0
nxE1 0/0.10
N13 el. GE 10.12.14.0 /24 GE Ring N x STM-1 ATM

11/0/X
W: 13/0/7
2/4/0 P: 12/0/7 RNC
10.11.12.0 /24
nxE1 0/0.10
ELP 1+1
el. GE 2/4/2.10
N11 2/4/1
10.20.22.0 /24
N12
10.12.20.0 /24 W: 9/0/6
IFC2 P: 8/0/6
el. GE
9/0/0
10.19.20.0 /24
10.20.21.0 /24
6/0/2.10 6/0/3.10 GE nxE1
nxE1 GE N20 0/0.10
0/0.10
nxEth N19
N21
86ST8000CLI-D Espoo
Preparing Nodes with the CLI

Console Connections
2
 Classroom Terminal Server aggregates all the Tellabs 8600

console port connections to one box
 Terminal Server IP address = 172.19.137.___
Hands-on Group Node-ID / Name TCP port in Terminal Server
11 / N11 __11
Group 1
12 / N12 __12
13 / N13 __13
Group 2
14 / N14 __14
15 / N15 __15
Group 3
17 / N17 __17
18 / N18 __18
Group 4
19 / N19 __19
20 / N20 __20
Group 5
21 / N21 __21
86ST8000CLI-D Espoo

Tellabs 8600 Smart Routers System Troubl PDF

Uploaded by

Copyright:

Available Formats

Tellabs 8600 Smart Routers System Troubl PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Tellabs 8600 Smart Routers System Troubl PDF

Uploaded by

Copyright:

Available Formats

Tellabs 8600 Smart Routers

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 3

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 4

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 5

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 6

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 7

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 8

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 9

05.5065.9039 Rev 11 00_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 10

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 1

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 2

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 3

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 4

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 5

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 6

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 7

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 8

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 9

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 10

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 11

High Timeconditional Threshold

High Unconditional Threshold

Critical Timeconditional Threshold

Critical Unconditional Threshold

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 12

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 13

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 14

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 15

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 16

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 17

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 18

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 19

Here WINGINE is an average of all of the processors.

MBUSx = memory bus number

QMEM1 = queue buffer

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 20

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 21

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 22

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 23

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 24

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 25

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 26

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 27

POST = Power On Self Test

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 28

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 29

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 30

Interface ge13/0/7 section:

Logical ge13/0/7 section:

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 31

Router(cfg-if[ge5/0/7.51])# vlan statistics non-qos in-qos

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 32

There are three MTU values specified in the output:

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 33

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 34

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 35

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 36

05.5065.9039 Rev 11 01_86ST8000CLI-D TELLABS CONFIDENTIAL & PROPRIETARY 37