Tecmpl 3201

Troubleshooting MPLS
– On All Cisco
Platforms
Vinit Jain – CCIE# 22854 @vinugenie
Brad Edgeworth – CCIE# 31574 @bradedgeworth
Michael Whitaker – CCIE# 51871
TECMPL - 3201
Cisco Spark
Questions?
Use Cisco Spark to chat with the
speaker after the session
How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
Cisco Spark spaces will be cs.co/ciscolivebot#TECMPL-3201

available until July 3, 2017.
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
We will be learning on how MPLS
works and in fact works so smoothly…
But, how can you learn if you can’t see it..
TECMPL-3201 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
• Troubleshooting MPLS TE
Agenda
• Fundamentals
• Troubleshooting LDP Issues
• BGP, LDP, RSVP
• Troubleshooting MPLS LSP
• OAM, Multipath Trace
• Troubleshooting MPLS L3 VPNs
• Inter-AS MPLS VPNs
• CsC
• Troubleshooting 6VPE
MPLS
Fundamentals
Why was MPLS created?
• With traditional routing, a router receives a packet and checks the header for the
destination IP address.
• It then locates the longest matching route in the forwarding table
• Performs recursive lookups to find the outbound interface and then forward the
packet out of that interface.
• This process continues for every hop (router) along the path to the packet’s
destination.
Visualizing the Problem (IP Routing)
Let me look that
up
Network Out Int Network Out Int Network Out Int Network Out Int
10.1.0.0/16 Gi0/2 10.1.0.0/16 Gi0/0 10.1.0.0/16 Gi0/1 10.1.0.0/16 Gi0/0
10.1.1.0/24 Gi0/0 10.1.1.0/24 Gi0/0 10.1.1.0/24 Gi0/1 10.1.1.0/24 Gi0/0
10.12.1.0/24 Gi0/0 10.12.1.0/24 Gi0/0 10.12.1.0/24 Gi0/1 10.12.1.0/24 Gi0/0
10.23.1.0/24 Gi0/0 10.23.1.0/24 Gi0/1 10.23.1.0/24 Gi0/1 10.23.1.0/24 Gi0/0
10.34.1.0/24 Gi0/0 10.34.1.0/24 Gi0/1 10.34.1.0/24 Gi0/0 10.34.1.0/24 Gi0/0
10.4.0.0/16 Gi0/0 10.4.0.0/16 Gi0/1 10.4.0.0/16 Gi0/0 10.4.0.0/16 Gi0/1
10.4.4.0/24 Gi0/0 10.4.4.0/24 Gi0/1 10.4.4.0/24 Gi0/0 10.4.4.0/24 Gi0/2
R1 R2 R3 R4
I need to goto
10.4.4.45
Creation of Multiprotocol Label Forwarding
• Multi-Protocol Label Switching (MPLS) forwarding reduces the lookup process

by all of the routers in the path of a packet.
• A router will assign a locally significant label (numerical value) for the directly
connected prefixes that is connected to it. A label exists for all of the routes for
all of the routers in the routing domain.
• Outside of the first packet, all forwarding occurs based on the MPLS label. The
local router contains a table that correlates its local label with the downstream
routers label, and outbound interface. Forwarding lookups are more explicit and
do not require subsequent analysis by routers in the middle of the packets path.
Visualizing the Solution (MPLS Forwarding)
Let me look that Let me look that Let me look that
up label up label up label
In Out In Out In Out

Label Label Network Out Int Label Label Network Out Int Label Label Network Out Int
100 - 10.1.0.0/16 Gi0/2 200 100 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
101 - 10.1.1.0/24 Gi0/1 201 101 10.1.1.0/24 Gi0/0 302 201 10.1.1.0/24 Gi0/0
102 - 10.12.1.0/24 Gi0/0 202 - 10.12.1.0/24 Gi0/0 303 202 10.12.1.0/24 Gi0/0
103 203 10.23.1.0/24 Gi0/0 203 - 10.23.1.0/24 Gi0/1 304 - 10.23.1.0/24 Gi0/0
104 204 10.34.1.0/24 Gi0/0 204 304 10.34.1.0/24 Gi0/1 305 - 10.34.1.0/24 Gi0/0
105 205 10.4.0.0/16 Gi0/0 205 305 10.4.0.0/16 Gi0/1 306 405 10.4.0.0/16 Gi0/1
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 307 406 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
I need to goto A MPLS packet A MPLS packet A MPLS packet

10.4.4.45 with label 206 with label 306 with label 406
Forwarding Performed by MPLS Labels
MPLS networks forward traffic based upon the outermost MPLS label of a packet.
None of the transit routers require the examination of the packet’s header or
payload as long as a label exists in the packet.
Provides a form of tunneling
100 - 10.1.0.0/16 Gi0/2 200 100 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
101 - 10.1.1.0/24 Gi0/1 201 101 10.1.1.0/24 Gi0/0 302 201 10.1.1.0/24 Gi0/0
102 - 10.12.1.0/24 Gi0/0 202 - 10.12.1.0/24 Gi0/0 303 202 10.12.1.0/24 Gi0/0
103 203 10.23.1.0/24 Gi0/0 203 - 10.23.1.0/24 Gi0/1 304 - 10.23.1.0/24 Gi0/0
104 204 10.34.1.0/24 Gi0/0 204 304 10.34.1.0/24 Gi0/1 305 - 10.34.1.0/24 Gi0/0
105 205 10.4.0.0/16 Gi0/0 205 305 10.4.0.0/16 Gi0/1 306 405 10.4.0.0/16 Gi0/1
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 307 406 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
MPLS Fundamentals
MPLS Architecture
• MPLS has two major components:
1. Control plane: Exchanges Layer 3 routing information and labels
2. Forwarding plane: Forwards packets based on labels
• Control plane contains complex mechanisms to exchange routing information,
such as OSPF, EIGRP, IS-IS, and BGP, and to exchange labels, such as TDP,
LDP, BGP, and RSVP.
• Forwarding plane forwards packets based on CEF
MPLS Fundamentals
Terminologies
• RIB is the Routing Information Base that is analogous to the IP routing table.
• FIB aka CEF is Forwarding information base that is derived from the IP routing
table.
• LIB is Label Information Base that contains all the label bindings learned via
LDP
• LFIB is Label Forwarding Information Base that is derived from FIB entries and
corresponding LIB entries.
• FEC ( Forwarding Equivalence Class)
• Group of IP packets forwarded in the same manner (e.g. over same forwarding path)
• A FEC can represent a: Destination IP prefix, VPN ID, ATM VC, VLAN ID, Traffic
Engineering tunnel, Class of Service.
MPLS Fundamentals
MPLS Label: Label Format
• MPLS uses a 32-bit label field that is inserted between Layer 2 and Layer 3
headers
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
Label COS S TTL
Label = 20 bits
COS/EXP = Class of Service, 3 bits
S = Bottom of Stack, 1 bit
TTL = Time to Live (Loop detection)
MPLS Fundamentals
MPLS Label: Data Packet
Dest Source MPLS MPLS MPLS Dest Source DSCP TTL Payload
MAC MAC Label EXP TTL IP IP
MPLS Router MPLS Header Fields Original Packet Header

Interface MAC
MPLS Fundamentals
MPLS Label: The Label Stack
• \
 An MPLS packet may have more than one label

 Frame Mode can handle a stack of two or more
labels, depending on the platform
 Bottom most label has the S-bit set to 1
 LSRs label switch packets are based ONLY on
the label at the top of the stack
MPLS Fundamentals
MPLS Label: The Label Stack
 The following scenarios may produce more than one label:
• MPLS L3 VPNs (two labels: The top label points to the egress router and the
second label identifies the VPN.)
• MPLS TE with Fast Reroute (FRR) (two or more labels: The top label is for
the backup tunnel and the second label points to the primary tunnel
destination.)
• MPLS VPNs combined with MPLS TE / FRR (three labels)
• Carrier Supporting Carrier (CSC) with MPLS TE / FRR (four labels)
MPLS Fundamentals
Label Switch Path (LSP)
IGP domain without a label IGP domain with a label

distribution protocol distribution protocol
LSP follows IGP shortest path LSP diverges from IGP shortest path
 LSPs are derived from IGP routing information

 LSPs may diverge from IGP shortest path
• LSP tunnels (explicit routing) with MPLS Traffic Engineering (TE)
 LSPs are unidirectional
Upstream and Downstream MPLS Routers
Relative terms in MPLS – Changes based on destination network
 Downstream – Router towards the direction of the destination.
Advertises the local label towards the source
 Upstream – Router towards the source of the packet. Labels the packet
with the downsteam router’s local label.
 R3 is downstream to R2 for the 10.4.4.5 prefix
R1 R2 R3 R4
10.4.4.5 Payload
MPLS Labeling Concepts
There are three main label actions that you should be aware of:
 Push, SWAP, and POP
Demonstrate with a packet forwarding to 10.4.4.5
3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
.. .. .. .. .. .. .. .. .. .. .. ..
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 307 POP 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
 Push – Placing a new label on to the packet (IP or MPLS)

3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
.. .. .. .. .. .. .. .. .. .. .. ..
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 307 POP 10.4.4.0/24 Gi0/1
206
R1 R2 R3 R4
206 10.4.4.5 Payload
 Swap – Removal of topmost label and placing a new label

3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
.. .. .. .. .. .. .. .. .. .. .. ..
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 307 POP 10.4.4.0/24 Gi0/1
306
R1 R2 R3 R4
206
306 10.4.4.5 Payload
 Swap – Removal of topmost label and placing a new label
 Pop – Removal of the topmost label
3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 301 200 10.1.0.0/16 Gi0/0
.. .. .. .. .. .. .. .. .. .. .. ..
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 306 POP 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
306 10.4.4.5 Payload
Implicit Null and Penultimate Hop Pop
Why did R3 have a POP label for 10.4.4.0/24 Network?
 R4 places an Implicit Null (MPLS Label #3) for the direct attached
network. This indicates that R4 does not need a label as the network is
directly attached, or that R4 is the last Label Switch Router (LSR) to the
destination
 R3 receives Implicit Null for that FEC and places a POP entry in the FIB
for that FEC. Penultimate Hop Pop is the LSR before the edge router.
(IE. R3)
In Out In Out
Label Label Network Out Int Label Label Network Out Int
307 POP 10.4.4.0/24 Gi0/1 3 - 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
MPLS Fundamentals
MPLS Architecture Incoming IP
Packet
Control Plane Data Plane

Routing
Routing Routing Forwarding
updates
Protocol Information Information
from peer
Database Base (RIB) Base (FIB)
router’s
Outgoing
MPLS/IP
Packet
Label Label Label
Local
Bindings Information Forwarding
Label
via LDP Base (LIB) Information
Bindings
peering Base (LFIB)
Incoming
MPLS Packet
MPLS Fundamentals
MPLS: Ethertype
 Ethertype 0x0800 refers to IP

 Ethertype 0x8847 refers to MPLS
 Based on the Ethertype, the packet is handed over to the appropriate
processing engine on the router
MPLS Fundamentals
Facts Check - Question
• Which protocols have signaling and labeling capabilities?
• OSPF / IS-IS
• RSVP
• LDP / TDP
• BGP
MPLS Trivia
Question
Fun with MPLS Trivia
R1, R2, R3, R4 and R5 all have OSPF and MPLS enabled.
What changes can be made on R2 and/or R3 to prevent only R1’s
Loopback (192.168.1.1) from pinging R5’s Loopback (192.168.5.5)?
We will explain some of the concepts that make this work.
R1 R2 R3 R4 R5
R1#ping 192.168.5.5 so 192.168.1.1

Lo0: 192.168.1.1 Type escape sequence to abort. Lo0: 192.168.5.5
Sending 5, 100-byte ICMP Echos to 192.168.5.5, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/4/5 ms
Configuring and
Basic LDP
Operations
Troubleshooting LDP Issues
MPLS LDP Configuration
IOS / IOS XE IOS XR NX-OS
install feature-set mpls

mpls label protocol ldp mpls ldp
feature-set mpls
! router-id x.x.x.x
feature mpls
interface Gig 0/0 interface gi 0/0/0/0
mpls ldp configuration
mpls ip interface gi 0/0/0/1
router-id x.x.x.x
mpls label protocol ldp
!
exit
interface ethernet 2/1
!
mpls ip
mpls ldp router-id
loopback0 force
Establishing Adjacency & Swapping Labels
Populating the RIB
• First the IGP (OSPF / IS-IS) is established and routes are exchanged between
all routers

N/A N/A 10.1.0.0/16 Gi0/2 N/A N/A 10.1.0.0/16 Gi0/0 N/A N/A 10.1.0.0/16 Gi0/0
R1 R2 R3 R4
Creating the Local Labels
• Local Labels are automatically generated for all prefixes in the RIB.
(MPLS Label 3 is reserved for Implicit-Null – directly connected routes)
• This includes local network prefixes
3 - 10.1.0.0/16 Gi0/2 200 N/A 10.1.0.0/16 Gi0/0 300 N/A 10.1.0.0/16 Gi0/0
3 - 10.1.1.0/24 Gi0/1 201 N/A 10.1.1.0/24 Gi0/0 301 N/A 10.1.1.0/24 Gi0/0
3 - 10.12.1.0/24 Gi0/0 3 - 10.12.1.0/24 Gi0/0 302 N/A 10.12.1.0/24 Gi0/0
103 N/A 10.23.1.0/24 Gi0/0 3 - 10.23.1.0/24 Gi0/1 3 - 10.23.1.0/24 Gi0/0
104 N/A 10.34.1.0/24 Gi0/0 204 N/A 10.34.1.0/24 Gi0/1 3 - 10.34.1.0/24 Gi0/0
105 N/A 10.4.0.0/16 Gi0/0 205 N/A 10.4.0.0/16 Gi0/1 305 N/A 10.4.0.0/16 Gi0/1
106 N/A 10.4.4.0/24 Gi0/0 206 N/A 10.4.4.0/24 Gi0/1 306 N/A 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
• Local Labels are exchanged with downstream routers
• Labels are all exchanged at the same time.
(This animation was done to show you the correlation of tables)

3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 300 200 10.1.0.0/16 Gi0/0
3 - 10.1.1.0/24 Gi0/1 201 POP 10.1.1.0/24 Gi0/0 301 201 10.1.1.0/24 Gi0/0
3 - 10.12.1.0/24 Gi0/0 3 - 10.12.1.0/24 Gi0/0 302 POP 10.12.1.0/24 Gi0/0
103 POP 10.23.1.0/24 Gi0/0 3 - 10.23.1.0/24 Gi0/1 3 - 10.23.1.0/24 Gi0/0
104 204 10.34.1.0/24 Gi0/0 204 POP 10.34.1.0/24 Gi0/1 3 - 10.34.1.0/24 Gi0/0
105 205 10.4.0.0/16 Gi0/0 205 305 10.4.0.0/16 Gi0/1 305 405 10.4.0.0/16 Gi0/1
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 306 406 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
Troubleshooting LDP
Issues
LDP Neighborship
 LDP neighborship is formed on TCP port 646
 Discovery Mechanism:
 Basic Discovery – Multicast UDP hellos for directly connected neighbors
 Extended Discovery – Targeted Unicast UDP hellos for non-directly connected
neighbors
• Parameters
• Session Keepalive = 60 sec. & Hold time = 180 Sec.
• Discover Hello interval = 5 sec. and Hold Time = 15 sec.
• Can be viewed using the command show mpls ldp parameters
Adjacency Requirements
 LDP Router-ID must have a specific routing entry in the RIB
 Authentication parameters must match
 Multiple L3 links between LDP devices
LDP Neighborship Negotiation
Verifying LDP Neighborship
PE1#sh mpls ldp neighbor

Peer LDP Ident: 10.13.1.101:0; Local LDP Ident 10.13.1.61:0
TCP connection: 10.13.1.101.11031 - 10.13.1.61.646
State: Oper; Msgs sent/rcvd: 58/60; Downstream
Up time: 00:39:27
LDP discovery sources:
Ethernet0/0, Src IP addr: 10.13.1.5
Ethernet1/0, Src IP addr: 10.13.1.9
Addresses bound to peer LDP Ident:
10.13.1.9 10.13.1.5 10.13.2.5 10.13.1.101
PE1#show tcp brief| i 646

43ABB020 10.13.1.101.11031 10.13.1.61.646 ESTAB
PE1#
Reachability and ACL verification
• Ensure reachability between the LDP router ID’s

PE1#ping 192.168.11.11 source lo0
Type escape sequence to abort.
..... Check Routing
Success rate is 0 percent (0/5) Configuration
• Verify no ACL in path blocking TCP port 646 and other Multicast traffic for LDP
Hello’s.
PE1#telnet 192.168.11.11 646 /source-interface lo0
Trying 192.168.11.11, 646 ... Verify ACLs in the path or
% Destination unreachable; gateway or host down on the routers itself
LDP Router-id
• If router-id is not set manually, router checks all operational interfaces on the
router(including loopbacks) and chooses the highest IP address as the LDP
router-id.
• LDP_ID should be hardcoded via
• “mpls ldp router-ID <interface>”
• The above configuration will not help unless:
• <interface> is UP when LDP gets started
• Existing LDP_ID (usually an interface) is shut
• Following avoids both shortcomings

• “mpls ldp router-ID <interface> force”
Troubleshooting LDP issues
Verifying LDP Connection
 “show mpls ldp discovery [detail]”

• Must show xmit/recv on LDP enabled interface
PE1#show mpls ldp discovery

Local LDP Identifier:
192.168.1.1:0 Local LDP_ID Xmited and
Recvd Hellos
Discovery Sources: on that
Interfaces: interface
GigabitEthernet0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0
Discovered
Neighbors’ LDP_ID
Problem with xmit / recv
Lo0=192.168.1.1 Lo0=192.168.11.11
PE1 P1
PE1#show mpls ldp discovery P1#show mpls ldp discovery

Local LDP Identifier: Local LDP Identifier:
192.168.1.1:0 192.168.11.11:0
Discovery Sources: Discovery Sources:
Interfaces: Interfaces:
GigabitEthernet0/1 (ldp): xmit GigabitEthernet0/1 (tdp): xmit
R1#debug mpls ldp transport connections

07:00:06.106: ldp: Scan listening TCBs
07:01:06.106: ldp: Scan listening TCBs Label Protocol
07:02:06.106: ldp: Scan listening TCBs is TDP
PE1(config-if)#mpls label protocol ldp
LDP No Route Problem
Lo0=192.168.1.1 Lo0=192.168.11.11
PE1 P1
PE1#show mpls ldp discovery P1#show mpls ldp discovery

192.168.1.1:0 192.168.11.11:0
Discovery Sources: Discovery Sources:
Interfaces: Interfaces:
Gi0/1 (ldp): xmit/recv Gi0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0; no route LDP Id: 192.168.1.1:0
PE1#show ip route 192.168.11.11

% Network not in table
Problem: Default route towards the peering router
Problem due to Summarization
PE1 P1
PE1#show mpls ldp neighbor 192.168.11.11 PE2#sh mpls ldp neighbor 192.168.1.1
PE1#show mpls ldp discovery PE2#show mpls ldp discovery

192.168.1.1:0 192.168.11.11:0
GigabitEthernet0/1 (ldp): xmit/recv GigabitEthernet0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0 LDP Id: 192.168.1.1:0
PE1#show ip route 192.168.11.11 PE2#show ip route 192.168.1.1
Routing entry for 192.168.11.11/32 Routing entry for 192.168.1.0/24
Known via "ospf 100", distance 110, metric 2, type Known via "bgp 100", distance 200, metric 0
intra area Tag 1, type internal
Last update from 10.1.111.11 on Gi0/1, 00:04:34 ago Last update from 192.168.1.12 20:10:38 ago
Routing Descriptor Blocks: Routing Descriptor Blocks:
* 10.1.111.11, from 192.168.11.11, 00:04:34 ago, * 192.168.1.12, from 192.168.12.12, 20:10:38
via GigabitEthernet0/1 ago
Route metric is 2, traffic share count is 1 Route metric is 0, traffic share count is 1
AS Hops 5
Troubleshooting LDP Issues Also good to check “show
MPLS LDP Trace on IOS XR mpls ldp trace discovery”
RP/0/0/CPU0:PE2#show mpls ldp trace peer last 20
0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9548, event=0, state
0 -> 1
0/0/CPU0 t1 [PEER]:581: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'TCP connection closed'
0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'TCP connection closed' ('Success')
0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg
0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9520, event=0, state
0 -> 1
0/0/CPU0 t1 [PEER]:575: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'Received Notification message

from peer' (more_info 'KeepAlive Timer Expired')
0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'Received Notification message

from peer' ('KeepAlive Timer Expired')
0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg
LDP & IGP Sync
• When a link comes up, LDP and IGP compete to converge; Labeled traffic drops
if IGP wins.
• When LDP session on a link drops, IGP may continue forwarding labeled traffic
to that link and cause traffic dropped.
LDP & IGP Sync – Solution
• Link up:
• If LDP peer is reachable (alternate route exists), defer IGP adjacency on the link.
• If LDP peer is not reachable (no alternate route), IGP advertise max-metric to reach
neighbor through the link.
• LDP session down:
• IGP advertises max-metric to reach neighbor through the link.
LDP & IGP Sync
• LDP IGP Sync feature is enabled under IGP (OSPF/ISIS)

• - “sync-igp-shortcuts” for TE tunnel interfaces, “sync” for all other types.
router (config-isis-if-af) # mpls ldp sync [ level <1-2> ]
router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if)
router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar)
LDP & IGP Sync
 LDP IGP Sync delays are configured under LDP
router (config-ldp) # igp sync delay on-session-up <sec>
router (config-ldp) # igp sync delay on-proc-restart <sec>
LDP Session Protection
• Problem:
I. When a link flaps (for a short time),
II. LDP hello adjacency over the link flaps
III. LDP session is torn down then re-setup
IV. LDP re-exchanges label bindings when LDP session is setup (i.e. LDP
re-convergence).
• Solution:
• When LDP session supported by link hello is setup, create a targeted hello to
protect the session.
• When link is down, the targeted hello remains through other path and keeps
the LDP session up.
• When link restores, re-discover neighbors, re-program forwarding.
LDP Session Protection
router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> | infinite } ]
router (config-ldp) # log session-protection
Case Study - 1
IP RAN
10.12.2.0/24
• 3 routing processes between
R1 and R2
192.168.1.1 192.168.2.2
• Lo0 defined as the LDP router- 10.12.1.0/24
id on both routers
R1 R2
• LDP adjacency is formed just
across one link, down on other
two CORE
10.12.3.0/24
R1#show mpls ldp neighbor
TCP connection: 10.12.3.2.646 - 192.168.1.1.18418
Up time: 00:19:22
GigabitEthernet4, Src IP addr: 10.12.3.2
10.255.0.85 10.12.1.2 10.12.2.2 10.12.3.2 192.168.2.2
R1#show mpls ldp neighbor

TCP connection: 192.168.2.2.25006 - 192.168.1.1.646
Up time: 00:19:22
10.255.0.85 10.12.1.2 10.12.2.2 10.12.3.2 192.168.2.2
*Jun 9 03:37:39.492: ldp: Opening listen port 646 for 192.168.2.2 (for hellos from
10.12.2.2)
*Jun 9 03:38:36.470: ldp: adj match for listen record
*Jun 9 03:38:36.470: ldp: lsn_closing is TRUE, adj was found for listen record
*Jun 9 03:38:36.470: ldp: removing/restarting errored listen socket (h_adj:192.168.2.2:0)
*Jun 9 03:38:36.470: ldp: {ldp listen 0.0.0.0:646=>192.168.2.2:0}: Delete listen TCB; tcb
0x7F1D2AD30548 [key 3090]; addr 192.168.2.2
*Jun 9 03:38:36.470: ldp: Unregistered from LDP TCB database tcb 0x7F1D2AD30548 [key
3090], total 1
*Jun 9 03:34:52.516: ldp: Opening ldp conn; adj 0x7F1D2AC4D9D8, 192.168.1.1

<-> 10.12.3.2; with normal priority
*Jun 9 03:34:52.517: ldp: :{ldp conn 192.168.1.1:49572=>10.12.3.2:646}:
Registered tcb 0x7F1D2AD2FB28 [key 2924] with LDP TCB database, total 2
*Jun 9 03:34:52.518: ldp: Conn failed (TCP connect notify)!; adj
0x7F1D2AC4D9D8, 10.12.3.2
*Jun 9 03:34:52.519: ldp: {ldp conn 192.168.1.1:49572=>10.12.3.2:646} (Gi4)
(adj 0x7F1D2AC4D9D8): processing transport close request
*Jun 9 03:34:52.519:cle mp ldp: Unregistered from LDP TCB database tcb
0x7F1D2AD2FB28 [key 2924], total 1
Case Study - 1
• Verify the TCP connection – You will find the clue
• Router-ID is configured with Lo0 (forced)
• If one of the interfaces is configured with mpls ldp discovery transport-
address interface, then this behavior can be noticed.
Troubleshooting LSP
Issues
Troubleshooting MPLS LSP
Reasons for LSP to Break
MP-IBGP – VPNv4
LDP + IGP
172.16.11.0/24 10.1.111.0/24 10.1.211.0/24 172.16.22.0/24
CE1 PE1 P1 PE2 CE2

Lo0=172.16.1.1/32 192.168.1.1/32 192.168.11.11/32 192.168.2.2/32 Lo0=172.16.2.2/32
• Broken LDP adjacency

• MPLS not enabled
• Mismatch labels
• Software/hardware corruption
Label Information Base (LIB)
• LIB stores local and remote bindings

• Local Binding:
• Prefix in own routing table + local label
• One binding
• Remote Binding:
• Prefix + remote label received from LDP neighbor
• Holds LDP router-id
• One binding per LDP neighbor
• LIB stores all labels from all LDP (BGP) neighbors, even the ones that are not
used for packet forwarding (now)
Looking at the LIB
RTR#show mpls ldp bindings detail
tib entry: 10.1.1.0/30, rev 10
local binding: tag: imp-null
Advertised to:
10.1.2.2:0 10.1.2.6:0 10.1.2.4:0
remote binding: tsr: 10.1.2.2:0, tag: imp-null
remote binding: tsr: 10.1.2.6:0, tag: 12304
remote binding: tsr: 10.1.2.4:0, tag: 12305
Label Forwarding Information Base (LFIB)
• The LFIB stores local and remote labels for prefixes that are used to forward
packets
• Prefixes that are used = prefixes in routing table (RIB)
• Labels are derived from LIB
LDP TDP
prefix, next-hop and in-
label, out-label prefix + next-hop
LIB LFIB RIB
(prefix, LDP Ident, get in- and out-label for (prefix,next-hop, (prefix, next-hop)
label) (prefix, next-hop) in-label, out-label)
Building the LFIB
P1#show ip route 3.3.3.4

Routing entry for 3.3.3.4/32
* 10.1.2.1, from 10.1.2.1, 13:28:32 ago, via Ethernet0/0
P1#show mpls ldp neighbor 10.1.2.1
P1#show mpls ldp binding 3.3.3.4 255.255.255.255
lib entry: 3.3.3.4/32, rev 18
remote binding: lsr: 3.3.3.3:0, label: imp-null
P1#show mpls forwarding 3.3.3.4
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
20 Pop Label 3.3.3.4/32 0 Et0/0 10.1.2.1
MPLS OAM
• Defined in RFC 4379
• LSP Ping and Traceroute provide ability to monitor MPLS Label Switched Paths
and quickly isolate MPLS forwarding problems.
• Two messages
• MPLS Echo Request:
MPLS labeled IPv4 or IPv6 UDP packet
• MPLS Echo Reply IPv4 or IPv6 UDP packet
• Ping mode: Connectivity check of an LSP

• Test if a particular “FEC” ends at the correct egress LSR
• Traceroute mode: Hop by Hop fault localization
• Packet follows data path
FEC Types Supported
• ping mpls ?
ipv4 Target specified as an IPv4 address
pseudowire Target VC specified as an IPv4 address and VC ID
traffic-eng Target specified as TE tunnel interface
• traceroute mpls ?
ipv4 Target specified as an IPv4 address
multipath LSP Multipath Traceroute
pseudowire Target VC specified as an IPv4 address and VC ID
traffic-eng Target specified as TE tunnel interface
LSP Ping (ping mpls . . . )
• Simple and efficient mechanism to detect data plane failures in MPLS LSPs
• Verify data plane against the control plane
• Sending “echo request” and receiving “echo reply”
• Verify that packets belonging to a FEC exit the LSP on the correct egress LSR
• Modelled after the well known IP ping and traceroute
• Ping verifies connectivity, traceroute verifies path
• LSP Ping/trace leave the LSR with the correct label stack for the LSP to be
tested
Packet Format
Version Number Must Be Zero
Message Type Reply Mode Return Code Return Subcode
Sender’s Handle
Sequence Number
Timestamp Sent (seconds)
Timestamp Sent (microseconds)
Timestamp Received (seconds)
Timestamp Received (microseconds)
TLV …
Packet Format
• Version number: 1
• Message Type
• MPLS Echo Request
• MPLS Echo Reply
• Reply Mode
1 Do not reply
2 Reply via an IPv4/IPv6 UDP packet
3 Reply via an IPv4/IPv6 UDP packet with Router Alert
4 Reply via application level control channel
• Timestamp
• Time-of-day in seconds and microseconds
Reply Modes
• Reply Mode – Do Not Reply
• This mode is useful for a keepalive application running at the remote end
• Such an application would trigger state changes if it does not receive
a LSP ping packet within a predefined time
• An MPLS echo request with “do not reply” may also be used by the receiving
router to log gaps in the sequence numbers and/or maintain delay/jitter statistics
Reply Modes
• Reply Mode – Reply via an IPv4 UDP Packet
• The Reply via UDP packet implies that an IP V4 UDP packet should be sent in
reply to an MPLS echo request
• This will be the most common reply mode for simple LSP pings sent to
periodically poll the integrity of an LSP
• This is the default reply mode
Reply Modes
• Reply Mode – Reply via an IPv4 UDP Packet with Router Alert
• In this mode when the destination router replies it appends a label of “1” to the
packet
• This forces all the intermediate routers, on the way back, to process switch the
reply
• This mode is CPU intensive and should generally be used if the reply fails for
“reply with IPv4 UDP packet”
• This mode is useful when we have inconsistency between IP and MPLS
Return Codes
Value Meaning
0 The Error Code Is Contained in the Error Code TLV
1 Malformed Echo Request Received
2 One Or More of the TLVs Was Not Understood
3 Replying Router Is an Egress for the FEC
4 Replying Router Has No Mapping for the FEC
5 Replying Router Is Not One of the “Downstream Routers”
Replying Router Is one of the “Downstream Routers”, and Its Mapping for this FEC on the
6 Received Interface Is the Given Label
MPLS Echo Request
R1#ping mpls ipv4 192.168.2.2/32 verbose

destination 127.0.0.2 repeat 1 exp 7 pad 0xFFFF
Sending 1, 100-byte MPLS Echos to 10.200.254.4/32,
timeout is 2 seconds, send interval is 0 msec:
Codes: '!' - success, 'Q' - request not transmitted,
'.' - timeout, 'U' - unreachable,
'R' - downstream router but not target
! Reply address 10.1.211.2, return code 3
MPLS Ping (Operational Theory)
• We use the same label stack as used by the LSP and this makes the echo to be
switched inband of LSP
• The IP header destination address field of the echo request is a 127/8 address
• An Echo reply, which may or may not be labelled, has the egress interface IP
address as the source; destination IP address/port are copied from the echo-
request’s source address/port
• Presence of the 127/8 address in the IP header destination address field causes
the packet to be consumed by any routers trying to forward the packet using the
ip header
• In this case P1 would not forward the echo-req to PE1 but rather consumes the
packet and sends a reply to PE2 accordingly
MPLS Ping Packet Capture
Operation
MPLS OAM Caveats
• For LSP ping we generate an MPLS echo request
• The payload includes the LDP/RSVP/L2 Circuit sub-TLV depending on the LSP
we use
• Echo request is appropriately labelled and sent out
• Ping mode: MPLS TTL = 255
• Traceroute mode: TTL = 1, 2 ,3 etc.
• MPLS Echo Request always has FEC Stack TLV
• The LSP ping sender sets the return code to 0.
• The replying router would set it accordingly based on the table shown previously
TTL Field in Labels
• Only the TTL field in the label at the top of the stack counts
• The outgoing TTL value is only a function of the incoming TTL value
• Outgoing TTL is one less than incoming TTL
• If outgoing TTL = 0, packet is not forwarded (not even stripped and forwarded
as an IP packet)
• When an IP packet is first labelled, the TTL field is copied from the IP header to
the MPLS header (after being decremented by 1)
• When the label stack is removed, the outgoing TTL value is copied to the TTL
field in the IP header
• Unless MPLS TTL > IP TTL
Operation
• Receiving LSR checks that label stack of received packet matches with the
received FECs in FEC Stack
• MPLS Echo Reply is sent in response to MPLS Echo Request
– Destination IP address is source IP address of Echo Request
– IP TTL = 255
– Reply Mode: (You do not control if return packet is sent over IP or MPLS)
• IPv4
• IPv4 with Router Alert (IP Option)
– If over MPLS, then Router Alert Label as topmost label is added in the label stack
– Hardware forwarding bypassed; packet is sent to RP process level forwarding
Traceroute in MPLS Network
In Prefix Output Out In Prefix Output Out
Label Interface Label Label Interfac Label
e
- 172.16.2.2/32 Y 19 24008 24008 172.16.2.2/32 Y -
16 172.16.1.1/32 X - - 172.16.1.1/32 X 22 16
Y Y
PE1 X P1 X PE2
192.168.1.1/32 192.168.2.2/32
In Prefix Output Out

Label Interface Label
CE1 CE2
22 192.168.1.1/32 X pop
172.16.1.1/32 19 192.168.2.2/32 Y pop 172.16.2.2/32
Traceroute in MPLS Network
Aggregate Outgoing
Label 19, TTL=1 Label, IP Lookup
done in CEF for VRF
Label 24008 Label 24008,
TTL=255
172.16.2.2 172.16.2.2 172.16.2.2
TTL=2 TTL=1 TTL=255, ICMP
UDP port 35678 UDP port 35678 TTL Exceeded
CE1 PE1 P1 PE2 CE2

172.16.1.1/32 192.168.1.1/32 192.168.2.2/32 172.16.2.2/32
Label 22, TTL=254
172.16.1.1 TTL=252 Label 16, TTL=253 Label 16

ICMP TTL Exceeded
172.16.1.1 TTL=254 172.16.1.1 TTL=254
ICMP TTL Exceeded ICMP TTL Exceeded
MPLS Trace
• The ICMP messages “TTL exceeded” are forwarded along the LSP until the end
of the LSP. So, the router does not lookup the source ip address in the global
routing table to return the ICMP message.
• Reason : P routers do not have knowledge of VPN prefixes : all traceroutes
initiated from within a VPN would fail
• ICMP messages are forwarded with EXP bits = 6
MPLS Trace Hiding
• This command prohibits the copying of the TTL from the IP header to the MPLS
shim header and vice versa (TTL is set to 255)
• It should be configured on the routers that do the label imposement (LSR edge
routers), which is the PE routers.
• Providers like to use it so that the customers see the MPLS network as one hop
when tracerouting
no mpls ip propagate-ttl forwarded
MPLS Trace Hiding
CE1#traceroute 172.16.2.2 source 172.16.1.1 (mpls ip propagate-ttl forwarded)

Tracing the route to 172.16.2.2
1 172.16.11.2 [AS 100] 3 msec 3 msec 3 msec local PE
2 10.1.111.11 [MPLS: Labels 19/24008 Exp 0] 122 msec 25 msec 19 msec P
3 10.1.211.2 [MPLS: Label 24008 Exp 0] 21 msec 16 msec 23 msec remote PE
4 172.16.12.1 [AS 100] 23 msec * 22 msec remote CE
CE1#traceroute 172.16.2.2 source 172.16.1.1 (no mpls ip propagate-ttl forwarded)

VRF info: (vrf in name/id, vrf out name/id)
1 172.16.11.2 [AS 100] 4 msec 3 msec 3 msec local PE
2 10.1.211.2 [MPLS: Label 24008 Exp 0] 25 msec 25 msec 31 msec remote PE
3 172.16.12.1 [AS 100] 24 msec * 28 msec remote CE
MPLS Trace with no mpls ip propagate-ttl on PE routers
Aggregate Outgoing
Label 19, TTL=1 Label
udp port
Label 24008 Label 24008,
35678?
TTL=255
172.16.2.2 172.16.2.2 172.16.2.2 172.16.2.2
TTL=2 TTL=1 TTL=1 TTL=1
UDP port 35678 UDP port 35678 UDP port 35678 UDP port 35678
CE1 PE1 P1 PE2 CE2

172.16.1.1/32 172.16.2.2/32
Label 22, TTL=255
172.16.1.1 TTL=254, 172.16.1.1
ICMP TTL=255, ICMP
Label 16, TTL=254 Label 16 Port Unreachable
Port Unreachable
172.16.1.1 TTL=254, 172.16.1.1 TTL=254,
ICMP ICMP
Port Unreachable Port Unreachable
Multipath MPLS Trace
• MPLS LSP ping / trace is useful tool to validate the health of a label switched
path
• In case of multiple paths, LSP ping may not serve useful to validate all the
available paths
• Multipath MPLS trace allows users to identify all LSP failures
• The multipath LSP trace, sends probe by setting the destination to loopback
address (127.x.x.x), which can help detect failure in LSP by avoiding the packet
to get IP routed.
192.168.2.2/32
Echo Request
1
SRC – 10.1.16.6
DEST – 127.0.0.0 1 R2
192.168.6.6/32 192.168.1.1/32 192.168.4.4/32
R6 R1 R4
2
192.168.3.3/32
Echo Reply
SRC – 10.1.16.1
2 DEST – 10.1.16.6
DS Mapping – 127.0.0.1
24002 - 10.1.13.3
DS Mapping – 127.0.0.0 R3
30002 - 10.1.12.2
192.168.2.2/32
Echo Request
3
SRC – 10.1.16.6
DEST – 127.0.0.0 4
R2
192.168.6.6/32 3 192.168.1.1/32 192.168.4.4/32
R6 R1 R4
Echo Reply
SRC – 10.1.12.2
192.168.3.3/32
4 DEST – 10.1.16.6
pop - 10.1.24.4
R3
Multipath MPLS Trace 192.168.2.2/32
Echo Request
5 R2
SRC – 10.1.16.6
DEST – 127.0.0.1
192.168.6.6/32 192.168.1.1/32 192.168.4.4/32
R6 5 R1 R4
192.168.3.3/32
Echo Reply 6
SRC – 10.1.13.3
6 DEST – 10.1.16.6
R3
pop - 10.1.34.4
PE1#traceroute mpls multipath ipv4 192.168.4.4/32
Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
<snip>
LL!
Path 0 found,
output interface Gi0/1 nexthop 10.1.16.1
source 10.1.16.6 destination 127.0.0.1
0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0
L 1 10.1.16.1 10.1.12.2 MRU 1500 [Labels: 30002 Exp: 0] ret code 8 multipaths 2
L 2 10.1.12.2 10.1.24.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1
! 3 10.1.24.4, ret code 3 multipaths 0
L!
Path 1 found,
output interface Gi0/1 nexthop 10.1.16.1
source 10.1.16.6 destination 127.0.0.0
0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0
L 1 10.1.16.1 10.1.13.3 MRU 1500 [Labels: 24002 Exp: 0] ret code 8 multipaths 2
L 2 10.1.13.3 10.1.34.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1
! 3 10.1.34.4, ret code 3 multipaths 0
Paths (found/broken/unexplored) (2/0/0)
Echo Request (sent/fail) (5/0)
Echo Reply (received/timeout) (5/0)
Total Time Elapsed 192 ms
Demo - Multipath MPLS Trace
MPLS Forwarding Plane
 With MPLS, the idea is to de-couple the forwarding from the IP header
 The forwarding decision is based on the MPLS header, not the IP header
 The above is true once the packet is inside the MPLS network
 Forwarding is still based on the IP header at the edge where the packet first
enters the MPLS network
 CEF must be configured on all the routers in a MPLS network.
 CEF takes care of the crucial “recursion” and “resolution” operations
What happens when CEF disabled?
PE1#show mpls forwarding-table

16 No Label 172.16.1.1/32 0 drop
17 No Label 192.168.12.12/32 0 drop
20 No Label 192.168.2.2/32 0 drop
21 No Label 10.1.212.0/24 0 drop
22 No Label 10.1.211.0/24 0 drop
23 No Label 192.168.11.11/32 0 drop
24 No Label 172.16.11.0/24 0 drop
25 No Label 172.16.14.0/24 0 drop
MPLS Forwarding Plane – Outgoing Labels
PE1#show mpls forwarding-table 192.168.2.2

Local Outgoing Prefix Bytes Label Outgoing NextHop
20 19 192.168.2.2/32 0 Gi0/1 10.1.111.11
PE1#
• Outgoing label also conveys what treatment the packet is going to

get. It could also be:
I. Pop - Pops the topmost label
II. Untagged - Untag the incoming MPLS packet
III. Aggregate - Untag and then do a FIB lookup
 Label values 0-15 are reserved.
MPLS Forwarding Plane: Outgoing Labels
PE1#sh mpls forwarding-table
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
16 2002 10.13.1.22/32 0 Et0/0 10.13.1.5
2002 10.13.1.22/32 0 Et1/0 10.13.1.9
18 Pop tag 10.13.1.101/32 0 Et1/0 10.13.1.9
Pop tag 10.13.1.101/32 0 Et0/0 10.13.1.5
19 Pop tag 10.13.2.4/30 0 Et1/0 10.13.1.9
Pop tag 10.13.2.4/30 0 Et0/0 10.13.1.5
20 Untagged 5.5.5.5/32[V] 0 Se2/0 point2point
21 Pop tag 10.13.21.4/30 0 Et1/0 10.13.1.9
Pop tag 10.13.21.4/30 0 Et0/0 10.13.1.5
24 Aggregate 200.1.61.4/30[V] 0
26 Untagged 30.30.30.1/32[V] 0 Se2/0 point2point
PE1#
MPLS Forwarding Plane: Outgoing Labels
 Untagged
• Convert the incoming MPLS packet to an IP packet and forward it.
 Pop
• Pop the top label from the label stack present in an incoming MPLS packet
and forward it as an MPLS packet.
• If there was only one label in the stack, then forward it as an IP packet. SAME
as imp-null label.
 Aggregate
• Convert the incoming MPLS packet to an IP packet and then do a FIB lookup
for it to find out the outgoing interface.
MPLS Forwarding Plane - Lookup
 Three cases in the MPLS forwarding:
1) Label Imposition - IP to MPLS conversion
2) Label swapping - MPLS to MPLS
3) Label disposition - MPLS to IP conversion
 So, depending upon the case, we need to check:

1) FIB - For IP packets that get forwarded as MPLS
2) LFIB - For MPLS packets that get forwarded as MPLS
3) LFIB - For MPLS packets that get forwarded as IP
MPLS Forwarding Plane: Loadsharing
 MPLS Loadsharing (due to multiple paths to a prefix) is no different from that of
IP
 Hashing-algorithm is still the typical ‘FIB based’ i.e per-dest loadsharing by
default **
 So the “show commands” are still relevant
• “Show ip cef exact-route <source> <dest>” etc.
 But the <dest> must be known in the FIB table, otherwise the command won’t
work.
• Won’t work on P routers for the VPN prefixes.
MPLS Forwarding Plane: MTU Setting
• “mpls mtu <bytes>” can be applied to an interface to change the
MPLS MTU size on the interface
• MPLS MTU size is checked by the router
• while converting an IP packet into a labeled packet or transmitting a labelled
packet
• Label imposition(s) increases the packet size by 4 bytes/label, hence the
outgoing packet size may exceed ‘interface MTU’ size, hence the need
to tune MTU
• ‘mpls mtu <bytes>” command has no effect on “interface or IP MTU” size.
• By default, MPLS MTU = interface MTU
• MPLS MTU setting doesn’t affect MTU handling for IP-to-IP packet switching
MPLS Forwarding Plane: MTU Setting
• If the label imposition makes the packet bigger than the

MPLS MTU size of an outgoing interface, then:
- If the DF bit set, then discard the packet and send ICMP reply
back (with code=4)
- If the DF bit is not set, then fragment the IP packet (say, into 2
packets), and then impose the same label(s) on both the packets,
and then transmit MPLS packets
MPLS Forwarding Plane: Show Commands
 “show mpls forwarding”
• Shows all LFIB entries (vpn, non-vpn, TE etc.)
 “show mpls forwarding <prefix>”
 LFIB lookup based on a prefix
 “show mpls forwaring label <label>”

 LFIB lookup based on an incoming label
 “show mpls forwarding <prefix> detail”

 Shows detailed info such as L2 encap etc
MPLS Forwarding Plane: Show Commands
R2#show mpls forwarding 10.13.1.11 detail
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
45 51 10.13.1.11/32 0 Fa1/1/1 10.13.7.33
MAC/Encaps=14/18, MRU=1500, Tag Stack{51}
0003FD1C828100044E7548298847 00033000
No output feature configured
Per-packet load-sharing
R2#
14/18 means that the L2 header is of 14 bytes, but

L2+label header is 18 bytes (one label is 4 bytes)
Introduction to
MPLS Tunneling:
BGP Free Core
The Need for Intermediary Forwarding
• Providing transit connectivity to other Autonomous Systems requires that all
routers between BGP Edge (R1 & XR4) know how to forward packets to the
appropriate device.
• Typically all devices participate with BGP
• The exception is redistributing routes on the edge (R1 & XR4) to the IGP
• Doesn’t scale well on the Internet (600K+ Routes)
?
CE6 R1 R2 XR3 XR4 CE7
AS 600 AS 100
AS 700
The Use of Tunnels
An alternate solution could use tunnels on intermediary routers to get connectivity
between Internet Edge routers.
• The problems with must tunnels:
• Adds significant packet overhead
• Requires configuration of encapsulating and encapsulating interfaces
• Doesn’t scale well.

AS 600 AS 100
AS 700
MPLS: An Alternative View to Tunnels
MPLS can provide tunnel functionality.
• Doesn’t require configure encapsulating/decapsulating interfaces.
• Scalable as additional Edge devices can be added without configuring other
devices in it.
• Packets are forwarded by the endpoint FEC

AS 600 AS 100
AS 700
BGP Free Core with MPLS
Configuration Requirements
A FEC must exist for the next-hop in the BGP table
R1#show ip bgp | b Network
Network Next Hop Metric LocPrf Weight Path
* 100.64.6.0/24 172.16.16.6 0 0 600 ?
* i 100.64.7.0/24 192.168.4.4 0 100 0 700 ?
* 172.16.16.0/24 172.16.16.6 0 0 600 ?
* i 172.16.47.0/24 192.168.4.4 0 100 0 700 ?

AS 600 AS 100
AS 700
BGP Free Core with MPLS
A FEC must exist for the next-hop in the BGP table
R1#show mpls forwarding-table
100 Pop Label 192.168.2.2/32 0 Gi0/1 10.12.1.2
102 Pop Label 10.23.1.0/24 0 Gi0/1 10.12.1.2
104 202 192.168.3.3/32 0 Gi0/1 10.12.1.2
105 203 10.34.1.0/24 0 Gi0/1 10.12.1.2
106 204 192.168.4.4/32 0 Gi0/1 10.12.1.2

AS 600 AS 100
AS 700
BGP Free Core Configuration
Configuration NX-OS
mpls ldp configuration
IOS / IOS XE router-id Lo0
interface Ethernet2/1
Interface GigabitEthernet 0/1 mpls ip
mpls ip ip address 10.25.1.5/24
router ospf 1 ip router ospf NXOS area 0.0.0.0
network 10.0.0.0 0.255.255.255 area 0 router bgp 100
network 192.168.0.0 0.0.255.255 area 0 address-family ipv4 unicast
router bgp 100 neighbor 172.16.16.6 remote-as 600
neighbor 172.16.16.6 remote-as 600
address-family ipv4 unicast
neighbor 192.168.4.4
neighbor 192.168.4.4 next-hop-self
remote-as 100
update-source loopback0
next-hop-self
BGP Free Core Configuration
Configuration IOS XR
router bgp 100

address-family ipv4 unicast router ospf 1
neighbor 172.16.47.7 area 0
remote-as 700 interface Loopback0
address-family ipv4 unicast interface GigabitEthernet0/0/0/0
route-policy PASSALL in !
route-policy PASSALL out mpls ldp
neighbor 192.168.1.1 router-id 192.168.4.4
remote-as 100 interface GigabitEthernet0/0/0/0
update-source Loopback0
next-hop-self
Verifying a BGP Free Core
CE6#trace 100.64.7.7
1 172.16.16.1 6 msec 3 msec 3 msec
2 10.12.1.2 [MPLS: Label 204 Exp 0] 10 msec 9 msec 8 msec
4 10.34.1.4 8 msec 10 msec 9 msec
5 172.16.47.7 [AS 700] 10 msec * 10 msec

AS 600 AS 100
AS 700
Confirming the labels
R1#show mpls forwarding-table | i 204|ing|witch
106 204 192.168.4.4/32 0 Gi0/1 10.12.1.2
R2#show mpls forwarding-table | i 204|ing|witch

204 24005 192.168.4.4/32 16936 Gi0/2 10.23.1.3

AS 600 AS 100
AS 700
MPLS Labels Case
Study
MPLS Labels Case Study
Case Study – MPLS Traffic Not Forwarded
• Customer reported traffic forwarding issue to the VRF’s attached to a newly
configured PE2 router
• The PE1 router has the VPN label which is being shared with the remote PE2
router
MP-IBGP – VPNv4
LDP + IGP
172.16.11.0/24 10.1.14.0/24 10.1.24.0/24 172.16.22.0/24
CE1 PE1 P1 PE2 CE2

Lo0=172.16.1.1/32 Lo0=192.168.1.1/32 Lo0=192.168.4.4/32 Lo0=192.168.2.2/32 Lo0=172.16.2.2/32
• On PE1, the CEF shows the correct forwarding output.
Troubleshooting Approach
• The first step in MPLS deployment is to verify if the LSP is complete or not.
• Use ping mpls ipv4 <dest-pe-loopback> <subnet_mask> to verify LSP Path
• Use traceroute mpls ipv4 <dest-pe-loopback> <subnet_mask> to verify
what is the path and see the point where MPLS packet is getting dropped
• The other option is to check the labeling and LFIB information hop by hop or at
least on the node where the MPLS trace is dropped.
Findings
• The MPLS PING failed
• MPLS Trace dropped on P-1 router
• Show mpls forwarding <PE2-loopback> output shows no label as outgoing label
P-1# show mpls forwarding 192.168.3.3
17 No Label 192.168.3.3/32 476193 Et0/0 23.23.23.2
• Verified that LDP was enabled between the two routers but there was no
bindings
Resolution
P-1(config)#no mpls ldp advertise-labels
P-1(config)#mpls ldp advertise-labels for LOOPBACK_ACL
• The P-1 router had an ACL to limit the allocation of labels for certain prefixes
• Sometimes, there are too many prefixes in the core due to which the labels get
exhausted
• To prevent such situations, LDP is configured to allocate labels for certain prefixes but
not all.
• PE2 loopback address was added in the ACL which fixed the problem
Label Filtering
• LDP setup LSPs to carry VPN traffic from PE to PE.
• VPN traffic is always destined to PE’s loopback address.
• Only label bindings for those prefixes will be useful in such scenario.
• IOS / IOS-XE
• Outbound Label Filtering
• IOS-XR
• Local Label Allocation Filtering
• NX-OS
• Inbound Label Filtering
• Local Label Allocation Filtering
Label Filtering
• Local label allocation filtering
• LDP allocates local labels for IGP /32 prefixes (by default)
• Can be configured to allocate labels for all or certain prefixes
• LDP accepts and keeps remote labels even no local labels exist
• Outbound label filtering

• Advertise local labels for some prefixes to some peers
• Multiple instances of configuration, label advertised as long as one instance allowed
• Inbound label filtering

• Accept remote labels for certain prefixes from a peer
• Per neighbor configuration
• When configuration is changed, may need to tear down and re-establish session to
receive labels denied prior to the change*
Label Filtering Configuration – NX-OS
• Inbound label filtering
NX-OS(config-ldp)# neighbor 10.0.0.22 labels accept ?
WORD Name of prefix list
• Outbound label filtering

NX-OS(config-ldp)# no advertise-labels
NX-OS(config-ldp)# advertise-labels for pfx1 to peer1?
WORD Name of prefix list
NX-OS(config-ldp)# advertise-labels for pfx1 to peer1
NX-OS(config-ldp)# advertise-labels for pfx2 to peer2
• Local label allocation filtering

NX-OS(config-ldp)# label allocate global ?
all-routes Allocate local labels for all routes
host-routes Allocate local labels for host routes only
prefix-list Specify a prefix-list for local label filtering
Troubleshooting
MPLS L3 VPNs
Troubleshooting MPLS L3 VPNs
Nodes and their Roles
• PE – Provider Edge router, connects to P and CE routers
• Maintains separate routing table per VRF
• Uses MP-BGP to exchange VRF routing information (RD + RT)
• Performs LFIB and FIB lookups, label imposition and disposition
• Exchanges IGP and LDP labels with the core
• P – Provider core router, connects to P and PE routers

• Does not need to run BGP with the PE’s
• Performs LFIB MPLS forwarding, label swap or PHP
• Exchanges IGP and LDP labels with other P routers and the PE’s
• CE – Customer edge router, connects to the CE network and the PE

• Forwards only IP packets – no awareness of the MPLS network is needed
• Routes between the CE internal network and the PE router
IP Addressing Concerns
Customer A Customer A
Site 1 Site 2
172.16.1.0/24 172.16.3.0/24
CE1 CE3
PE1
172.16.2.0/24 172.16.4.0/24
CE2 CE4
Customer B Customer B
Site 1 Site 2
Isolation Through the Use of VRFs
Customer A Customer A
Site 1 Site 2
172.16.1.0/24 172.16.3.0/24
CE1 VRF VPN01 CE3
PE1
172.16.2.0/24 VRF VPN02 172.16.4.0/24

CE2 CE4
Customer B Customer B
Site 1 Site 2
VRF Overview
• VRF = VPN Routing Forwarding instance
• Isolated routing table, kind of like a VM
• Easiest to think of each VRF like a different physical box
• Interfaces are assigned to a VRF
• Everything not in a VRF is in “the global” (routing table)

• In MPLS-VPN each customer has a VRF
• VRFs for customers, global for the Provider
vrf global
Customer ISP
Network PE mpls
VRF Overview
 Because each RIB is isolated, overlapping address are allowed
 VRF-aware features add “vrf <name>” to commands
 Commands without VRF keyword reference the global RIB
e0 e1
ip vrf forwarding red ip vrf forwarding red
ip address 1.1.1.1/24 ip address 2.2.2.2/24
e2
ip address 1.1.1.1/24
VRF Overview
e0 e1
ip vrf forwarding red ip vrf forwarding red
ip address 1.1.1.1/24 ip address 2.2.2.2/24
e2
ip address 1.1.1.1/24
PE1#show ip route 2.2.2.0

% Network not in table
PE1#show ip route vrf red 2.2.2.0

Routing Table: red
Routing entry for 2.2.2.0/24
Known via "connected“
* directly connected, via Ethernet1
L3VPN by Parts
The Edge:
• Any routing protocol between the PE and CE
The Core:
• BGP between PEs
• LDP
• IGP (mainly to get between PEs)
PE-CE Protocol PE-CE Protocol

MP-EBGP
CE PE CE
P2 PE
LDP + IGP TECMPL-3201 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
Visualizing Data Flow
MP-EBGP
CE PE CE
P2 PE
100 20
100.64.6.6
1. Packet is received on local PE

2. Remote VPN Label is assigned
3. Remote PE Label is assigned
Visualizing Data Flow
MP-EBGP
CE PE CE
P2 PE
100 20 100.64.6.6 100 20 100.64.6.6 20 100.64.6.6
1. The P router next to destination PE router POPs the

outer label
2. The packet is forwarded onto the Destination PE router
3. The VPN Label is examined and POP’d
4. The packet is forwared out to the VRF
MP-BGP (Multi Protocol BGP)
• MP-BGP extends BGP to carry more than just IPv4 prefixes
• Introduced “address family” style configuration
• Allows for IPv6, MPLS and other information in same BGP session
• When session is established the capabilities are negotiated
• No new rules, still requires full mesh or RRs

• RRs need to support additional capabilities
• For MPLS only PEs need to speak BGP or know CE routes
• L3VPN Relies on Extended Communities
• Extended Communities are arbitrary TLVs attached to BGP prefixes
MP-BGP: Address-Families
• Address-family (AFI) “vpnv4”, “ipv4 unicast vrf” introduced
• vpnv4 AFI for PE to PE (label information)
• ipv4 unicast vrf for PE to CE

• Neighbor must be “activated” for each AFI supported
router bgp 100
!
address-family vpnv4
neighbor 192.168.3.3 activate Remote PE
neighbor 192.168.3.3 send-community
extended
!
address-family ipv4 unicast vrf red
neighbor 192.168.4.4 remote-as 400 Local CE
neighbor 192.168.4.4 activate
MP-BGP: Advertising CE Routes
 BGP maintains a table for each AFI (vpnv4, ipv4, vrf…)
 CE routes are placed into the vpnv4 BGP table
• BGP routes in a vrf AFI are automatically turned into vpnv4 routes
• If BGP is not PE-CE protocol routes must be redistributed into ipv4 vrf AFI
 All vpnv4 routes get an assigned label

 vpnv4 routes are exchanged between vpnv4 peers (PEs)
RTs and RDs: Creating the VRF
• VRFs have 3 parts:

1. VRF name (case sensitive)
2. Route Distinguisher (RD)
3. Route Target(s) (RT)
ip vrf red
• RD and RT are for MPLS; RD must always rd 100:100
route-target import 200:200
be defined route-target export 201:201
• RD must be unique to the VRFs on the
local PE
• If there is no MPLS, called “VRF-lite”
Understanding the RT
• Route Target
• RT is a BGP extended community (extra
information on the update)
ip vrf red
• “route-target export” adds the rd 1:1
community to the outbound update route-target import 100:100
• “route-target import” defines which route-target import 200:200
routes to bring into the VRF route-target export 201:201
• Multiple imports and exports allowed route-target export 44:313
Understanding RDs
• Route Distinguisher
• There is only one VPNv4 table
• How are routes distinguished from another?
• Prepending the RD to the route to creates a VPNv4 route
• Only used to make routes unique VPNv4 prefixes
IPv4 Route: 192.168.1.0/24

RD: 100:100
VPNv4 Route: 100:100:192.168.10/24
RT in Action
ip vrf red
rd 1:1
route-target import 100:100
route-target export 201:201
66:66:2.2.2.0/24
VRF Red RIB
RT: 100:100
BGP 2.2.2.0/24
55:55:1.1.1.0/24 3.3.3.0/24
Update
RT: 201:201
44:44:3.3.3.0/24
RT: 100:100
vrf definition VPN01
rd 200:1
route-target export 200:1
Locally Assigned Label
RD
Prefix
Route Target
Example Topology
AS500 AS200 AS500
MP-EBGP
CE5 172.16.15.0/24 PE1 10.23.1.0/24 172.32.36.0/24 CE6

10.12.1.0/24 P2 PE3
Lo0: 192.168.1.1 Lo0: 192.168.2.2 Lo0: 192.168.3.3
Lo0: 100.64.5.5 Lo0: 100.64.6.6
Our Example scenario, CE5 (100.64.5.5) wants to ping CE 6 (100.64.6.6)
Troubleshooting CE to PE
AS500 AS200 AS500
MP-EBGP
CE5 172.16.15.0/24 PE1 10.23.1.0/24 172.32.36.0/24 CE6

10.12.1.0/24 P2 PE3
Route Exchange Route Exchange
CE  PE
• Check Local PE for receipt of local CE Routes
• Check Remote PE for receipt of remote CE routes
• Can the CE ping the PE?
Troubleshooting PE to CE
AS500 AS200 AS500
MP-EBGP
CE5 172.16.15.0/24 PE1 10.23.1.0/24 172.32.36.0/24 CE6

10.12.1.0/24 P2 PE3
Route Exchange Route Exchange
CE  PE
• Check Local PE for receipt of remote CE Routes
• Check Remote CE for receipt of local CE routes
• Can the PE ping the CE?
Checking the PE VRF Routing Table
R1#show ip route vrf VPN01 | b Gateway
Gateway of last resort is not set
100.0.0.0/32 is subnetted, 2 subnets

B 100.64.5.5 [20/0] via 172.16.15.5, 00:08:03
B 100.64.6.6 [200/0] via 192.168.3.3, 00:01:19
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.16.15.0/24 is directly connected, GigabitEthernet0/1
RP/0/0/CPU0:XR3#show route vrf VPN01 | b Gateway

B 100.64.5.5/32 [200/0] via 192.168.1.1 (nexthop in vrf default), 00:05:14

B 100.64.6.6/32 [20/0] via 172.32.36.6, 00:04:03
C 172.32.36.0/24 is directly connected, 00:07:52, GigabitEthernet0/0/0/1
Troubleshooting Control Plane
AS500 AS200 AS500
MP-EBGP
CE5 172.16.15.0/24 PE1 10.23.1.0/24 172.32.36.0/24 CE6

10.12.1.0/24 P2 PE3
VPNv4 Route Exchange
PE  PE (or PE  RR)
• Check LSP Path between PE routers
• Check for route exchange?
• If routes are missing, did you check the export RT vs.
Import RT
Verify PE to PE LSP
R1#ping mpls ipv4 192.168.3.3 255.255.255.255
.....
Success rate is 0 percent (0/5)
RP/0/0/CPU0:XR3(config)#mpls oam
RP/0/0/CPU0:XR3(config-oam)#commit
R1#ping mpls ipv4 192.168.2.2 255.255.255.255

!!!!!
Checking the PE VPNv4 Routing Table
R1#show bgp vpnv4 unicast all | b Network
Route Distinguisher: 200:1 (default for vrf VPN01)
*> 100.64.5.5/32 172.16.15.5 0 0 500 i
*>i 100.64.6.6/32 192.168.3.3 0 100 0 500 i
RP/0/0/CPU0:XR3#show bgp vpnv4 unicast | b Network

*>i100.64.5.5/32 192.168.1.1 0 100 0 500 i
*> 100.64.6.6/32 172.32.36.6 0 0 500 i
Verifying VPN Label
R1#show bgp vpnv4 unicast vrf VPN01 100.64.5.5
BGP routing table entry for 200:1:100.64.5.5/32, version 2
Paths: (1 available, best #1, table VPN01)
Advertised to update-groups:
2
Refresh Epoch 1
500
172.16.15.5 (via vrf VPN01) from 172.16.15.5 (100.64.5.5)
Origin IGP, metric 0, localpref 100, valid, external, best
Extended Community: RT:200:1
mpls labels in/out 103/nolabel
rx pathid: 0, tx pathid: 0x0
Local VPN Label
Verifying Remote VPN Label
RP/0/0/CPU0:XR3#show bgp vpnv4 unicast vrf VPN01 100.64.5.5
BGP routing table entry for 100.64.5.5/32, Route Distinguisher: 200:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 4 4
Paths: (1 available, best #1)
Not advertised to any peer
500
192.168.1.1 (metric 3) from 192.168.1.1 (192.168.1.1)
Received Label 103 RemoteVPN Label
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best,
import-candidate, imported
Received Path ID 0, Local Path ID 1, version 4
Extended community: RT:200:1
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route
Distinguisher: 200:1
Verifying Labels (The Easy Way)
R1#show bgp vpnv4 unicast all labels
Network Next Hop In label/Out label
Route Distinguisher: 200:1 (VPN01)
100.64.5.5/32 172.16.15.5 103/nolabel
100.64.6.6/32 192.168.3.3 nolabel/33003
CE6 Route
CE5
RP/0/0/CPU0:XR3#show bgp vpnv4 unicast labels | b Network CE6 Route
Network Next Hop Rcvd Label Local Label
*>i100.64.5.5/32 192.168.1.1 103 nolabel
*> 100.64.6.6/32 172.32.36.6 nolabel 33003
In Label represents local label and Rcvd/Out Label represents remote label
Viewing the local VPN labels in the LFIB
R1#show mpls forwarding-table
100 No Label 100.64.5.5/32[V] 1129814 Gi0/1 172.16.15.5
101 Pop Label 192.168.2.2/32 0 Gi0/2 10.12.1.2
102 Pop Label 10.23.1.0/24 0 Gi0/2 10.12.1.2
103 24001 192.168.3.3/32 0 Gi0/2 10.12.1.2
RP/0/0/CPU0:XR3#show mpls forwarding

Local Outgoing Prefix OutgoingLocal VPN
NextLabel Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ----------
33000 24000 192.168.1.1/32 Gi0/0/0/0 10.23.1.2 1164391
33001 Pop 192.168.2.2/32 Gi0/0/0/0 10.23.1.2 46706
33002 Pop 10.12.1.0/24 Gi0/0/0/0 10.23.1.2 0
33003 Unlabelled 100.64.6.6/32[V] Gi0/0/0/1 172.32.36.6 1101658
33004 Aggregate VPN01: Per-VRF Aggr[V] \
VPN01 © 2017 Cisco and/or its affiliates. All rights52000
reserved. Cisco Public
CE Routing Tables
AS500 AS200 AS500
MP-EBGP
CE5#show ip route | b Gateway

CE5 Gateway of last PE1
resort is not set CE6
172.16.15.0/24 10.12.1.0/24 10.23.1.0/24 172.32.36.0/24
P2 PE3
C 100.64.5.5 is directly connected, Loopback0
L Connectivity Between CE5 and CE6
172.16.15.5/32 is directly connected, GigabitEthernet0/1
--- Does Not Exist!

CE5#show bgp sum
BGP router identifier 100.64.5.5, local AS number 500
Neighbor V Why?TblVer
AS MsgRcvd MsgSent InQ OutQ Up/Down State/PfxRcd
172.16.15.1 4 200 158 157 2 0 0 02:18:27 0
CE Routing Tables
AS500 AS200 AS500
MP-EBGP
CE6#show ip route | b Gateway

Gateway
CE5 of last resort is not set
172.16.15.0/24 PE1 10.12.1.0/24 10.23.1.0/24 172.32.36.0/24 CE6
P2 PE3
C 100.64.6.6 is directly connected, Loopback0
L 172.32.36.6/32 is directly connected, GigabitEthernet0/1
---
CE6#show bgp sum
BGP router identifier 100.64.6.6, local AS number 500
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

172.32.36.3 4 200 6 8 2 0 0 00:03:51 0
Deployment Tips and Tricks
• Advertise the PE – CE Network Link on the PE Router.

• Allows for PE to PE Connectivity tests without involving CEs
• Allows for Local PE to Remote CE Connectivity Test
• Allows for Local CE to Remote PE Connectivity Test
R1 XR3
router bgp 200 router bgp 200
address-family ipv4 vrf VPN01 vrf VPN01
redistribute connected address-family ipv4 unicast
redistribute connected
CE Routing Tables
AS500 AS200 AS500
CE5#show ip route | b Gateway MP-EBGP
CE5 100.0.0.0/32
PE1 is 10.12.1.0/24
subnetted, 1 subnets CE6
172.16.15.0/24 10.23.1.0/24 172.32.36.0/24
C 100.64.5.5 is directly connected,
P2 Loopback0 PE3
L 172.16.15.5/32 is directly connected, GigabitEthernet0/1
B 172.32.36.0 [20/0] via 172.16.15.1, 00:01:04
---
Look at the topology, and think about
what the problem can be?
CE5#ping 172.32.36.3
!!!!!
Fixing the BGP AS_Path Problem
• AS_Path is a loop prevention mechanism

• PE routers can use a special feature called AS-Override.
• Any prefixes with the same AS that the is used by the CE is changed to the AS of
the PE
R1 XR3
router bgp 200 router bgp 200
address-family ipv4 vrf VPN01 neighbor 172.32.36.6
redistribute connected remote-as 500
neighbor 172.16.15.5 remote-as 500 address-family ipv4 unicast
neighbor 172.16.15.5 activate route-policy PASSALL in
neighbor 172.16.15.5 as-override route-policy PASSALL out
as-override
CE Routing Tables
AS500 AS200 AS500
MP-EBGP
CE5#show bgp ipv4 unicast | b Network

CE5 Network PE1 10.12.1.0/24
172.16.15.0/24 Next HopP2 Metric LocPrf Weight
10.23.1.0/24 Path
172.32.36.0/24 CE6
PE3
*> 100.64.5.5/32 0.0.0.0 0 32768 i
*> 100.64.6.6/32 172.16.15.1 0 200 200 i
r> 172.16.15.0/24 172.16.15.1 0 0 200 ?
*> 172.32.36.0/24 172.16.15.1 0 200 ?
---
CE5#ping 100.64.6.6 so lo0
!!!!!
CE Routing Tables
AS500 AS200 AS500
MP-EBGP
CE5 172.16.15.0/24 PE1 10.23.1.0/24 172.32.36.0/24 CE6

10.12.1.0/24 P2 PE3
CE5#trace 100.64.6.6 so lo0

1 172.16.15.1 [AS 200] 3 msec 3 msec 3 msec
2 10.12.1.2 [MPLS: Labels 24001/33003 Exp 0] 10 msec 9 msec 10 msec
4 172.32.36.6 [AS 200] 9 msec * 12 msec
Route Reflectors
RR
AS500 AS200 AS500
CE5 PE1 CE6

P2 PE3
CE5#trace 100.64.6.6 so lo0

1 172.16.15.1 [AS 200] 3 msec 3 msec 3 msec
4 172.32.36.6 [AS 200] 9 msec * 12 msec
Route Reflectors
RR
AS500 AS500
CE5 PE1 CE6

P2 PE3
Route Reflectors are used to solve problems with scale. When route-reflectors
are used, they need to be checked as part of the path of the VPNv4 router
advertisement.
• A RR disables RT Filtering
• VRFs do not exist on RR; so you can not issue commands specific to a VRF
• Commands are based on RT and RD
Viewing MPLS L3 VPNs by RT
IOS-RR#config t
IOS-RR(config)#ip extcommunity-list 1 permit rt 200:1
IOS-RR(config)#exit
IOS-RR#show bgp vpnv4 unicast all extcommunity-list 1
*> 100.64.5.5/32 172.16.15.5 0 0 500 I
*>i 100.64.6.6/32 192.168.3.3 0 100 0 500 i
RP/0/0/CPU0:XR-RR#conf t
RP/0/0/CPU0:XR-RR(config)#route-policy RT
RP/0/0/CPU0:XR-RR(config-rpl)#if extcommunity rt matches-any ( 1:10) then
RP/0/0/CPU0:XR-RR(config-rpl)# pass endif
RP/0/0/CPU0:XR-RR(config-rpl)#end-policy
RP/0/0/CPU0:XR-RR(config)#commit
RP/0/0/CPU0:XR-RR(config)#end
RP/0/0/CPU0:XR-RR#show bgp vpnv4 unicast route-policy RT
*> 100.64.5.5/32 172.16.15.5 0 0 500 I
*>i 100.64.6.6/32 192.168.3.3 0 100 © 2017 Cisco and/or
0 its500
affiliates. Alli
rights reserved. Cisco Public
Live Troubleshooting Demo
Route Reflectors
RR
AS500 AS500
CE5 PE1 CE6

P2 PE3
• CE5 cannot ping CE6

• IP Addressing is exactly the same as before.
• PE1 and PE3 now connect to a Route Reflector (192.168.10.10)
What do we do first and why?

Inter-AS
MPLS VPNs
Inter-AS MPLS VPNs
Flavors
• Previous section – VPNs within Single-AS boundary
• Inter-AS MPLS VPN – VPNs spanning across multiple AS boundaries
• Types:
• Option A – Back to Back VRF
• Option B – Inter-Provider VPNs using ASBR-to-ASBR approach
1. Next-Hop-Self Method
2. Redistribute Connected Method
3. Multi-hop EBGP between ASBRs
• Option C – MP-EBGP between RR and EBGP between ASBR
Inter-AS VPN Topology RR
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
PE
R5 XR6 PE
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN Option A: Back to Back VRFs RR
RR
R3 ASBR XR4
ASBR VRF VPN01 VRF VPN02 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1 XR2
VPN02 VPN02
• Terminate VRFs on ASBRs

PE
R5 XR6 PE
• Advertise Peering Link to VRF/BGP
• Exchange routes across peering link
• Simple
CE
R7 • Doesn’t Scale Well R8 CE
AS 700 AS 700
100.64.7.7 100.64.8.8
RR
R3 router bgp 100 ASBR XR4

ASBR 192.168.2.2
192.168.1.1 bgp log-neighbor-changes
no bgp default ipv4-unicast
AS100 neighbor 192.168.3.3 remote-as 100
ASBR AS200
VPN02 R1 neighbor 192.168.3.3 update-source
XR2Loopback0
VPN02
!
neighbor 192.168.3.3 send-community extended
PE exit-address-family
R5 XR6 PE
!
address-family ipv4 vrf VPN01
CE exit-address-family
R7 R8 CE
AS 700 AS 700
100.64.7.7 100.64.8.8
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

router R1
VPN02 bgp 200 XR2 VPN02
vrf VPN02
rd 200:1
PE
R5 ! XR6 PE
remote-as 100
route-policy PASSALL in
route-policy PASSALL out
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

CE7#trace
VPN02 100.64.8.8
R1 XR2 VPN02
1 172.16.57.5 3 msec 3 msec 3 msec
PE 2 172.16.12.1 [AS 100] [MPLS: Label 204 Exp 0] 4 msec 4 msec 5 msec
R5 XR6 PE
3 172.16.12.2 [AS 100] 5 msec 5 msec 4 msec
5 172.32.68.8 [AS 200] 11 msec * 11 msec
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN Option B1: ASBR to ASBR w/ Next-Hop-Self RR
RR
Next-Hop-Self Next-Hop-Self
R3 ASBR XR4
ASBR MP-EBGP 192.168.2.2
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
No LDP or IGP required on the link between the two ASBRs.

CE Configure no bgp default route-target filter on ASBRs CE
R7 R8
AS 700 ASBRs advertise to RRs with Next-Hop-Self AS 700
100.64.7.7 100.64.8.8
Inter-AS MPLS VPNs
Problems with Route Filtering
• Enabled by default by all non-RRs.
R1#debug bgp vpnv4 unicast updates
BGP updates debugging is on for address family: VPNv4 Unicast
R1#clear bgp vpnv4 unicast * sof
*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1) rcvd
Refresh Start-of-RIB
*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1)

refresh_epoch is 3
*Jun 20 19:35:50.711: BGP(4): 192.168.3.3 rcvd UPDATE w/ attr: nexthop 192.168.5.5, origin ?,
localpref 100, metric 0, originator 192.168.5.5, clusterlist 192.168.3.3, merged path 700, AS_PATH
, extended community RT:100:1
*Jun 20 19:35:50.714: BGP(4): 192.168.3.3 rcvd 100:1:100.64.7.0/24, label 5003 - DENIED due to:
extended community not supported;
RR interface GigabitEthernet0/2
ip address 172.16.12.1 255.255.255.0
R3 mpls bgp forwarding ASBR XR4
ASBR 192.168.2.2
192.168.1.1
! MP-EBGP
router bgp 100
AS100 bgp log-neighbor-changes ASBR AS200
VPN02 R1 no bgp default ipv4-unicast XR2 VPN02
no bgp default route-target filter
neighbor 192.168.3.3 update-source Loopback0
PE !
R5 XR6 PE
CE neighbor 192.168.3.3 next-hop-self
R7 R8 CE
AS 700 AS 700
100.64.7.7 100.64.8.8
RR

bgp ASBR
router-id 192.168.2.2 MP-EBGP 192.168.2.2
192.168.1.1
address-family vpnv4 unicast
retain route-target all
AS100! ASBR AS200
R1 XR2
VPN02neighbor 172.16.12.1 VPN02
remote-as 100
PE route-policy PASSALL out
R5 ! XR6 PE
!
remote-as 200
CE next-hop-self CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS MPLS VPNs
Problems with Route Installation at Remote PEs
Routes will not install on remote Pes if they have different

RTs
• AS 100 was using 100:1
• AS 200 was using 200:1
Check to see if the routes make it on ASBRs or RRs
Inter-AS MPLS VPNs
Problems with Route Installation: Checking on the RRs
R3#show bgp vpnv4 unicast all | b Netw
Route Distinguisher: 100:1
*>i 100.64.7.0/24 192.168.5.5 0 100 0 700 ?
*>i 172.16.57.0/24 192.168.5.5 0 100 0 ?
*>i 100.64.8.8/32 192.168.1.1 0 100 0 200 700 ?
*>i 172.32.68.0/24 192.168.1.1 0 100 0 200 ?
RP/0/0/CPU0:XR4#show bgp vpnv4 unicast | b Netw

*>i100.64.7.0/24 192.168.2.2 100 0 100 700 ?
*>i172.16.57.0/24 192.168.2.2 100 0 100 ?
*>i100.64.8.8/32 192.168.6.6 0 100 0 700 ?
*>i172.32.68.0/24 192.168.6.6 0 100 0 ?
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 1 – Additional Import Statements
Simple Solution, but does it scale?

R3 (IOS PEs) XR4 (IOS XR PEs)
vrf definition VPN01 vrf VPN02
rd 100:1 address-family ipv4 unicast
route-target export 100:1 import route-target
route-target import 100:1 200:1
!
export route-target
200:1
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 2 – Route Target ReWrite on ASBRs
IOS ASBRs (R1)

ip extcommunity-list 1 permit rt 200:1
route-map REWRITE permit 10
match extcommunity 1
set extcomm-list 1 delete
set extcommunity rt 100:1 additive
!
router bgp 100
neighbor 172.16.12.2 route-map REWRITE in
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 2 – Route Target Re-Write on ASBRs
IOS XR ASBRs (XR2)

route-policy REWRITE
if extcommunity rt matches-any AS100VPN01
then
set extcommunity rt AS200VPN02
endif
pass
end-policy
!
extcommunity-set rt AS100VPN01
100:1 router bgp 200
end-set neighbor 172.16.12.1
! remote-as 100
extcommunity-set rt AS200VPN01 address-family vpnv4 unicast
200:1 route-policy REWRITE in
end-set route-policy PASSALL out
RR

bgp ASBR
router-id 192.168.2.2 MP-EBGP 192.168.2.2
192.168.1.1
AS100! ASBR AS200
R1 XR2
VPN02neighbor 172.16.12.1 VPN02
remote-as 100
R5 ! XR6 PE
!
remote-as 200
CE next-hop-self CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02

Local Outgoing Prefix Outgoing Next Hop Bytes
------ ----------- ------------------ ------------
PE --------------- ------------
24000 R5Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 796 XR6 PE
24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12010
24003 60003 200:1:100.64.8.8/32 \
192.168.6.6 0
24004 60004 200:1:172.32.68.0/24 \
192.168.6.6 208
24005 Aggregate 172.16.12.0/24 default 0
CE
24006 R7206 100:1:100.64.7.0/24 \ R8 CE
172.16.12.1 0
AS 700
24007 207 100:1:172.16.57.0/24 \ AS 700
100.64.7.7 172.16.12.1 0 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
router static
172.16.12.1/32 GigabitEthernet0/0/0/1
PE
R5 XR6 PE
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02

------ ----------- ------------------ ------------
PE --------------- ------------
24000 R5Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 1070 XR6 PE
24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12383
24003 60003 200:1:100.64.8.8/32 \
192.168.6.6 0
24004 60004 200:1:172.32.68.0/24 \
192.168.6.6 20176
24006 206 100:1:100.64.7.0/24 \
CE Gi0/0/0/1 172.16.12.1 0 CE
R7 R8
24007 207 100:1:172.16.57.0/24 \
AS 700 Gi0/0/0/1 172.16.12.1 0 AS 700
100.64.7.7
24008 Pop 172.16.12.1/32 Gi0/0/0/1 172.16.12.1 0 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
CE7#trace 100.64.8.8
PE Tracing the route to 100.64.8.8
R5 VRF info: (vrf in name/id, vrf out name/id) XR6 PE
1 172.16.57.5 3 msec 2 msec 3 msec
5 172.32.68.8 [AS 200] 16 msec * 18 msec
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN Option B2: Advertise Peering Link RR
RR
Redistribute Redistribute
R3
Connected Static Route ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
No LDP or IGP required on the link between the two ASBRs.

CE Configure no bgp default route-target filter on ASBRs CE
R7 R8
AS 700 ASBRs redistribute/advertise peering link into IGP AS 700
100.64.7.7 100.64.8.8
RR interface GigabitEthernet0/2
ip address 172.16.12.1 255.255.255.0
R3 mpls bgp forwarding ASBR XR4
ASBR ! MP-EBGP 192.168.2.2
192.168.1.1
router ospf 1
redistribute connected subnets
AS100 ASBR AS200
R1 network 10.0.0.0 0.255.255.255 area XR2 100
VPN02 VPN02
network 192.168.0.0 0.0.255.255 area 100
!
router bgp 100
PE no bgp default route-target filter
R5 neighbor 172.16.12.2 remote-as 200 XR6 PE
!
CE neighbor 172.16.12.2 send-community extended R8 CE
R7
AS 700 AS 700
100.64.7.7 neighbor 192.168.3.3 send-community extended 100.64.8.8
RR router ospf 1
redistribute static
R3 area 200 ASBR XR4
ASBR
interface Loopback0 MP-EBGP 192.168.2.2
192.168.1.1
interface GigabitEthernet0/0/0/0
AS100 ASBR AS200
! R1 XR2
VPN02 VPN02
router bgp 200
!
PE neighbor 172.16.12.1
R5 remote-as 100 XR6 PE
route-policy REWRITE in
!
CE remote-as 200 CE
R7 R8
AS 700 AS 700
100.64.7.7 address-family vpnv4 unicast 100.64.8.8
Inter-AS VPN Option B3: Multi-hop RR
RR
Redistribute Redistribute
R3
Connected Connected ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1 XR2
VPN02 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
Static route on ASBRs to reach remote ASBR loopback

ASBRs peer with each other via Loopback interface.
Requires EBGP Multi-Hop
CE CE
R7 Configure no bgp default route-target filter on ASBRs R8
AS 700 AS 700
100.64.7.7 ASBRs advertise remote loopback into BGP 100.64.8.8
Inter-AS VPN Option B3: MultiHop RR
RR ip route 192.168.2.2 255.255.255.255 172.16.12.2
interface GigabitEthernet0/2
R3 ip address 172.16.12.1 255.255.255.0 ASBR XR4
ASBR mpls ip MP-EBGP 192.168.2.2
192.168.1.1
!
router ospf 1
AS100 ASBR AS200
R1 redistribute static subnets XR2
VPN02
network 10.0.0.0 0.255.255.255 area 100 VPN02
network 192.168.0.0 0.0.255.255 area 100
!
router bgp 100
PE no bgp default ipv4-unicast
R5 no bgp default route-target filter XR6 PE
neighbor 192.168.2.2 ebgp-multihop 255
!
CE neighbor 192.168.2.2 activate CE
R7 R8
AS 700 AS 700
100.64.7.7 neighbor 192.168.2.2 route-map REWRITE in 100.64.8.8
Inter-AS VPN Option B3: Multihop RR
RR router ospf 1
redistribute static
R3 area 200 ASBR XR4
.. ASBR MP-EBGP 192.168.2.2
192.168.1.1
!
router bgp 200
AS100address-family vpnv4 unicast ASBR AS200
R1 XR2
VPN02 VPN02
!
remote-as 100
PE ebgp-multihop 255
R5 update-source Loopback0 XR6 PE
!
CE remote-as 200 CE
R7 R8
AS 700 AS 700
100.64.7.7 address-family vpnv4 unicast TECMPL-3201 100.64.8.8
187
Inter-AS VPN Option C: MBGP between RRs RR
RR MP-EBGP
VPNv4 Routes/Labels
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
MP-EBGP
PE & RR Labels
AS100 ASBR AS200
R1 XR2
VPN02 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
VPNv4 session is established between RRs

RRs use Next-Hop-Unchanged
CE ASBRs exchange RRs and PE loopbacks as labeled routes
R7 R8 CE
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN Option C: RRs Peer Direct RR
RR
router ospf 1
R3 redistribute bgp 100 subnets ASBR XR4
ASBR network MP-EBGP
10.0.0.0 0.255.255.255 area 100
192.168.2.2
192.168.1.1
network 192.168.0.0 0.0.255.255 area 100
!
AS100 ASBR AS200
VPN02 R1 router bgp 100 XR2 VPN02
bgp log-neighbor-changes
!
PE address-family ipv4
R5 network 192.168.1.1 mask 255.255.255.255 XR6 PE
network 192.168.3.3 mask 255.255.255.255
network 192.168.5.5 mask 255.255.255.255
neighbor 172.16.12.2 send-label
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 TECMPL-3201 100.64.8.8
RR
router ospf 1
redistribute bgp 200
R3 areaASBR
200 ASBR XR4
MP-EBGP 192.168.2.2
interface
192.168.1.1 Loopback0
AS100 interface GigabitEthernet0/0/0/2 ASBR AS200
R1 XR2
!
VPN02 VPN02
router bgp 200
bgp router-id 192.168.2.2
network 192.168.2.2/32
PE network 192.168.4.4/32
R5 XR6 PE
network 192.168.6.6/32
allocate-label all
!
remote-as 100
CE address-family ipv4 labeled-unicast CE
R7 R8
AS 700 route-policy PASSALL out AS 700
100.64.7.7 100.64.8.8
RR
router bgp 100
R3 ASBR XR4
ASBR bgp log-neighbor-changes
MP-EBGP 192.168.2.2
192.168.1.1
AS100 neighbor 192.168.4.4 ebgp-multihop 255 ASBR AS200
R1neighbor 192.168.4.4 update-source Loopback0XR2
VPN02 VPN02
!
PE neighbor 192.168.4.4 activate
R5 XR6 PE
neighbor 192.168.4.4 next-hop-unchanged
neighbor 192.168.4.4 route-map REWRITE in
CE neighbor 192.168.5.5 route-reflector-client CE
R7 R8
AS 700 AS 700
100.64.7.7 TECMPL-3201 100.64.8.8
RR
router bgp 200
R3 ASBR XR4
ASBRbgp router-id 192.168.4.4
MP-EBGP 192.168.2.2
192.168.1.1
!
AS100 neighbor 192.168.3.3 ASBR AS200
R1 XR2
VPN02 remote-as 100 VPN02
ebgp-multihop 255
R5 XR6 PE
next-hop-unchanged
!
!
remote-as 200
CE update-source Loopback0 CE
R7 R8
AS 700 route-reflector-client AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN: Examining the VPNv4 Routes RR
RR
Check Routes Check Routes

R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1 XR2
VPN02 VPN02
Check Routes
Check Routes
PE
R5 XR6 PE
Verify the source/destination routes at entry/remote

PE and local/remote ASBR
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 100.64.8.8
Inter-AS VPN: Examining the MPLS FECs RR
RR Check MPLS Check MPLS
Forwarding Forwarding
R3 Table Table ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1 XR2
VPN02 VPN02
Check MPLS Check MPLS
Forwarding Forwarding
Table Table
PE
R5 XR6 PE
Verify the source/destination is label switched towards

destination with numbered labeles or ASBRs have
POP
CE
R7
Remember IOS XR needs a /32 entry for the FEC to R8 CE
populate.
AS 700 AS 700
100.64.7.7 100.64.8.8
Troubleshooting Inter-AS VPN: Tip RR
RR
Add a VRF Add a VRF
R3
Check Here Check Here ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1 XR2
VPN02 VPN02
PE
R5 XR6 PE
Sometimes the issue may not appear directly.

Add a loopback interface on ASBR, and place into a
VRF.
CE CE
R7 Provides a method of checking connectivity across the R8
AS 700 ASBR link. AS 700
100.64.7.7 100.64.8.8
MPLS Carrier
Supporting
Carrier (CSC)
Carrier Supporting Carrier (CSC)
• CSC allows MPLS services across discontiguous areas. Typically when MPLS
services cannot be provided end-to-end because of geography reasons.
Service Provider 1
Service Provider 2 Service Provider 2
CE CE
Customer Customer
Carrier Supporting Carrier (CSC) Roles
CSC-PE CSC-PE
R1 CSC-CE
CSC-CE XR2
R3 XR4
Backbone
Carrier
Customer Customer
Carrier Carrier
R5
PE PE XR6
CE CE
R7 R8
AS 700 AS 800
CSC is not running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
Labeled BGP LDP + IGP XR2 BGP
R3 XR4
IBGP + RR Client
IBGP
IBGP
Customer
Carrier
R5 XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
MP-IBGP
LDP + IGP
R1
Labeled
LDP + IGP XR2 BGP
R3 description to R3 XR4
vrf forwarding CORE
IBGP + RR Client
IBGP
IBGP
ip address 172.16.13.1 255.255.255.0

mpls ip
! Customer
router ospf Carrier
10 vrf CORE
R5 redistribute bgp 100 subnets XR6
network 172.16.0.0 0.0.255.255 area 200
!
EBGP EBGP
router bgp 100
CE7 address-family ipv4 vrf CORE CE7
AS 700 redistribute ospf 10
AS 800
MP-IBGP
Labeled BGP Labeled

R1
LDP + IGP XR2 BGP
R3 description to R3 XR4
vrf forwarding CORE
IBGP + RR Client
IBGP
IBGP
ip address 172.16.13.1 255.255.255.0

mpls bgp forwarding
! Customer
router bgp 100
Carrier
R5 address-family ipv4 vrf CORE XR6
network 172.16.13.0 mask 255.255.255.0
EBGP EBGP
CE7 neighbor 172.16.13.3 as-override CE7
AS 700 neighbor 172.16.13.3 send-label
AS 800
MP-IBGP
LDP + IGP
or Labeled
R1
router bgp 100 XR4

R3
vrf CORE
rd 100:1 IBGP + RR Client
IBGP
IBGP

allocate-label all Customer
Carrier
!
R5 XR6
remote-as 200
EBGP ipv4 labeled-unicast
address-family EBGP
CE7 CE7
AS 700
as-override AS 800
Where is MP-IBGP
172.32.24.4/32
LDP + IGP
or Labeled
R1
R3 mpls forwarding XR4

RP/0/0/CPU0:XR2#show
Local Outgoing Prefix Outgoing Next Hop
IBGP + RR Client Bytes
IBGP
IBGP

------ ----------- ------------------ ------------ --------------- ------------
24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 1271
Customer
24002 100 172.16.13.0/24[V] 192.168.1.1 0
24003 104 172.16.35.0/24[V]
Carrier192.168.1.1 0
24004 Aggregate R5
CORE: Per-VRF Aggr[V] \ XR6
CORE 4280
24005 103
EBGP 192.168.3.3/32[V] 192.168.1.1 0 EBGP
24006 106 192.168.5.5/32[V] 192.168.1.1 1022
24007
CE7 Pop 172.32.46.0/24[V] 172.32.24.4 0 CE7
24008 Pop 192.168.4.4/32[V] 172.32.24.4 49920
AS 70044005
24009 192.168.6.6/32[V] 172.32.24.4 8312796 AS 800
router static MP-IBGP

vrf CORE
LDPipv4
address-family + IGP
unicast
or
172.32.24.4/32 GigabitEthernet0/0/0/0 Labeled
R1

R3 XR4
Label Label or ID Interface
IBGP + RR Client Switched
IBGP
IBGP
------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 296
24001 Pop 172.32.24.4/32[V] Gi0/0/0/0 172.32.24.4 1210
Customer
24002 100 172.16.13.0/24[V] 192.168.1.1 0
24003 104 172.16.35.0/24[V]
Carrier 192.168.1.1 0
24004 Aggregate R5 CORE: Per-VRF Aggr[V] \ XR6
CORE 4280
24005 103
EBGP 192.168.3.3/32[V] 192.168.1.1 0 EBGP
24006 106 192.168.5.5/32[V] 192.168.1.1 252
24007
CE7 Pop 172.32.46.0/24[V] Gi0/0/0/0 172.32.24.4 0 CE7
24008 Pop 192.168.4.4/32[V] Gi0/0/0/0 172.32.24.4 48880
AS 70044005
24009 192.168.6.6/32[V] Gi0/0/0/0 172.32.24.4 8092044 AS 800
MP-IBGP
LDP + IGP
or Labeled
R1
CE7#trace 100.64.8.8
Tracing the route R3
to 100.64.8.8 XR4
1 172.16.57.5 3 msec 3 msec 3 msec IBGP + RR Client
IBGP
IBGP
2 172.16.35.3 [AS 200] [MPLS: Label 3005 Exp 0] 16 msec 18 msec 15 msec
4 10.12.1.2 [MPLS: Label 24009 Exp 0] 14 Customer
msec 16 msec 16 msec
5 172.32.24.4 [MPLS: Label 44005 Exp 0] 14Carrier
msec 14 msec 16 msec
6 172.32.46.6 [AS
R5 200] 15 msec 15 msec 16 msec
XR6
7 172.32.68.8 [AS 200] 16 msec * 19 msec
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
R3
Customer XR4
Carrier
IGP
LDP
IGP
LDP
R5 IBGP XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
CE7#trace 100.64.8.8
Tracing the route R3
to 100.64.8.8 Customer XR4
VRF info: (vrf in name/id, vrf out name/id)Carrier
1 172.16.57.5 3 msec 3 msec 3 msec
IGP
LDP
IGP
LDP
2 172.16.35.3 [AS 200] [MPLS: Label 3005 Exp 0] 16 msec 18 msec 15 msec
6 172.32.46.6 [AS
R5 200] 15 msec 15 msec 16 IBGP
msec
XR6
7 172.32.68.8 [AS 200] 16 msec * 19 msec
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS VPN inside its POP Sites
MP-IBGP
LDP + IGP
or R1
Labeled
R3
Customer XR4
Carrier
IGP
LDP
IGP
LDP
R5 MP-IBGP XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS VPN inside its POP Sites
CE7# trace 100.64.8.8 so lo0
MP-IBGP
LDP + IGP
VRF info: (vrf in name/id, vrf out name/id) Labeled
or R1 4 msec
1 172.16.57.5 5 msec 3 msec LDP + IGP XR2 BGP
Labeled BGP
2 172.16.35.3
Backbone Carrier [MPLS: Labels 3005/60005 Exp 0] 18 msec 14 msec 24 msec
3Forwarding
172.16.13.1
Label [MPLS: Labels 112/60005 Exp 0] 15 msec 15 msec 16 msec
Customer XR4msec
4 10.12.1.2 [MPLS:
R3 Labels 24009/60005 Exp 0] 14 msec 12 msec 18
Carrier
IGP
LDP
IGP
LDP

7 172.32.68.8 [AS 800] 26 msec * 19 msec
R5 MP-IBGP XR6
EBGP EBGP
CE7 Customer Carrier CE7
VPN Label
AS 700 AS 800
Troubleshooting
IPv6 VPNs
Troubleshooting 6VPE
Reference Topology
IPv4 – 192.168.1.1/32
IPv6 – 2001:DB8::1/128
AS 100
Service Provider Core
PE1 IPv4 – IGP
MPLS
IPv4 – 192.168.2.2/32
IPv6 – 2001:DB8::2/128
IPv4 – 192.168.5.5/32
IPv6 – 2001:DB8::6/128 IPv6 – 2001:DB8::7/128
IPv6 – 2001:DB8::5/128
CE1 RR-P PE5 CE2

PE2
AS 200 IPv4 – 192.168.4.4/32 AS 300
VRF Configuration
• IPv6 enabled VRF’s are configured in the same way as IPv4 VRF’s
• On Cisco IOS, use command vrf definition to configure both IPv4 and IPv6
capable VRF’s
vrf definition VPN01 vrf VPN01
rd 1:1 address-family ipv6 unicast
address-family ipv6 unicast import route-target
route-target export 1:1 2:2
route-target import 2:2 export route-target
address-family ipv4 unicast 1:1
. . . address-family ipv4 unicast
interface Gi0/0 . . .
vrf forwarding VPN01 interface Gi0/0/0/0
ipv6 address xx:xx:xx::y/64 vrf VPN01
ipv6 address xx:xx:xx::y/64
6VPE Configuration – Cisco IOS
router bgp 100
bgp log-neighbor-changes
!
neighbor 192.168.4.4 next-hop-self
!
address-family ipv6 vrf red
neighbor 2001:DB8:0:16::6 remote-as 200
neighbor 2001:DB8:0:16::6 activate
exit-address-family
6VPE Configuration – IOS XR
router bgp 100
!
remote-as 100
next-hop-self
!
vrf red
rd 100:1
!
neighbor 2001:db8:0:26::6
remote-as 200
route-policy pass in
route-policy pass out
Verifying Control Plane
• Since both control plane and data plane works in opposite direction, verify the
IPv6 VPN prefix on PE5.
PE5#show ipv6 route vrf red

! Output omitted for brevity
B 2001:DB8::6/128 [200/0]
via 192.168.1.1%default, indirectly connected
B 2001:DB8::7/128 [20/0]
via FE80::7, GigabitEthernet0/2
• Verify the VPNv6 prefix in BGP along with the local label
PE5#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for [100:5]2001:DB8::7/128, version 38
Paths: (1 available, best #1, table red)
2
Refresh Epoch 1
300
2001:DB8:0:57::7 (FE80::7) (via vrf red) from 2001:DB8:0:57::7
(192.168.7.7)
Origin IGP, metric 0, localpref 100, valid, external, best
mpls labels in/out 23/nolabel
• The remote IOS PE - PE1, receives the VPNv6 prefix as the out label of 23.
PE1#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for [100:1]2001:DB8::7/128, version 7
Paths: (1 available, best #1, table red)
1
Refresh Epoch 1
300, imported path from [100:5]2001:DB8::7/128 (global)
::FFFF:192.168.5.5 (metric 3) (via default) from 192.168.4.4 (192.168.4.4)
Origin IGP, metric 0, localpref 100, valid, internal, best
Originator: 192.168.5.5, Cluster list: 192.168.4.4
mpls labels in/out nolabel/23
RP/0/0/CPU0:PE2#show bgp vpnv6 unicast vrf red 2001:db8::7/128
BGP routing table entry for 2001:db8::7/128, Route Distinguisher: 100:1
Last Modified: Feb 4 22:46:29.408 for 1d05h
Paths: (1 available, best #1)
Path #1: Received by speaker 0
300
192.168.5.5 (metric 3) from 192.168.4.4 (192.168.5.5)
Received Label 23
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best,
import-candidate, imported
Received Path ID 0, Local Path ID 1, version 5
Extended community: RT:100:1
Originator: 192.168.5.5, Cluster list: 192.168.4.4
Source VRF: default, Source Route Distinguisher: 100:5
Verifying Data Plane
PE1#show ipv6 cef vrf red 2001:db8::7/128 detail

2001:DB8::7/128, epoch 0, flags [rib defined all labels]
recursive via 192.168.5.5 label 23
nexthop 10.1.14.4 GigabitEthernet0/2 label 19
PE1#show mpls forwarding-table 192.168.5.5

21 19 192.168.5.5/32 0 Gi0/2 10.1.14.4
Verifying Data Plane on IOS XR
RP/0/0/CPU0:PE2#show cef vrf red ipv6 2001:db8::7/128
2001:db8::7/128, version 7, internal 0x5000001 0x0 (ptr 0xa140c5f4) [1],
0x0 (0x0), 0x208 (0xa14db230)
Updated Feb 4 22:46:29.731
Prefix Len 128, traffic index 0, precedence n/a, priority 3
via ::ffff:192.168.5.5, 3 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0xa176b0bc 0x0]
recursion-via-/128
next hop VRF - 'default', table - 0xe0000000
next hop ::ffff:192.168.5.5 via ::ffff:192.168.5.5:0
next hop 10.1.24.4/32 Gi0/0/0/1 labels imposed {19 23}
RP/0/0/CPU0:PE2#show mpls forwarding-table prefix 192.168.5.5/32

24001 19 192.168.5.5/32 0 Gi0/0/0/1 10.1.24.4
Verifying Ingress Hardware Programming – IOS XR
PE2#show cef vrf red ipv6 2001:db8::7/128 hardware ingress detail loc0/0/CPU0
0x0 (0x0), 0x208 (0xa14db230)
Updated Feb 4 22:46:29.730
[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Feb 4 22:46:29.730
LDI Update time Feb 4 22:46:29.730
recursion-via-/128
Ingress platform showdata is not available.
Load distribution: 0 (refcount 1)
Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0
Verifying Egress Hardware Programming – IOS XR
PE2#show cef vrf red ipv6 2001:db8::7/128 hard egr det loc 0/0/CPU0
0x0 (0x0), 0x208 (0xa14db230)
[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Feb 4 22:46:29.730
LDI Update time Feb 4 22:46:29.730
recursion-via-/128
Egress platform showdata is not available.
Load distribution: 0 (refcount 1)
Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0
Troubleshooting 6VPE / MPLS
Verifying Counters on Interface
• Verify the interface counters for mpls forwarding
• If there is forwarding problem, check the counters and ensure they are not
increasing.
• Initiate the VPNv6 prefix ping and verify the counters again to see if they
increased
RP/0/0/CPU0:PE2#show interface gigabitethernet0/0/0/1 accounting
GigabitEthernet0/0/0/1
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 261333 20337753 46929 2305821
IPV6_UNICAST 21017 2062274 20995 1964348
MPLS 10 1180 14426 968553
ARP 84 5040 84 3528
IPV6_ND 13296 1193736 10306 742016
Troubleshooting
MPLS
Traffic-Engineering
Troubleshooting MPLS TE
The “Fish” Problem
35
M
Tra
ffi c
100
60 40
50
X
ffi c
Tra
M
10
80
45 25
70
Motivation
• Increase efficiency of bandwidth resources
• Prevent over-utilised (congested) links whilst other links are under-utilised
• Ensures the most desirable/appropriate path for certain traffic types based on
certain policies
• Override the shortest path selected by the IGP
• The ultimate goal is COST SAVING
CSPF – The TE Algorithm
Dijkstra(G, w, s):
• CSPF (executed at ingress) – Initialize-single-source(G,s);
computes an optimal explicit path S = empty set;
based on constraints Q = V[G];
While Q is not empty {
• Bandwidth requirements u = Extract-Min(Q);
• Hop limitations S = S union {u};
for each vertex v in Adj[u] {
• Administrative groups (link colors) relax(u, v, w);
}
• Priority (setup and hold) }
• Explicit route In which:
• Link attributes G: the graph, represented in some way (e.g.
• Reservable bandwidth of the links adjacency list)
w: the distance (weight) for each edge (u,v)
(static bandwidth minus the currently s (small s): the starting vertex (source)
reserved bandwidth S (big S): a set of vertices whose final
shortest path from s have already been
determined
Q: set of remaining vertices, Q union S = V
draft-manayya-cspf-00
1. CSPF process begins at ingress router with parameters of bandwidth, setup priority, hold priority
and method used incase of equal cost multipath such as random, least fill or most-fill. It determines
the final destination (Egress router).
2. It checks for maximum hop count, include and exclude constraints configured.
3. Check each node for metric and hop count starting with Ingress.
4. For each node check if endpoint is already visited ,if yes then skip the verification. if not check the
link for metric, color and bandwidth (for constraints). The information on each node includes
administrative groups (Color), metrics, static bandwidth, reservable bandwidth, and available
bandwidth priority level. The information contained in the traffic engineering database should be the
same across all routers in the same traffic engineering domain.
5. If it fails then remove this link.
6. If it passes then select the link with shortest path to neighbor router, go to next link and repeat the
step 4.
draft-manayya-cspf-00 (contd…)
• Repeat the steps 3 to 5 for all nodes
• The result of the CSPF algorithm is formed into a strict-hop ERO (Explicit Route Object)
• When the ERO is completed, the ERO is passed to the RSVP (Resource Reservation Protocol)
process, where it is used for signaling and establishing the LSP in the network.
• If it is not possible to find the path then indicate about not finding a route then retry after retry interval.
RSVP Overview
• Once the path is calculated, it must be signaled across the network
• Reserve any bandwidth to avoid “double booking” from other TE reservations
• Priority can be used to pre-empt low priority existing tunnels
• RSVP used to setup TE LSP

• PATH messages (from head to tail) carries LABEL_REQUEST
• RESV messages (from tail to head) carries LABEL
• When RESV reaches headend, tunnel interface = UP

• RSVP messages exist for LSP teardown & error sig
Headend Midpoint Tailend
RSVP Overview – Admission Control
• On receipt of PATH message
• Router will check there is bandwidth available to honour the reservation
• If bandwidth available then RSVP accepted
• On receipt of a RESV message

• Router actually reserves the bandwidth for the TE LSP
• If preemption is required lower priority LSP are torn down
• OSPF/ISIS updates are triggered
Does RSVP actually allocates the b/w across the path for TE tunnel?
RSVP Overview – Admission Control
100
60 40
50
RSVP Path
Message
(10M)
PATH RSVP
80
45 25 RESV
BW=10 30 Message
70
Configuration / Feature requirements
• RSVP should be enabled on relevant interfaces
• mpls traffic-eng should be enabled
• Globally 100
• Interface level 60 40
50
• IGP Level
• Tunnel Interface Configuration

80
• Allowing traffic through TE Tunnel 45 25
70
• Decision on Path Selection Process
• Dynamic
• Explicit-path
Autoroute Announce
• Used to include TE LSP in SPF calculations
• Tunnel is treated as a directly connected link to the tail
• IGP adjacency is NOT run over the tunnel!
• Using autoroute announce, all nodes behind the headend are routed via tunnel
IOS – IOS-XE (Config under Tunnel Interface)
tunnel mpls traffic-eng autoroute announce
IOS-XR (Configuration under Tunnel-te Interface)

autoroute announce
NX-OS (Configuration under Tunnel-te Interface)

autoroute announce
Forwarding Adjacency
• Autoroute does not advertise the LSP into the IGP
• There may be requirement to advertise the existence of TE tunnels to upstream
routers
• Allow upstream routers to compute a better path to destination a over downstream TE
tunnel
R1 R4 R5
All links have metric of 10
R3 R8
R2 R6 R7
Verification Commands
• Verifying RSVP Interfaces
• Show ip rsvp interface
• Verifying TE Tunnels
• Show mpls traffic-eng tunnels tunnel <num>
• Show mpls traffic-eng forwarding (XR)
• Show mpls traffic-eng forwarding-adjacency
• Verifying FRR Database

• Show mpls traffic-eng fast-reroute database
RSVP Troubleshooting
RP/0/0/0:R1#sh rsvp counters messages summary
All RSVP Interfaces Recv Xmit Recv Xmit
Path 0 25 Resv 30 0
PathError 0 0 ResvError 0 1
PathTear 0 30 ResvTear 12 0
ResvConfirm 0 0 Ack 24 37
Bundle 0 Hello 0 5099
SRefresh 8974 9012 OutOfOrder 0
Retransmit 20 Rate Limited 0
IOS - Show ip rsvp counters summary
IP proto 0x2e – Can use this for performing packet capture
Verify Basic TE Tunnel Forwarding
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels 400
Name: tunnel-te400 Destination: 192.168.4.4 Ifhandle:0x580
Status:
Admin: up Oper: up Path: valid Signalling: connected
path option 1, type dynamic (Basis for Setup, path weight 1)
G-PID: 0x0800 (derived from egress interface properties)
Bandwidth Requested: 0 kbps CT0
Creation Time: Thu Jun 15 19:22:40 2017 (00:15:46 ago)
Config Parameters:
Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff
Metric Type: TE (global)
Path Selection:
<snip>
Fast Reroute: Disabled, Protection Desired: None
Path Protection: Not Enabled BFD Fast Detection: Disabled
Reoptimization after affinity failure: Enabled
Soft Preemption: Disabled
History: Tunnel has been up for: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)
Current LSP: Uptime: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)
Path info (OSPF 100 area 0):
Node hop count: 1
Hop0: 10.24.1.4
Hop1: 192.168.4.4
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels brief

TUNNEL NAME DESTINATION STATUS STATE
tunnel-te400 192.168.4.4 up up
PE1_t100 192.168.2.2 up up
PE4_t100 192.168.2.2 up up
PE1_t101 192.168.2.2 up up
Re-optimization Configs
• Configuration
• Logging
• Logging events lsp-status reoptimize (XR TE Tunnel interface config)
• Logging events lsp-status reroute (XR TE Tunnel interface config)
Troubleshooting : TE Tunnel does not come up
RP/0/RP0/CPU0:PE2# show mpls traffic-eng tunnel 400 detail
Wed May 29 14:07:50.428 UTC
Name: tunnel-te 400 Destination: 0.0.0.0
Status:
Admin: up Oper: down Path: not valid Signalling: Down
ospf 100 area 0
G-PID: 0x0800 (internally specified)
Config Parameters:
Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff Very verbose
Metric Type: TE (default) reason given here
AutoRoute: disabled LockDown: disabled on this line for
Loadshare: 0 equal loadshares config errors
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 0
Direction: unidirectional
Endpoint switching capability: unknown, encoding type: unassigned
Transit switching capability: unknown, encoding type: unassigned
Reason for the tunnel being down: No destination is configured
History:
Prior LSP:
ID: path option 10 [13]
Removal Trigger: signalling shutdown No Destination
configured under
Tunnel interface
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detail
Name: tunnel-te400 Destination: 192.168.4.4
Status:
ospf 100 area 0
Config Parameters:
Metric Type: TE (default)
AutoRoute: enabled LockDown: disabled
Loadshare: 0 equal loadshares
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 1
Direction: unidirectional
Endpoint switching capability: unknown, encoding type: unassigned
Transit switching capability: unknown, encoding type: unassigned Insufficient RSVP b/w.
History: Bandwidth command not
Prior LSP: configured under rsvp.
ID: path option 1 [21] or
Removal Trigger: path verification failed is misconfigured
Last Error:
PCALC:: No path to destination, 192.168.4.4(bw)
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detail
Name: tunnel-te400 Destination: 192.168.4.4
Status:
ospf 100 area 0
Config Parameters:
Metric Type: TE (default)
AutoRoute: enabled LockDown: disabled
Loadshare: 0 equal loadshares
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 1 Tunnel has no
Direction: unidirectional alternative path
Endpoint switching capability: unknown, encoding type: unassigned Or
Transit switching capability: unknown, encoding type: unassigned Explicit path is
History: misconfigured.
Prior LSP:
ID: path option 1 [21]
Removal Trigger: path verification failed
Last Error:
PCALC:: No path to destination, 192.168.4.4(reverselink or exclude-link)
TE Tunnel not up (Summary)
• RSVP Signaling in progress
• Show rsvp sessions dst-port
• No path available
• Show mpls traffic-eng igp-area
• Show mpls traffic-eng topology model-type rdm|mam (Russian Dolls
/ Maximum allocation)
• Show mpls traffic-eng link-management interface x/y
• Cannot reach dst x.x.x.x from y.y.y.y
• Show rsvp interface
• Or check TE topology database
Class-Based Tunnel Selection – CBTS
Destination NH: PE2 EXP: 4
PE2
Destination NH: PE2 EXP: Default

PE1 Destination NH: PE3 EXP: 3,4 PE3

PE4
• EXP-based selection between multiple tunnels to same destination

• Local Tunnels (Head-end) configured with allowable EXP values
• Tunnels may be configured as default
• No IGP extensions, VRF aware
• Simplifies use of DS-TE tunnels & similar to PVC Bundling
Russian Dolls Model (RDM)
• BW pool applies to one
or more classes All
BC0 Classes
• Global BW pool (BC0) (Class0 Maximum
+ Reservable
equals MRB BC1 Class1 Bandwidth
Class1 +
+ Class2) (MRB)
• BC0..BCn used for Class2
BC2 Class2
computing unreserved
BW for class n
• Current implementation supports BC0 and BC1
• BC0 – Global Pool
• BC1 – Sub Pool
• Supported by Traditional and IETF implementation
Maximum Allocation Model (MAM)
• BW pool applies
to one class
BC0
Class0
• Sum of BW pools may exceed MRB
Maximum
• Sum of total BC1 Class1 All Reservable
Classes Bandwidth
reserved BW may (MRB)
not exceed MRB BC2 Class2
• Current implementation supports BC0

and BC1
• Supported by IETF Implementation
only
CBTS – Configuration Example
Both tunnels to same
interface Tunnel65
destination but different QoS
ip numbered loopback0
tunnel destination 192.168.2.2
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng bandwidth sub-pool 30000
tunnel mpls traffic-eng exp 5
interface Tunnel66
ip numbered loopback0
tunnel destination 192.168.2.2
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng bandwidth 50000
tunnel mpls traffic-eng exp default
Traces to collect on IOS XR
Module Trace commands

MPLS TE Control Show tech support mpls traffic-eng
RSVP Show tech support rsvp
CEF (forwarding) Show cef mpls trace location <line card location>
Show cef platform trace all all location <line card location>
Show cef trace location <line card location>
Show mpls forwarding tunnel detail
Show mpls forwarding labels hardware ingress/egress detail loc
Show cef mpls adj tunnel-te <> hardware ingress/egress detail loc <>
SONET Show sonet-local trace location <line card location>
Bundles Show tech-support bundles
Interface Manager Show tech-support pfi
Tunnel Protection
• Mechanism to mitigate packet loss during a failure
• Pre-provisioned protection tunnels that carry traffic when a protected link or
node goes down
• MPLS TE protection also known as FAST REROUTE
• Protects against LINK FAILURE
• For example, Fibre cut, Carrier Loss, ADM failure
• Protects against NODE FAILURE
• For example, power failure, hardware crash, maintenance
Categories of FRR
• Local Protection
• Link Protection
• Node Protection
• Protects a segment of the tunnel (Node or Link)
• 1:N Scalability
• Faster failure recovery
• Path Protection
• Protects individual tunnels
• 1:1 Scalability
• More resource consumption
Link Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4

TE Label
Troubleshooting MPLS TE PLR
FRR kicks in…
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4

TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4

TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4

TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4

TE Label
MPLS Traffic-Engineering
Demo
Troubleshooting
MPLS with NX-OS
Troubleshooting MPLS with NX-OS
Software Architecture
IM/OIM/
MPLS-Mgr L3VM
CLI URIB
SNMP ULIB
System
LDP IGP
Manager
Feature
Netstack
Manager
License
Manager PSS
MTS
Shared Memory
Message Queue
Component Functions
• Interact with ULIB
• Allocate local label for prefixes
• Interact with URIB
• Learn routes
• Program outgoing labels
• Interact with Netstack

• UDP socket (Hello messages)
• TCP sockets (Other LDP messages)
• Interact with IM/OIM/MPLS_mgr

• Learn interface status
• Learn interface address
• Enable MPLS forwarding on interface
Component Functions (contd…)
• Interact with L3VM
• VRF table id
• Interact with IGP
• LDP-IGP sync
• LDP auto-configuration
• Interact with platform services

• Enabling LDP feature enables multiple services:
• LDP service
• ULIB service
• mpls_mgr service
• mpls_oam service
MPLS Packet Flow
L2FT FIB
LDB ILM ADJ ELM RIT
(DMAC) TCAM
• LDB – L2 Features; Perform LDP lookup to derive LIF / BD for ingress packet
• L2FT – Perform SMAC and DMAC lookup; DMAC should be router MAC
• ILM – Lookup ingress LIF MAP table and identify feature enabled, i.e. MPLS
• FIB – Deals with both PI and PD programming
• ADJ – FIB results provides adjacency points to egress LIF
• ELM – Egress LIF has the DI for egress interface
• RIT – Generate the rewrite (SMAC, DMAC and Label rewrite [push, pop, swap])
LDB – Check if the router BD is set in the LDP entry
module-1# show hardware internal forwarding interface e1/1
Software Tables:
Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002
State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)
Number of Member Ports(0x0)
LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)
Hardware Tables:
Instance: 0x1
L2-LIF-MAP entry with index = 0x422
ldb_base = 0xc801 add_vlan = 0
Instance: 0x1
L2-LIF entry with index = 0xc801
pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0
ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002
<snip>
Check if the router BD is set in the LDP entry
module-1# show hardware internal forwarding interface e1/1
Software Tables:
Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002
State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)
Number of Member Ports(0x0)
LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)
Hardware Tables:
Instance: 0x1
L2-LIF-MAP entry with index = 0x422
ldb_base = 0xc801 add_vlan = 0
Instance: 0x1
L2-LIF entry with index = 0xc801
pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0
ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002
<snip>
Verify L2FT and ILM
L2FT
show hardware mac address-table
FE | Valid| PI| BD | MAC | Index| Stat| SW | Modi| Age| Tmr| GM| Sec| TR| NT| RM| RMA| Cap| Fld|Always
| | | | | | ic | | fied|Byte| Sel| | ure| AP| FY| | |TURE| | Learn
---+------+---+------+---------------+-------+-----+-----+-----+----+----+---+----+---+---+---+----+----+----+------
0 1 1 2 0022.557a.32c1 0x00400 1 0x000 0 6 0 1 0 0 0 0 0 0 0 0
0 1 0 1 0100.0cff.fffe 0x00421 1 0x001 0 6 0 0 0 0 0 0 0 0 0 0
ILM
NX-OS# show hardware internal forwarding interface Ethernet 1/1 module 10 | inc mpls_en
l2l3_lkup_cfg = 0 mpls_en = 1 sm_en = 0 red_ids_chk_fail_en = 1 v4_rpfv3_en = 0
ipv4_en = 1 eompls_en = 0 mpls_en = 1
N7k-1# show hardware internal forwarding interface e1/2 module 1 | in mpls_en

mpls_vpn_sel : 0x0 l2_tunnel_type : 0x0 mpls_en : 0x1
Verifying FIB - PI
N7k-1# show forwarding route module 1
----------------+----------------------------------------+----------------------+-----------------
Prefix | Next-hop | Interface | Labels
----------------+----------------------------------------+----------------------+-----------------
<snip>
192.168.2.2/32 nxthop 10.12.1.2 Ethernet1/2 NO-OP
192.168.3.3/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 21
192.168.4.4/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 22
N7k-1# show forwarding route detail

Prefix 192.168.2.2/32,
No of paths : 1 Update time: Wed Jun 14 08:46:21 2017
nxthop 10.12.1.2 Ethernet1/2 NO-OP DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Prefix 192.168.3.3/32,
nxthop 10.12.1.2 Ethernet1/2 PUSH 21 DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Prefix 192.168.4.4/32,
nxthop 10.12.1.2 Ethernet1/2 PUSH 22 DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Verifying FIB – PI – Forwarding and Adjacency Info
N7k-1# show forwarding mpls module 1
--------+-----------+-------------------+----------------+-------------+-------
Local |Prefix |FEC |Next-Hop |Interface |Out
Label |Table Id |(Prefix/Tunnel id) | | |Label
--------+-----------+-------------------+----------------+-------------+-------
18 |0x1 |192.168.2.2/32 |10.12.1.2 |Ethernet1/2 |Pop Label
19 |0x1 |192.168.3.3/32 |10.12.1.2 |Ethernet1/2 |21
20 |0x1 |192.168.4.4/32 |10.12.1.2 |Ethernet1/2 |22
N7k-1# show forwarding adjacency

IPv4 adjacency information
next-hop rewrite info interface
-------------- --------------- -------------
10.1.12.2 001b.54c2.3342 Ethernet1/2
N7k-1# show forwarding adjacency mpls

IPv4 adjacency information, adjacency count 1
next-hop rewrite info interface
-------------- --------------- -------------
10.1.12.2 Ethernet1/2 001b.54c2.3342 NO-OP 3
10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 21
10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 22
Verifying FIB – PD – MPLS Programming
N7k-1# show system internal forwarding mpls detail
Table id = 0x1
------------------
----+--------+--------+------------+----------+----------+-----------+--------+
Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op
----+--------+--------+------------+----------+----------+-----------+--------+
0 0x5624 0x23c2 16 0x5c 0x1fe0 0 POP ONE
0 0x5625 0x23c3 17 0x5c 0x1fe0 0 POP ONE
0 0x5224 0x23c4 18 0x62 0x2 3 POP ONE
0 0x5225 0x23c5 19 0x60 0x2 21 SWAP ONE
0 0x5c24 0x23c6 20 0x64 0x2 22 SWAP ONE
0 0x5c25 0x23c7 21 0x65 0x3 0 POP ONE
Table id = 0x2a
------------------
----+--------+--------+------------+----------+----------+-----------+--------+
Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op
----+--------+--------+------------+----------+----------+-----------+--------+
No labels in table
Aggregate Table id = 0x2a
------------------
--------+--------+
label | vpn_id
--------+--------+
0 492287 0x2a
Verify Label Information in Hardware
pe1# show system internal forwarding mpls label
show system internal forwarding mpls
Table id = 1
------------------
----+--------+------------+----------+----------+-----------+--------+
Dev | Index | In-label | AdjIndex | LIF | Out-label | Op
----+--------+------------+----------+----------+-----------+--------+
0 0x1ffa9 18 0x62 0x2 3 POP ONE
0 0x5225 19 0x60 0x2 21 SWAP ONE
0 0x5c24 20 0x64 0x2 20 SWAP ONE
FIB DRAM
FIB TCAM Egress LIF
Adjacency
Index (LTL)
Index
Route Update PD Verification
• Use the following command to check the route in FIB PD
• Show system internal forwarding route
• Use the following command to check the adjacency in FIB PD
• Show system internal forwarding adjacency
• Use the following command to check the MPLS adjacency in LFIB PD
• Show system internal forwarding mpls adjacency
• Use the following command to check the hardware adjacency to verify if
the packet is getting forwarding out correct interface
• Show system internal forwarding adjacency entry <adj>
detail
Troubleshooting L3VPN VRF Issues
• Check for L3VM process for the event-traces to verify the events that occurrent
for the VRF
N7k-1# show system internal l3vm event-history vrf
VRF events for L3VM Process - Bufsize 1000 KB2017
2017 Jun 14 09:10:02.139925 l3vm [5710]: [5830]: Updated interface Ethernet1/1 cmd <vrf member TEST>
2017 Jun 14 09:10:02.139757 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) changing from VRF default to VRF TEST - Count 1
2017 Jun 14 09:10:02.139728 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) will be down, VRF default UP-IF count 1
2017 Jun 14 09:10:02.139680 l3vm [5710]: [5830]: Moving Ethernet1/1 (ifindex: 0x1a000000 iod: 37) from VRF default to VRF TEST
2017 Jun 14 09:10:02.139522 l3vm [5710]: [5830]: Deleting all L3VM_PSS_IF_KEY config for interface Ethernet1/1
2017 Jun 14 09:10:02.137418 l3vm [5710]: [5830]: [VSH] Process interface Eth1/1 cmd <vrf member TEST>
2017 Jun 14 09:06:24.460917 l3vm [5710]: [5830]: Updated vrf TEST cmd <address-family ipv4 unicast>
2017 Jun 14 09:06:24.460771 l3vm [5710]: [5830]: [VSH] Process vrf TEST cmd <address-family ipv4 unicast>
2017 Jun 14 09:06:24.426293 l3vm [5710]: [5830]: l3vm_pd_process_l3vm_mts_msg_from_ctrl: Received l3vm notification (mtype: 4)
2017 Jun 14 09:06:24.424511 l3vm [5710]: [5829]: VRF TEST:ipv4:base table (Up:--) sending: Table create
2017 Jun 14 09:06:24.424372 l3vm [5710]: [5829]: VRF TEST:ipv6:base table (Up:--) sending: Table create
2017 Jun 14 09:06:24.424256 l3vm [5710]: [5829]: VRF TEST (Up:--) sending: VRF create
2017 Jun 14 09:06:24.424006 l3vm [5710]: [5829]: VRF TEST - Created
2017 Jun 14 09:06:24.424002 l3vm [5710]: [5829]: VRF TEST (Up:--) sdb ack
2017 Jun 14 09:06:24.423008 l3vm [5710]: [5829]: gsdb_op_callback() - gsdb context 0x0003ce86
2017 Jun 14 09:06:24.421933 l3vm [5710]: [5830]: Updated cmd <vrf context TEST>
Complete Your Online
Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 gift card.
• Complete your session surveys
through the Cisco Live mobile
app or on www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be

available for viewing on demand after the
event at www.CiscoLive.com/Online.
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions
Thank you
SP SDN –
Segment Routing
Segment Routing
Path towards Segment Routing
• LDP had its own challenges
• Extra process required (LDP) + It creates complicated interaction with IGP (LDP-IGP
Sync)
• RSVP-TE – Deployment and scalability issues (Only 10% SP space uses
RSVP-TE and that too with FRR use-case)
• Always-on Feature, even when TE is not required in the network
• Need network that could understand application requirements
Segment Routing
Overview
• SR originally meant “Strade Romane” – network of roads which were built by
Roman Empire
• The name was later changed to Segment Routing
• SR is nothing but Application Engineered Routing, where application makes
request to the network (controller) to provide it a path that would serve the
needs of the application
• SR is a source based routing, where the source chooses a path based on the
application requirements
• The chosen path is encoded in the packet header as an ordered list of segments
• Segment – ID for any type of instruction
• Forwarding or service
Segment Routing
Scalability and Virtualization
Millions of
• Each engineered application flow is mapped on a path Application
flow paths
• Multiple possible paths are available in the network
A path is
• A path is expressed as an ordered list of segments mapped on a
list of
• The network maintains segments segments
• thousands of segments
The network
• completely independent of application size/frequency only
maintains
segments
• Excellent scaling and virtualization No per-flow
• the application state is no longer within the router but application
state
within the packet
Segment Routing
Data Plane
• MPLS: an ordered list of segments is represented as a stack of labels
• Segment Routing re-uses MPLS data plane without any change
• Segment represented as MPLS label
• Applicable to IPv4 and IPv6 address families
• IPv6: Source routing capability through the use of extension header

• Full interoperability with non-source routing nodes (with no signaling)
Segment Routing PHP
Packet Flow
16003
24037 24037
16009 16009
Global label
16003
A B C D
segment 1 16009
segment 2 I Global label
16009
E F G H
Adjacency segment 3
label 24037 16009
Segment Routing
SRGB
• Segment Routing Global Block SR1(config)#segment-routing mpls
• Range of labels reserved for Segment SR1(config-srmpls)#global-block 18000 19999
Routing Global Segments SR1(config-srmpls)#
• Default SRGB is 16,000 – 23,999
• Best practice: same SRGB on all nodes

• A non-default SRGB can be configured
• All protocols use the same SRGB
• SRGB is allocated as a block of labels under
control of SR-APP
• Modifying a SRGB configuration is
disruptive for traffic
Segment Routing
IGP Segments
• Prefix SID
• Shortest-path to the IGP prefix
• Equal Cost MultiPath (ECMP)-aware
• Global Segment
• Label = 16000 + Index
• Distributed by ISIS/OSPF
• Adjacency SID
• Forward on the IGP adjacency
• Local Segment
• Advertised as label value
Segment Routing
Migration from LDP to SR
• Configuring SR does not ensure that SR labels will be used by the routers for
forwarding purpose
• By default, the LDP bindings will be used for forwarding decisions
• To make SR to be preferred over LDP, use below configuration
IOS XE IOS XR
segment-routing mpls router isis SR-AS
! !
set-attributes address-family ipv4 unicast
address-family ipv4 segment-routing mpls sr-prefer
sr-label-preferred !
exit-address-family
!
Complete Your Online
Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 gift card.
• Complete your session surveys
through the Cisco Live mobile
app or on www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be

available for viewing on demand after the
event at www.CiscoLive.com/Online.
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions
Thank you

Tecmpl 3201

Uploaded by

Copyright:

Available Formats

Tecmpl 3201

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Tecmpl 3201

Uploaded by

Copyright:

Available Formats

Troubleshooting MPLS

Cisco Spark spaces will be cs.co/ciscolivebot#TECMPL-3201

But, how can you learn if you can’t see it..

• Multi-Protocol Label Switching (MPLS) forwarding reduces the lookup process

In Out In Out In Out

I need to goto A MPLS packet A MPLS packet A MPLS packet

Label COS S TTL

MPLS Router MPLS Header Fields Original Packet Header

 An MPLS packet may have more than one label

 The following scenarios may produce more than one label:

IGP domain without a label IGP domain with a label

 LSPs are derived from IGP routing information

 R3 is downstream to R2 for the 10.4.4.5 prefix

In Out In Out In Out

206 10.4.4.5 Payload

In Out In Out In Out

306 10.4.4.5 Payload

307 POP 10.4.4.0/24 Gi0/1 3 - 10.4.4.0/24 Gi0/1

Control Plane Data Plane

 Ethertype 0x0800 refers to IP

R1#ping 192.168.5.5 so 192.168.1.1

IOS / IOS XE IOS XR NX-OS

install feature-set mpls

In Out In Out In Out

In Out In Out In Out

PE1#sh mpls ldp neighbor

PE1#show tcp brief| i 646

• Ensure reachability between the LDP router ID’s

• Following avoids both shortcomings

 “show mpls ldp discovery [detail]”

PE1#show mpls ldp discovery

PE1#show mpls ldp discovery P1#show mpls ldp discovery

R1#debug mpls ldp transport connections

PE1(config-if)#mpls label protocol ldp

PE1#show mpls ldp discovery P1#show mpls ldp discovery

PE1#show ip route 192.168.11.11

Problem: Default route towards the peering router

PE1#show mpls ldp discovery PE2#show mpls ldp discovery

RP/0/0/CPU0:PE2#show mpls ldp trace peer last 20

0/0/CPU0 t1 [PEER]:581: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'TCP connection closed'

0/0/CPU0 t1 [PEER]:575: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'Received Notification message

0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'Received Notification message

• LDP IGP Sync feature is enabled under IGP (OSPF/ISIS)

router (config-isis-if-af) # mpls ldp sync [ level <1-2> ]

router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if)

router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar)

 LDP IGP Sync delays are configured under LDP

router (config-ldp) # igp sync delay on-session-up <sec>

router (config-ldp) # igp sync delay on-proc-restart <sec>

router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> | infinite } ]

router (config-ldp) # log session-protection

R1#show mpls ldp neighbor

*Jun 9 03:34:52.516: ldp: Opening ldp conn; adj 0x7F1D2AC4D9D8, 192.168.1.1

CE1 PE1 P1 PE2 CE2

• Broken LDP adjacency

• LIB stores local and remote bindings

P1#show ip route 3.3.3.4

• Ping mode: Connectivity check of an LSP

Version Number Must Be Zero