Mpls l3vpn Tecmpl-3201

#CLUS
Troubleshooting
MPLS – On All Cisco
Platforms
Vinit Jain – CCIE# 22854
@vinugenie
Brad Edgeworth – CCIE# 31574
@bradedgeworth
TECMPL-3201
#CLUS
Agenda
• Troubleshooting LDP Issues
• BGP, LDP, RSVP
• Troubleshooting MPLS LSP
• OAM, Multipath Trace
• Troubleshooting MPLS L3 VPNs
• Troubleshooting PE-CE Interaction (RD, RT, VPN Services)
• Interactions with Traffic Engineering
• Segment Routing
• Migration
• On Demand Next-Hop (ODN)
#CLUS TECMPL-3201 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session
How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
Webex Teams will be moderated cs.co/ciscolivebot#TECMPL-3201

by the speaker until June 16, 2019.
#CLUS © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
MPLS Trivia
Question
Fun with MPLS Trivia
R1, R2, R3, R4 and R5 all have OSPF and MPLS enabled.
What changes can be made on R2 and/or R3 to prevent only R1’s
Loopback (192.168.1.1) from pinging R5’s Loopback (192.168.5.5)?
We will explain some of the concepts that make this work.
R1 R2 R3 R4 R5
R1#ping 192.168.5.5 so 192.168.1.1

Lo0: 192.168.1.1 Type escape sequence to abort. Lo0: 192.168.5.5
Sending 5, 100-byte ICMP Echos to 192.168.5.5, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/4/5 ms
Configuring and
Basic LDP
Operations
Troubleshooting LDP Issues
MPLS LDP Configuration
IOS / IOS XE IOS XR NX-OS
install feature-set mpls

mpls label protocol ldp mpls ldp
feature-set mpls
! router-id x.x.x.x
feature mpls
interface Gig 0/0 interface gi 0/0/0/0
mpls ldp configuration
mpls ip interface gi 0/0/0/1
router-id x.x.x.x
mpls label protocol ldp
!
exit
interface ethernet 2/1
!
mpls ip
mpls ldp router-id
loopback0 force
Establishing Adjacency & Swapping Labels
Populating the RIB
• First the IGP (OSPF / IS-IS) is established and routes are
exchanged between all routers
In Out In Out In Out

Label Label Network Out Int Label Label Network Out Int Label Label Network Out Int
N/A N/A 10.1.0.0/16 Gi0/2 N/A N/A 10.1.0.0/16 Gi0/0 N/A N/A 10.1.0.0/16 Gi0/0
R1 R2 R3 R4
Creating the Local Labels
• Local Labels are automatically generated for all prefixes in the RIB.
(MPLS Label 3 is reserved for Implicit-Null – directly connected routes)
• This includes local network prefixes
3 - 10.1.0.0/16 Gi0/2 200 N/A 10.1.0.0/16 Gi0/0 300 N/A 10.1.0.0/16 Gi0/0
3 - 10.1.1.0/24 Gi0/1 201 N/A 10.1.1.0/24 Gi0/0 301 N/A 10.1.1.0/24 Gi0/0
3 - 10.12.1.0/24 Gi0/0 3 - 10.12.1.0/24 Gi0/0 302 N/A 10.12.1.0/24 Gi0/0
103 N/A 10.23.1.0/24 Gi0/0 3 - 10.23.1.0/24 Gi0/1 3 - 10.23.1.0/24 Gi0/0
104 N/A 10.34.1.0/24 Gi0/0 204 N/A 10.34.1.0/24 Gi0/1 3 - 10.34.1.0/24 Gi0/0
105 N/A 10.4.0.0/16 Gi0/0 205 N/A 10.4.0.0/16 Gi0/1 305 N/A 10.4.0.0/16 Gi0/1
106 N/A 10.4.4.0/24 Gi0/0 206 N/A 10.4.4.0/24 Gi0/1 306 N/A 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
• Local Labels are exchanged with downstream routers
• Labels are all exchanged at the same time.
(This animation was done to show you the correlation of tables)

3 - 10.1.0.0/16 Gi0/2 200 POP 10.1.0.0/16 Gi0/0 300 200 10.1.0.0/16 Gi0/0
3 - 10.1.1.0/24 Gi0/1 201 POP 10.1.1.0/24 Gi0/0 301 201 10.1.1.0/24 Gi0/0
3 - 10.12.1.0/24 Gi0/0 3 - 10.12.1.0/24 Gi0/0 302 POP 10.12.1.0/24 Gi0/0
103 POP 10.23.1.0/24 Gi0/0 3 - 10.23.1.0/24 Gi0/1 3 - 10.23.1.0/24 Gi0/0
104 204 10.34.1.0/24 Gi0/0 204 POP 10.34.1.0/24 Gi0/1 3 - 10.34.1.0/24 Gi0/0
105 205 10.4.0.0/16 Gi0/0 205 305 10.4.0.0/16 Gi0/1 305 405 10.4.0.0/16 Gi0/1
106 206 10.4.4.0/24 Gi0/0 206 306 10.4.4.0/24 Gi0/1 306 406 10.4.4.0/24 Gi0/1
R1 R2 R3 R4
Troubleshooting
LDP Issues
LDP Neighborship
 LDP neighborship is formed on TCP port 646
 Discovery Mechanism:
 Basic Discovery – Multicast UDP hellos for directly connected neighbors
 Extended Discovery – Targeted Unicast UDP hellos for non-directly
connected neighbors
• Parameters
• Session Keepalive = 60 sec. & Hold time = 180 Sec.
• Discover Hello interval = 5 sec. and Hold Time = 15 sec.
• Can be viewed using the command show mpls ldp parameters
Adjacency Requirements
 LDP Router-ID must have a specific routing entry in the RIB
 Authentication parameters must match
 Multiple L3 links between LDP devices
LDP Neighborship Negotiation
Verifying LDP Neighborship
PE1#sh mpls ldp neighbor
Peer LDP Ident: 10.13.1.101:0; Local LDP Ident 10.13.1.61:0
TCP connection: 10.13.1.101.11031 - 10.13.1.61.646
State: Oper; Msgs sent/rcvd: 58/60; Downstream
Up time: 00:39:27
LDP discovery sources:
Ethernet0/0, Src IP addr: 10.13.1.5
Ethernet1/0, Src IP addr: 10.13.1.9
Addresses bound to peer LDP Ident:
10.13.1.9 10.13.1.5 10.13.2.5 10.13.1.101
PE1#show tcp brief| i 646

43ABB020 10.13.1.101.11031 10.13.1.61.646 ESTAB
PE1#
Reachability and ACL verification
• Ensure reachability between the LDP router ID’s
PE1#ping 192.168.11.11 source lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.11.11, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
..... Check Routing
Success rate is 0 percent (0/5) Configuration
• Verify no ACL in path blocking TCP port 646 and other Multicast
traffic for LDP Hello’s.
PE1#telnet 192.168.11.11 646 /source-interface lo0

Trying 192.168.11.11, 646 ... Verify ACLs in the path or
% Destination unreachable; gateway or host down on the routers itself
LDP Router-id
• If router-id is not set manually, router checks all operational
interfaces on the router(including loopbacks) and chooses the
highest IP address as the LDP router-id.
• LDP_ID should be hardcoded via
• “mpls ldp router-ID <interface>”
• The above configuration will not help unless:
• <interface> is UP when LDP gets started
• Existing LDP_ID (usually an interface) is shut
• Following avoids both shortcomings

• “mpls ldp router-ID <interface> force”
Troubleshooting LDP issues
Verifying LDP Connection
 “show mpls ldp discovery [detail]”
• Must show xmit/recv on LDP enabled interface
PE1#show mpls ldp discovery

Local LDP Identifier:
192.168.1.1:0 Local LDP_ID Xmited and
Recvd Hellos
Discovery Sources: on that
Interfaces: interface
GigabitEthernet0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0
Discovered
Neighbors’ LDP_ID
Problem with xmit / recv
Lo0=192.168.1.1 Lo0=192.168.11.11
PE1 P1
PE1#show mpls ldp discovery P1#show mpls ldp discovery

Local LDP Identifier: Local LDP Identifier:
192.168.1.1:0 192.168.11.11:0
Discovery Sources: Discovery Sources:
Interfaces: Interfaces:
GigabitEthernet0/1 (ldp): xmit GigabitEthernet0/1 (tdp): xmit
R1#debug mpls ldp transport connections

07:00:06.106: ldp: Scan listening TCBs
07:01:06.106: ldp: Scan listening TCBs Label Protocol
07:02:06.106: ldp: Scan listening TCBs is TDP
PE1(config-if)#mpls label protocol ldp
LDP No Route Problem
Lo0=192.168.1.1 Lo0=192.168.11.11
PE1 P1
PE1#show mpls ldp discovery P1#show mpls ldp discovery

192.168.1.1:0 192.168.11.11:0
Discovery Sources: Discovery Sources:
Interfaces: Interfaces:
Gi0/1 (ldp): xmit/recv Gi0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0; no route LDP Id: 192.168.1.1:0
PE1#show ip route 192.168.11.11

% Network not in table
Problem: Default route towards the peering router
Problem due to Summarization
PE1 P1
PE1#show mpls ldp neighbor 192.168.11.11 PE2#sh mpls ldp neighbor 192.168.1.1
PE1#show mpls ldp discovery PE2#show mpls ldp discovery

192.168.1.1:0 192.168.11.11:0
GigabitEthernet0/1 (ldp): xmit/recv GigabitEthernet0/1 (ldp): xmit/recv
LDP Id: 192.168.11.11:0 LDP Id: 192.168.1.1:0
PE1#show ip route 192.168.11.11 PE2#show ip route 192.168.1.1
Routing entry for 192.168.11.11/32 Routing entry for 192.168.1.0/24
Known via "ospf 100", distance 110, metric 2, type Known via "bgp 100", distance 200, metric 0
intra area Tag 1, type internal
Last update from 10.1.111.11 on Gi0/1, 00:04:34 ago Last update from 192.168.1.12 20:10:38 ago
Routing Descriptor Blocks: Routing Descriptor Blocks:
* 10.1.111.11, from 192.168.11.11, 00:04:34 ago, * 192.168.1.12, from 192.168.12.12, 20:10:38
via GigabitEthernet0/1 ago
Route metric is 2, traffic share count is 1 Route metric is 0, traffic share count is 1
AS Hops 5
Troubleshooting LDP Issues Also good to check “show
mpls ldp trace discovery”
MPLS LDP Trace on IOS XR
RP/0/0/CPU0:PE2#show mpls ldp trace peer last 20
0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9548, event=0, state 0 ->
1
0/0/CPU0 t1 [PEER]:581: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'TCP connection closed'
0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'TCP connection closed' ('Success')
0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg
0/0/CPU0 t1 [PEER]:506: VRF(0x60000000): Peer(192.168.11.11:0): Peer FSM: Stepped, pp=0x102d9520, event=0, state 0 ->
1
0/0/CPU0 t1 [PEER]:575: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'Received Notification message from peer'
(more_info 'KeepAlive Timer Expired')
0/0/CPU0 t1 [PEER]:3262: VRF(0x60000000): Release Peer(192.168.11.11:0): rsn 'Received Notification message from peer'
('KeepAlive Timer Expired')
0/0/CPU0 t1 [PEER]:3625: Peer(192.168.11.11:0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg
LDP & IGP Sync
• When a link comes up, LDP and IGP compete to converge; Labeled
traffic drops if IGP wins.
• When LDP session on a link drops, IGP may continue forwarding
labeled traffic to that link and cause traffic dropped.
LDP & IGP Sync – Solution
• Link up:
• If LDP peer is reachable (alternate route exists), defer IGP adjacency on
the link.
• If LDP peer is not reachable (no alternate route), IGP advertise max-
metric to reach neighbor through the link.
• LDP session down:
• IGP advertises max-metric to reach neighbor through the link.
LDP & IGP Sync
• LDP IGP Sync feature is enabled under IGP (OSPF/ISIS)
• - “sync-igp-shortcuts” for TE tunnel interfaces, “sync” for all other types.
router (config-isis-if-af) # mpls ldp sync [ level <1-2> ]
router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if)
router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar)
LDP & IGP Sync
 LDP IGP Sync delays are configured under LDP
router (config-ldp) # igp sync delay on-session-up <sec>
router (config-ldp) # igp sync delay on-proc-restart <sec>
LDP Session Protection
• Problem:
I. When a link flaps (for a short time),
II. LDP hello adjacency over the link flaps
III. LDP session is torn down then re-setup
IV. LDP re-exchanges label bindings when LDP session is setup (i.e. LDP re-
convergence).
• Solution:
• When LDP session supported by link hello is setup, create a targeted hello to protect
the session.
• When link is down, the targeted hello remains through other path and keeps the LDP
session up.
• When link restores, re-discover neighbors, re-program forwarding.
LDP Session Protection
router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> | infinite } ]
router (config-ldp) # log session-protection
Case Study - 1
IP RAN
10.12.2.0/24
• 3 routing processes
between R1 and R2 192.168.1.1 192.168.2.2
• Lo0 defined as the LDP 10.12.1.0/24
router-id on both routers R2

R1
• LDP adjacency is formed

just across one link, down CORE
on other two 10.12.3.0/24
Troubleshooting MPLS LSP
Looking at the LIB
RTR#show mpls ldp bindings detail
tib entry: 10.1.1.0/30, rev 10
local binding: tag: imp-null
Advertised to:
10.1.2.2:0 10.1.2.6:0 10.1.2.4:0
remote binding: tsr: 10.1.2.2:0, tag: imp-null
remote binding: tsr: 10.1.2.6:0, tag: 12304
Label Forwarding Information Base (LFIB)
• The LFIB stores local and remote labels for prefixes that are used to
forward packets
• Prefixes that are used = prefixes in routing table (RIB)
• Labels are derived from LIB
LDP TDP
prefix, next-hop and in-
label, out-label prefix + next-hop
LIB LFIB RIB
(prefix, LDP Ident, get in- and out-label for (prefix,next-hop, (prefix, next-hop)
label) (prefix, next-hop) in-label, out-label)
Case Study - 1
• Verify the TCP connection – You will find the clue
• Router-ID is configured with Lo0 (forced)
• If one of the interfaces is configured with mpls ldp discovery
transport-address interface, then this behavior can be
noticed.
Troubleshooting
LSP Issues
Reasons for LSP to Break
MP-IBGP – VPNv4
LDP + IGP
172.16.11.0/24 10.1.111.0/24 10.1.211.0/24 172.16.22.0/24
CE1 PE1 P1 PE2 CE2

Lo0=172.16.1.1/32 192.168.1.1/32 192.168.11.11/32 192.168.2.2/32 Lo0=172.16.2.2/32
• Broken LDP adjacency
• MPLS not enabled
• Mismatch labels
• Software/hardware corruption
Label Information Base (LIB)
• LIB stores local and remote bindings
• Local Binding:
• Prefix in own routing table + local label
• One binding
• Remote Binding:
• Prefix + remote label received from LDP neighbor
• Holds LDP router-id
• One binding per LDP neighbor
• LIB stores all labels from all LDP (BGP) neighbors, even the ones that are not
used for packet forwarding (now)
Looking at the LIB
RTR#show mpls ldp bindings detail
tib entry: 10.1.1.0/30, rev 10
local binding: tag: imp-null
Advertised to:
10.1.2.2:0 10.1.2.6:0 10.1.2.4:0
remote binding: tsr: 10.1.2.2:0, tag: imp-null
Label Forwarding Information Base (LFIB)
• The LFIB stores local and remote labels for prefixes that are used to
forward packets
• Prefixes that are used = prefixes in routing table (RIB)
• Labels are derived from LIB
LDP TDP
prefix, next-hop and in-
label, out-label prefix + next-hop
LIB LFIB RIB
(prefix, LDP Ident, get in- and out-label for (prefix,next-hop, (prefix, next-hop)
label) (prefix, next-hop) in-label, out-label)
Building the LFIB
P1#show ip route 3.3.3.4
Routing entry for 3.3.3.4/32
* 10.1.2.1, from 10.1.2.1, 13:28:32 ago, via Ethernet0/0
P1#show mpls ldp neighbor 10.1.2.1
Peer LDP Ident: 3.3.3.3:0; Local LDP Ident 2.2.2.2:0
P1#show mpls ldp binding 3.3.3.4 255.255.255.255
lib entry: 3.3.3.4/32, rev 18
remote binding: lsr: 3.3.3.3:0, label: imp-null
P1#show mpls forwarding 3.3.3.4
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
20 Pop Label 3.3.3.4/32 0 Et0/0 10.1.2.1
MPLS OAM
• Defined in RFC 4379
• LSP Ping and Traceroute provide ability to monitor MPLS Label Switched Paths and
quickly isolate MPLS forwarding problems.
• Two messages
• MPLS Echo Request:
MPLS labeled IPv4 or IPv6 UDP packet
• MPLS Echo Reply IPv4 or IPv6 UDP packet
• Ping mode: Connectivity check of an LSP

• Test if a particular “FEC” ends at the correct egress LSR
• Traceroute mode: Hop by Hop fault localization

• Packet follows data path
FEC Types Supported
• ping mpls ?
ipv4 Target specified as an IPv4 address
pseudowire Target VC specified as an IPv4 address and VC ID
traffic-eng Target specified as TE tunnel interface
• traceroute mpls ?
ipv4 Target specified as an IPv4 address
multipath LSP Multipath Traceroute
pseudowire Target VC specified as an IPv4 address and VC ID
traffic-eng Target specified as TE tunnel interface
LSP Ping (ping mpls . . . )
• Simple and efficient mechanism to detect data plane failures in MPLS LSPs
• Verify data plane against the control plane

• Sending “echo request” and receiving “echo reply”
• Verify that packets belonging to a FEC exit the LSP on the correct egress
LSR
• Modelled after the well known IP ping and traceroute
• Ping verifies connectivity, traceroute verifies path
• LSP Ping/trace leave the LSR with the correct label stack for the LSP to be
tested
Packet Format
Version Number Must Be Zero
Message Type Reply Mode Return Code Return Subcode
Sender’s Handle
Sequence Number
Timestamp Sent (seconds)
Timestamp Sent (microseconds)
Timestamp Received (seconds)
Timestamp Received (microseconds)
TLV …
Packet Format
• Version number: 1
• Message Type
• MPLS Echo Request
• MPLS Echo Reply
• Reply Mode
1 Do not reply
2 Reply via an IPv4/IPv6 UDP packet
3 Reply via an IPv4/IPv6 UDP packet with Router Alert
4 Reply via application level control channel
• Timestamp
• Time-of-day in seconds and microseconds
Reply Modes
• Reply Mode – Do Not Reply
• This mode is useful for a keepalive application running at the
remote end
• Such an application would trigger state changes if it does not
receive
a LSP ping packet within a predefined time
• An MPLS echo request with “do not reply” may also be used by the
receiving router to log gaps in the sequence numbers and/or
maintain delay/jitter statistics
Reply Modes
• Reply Mode – Reply via an IPv4 UDP Packet
• The Reply via UDP packet implies that an IP V4 UDP packet should
be sent in reply to an MPLS echo request
• This will be the most common reply mode for simple LSP pings
sent to periodically poll the integrity of an LSP
• This is the default reply mode
Reply Modes
• Reply Mode – Reply via an IPv4 UDP Packet with Router Alert
• In this mode when the destination router replies it appends a label
of “1” to the packet
• This forces all the intermediate routers, on the way back, to
process switch the reply
• This mode is CPU intensive and should generally be used if the
reply fails for “reply with IPv4 UDP packet”
• This mode is useful when we have inconsistency between IP and
MPLS
Return Codes
Value Meaning
0 The Error Code Is Contained in the Error Code TLV
1 Malformed Echo Request Received
2 One Or More of the TLVs Was Not Understood
3 Replying Router Is an Egress for the FEC
4 Replying Router Has No Mapping for the FEC
5 Replying Router Is Not One of the “Downstream Routers”
Replying Router Is one of the “Downstream Routers”, and Its Mapping for this FEC on the
6 Received Interface Is the Given Label
MPLS Echo Request
R1#ping mpls ipv4 192.168.2.2/32 verbose
destination 127.0.0.2 repeat 1 exp 7 pad 0xFFFF
Sending 1, 100-byte MPLS Echos to 10.200.254.4/32,
timeout is 2 seconds, send interval is 0 msec:
Codes: '!' - success, 'Q' - request not transmitted,
'.' - timeout, 'U' - unreachable,
'R' - downstream router but not target
! Reply address 10.1.211.2, return code 3
MPLS Ping (Operational Theory)
• We use the same label stack as used by the LSP and this makes the echo
to be switched inband of LSP
• The IP header destination address field of the echo request is a 127/8
address
• An Echo reply, which may or may not be labelled, has the egress interface
IP address as the source; destination IP address/port are copied from the
echo-request’s source address/port
• Presence of the 127/8 address in the IP header destination address field
causes the packet to be consumed by any routers trying to forward the
packet using the ip header
• In this case P1 would not forward the echo-req to PE1 but rather
consumes the packet and sends a reply to PE2 accordingly
MPLS Ping Packet Capture
Operation
MPLS OAM Caveats
• For LSP ping we generate an MPLS echo request
• The payload includes the LDP/RSVP/L2 Circuit sub-TLV depending
on the LSP we use
• Echo request is appropriately labelled and sent out
• Ping mode: MPLS TTL = 255
• Traceroute mode: TTL = 1, 2 ,3 etc.
• MPLS Echo Request always has FEC Stack TLV
• The LSP ping sender sets the return code to 0.
• The replying router would set it accordingly based on the table
shown previously
TTL Field in Labels
• Only the TTL field in the label at the top of the stack counts
• The outgoing TTL value is only a function of the incoming TTL value
• Outgoing TTL is one less than incoming TTL
• If outgoing TTL = 0, packet is not forwarded (not even stripped and
forwarded as an IP packet)
• When an IP packet is first labelled, the TTL field is copied from the IP
header to the MPLS header (after being decremented by 1)
• When the label stack is removed, the outgoing TTL value is copied to the
TTL field in the IP header
• Unless MPLS TTL > IP TTL
Operation
• Receiving LSR checks that label stack of received packet matches
with the received FECs in FEC Stack
• MPLS Echo Reply is sent in response to MPLS Echo Request
– Destination IP address is source IP address of Echo Request
– IP TTL = 255
– Reply Mode: (You do not control if return packet is sent over IP or MPLS)
• IPv4
• IPv4 with Router Alert (IP Option)
– If over MPLS, then Router Alert Label as topmost label is added in the label stack
– Hardware forwarding bypassed; packet is sent to RP process level forwarding
Traceroute in MPLS Network
In Label Prefix Output Out Label In Label Prefix Output Out Label
Interface Interface
- 172.16.2.2/32 Y 19 24008 24008 172.16.2.2/32 Y -
16 172.16.1.1/32 X - - 172.16.1.1/32 X 22 16
Y Y
PE1 X P1 X PE2
192.168.1.1/32 192.168.2.2/32
In Label Prefix Output Out Label

Interface
CE1 22 192.168.1.1/32 X pop CE2

172.16.1.1/32 19 192.168.2.2/32 Y pop 172.16.2.2/32
Traceroute in MPLS Network
Label 19, Aggregate Outgoing
TTL=1 Label, IP Lookup
done in CEF for VRF
Label 24008 Label 24008,
TTL=255
172.16.2.2 172.16.2.2
TTL=2 TTL=1 172.16.2.2
UDP port UDP port TTL=255, ICMP
35678 35678 TTL Exceeded
CE1 PE1 P1 PE2 CE2

172.16.1.1/32 192.168.1.1/32 192.168.2.2/32 172.16.2.2/32
Label 22, TTL=254
172.16.1.1 TTL=252 Label 16, TTL=253 Label 16

ICMP TTL Exceeded
172.16.1.1 TTL=254 172.16.1.1 TTL=254
ICMP TTL Exceeded ICMP TTL Exceeded
MPLS Trace
• The ICMP messages “TTL exceeded” are forwarded along the LSP
until the end of the LSP. So, the router does not lookup the source
ip address in the global routing table to return the ICMP message.
• Reason : P routers do not have knowledge of VPN prefixes : all
traceroutes initiated from within a VPN would fail
• ICMP messages are forwarded with EXP bits = 6
MPLS Trace Hiding
• This command prohibits the copying of the TTL from the IP header to
the MPLS shim header and vice versa (TTL is set to 255)
• It should be configured on the routers that do the label imposement
(LSR edge routers), which is the PE routers.
• Providers like to use it so that the customers see the MPLS network
as one hop when tracerouting
no mpls ip propagate-ttl forwarded
MPLS Trace Hiding
CE1#traceroute 172.16.2.2 source 172.16.1.1 (mpls ip propagate-ttl forwarded)
Tracing the route to 172.16.2.2
1 172.16.11.2 [AS 100] 3 msec 3 msec 3 msec local PE
2 10.1.111.11 [MPLS: Labels 19/24008 Exp 0] 122 msec 25 msec 19 msec P
3 10.1.211.2 [MPLS: Label 24008 Exp 0] 21 msec 16 msec 23 msec remote PE
4 172.16.12.1 [AS 100] 23 msec * 22 msec remote CE
CE1#traceroute 172.16.2.2 source 172.16.1.1 (no mpls ip propagate-ttl forwarded)

VRF info: (vrf in name/id, vrf out name/id)
1 172.16.11.2 [AS 100] 4 msec 3 msec 3 msec local PE
2 10.1.211.2 [MPLS: Label 24008 Exp 0] 25 msec 25 msec 31 msec remote PE
3 172.16.12.1 [AS 100] 24 msec * 28 msec remote CE
MPLS Trace with no mpls ip propagate-ttl on PE routers
Aggregate Outgoing
Label 19, TTL=1 Label
udp port
Label 24008 Label 24008, 35678?
TTL=255
172.16.2.2 172.16.2.2 172.16.2.2 172.16.2.2
TTL=2 TTL=1 TTL=1 TTL=1
UDP port 35678 UDP port 35678 UDP port 35678 UDP port 35678
CE1 PE1 P1 PE2 CE2

172.16.1.1/32 172.16.2.2/32
Label 22, TTL=255
172.16.1.1 TTL=254, 172.16.1.1
ICMP TTL=255, ICMP
Port Unreachable Label 16, TTL=254 Label 16 Port Unreachable
172.16.1.1 TTL=254, 172.16.1.1 TTL=254,

ICMP ICMP
Port Unreachable Port Unreachable
Multipath MPLS Trace
• MPLS LSP ping / trace is useful tool to validate the health of a label
switched path
• In case of multiple paths, LSP ping may not serve useful to validate
all the available paths
• Multipath MPLS trace allows users to identify all LSP failures
• The multipath LSP trace, sends probe by setting the destination to
loopback address (127.x.x.x), which can help detect failure in LSP
by avoiding the packet to get IP routed.
192.168.2.2/32
Echo Request
1
SRC – 10.1.16.6
DEST – 127.0.0.0 1 R2
192.168.6.6/32 192.168.1.1/32 192.168.4.4/32
R6 R1 R4
2
192.168.3.3/32
Echo Reply
SRC – 10.1.16.1
2 DEST – 10.1.16.6
DS Mapping – 127.0.0.1
24002 - 10.1.13.3
DS Mapping – 127.0.0.0 R3
30002 - 10.1.12.2
192.168.2.2/32
Echo Request
3
SRC – 10.1.16.6
DEST – 127.0.0.0 4
R2
192.168.6.6/32 3 192.168.1.1/32 192.168.4.4/32
R6 R1 R4
Echo Reply
SRC – 10.1.12.2
192.168.3.3/32
4 DEST – 10.1.16.6
pop - 10.1.24.4
R3
Multipath MPLS Trace 192.168.2.2/32
Echo Request
5 R2
SRC – 10.1.16.6
DEST – 127.0.0.1
192.168.6.6/32 192.168.1.1/32 192.168.4.4/32
R6 5 R1 R4
192.168.3.3/32
Echo Reply 6
SRC – 10.1.13.3
6 DEST – 10.1.16.6
R3
pop - 10.1.34.4
PE1#traceroute mpls multipath ipv4 192.168.4.4/32
Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
<snip>
LL!
Path 0 found,
output interface Gi0/1 nexthop 10.1.16.1
source 10.1.16.6 destination 127.0.0.1
0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0
L 1 10.1.16.1 10.1.12.2 MRU 1500 [Labels: 30002 Exp: 0] ret code 8 multipaths 2
L 2 10.1.12.2 10.1.24.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1
! 3 10.1.24.4, ret code 3 multipaths 0
L!
Path 1 found,
output interface Gi0/1 nexthop 10.1.16.1
source 10.1.16.6 destination 127.0.0.0
0 10.1.16.6 10.1.16.1 MRU 1500 [Labels: 18 Exp: 0] multipaths 0
L 1 10.1.16.1 10.1.13.3 MRU 1500 [Labels: 24002 Exp: 0] ret code 8 multipaths 2
L 2 10.1.13.3 10.1.34.4 MRU 1500 [Labels: implicit-null Exp: 0] ret code 8 multipaths 1
! 3 10.1.34.4, ret code 3 multipaths 0
Paths (found/broken/unexplored) (2/0/0)
Echo Request (sent/fail) (5/0)
Echo Reply (received/timeout) (5/0)
Total Time Elapsed 192 ms
Demo - Multipath MPLS Trace
MPLS Forwarding Plane
 With MPLS, the idea is to de-couple the forwarding from the IP
header
 The forwarding decision is based on the MPLS header, not the IP
header
 The above is true once the packet is inside the MPLS network
 Forwarding is still based on the IP header at the edge where the
packet first enters the MPLS network
 CEF must be configured on all the routers in a MPLS network.
 CEF takes care of the crucial “recursion” and “resolution” operations
What happens when CEF disabled?
PE1#show mpls forwarding-table
16 No Label 172.16.1.1/32 0 drop
17 No Label 192.168.12.12/32 0 drop
20 No Label 192.168.2.2/32 0 drop
21 No Label 10.1.212.0/24 0 drop
22 No Label 10.1.211.0/24 0 drop
23 No Label 192.168.11.11/32 0 drop
24 No Label 172.16.11.0/24 0 drop
25 No Label 172.16.14.0/24 0 drop
MPLS Forwarding Plane – Outgoing Labels
PE1#show mpls forwarding-table 192.168.2.2

Local Outgoing Prefix Bytes Label Outgoing NextHop
20 19 192.168.2.2/32 0 Gi0/1 10.1.111.11
PE1#
• Outgoing label also conveys what treatment the packet is going

to get. It could also be:
I. Pop - Pops the topmost label
II. Untagged - Untag the incoming MPLS packet
III. Aggregate - Untag and then do a FIB lookup
 Label values 0-15 are reserved.
MPLS Forwarding Plane: Outgoing Labels
PE1#sh mpls forwarding-table
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
16 2002 10.13.1.22/32 0 Et0/0 10.13.1.5
2002 10.13.1.22/32 0 Et1/0 10.13.1.9
18 Pop tag 10.13.1.101/32 0 Et1/0 10.13.1.9
Pop tag 10.13.1.101/32 0 Et0/0 10.13.1.5
19 Pop tag 10.13.2.4/30 0 Et1/0 10.13.1.9
Pop tag 10.13.2.4/30 0 Et0/0 10.13.1.5
20 Untagged 5.5.5.5/32[V] 0 Se2/0 point2point
21 Pop tag 10.13.21.4/30 0 Et1/0 10.13.1.9
Pop tag 10.13.21.4/30 0 Et0/0 10.13.1.5
24 Aggregate 200.1.61.4/30[V] 0
26 Untagged 30.30.30.1/32[V] 0 Se2/0 point2point
PE1#
MPLS Forwarding Plane: Outgoing Labels
 Untagged
• Convert the incoming MPLS packet to an IP packet and forward it.
 Pop
• Pop the top label from the label stack present in an incoming MPLS
packet and forward it as an MPLS packet.
• If there was only one label in the stack, then forward it as an IP packet.
SAME as imp-null label.
 Aggregate
• Convert the incoming MPLS packet to an IP packet and then do a FIB
lookup for it to find out the outgoing interface.
MPLS Forwarding Plane - Lookup
 Three cases in the MPLS forwarding:
1) Label Imposition - IP to MPLS conversion
2) Label swapping - MPLS to MPLS
3) Label disposition - MPLS to IP conversion
 So, depending upon the case, we need to check:

1) FIB - For IP packets that get forwarded as MPLS
2) LFIB - For MPLS packets that get forwarded as MPLS
3) LFIB - For MPLS packets that get forwarded as IP
MPLS Forwarding Plane: Loadsharing
 MPLS Loadsharing (due to multiple paths to a prefix) is no different
from that of IP
 Hashing-algorithm is still the typical ‘FIB based’ i.e per-dest
loadsharing by default **
 So the “show commands” are still relevant
• “Show ip cef exact-route <source> <dest>” etc.
 But the <dest> must be known in the FIB table, otherwise the
command won’t work.
• Won’t work on P routers for the VPN prefixes.
MPLS Forwarding Plane: MTU Setting
• “mpls mtu <bytes>” can be applied to an interface to change the MPLS
MTU size on the interface
• MPLS MTU size is checked by the router
• while converting an IP packet into a labeled packet or transmitting a labelled
packet
• Label imposition(s) increases the packet size by 4 bytes/label, hence the
outgoing packet size may exceed ‘interface MTU’ size, hence the need to
tune MTU
• ‘mpls mtu <bytes>” command has no effect on “interface or IP MTU” size.
• By default, MPLS MTU = interface MTU
• MPLS MTU setting doesn’t affect MTU handling for IP-to-IP packet switching
MPLS Forwarding Plane: MTU Setting
• If
the label imposition makes the packet bigger than
the MPLS MTU size of an outgoing interface, then:
- If the DF bit set, then discard the packet and send ICMP
reply back (with code=4)
- If the DF bit is not set, then fragment the IP packet (say,
into 2 packets), and then impose the same label(s) on both
the packets, and then transmit MPLS packets
MPLS Forwarding Plane: Show Commands
 “show mpls forwarding”
• Shows all LFIB entries (vpn, non-vpn, TE etc.)
 “show mpls forwarding <prefix>”
 LFIB lookup based on a prefix
 “show mpls forwaring label <label>”
 LFIB lookup based on an incoming label
 “show mpls forwarding <prefix> detail”
 Shows detailed info such as L2 encap etc
MPLS Forwarding Plane: Show Commands
R2#show mpls forwarding 10.13.1.11 detail
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
45 51 10.13.1.11/32 0 Fa1/1/1 10.13.7.33
MAC/Encaps=14/18, MRU=1500, Tag Stack{51}
0003FD1C828100044E7548298847 00033000
No output feature configured
Per-packet load-sharing
R2#
14/18 means that the L2 header is of 14 bytes, but

L2+label header is 18 bytes (one label is 4 bytes)
MPLS Labels Case
Study
BGP 3107
Mobile Transport Market Conditions
• High Capacity requirements from Edge to Core
• 100Mbps eNB, 1Gbps Access, 10Gbps Aggregation, 100Gbps Core
• Higher scale as LTE drives ubiquitous mobile broadband
• Tens- to hundred-of-thousands of LTE eNBs and associated CSGs
• Support for multiple and mixed topologies
• Fiber and microwave rings in access, fiber rings, hub and spoke in aggregation and core
networks
• Need for graceful service integration and integration into existing infrastructure
• Need to support transport for all services from all locations
• Optimized operations with consistent packet transport
MPLS as Network Convergence Technology
Optimizing Service Delivery
Access Aggregation Edge Core
Cross-Domain Convergence IP/MPLS
LS Challenges with differing Access technologies

• Complexity of achieving 50 millisecond convergence with TE-FRR
• Splitting large networks into domains while delivering services end-to-end
• Common end-to-end convergence and resiliency mechanisms
• End-to-end provisioning and troubleshooting across multiple domain
Unified MPLS addresses these challenges

with elegant simplicity and scale
Seamless MPLS Overview
• An efficient MPLS transport architecture
• Virtualized to support many services on one infrastructure
• Relying on an intelligent hierarchy to scale to new challenges
• Enabling seamless operation for network and service resilience
• Separating transport from service operations with single touch point service
enablement and contiguous OAM
• Integrating alternate access technologies on same infrastructure while still

enabling Fixed and Mobile Services
Seamless MPLS Operation
Transport & Service Decoupling
Operational Points
LER LSR LER

Access AGG AGG MPLS AGG AGG Access
MPLS Unified MPLS MPLS
Typically, a service has to be configured on every network element via operational points. The
management system has to know the topology.
• Goal is to minimize the number of operational points

• Only with the integration of all MPLS islands, the minimum number of operational points is
possible
Service provisioning only at the Edge

Unified MPLS = Classical MPLS with a few additions
Classical
MPLS
IGP/LDP
Domain RFC BGP Flex LFA BGP E2E
isolation 3107 filtering Access R-LFA PIC OAM
L2/IGP/BGP/MPLS-
TP/LDP DoD
Unified
MPLS
Architecture
Scalability Security Simplification Multi-Service
RFC-3107
• RFC 3107 was approved May 2001, main purpose being scaling of MPLS
• RFC 3107 is BGP IPv4 with the ability to distribute labels
• BGP Filtering supported via BGP Communities in a secure manner
RFC 3107 basis:

• BGP can be used to distribute MPLS labels in the same way it can distribute a route
• The label mapping information for a particular route is piggybacked in the same
BGP Update message that is used to distribute the route itself.
• If two immediately adjacent Label Switched Routers (LSRs) are also BGP peers, then label
distribution can be done without the need for any other label distribution protocol.
LFA & R-LFA
• What is LFA FRR?
• RFC 5286 basic fast re-route mechanism with local protection in pure IP and MPLS/LDP
networks
• Pre-computing available paths at source node that do not create loops
• Gives benefits of TE-FRR, but no configuration or design required
• What is Remote LFA?
 Defined in draft “http://tools.ietf.org/html/draft-shand-remote-lfa”
 Remote LFA uses automated IGP/LDP behavior to extend basic LFA FRR to arbitrary
topologies
 A node dynamically computes its remote loop free alternate node(s)
– Done during SFP calculations using PQ algorithm (see draft)
 Automatically establishes a directed LDP session to it
– The directed LDP session is used to exchange labels for the FEC in question
 On failure, the node uses label stacking to tunnel traffic to the Remote LFA node, which in
turn forwards it to the destination
Remote LFA FRR - Protection
• C2’s LIB
• C1’s label for FEC A1 = 20 Backbone
• C3’s label for FEC C5 = 99

A1 A2
• C5’s label for FEC A1 = 21
• On failure, C2 sends A1-destined traffic onto an LSP C1 Directed LDP C5

E1
destined to C5 20 session
21
• Swap per-prefix label 20 with 21 that is expected by C5 for C2 21 C4

that prefix, and push label 99
99 C3
• When C5 receives the traffic, the top label 21 is the 21 X
one that it expects for that prefix and hence it 21 99

Access Region
forwards it onto the destination using the shortest-
path avoiding the link C1-C2.
BGP Prefix-Independent Protection (PIC)/BGP FRR
• BGP Fast Reroute (BGP FRR)
enables BGP to use alternate paths
• Algorithm uses a pointer to move all
prefixes to new next hop, not a hop by
hop rewrite
• ~ 100 msec protection
• Prefix-Independent
• Default behavior, entirely automated

computation
• Enables 3107 BGP+labels operation to
scale via hierarchy while maintaining fast
convergence characteristics
Unified MPLS Architecture Models
• Architecture Models based on:
• Access Type: Ethernet TDM or MPLS access
• Network Size: Small/Medium (1000 nodes or less) or Large
• End to Labeled Switch Path
Deployment Network Size Access Type Core/Aggregation LSP

Model
1 Small/Medium Ethernet/TDM Flat LDP
2 Small/Medium MPLS Hierarchical Labeled BGP
3 Large Ethernet Hierarchical Labeled BGP
4 Large MPLS Hierarchical Labeled BGP for Core,
Aggregation and Access
5 Large MPLS Hierarchical Labeled BGP for Core,
Aggregation with redistribution in
Access
1 – Small Network: Ethernet/TDM Access
Flat LDP LSP across Core and Aggregation Networks
Mobile
Core Node Transport GW Core Node
Aggregation Node Aggregation Node CSG
IP/Ethernet
Aggregation Core and Aggregation Pre-Aggregation
Node Node Business
Distributio IP/MPLS Domain
n Node
Aggregation Node Mobile Aggregation Node
TDM and Packet Fiber and Microwave
Microwave, 2G/3G/LTE 3G/LTE
IGP/LDP domain
• Core and Aggregation Networks form one IGP and LDP domain.
• Scale recommendation is less than 1000 IGP/LDP nodes
• Packet Microwave links aggregated in Aggregation Nodes
• Mobile Access is based on TDM
• All services –Mobile and Wireline– enabled by Aggregation Nodes
2 – Small Network: MPLS Access
Hierarchical BGP LSP Across Core + Aggregation and Access Networks
Aggregation Node Aggregation Node
Core Node Mobile Core Node
Transport GW CSG
CSG
RAN
RAN Core and Aggregation IP/MPLS Domain
IP/MPLS Domain Pre-Aggregation Pre-Aggregation
Node IP/MPLS domain Node CSG
CSG IGP Area
Mobile
Transport GW
Core Node Core Node CSG
CSG Aggregation Node
Aggregation Node
iBGP Hierarchical LSP
LDP LSP LDP LSP LDP LSP
• The Core and Aggregation form a relatively small IGP/LDP domain (1000 nodes)
• MPLS enabled RAN, each RAN forms a different IGP/LDP domain
• The Core/Aggregation and RAN Access Networks are integrated with labelled BGP LSP
• The Access Network Nodes learn only the MPC labelled BGP prefixes and selectively and optionally
the neighbouring RAN networks labelled BGP prefixes.
3 – Large Network: Ethernet/TDM access
Hierarchical BGP LSP Across Core Network and Aggregation Networks
Aggregation Node
Aggregation Node
Mobile
Transport GW CSG
Core Core
Aggregation Network Node Node Aggregation Network IP/Ethernet
IP/MPLS Core Network IP/MPLS
Domain IP/MPLS Domain Domain
Core CSG
Aggregation Core Pre-Aggregation
Node Node Mobile Node Node
Transport GW
Aggregation Node
TDM and Packet Aggregation Node Fiber and Microwave
Microwave, 2G/3G/LTE 3G/LTE
iBGP (eBGP across ASes) Hierarchical LSP
• Core and Aggregation Networks enable Unified MPLS Transport

• Core and Aggregation Networks are organized as independent IGP/LDP domains
• Core and Aggregation Networks may be in same or different Autonomous Systems
• The network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP
IPv4+labels
• No MPLS in Access Domain
• Aggregation Node enable Mobile and Wireline Services over Unified MPLS transport.
4 – Large Network: MPLS Access
Hierarchical BGP LSP Across Core, Aggregation and Access Networks
Aggregation Node
Aggregation Node
Mobile
Transport GW Core Node CSG
CSG Core Node
Core Core
Aggregation Network Node Aggregation Network RAN
RAN Node IP/MPLS
IP/MPLS IP/MPLS Core Network IP/MPLS domain
domain Domain IP/MPLS Domain Domain
Pre-Aggregation CSG
CSG Core Core Pre-Aggregation
Node Node Mobile Node Node
CSG
Aggregation Node
iBGP (eBGP across ASes) Hierarchical LSP
LDP LSP LDP LSP LDP LSP LDP LSP LDP LSP
• Core, Aggregation, Access Network enable Unified MPLS Transport

• Core, Aggregation, Access are organized as independent IGP/LDP domains
• Core and Aggregation Networks may be in same or different Autonomous Systems
• Network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP IPv4+labels.
• Intra domain connectivity is based on LDP LSPs
• The Access Network Nodes learn only the required labelled BGP FECs
5 - Large Network, MPLS Access
Hierarchical BGP LSP with IGP/LDP Redistribution in Access Network
Aggregation Node
Aggregation Node
Mobile
MPC iBGP community Transport GW Core
Core
Node MPC iBGP community CSG
CSG Core
Core
Node
Core Core
into RAN IGP into RAN IGP
RAN
Aggregation Network Node Node Aggregation Network
RAN
MPLS/IP IP/MPLS Core Network IP/MPLS MPLS/IP
Domain IP/MPLS Domain Domain
IGP Area/Process Pre-Aggregation Pre-Aggregation IGP Area/ProcessCSG
CSG Core Core Node
Node
Node Core Mobile Node
RAN IGP CSN Loopbacks Core
Core
Node RAN IGP CSN Loopbacks
Core Node Transport GW
into iBGP into iBGP
CSG
Aggregation Node
i/eBGP Hierarchical LSP
LDP LSP LDP LSP
• Core and Aggregation are distinct IGP/LDP domains that enable inter domain hierarchical LSPs
• Core and Aggregation Networks may be in same of different Autonomous Systems
• Redistribution of Core/Aggregation LSPs into Access Networks IGP
Unified MPLS Architecture
Summary
Cell Access Pre-Aggregation Aggregation PGW SGW
Core
Site Layer Layer Layer Layer
Simplified
Aggregation Distribution Core
Cell site MPLS Transport with E2E OAM, performance

node node node
management, provisioning with seamless resiliency

Router
Flexible L2 & L3 transport virtualisation to support GSM, 3G &

Sample Routing Architecture
iBGP/eBGP
Aggregation Node Core ABR EPC Gateway
Access Node
Node LTE, wholesale & retail options
Pre-Aggregation
Access Aggregation Network Core Network

Network
New levels of Scale for MPLS transport and optimal

Access Node Core ABR
routing
IGP/LDP Aggregation Node Centralised RR
IGP/LDP
IGP/LDP
L2
through RFC 3107 with BGP hierarchical LSPs
Demo –LDP
interop with BGP
3107
Troubleshooting
MPLS L3 VPNs
Troubleshooting MPLS L3 VPNs
Nodes and their Roles
• CE – Customer edge router, connects to the CE network and the PE
• Forwards only IP packets – no awareness of the MPLS network is needed
• Routes between the CE internal network and the PE router
• PE – Provider Edge router, connects to P and CE routers

• Maintains separate routing table per VRF
• Uses MP-BGP to exchange VRF routing information (RD + RT)
• Performs LFIB and FIB lookups, VPN label imposition and disposition
• P – Provider core router, connects to P and PE routers

• Does not need to run BGP with the PE’s
• Performs LFIB MPLS forwarding for outer label traffic (PE to PE)
IP Addressing Concerns
Customer A Customer A
Site 1 Site 2
172.16.1.0/24 172.16.3.0/24
CE1 CE3
PE1
172.16.2.0/24 172.16.4.0/24
CE2 CE4
Customer B Customer B
Site 1 Site 2
Isolation Through the Use of VRFs
Customer A Customer A
Site 1 Site 2
172.16.1.0/24 172.16.3.0/24
CE1 VRF VPN01 CE3
PE1
172.16.2.0/24 VRF VPN02 172.16.4.0/24

CE2 CE4
Customer B Customer B
Site 1 Site 2
L3VPN By Parts
The Edge:
• VRF = VPN Routing Forwarding instance
Isolated routing table, kind of like a VM
• Any routing protocol between the PE and CE
The Core:
• BGP VPNv4 and/or VPNv6 between PEs
• Labeled Switch Path between PEs
PE-CE Protocol PE-CE Protocol
MP-EBGP
CE PE CE
P2 PE
MP-BGP (Multiprotocol BGP) for MPLS VPNs
MP-BGP (Multi Protocol BGP)
• No new rules, still requires full mesh or RRs
• RRs need to support additional capabilities
• For MPLS only PEs need to speak BGP or know CE routes
• L3VPN Relies on Extended Communities
• Extended Communities are arbitrary TLVs attached to BGP prefixes
• BGP is used to Exchange the MPLS Label specific to the VPN prefix
• Outer MPLS Label is used to forward traffic between PEs
Visualizing Data Flow
MP-EBGP
CE PE CE
P2 PE
100 20
100.64.6.6
1. Packet is received on local PE

2. Remote VPN Label is assigned
3. Remote PE Label is assigned
Visualizing Data Flow
MP-EBGP
CE PE CE
P2 PE
100 20 100.64.6.6 100 20 100.64.6.6 20 100.64.6.6
1. The P router next to destination PE router POPs the outer

label
2. The packet is forwarded onto the Destination PE router
3. The VPN Label is examined and POP’d
4. The packet is forwarded out to the VRF
MP-BGP: Address-Families
• Address-family (AFI) “vpnv4”, “ipv4 unicast vrf” introduced
• vpnv4 AFI for PE to PE (label information)
• ipv4 unicast vrf for PE to CE
• Neighbor must be “activated” for each AFI supported
router bgp 100
neighbor 192.168.3.3 remote-as 100
!
address-family vpnv4
neighbor 192.168.3.3 activate Remote PE
neighbor 192.168.3.3 send-community
extended
!
address-family ipv4 unicast vrf red
neighbor 192.168.4.4 remote-as 400 Local CE
neighbor 192.168.4.4 activate
MP-BGP: Advertising CE Routes
 BGP maintains a table for each AFI (vpnv4, ipv4, vrf…)
 CE routes are placed into the vpnv4 BGP table
• BGP routes in a vrf AFI are automatically turned into vpnv4 routes
• If BGP is not PE-CE protocol routes must be redistributed into ipv4 vrf
AFI
 All vpnv4 routes get an assigned label
 vpnv4 routes are exchanged between vpnv4 peers (PEs)
RTs and RDs: Creating the VRF
• VRFs have 3 parts:
1. VRF name (case sensitive) ip vrf red
rd 100:100
2. Route Distinguisher (RD) route-target import 200:200
3. Route Target(s) (RT)
route-target export 201:201
• RD and RT are for MPLS; RD must always be defined

• RD must be unique to the VRFs on the local PE
vrf definition VPN01
rd 200:1
Locally Assigned Label
RD
Prefix
Route Target
Understanding RDs
• Route Distinguisher
• There is only one VPNv4 table
• How are routes distinguished from another?
• Prepending the RD to the route to creates a VPNv4 route
• Only used to make routes unique VPNv4 prefixes
IPv4 Route: 192.168.1.0/24
RD: 100:100
VPNv4 Route: 100:100:192.168.10/24
Let’s Investigate This Further
Understanding RDs
Route
Reflector
172.31.31.31
Understanding the RT
• Route Target
• RT is a BGP extended community (extra information on the update)
• “route-target export” adds the community to the outbound update
• “route-target import” defines which routes to bring into the VRF
• Multiple imports and exports allowed ip vrf red
rd 1:1
route-target import 100:100
route-target import 200:200

Let’s Investigate This Further route-target export 44:313
VPN Services
Understanding the RT
Fixing the BGP AS_Path Problem
• AS_Path is a loop prevention mechanism
• PE routers can use a special feature called AS-Override.
• Any prefixes with the same AS that the is used by the CE is changed
to the AS of the PE
R1 XR3
router bgp 200 router bgp 200
address-family ipv4 vrf VPN01 neighbor 172.32.36.6
redistribute connected remote-as 500
neighbor 172.16.15.5 remote-as 500 address-family ipv4 unicast
neighbor 172.16.15.5 activate route-policy PASSALL in
neighbor 172.16.15.5 as-override route-policy PASSALL out
as-override
Live
Troubleshooting
Demo
Route Reflectors
RR
AS500 AS500
CE5 PE1 CE6

P2 PE3
• CE5 cannot ping CE6

• IP Addressing is exactly the same as before.
• PE1 and PE3 now connect to a Route Reflector (192.168.10.10)
What do we do first and why?

Inter-AS MPLS
VPNs
Inter-AS MPLS VPNs
Flavors
• Previous section – VPNs within Single-AS boundary
• Inter-AS MPLS VPN – VPNs spanning across multiple AS
boundaries
• Types:
• Option 1 – Back to Back VRF
• Option 2 – Inter-Provider VPNs using ASBR-to-ASBR approach
A. Next-Hop-Self Method
B. Redistribute Connected Method
C. Multi-hop EBGP between ASBRs
• Option 3 – MP-EBGP between RR and EBGP between ASBR
Inter-AS MPLS VPNs
Option 1 - Back-to-Back VRF Method
VRF- ABCVRF- XYZ

RR-P1 RR-P2
AS100 AS200
PE1 IPv4 + IGP/BGP PE2
PE-ASBR1 PE-ASBR2
Lo0-11.11.11.11/32Lo0-22.22.22.22/32
CE1 CE2
Inter-AS MPLS VPNs
Option 2a – ASBR-to-ASBR with Next-Hop-Self Method
172.16.1.1 v1
RR-P1 RR-P2
AS100 MP-eBGP AS200

PE1 PE2
PE-ASBR1 PE-ASBR2
Lo0-11.11.11.11/32Lo0-22.22.22.22/32
neighbor x.x.x.x next-hop-self
CE1 CE2
172.16.1.1 172.16.2.2
• No LDP or IGP required on the link between the two ASBRs.
• Configure no bgp default route-target filter on ASBRs
Inter-AS MPLS VPNs
Option 2a – ASBR-to-ASBR with Next-Hop-Self Method
• Both ASBRs allocate VPN labels for prefixes received from the
other AS.
• When MP-eBGP peering is configured between ASBRs, below
configuration is done to complete LSP
• mpls bgp forwarding – on Cisco IOS devices
• no bgp default route-target filter configured on ASBR
not having VRF configured.
• Default behavior – deny vpnv4 prefixes that are not imported in any local
VRF
• On XR – retain route-target all
Inter-AS MPLS VPNs
Option 2b – ASBR-to-ASBR with Redistribute Connected Method
172.16.1.1 v1
RR-P1 RR-P2
AS100 MP-eBGP AS200

PE1 PE2
PE-ASBR1 PE-ASBR2
Lo0-11.11.11.11/32 Lo0-22.22.22.22/32
CE1 CE2
172.16.1.1 172.16.2.2
• No LDP or IGP required on the link between the two ASBRs.
Inter-AS MPLS VPNs
Option 2b – ASBR-to-ASBR with Redistribute Connected Method
• Redistribute the link between ASBR into IGP in local AS
• Required on both ASBR routers.
other AS.
• VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200.
• Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with
V2 and advertises it towards the core.
Inter-AS MPLS VPNs
Option 2c – ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method
172.16.1.1 v1
RR-P1 RR-P2
AS100 AS200
PE1 MP-eBGP PE2
PE-ASBR1 PE-ASBR2
Lo0-11.11.11.11/32Lo0-22.22.22.22/32
CE1 CE2
172.16.1.1 • Loopback to loopback peering between ASBRs 172.16.2.2
Inter-AS MPLS VPNs
Option 2c – ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method
• Loopback to loopback MP-EBGP peering between ASBRs.
• IGP or static route required between the ASBR link
other AS.
• VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200.
• Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with
V2 and advertises it towards the core.
Inter-AS MPLS VPNs
Option 3 – Multi-Hop MP-EBGP between RR and EBGP between ASBRs
MP-eBGP
RR-P1 RR-P2
eBGP +
AS100 Send-label AS200
PE1 PE2
PE-ASBR1 PE-ASBR2
Lo0-11.11.11.11/32Lo0-22.22.22.22/32
CE1 CE2
172.16.1.1 172.16.2.2
• Neighbor send-label required on eBGP peers on ASBR.

Inter-AS MPLS VPNs
Option 3 – Multi-Hop MP-EBGP between RR and EBGP between ASBRs
• RR & ASBR loopbacks are advertised via EBGP on ASBR
• The remote ASBR redistributes the received loopbacks into local IGP
• MP-EBGP peering configured between RR’s on each AS
• Configure neighbor next-hop-unchanged
Troubleshooting
IPv6 VPNs
Troubleshooting 6VPE
Reference Topology
IPv4 – 192.168.1.1/32
IPv6 – 2001:DB8::1/128
AS 100
Service Provider Core
PE1 IPv4 – IGP
MPLS
IPv4 – 192.168.2.2/32
IPv6 – 2001:DB8::2/128
IPv4 – 192.168.5.5/32
IPv6 – 2001:DB8::6/128 IPv6 – 2001:DB8::7/128
IPv6 – 2001:DB8::5/128
CE1 RR-P PE5 CE2

PE2
AS 200 IPv4 – 192.168.4.4/32 AS 300
VRF Configuration
• IPv6 enabled VRF’s are configured in the same way as IPv4 VRF’s
• On Cisco IOS, use command vrf definition to configure both IPv4 and IPv6 capable
VRF’s
vrf definition VPN01 vrf VPN01
rd 1:1 address-family ipv6 unicast
address-family ipv6 unicast import route-target
route-target import 1:1 1:1
route-target export 1:1 2:2
route-target import 2:2 export route-target
address-family ipv4 unicast 1:1
. . . address-family ipv4 unicast
interface Gi0/0 . . .
vrf forwarding VPN01 interface Gi0/0/0/0
ipv6 address xx:xx:xx::y/64 vrf VPN01
ipv6 address xx:xx:xx::y/64
6VPE Configuration – Cisco IOS
router bgp 100
bgp router-id 192.168.1.1
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 192.168.4.4 update-source Loopback0
!
neighbor 192.168.4.4 send-community extended
neighbor 192.168.4.4 next-hop-self
!
address-family ipv6 vrf red
neighbor 2001:DB8:0:16::6 remote-as 200
neighbor 2001:DB8:0:16::6 activate
exit-address-family
6VPE Configuration – IOS XR
router bgp 100
address-family vpnv6 unicast
!
neighbor 192.168.4.4
remote-as 100
update-source Loopback0
next-hop-self
!
vrf red
rd 100:1
address-family ipv6 unicast
!
neighbor 2001:db8:0:26::6
remote-as 200
route-policy pass in
route-policy pass out
Verifying Control Plane
• Since both control plane and data plane works in opposite
direction, verify the IPv6 VPN prefix on PE5.
PE5#show ipv6 route vrf red

! Output omitted for brevity
B 2001:DB8::6/128 [200/0]
via 192.168.1.1%default, indirectly connected
B 2001:DB8::7/128 [20/0]
via FE80::7, GigabitEthernet0/2
• Verify the VPNv6 prefix in BGP along with the local label
PE5#show bgp vpnv6 unicast vrf red 2001:db8::7/128

BGP routing table entry for [100:5]2001:DB8::7/128, version 38
Paths: (1 available, best #1, table red)
Advertised to update-groups:
2
Refresh Epoch 1
300
2001:DB8:0:57::7 (FE80::7) (via vrf red) from 2001:DB8:0:57::7
(192.168.7.7)
Origin IGP, metric 0, localpref 100, valid, external, best
Extended Community: RT:100:1
mpls labels in/out 23/nolabel
rx pathid: 0, tx pathid: 0x0
• The remote IOS PE - PE1, receives the VPNv6 prefix as the out
label of 23.
PE1#show bgp vpnv6 unicast vrf red 2001:db8::7/128
BGP routing table entry for [100:1]2001:DB8::7/128, version 7
Paths: (1 available, best #1, table red)
Advertised to update-groups:
1
Refresh Epoch 1
300, imported path from [100:5]2001:DB8::7/128 (global)
::FFFF:192.168.5.5 (metric 3) (via default) from 192.168.4.4 (192.168.4.4)
Origin IGP, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1
Originator: 192.168.5.5, Cluster list: 192.168.4.4
mpls labels in/out nolabel/23
rx pathid: 0, tx pathid: 0x0
RP/0/0/CPU0:PE2#show bgp vpnv6 unicast vrf red 2001:db8::7/128
BGP routing table entry for 2001:db8::7/128, Route Distinguisher: 100:1
Last Modified: Feb 4 22:46:29.408 for 1d05h
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
300
192.168.5.5 (metric 3) from 192.168.4.4 (192.168.5.5)
Received Label 23
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best,
import-candidate, imported
Received Path ID 0, Local Path ID 1, version 5
Extended community: RT:100:1
Originator: 192.168.5.5, Cluster list: 192.168.4.4
Source VRF: default, Source Route Distinguisher: 100:5
Verifying Data Plane
PE1#show ipv6 cef vrf red 2001:db8::7/128 detail

2001:DB8::7/128, epoch 0, flags [rib defined all labels]
recursive via 192.168.5.5 label 23
nexthop 10.1.14.4 GigabitEthernet0/2 label 19
PE1#show mpls forwarding-table 192.168.5.5

21 19 192.168.5.5/32 0 Gi0/2 10.1.14.4
Verifying Data Plane on IOS XR
RP/0/0/CPU0:PE2#show cef vrf red ipv6 2001:db8::7/128
2001:db8::7/128, version 7, internal 0x5000001 0x0 (ptr 0xa140c5f4) [1],
0x0 (0x0), 0x208 (0xa14db230)
Updated Feb 4 22:46:29.731
Prefix Len 128, traffic index 0, precedence n/a, priority 3
via ::ffff:192.168.5.5, 3 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0xa176b0bc 0x0]
recursion-via-/128
next hop VRF - 'default', table - 0xe0000000
next hop ::ffff:192.168.5.5 via ::ffff:192.168.5.5:0
next hop 10.1.24.4/32 Gi0/0/0/1 labels imposed {19 23}
RP/0/0/CPU0:PE2#show mpls forwarding-table prefix 192.168.5.5/32

24001 19 192.168.5.5/32 0 Gi0/0/0/1 10.1.24.4
Verifying Ingress Hardware Programming – IOS XR
PE2#show cef vrf red ipv6 2001:db8::7/128 hardware ingress detail loc0/0/CPU0
0x0 (0x0), 0x208 (0xa14db230)
Updated Feb 4 22:46:29.730
[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Feb 4 22:46:29.730
LDI Update time Feb 4 22:46:29.730
recursion-via-/128
Ingress platform showdata is not available.
Load distribution: 0 (refcount 1)
Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0
Verifying Egress Hardware Programming – IOS XR
PE2#show cef vrf red ipv6 2001:db8::7/128 hard egr det loc 0/0/CPU0
0x0 (0x0), 0x208 (0xa14db230)
[1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)]
LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0]
gateway array update type-time 1 Feb 4 22:46:29.730
LDI Update time Feb 4 22:46:29.730
recursion-via-/128
Egress platform showdata is not available.
Load distribution: 0 (refcount 1)
Hash OK Interface Address

0 Y Unknown ::ffff:192.168.5.5:0
Troubleshooting 6VPE / MPLS
Verifying Counters on Interface
• Verify the interface counters for mpls forwarding
• If there is forwarding problem, check the counters and ensure they are not
increasing.
• Initiate the VPNv6 prefix ping and verify the counters again to see if they
increased
RP/0/0/CPU0:PE2#show interface gigabitethernet0/0/0/1 accounting
GigabitEthernet0/0/0/1
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 261333 20337753 46929 2305821
IPV6_UNICAST 21017 2062274 20995 1964348
MPLS 10 1180 14426 968553
ARP 84 5040 84 3528
IPV6_ND 13296 1193736 10306 742016
Troubleshooting
MPLS
Traffic-
Engineering
Troubleshooting MPLS TE
The “Fish” Problem
35
M
Tra
ffi c
100
60 40
50
X
ffi c
Tra
M
10
80
45 25
70
Motivation
• Increase efficiency of bandwidth resources
• Prevent over-utilised (congested) links whilst other links are under-
utilised
• Ensures the most desirable/appropriate path for certain traffic types
based on certain policies
• Override the shortest path selected by the IGP
• The ultimate goal is COST SAVING
CSPF – The TE Algorithm
Dijkstra(G, w, s):
• CSPF (executed at ingress) – Initialize-single-source(G,s);
computes an optimal explicit path S = empty set;
Q = V[G];
based on constraints While Q is not empty {
• Bandwidth requirements u = Extract-Min(Q);
S = S union {u};
• Hop limitations for each vertex v in Adj[u] {
relax(u, v, w);
• Administrative groups (link colors) }
}
• Priority (setup and hold) In which:
• Explicit route
G: the graph, represented in some way (e.g.
• Link attributes adjacency list)
w: the distance (weight) for each edge (u,v)
• Reservable bandwidth of the links s (small s): the starting vertex (source)
(static bandwidth minus the S (big S): a set of vertices whose final
shortest path from s have already been
currently reserved bandwidth determined
Q: set of remaining vertices, Q union S = V
draft-manayya-cspf-00
1. CSPF process begins at ingress router with parameters of bandwidth, setup priority, hold
priority and method used incase of equal cost multipath such as random, least fill or most-fill. It
determines the final destination (Egress router).
2. It checks for maximum hop count, include and exclude constraints configured.
3. Check each node for metric and hop count starting with Ingress.
4. For each node check if endpoint is already visited ,if yes then skip the verification. if not check
the link for metric, color and bandwidth (for constraints). The information on each node includes
administrative groups (Color), metrics, static bandwidth, reservable bandwidth, and available
bandwidth priority level. The information contained in the traffic engineering database should be
the same across all routers in the same traffic engineering domain.
5. If it fails then remove this link.
6. If it passes then select the link with shortest path to neighbor router, go to next link and repeat
the step 4.
draft-manayya-cspf-00 (contd…)
• Repeat the steps 3 to 5 for all nodes
• The result of the CSPF algorithm is formed into a strict-hop ERO (Explicit Route Object)
• When the ERO is completed, the ERO is passed to the RSVP (Resource Reservation Protocol)
process, where it is used for signaling and establishing the LSP in the network.
• If it is not possible to find the path then indicate about not finding a route then retry after retry
interval.
RSVP Overview
• Once the path is calculated, it must be signaled across the network
• Reserve any bandwidth to avoid “double booking” from other TE reservations
• Priority can be used to pre-empt low priority existing tunnels
• RSVP used to setup TE LSP

• PATH messages (from head to tail) carries LABEL_REQUEST
• RESV messages (from tail to head) carries LABEL
• When RESV reaches headend, tunnel interface = UP

• RSVP messages exist for LSP teardown & error sig
Headend Midpoint Tailend
RSVP Overview – Admission Control
• On receipt of PATH message
• Router will check there is bandwidth available to honour the reservation
• If bandwidth available then RSVP accepted
• On receipt of a RESV message

• Router actually reserves the bandwidth for the TE LSP
• If preemption is required lower priority LSP are torn down
• OSPF/ISIS updates are triggered
Does RSVP actually allocates the b/w across the path for TE tunnel?
RSVP Overview – Admission Control
100
60 40
50
RSVP Path
Message
(10M)
PATH RSVP
80
45 25 RESV
BW=10 30 Message
70
Configuration / Feature requirements
• RSVP should be enabled on relevant interfaces
• mpls traffic-eng should be enabled
• Globally 100
• Interface level 60 40
50
• IGP Level
• Tunnel Interface Configuration

• Allowing traffic through TE Tunnel 80
45 25
• Decision on Path Selection Process 70
• Dynamic
• Explicit-path
Autoroute Announce
• Used to include TE LSP in SPF calculations
• Tunnel is treated as a directly connected link to the tail

• IGP adjacency is NOT run over the tunnel!
• Using autoroute announce, all nodes behind the headend are routed via
tunnel
IOS – IOS-XE (Config under Tunnel Interface)
tunnel mpls traffic-eng autoroute announce
IOS-XR (Configuration under Tunnel-te Interface)

autoroute announce
NX-OS (Configuration under Tunnel-te Interface)

autoroute announce
Forwarding Adjacency
• Autoroute does not advertise the LSP into the IGP
• There may be requirement to advertise the existence of TE tunnels to

upstream routers
• Allow upstream routers to compute a better path to destination a over downstream
TE tunnel
R1 R4 R5
All links have metric of 10
R3 R8
R2 R6 R7
Verification Commands
• Verifying RSVP Interfaces
• Show ip rsvp interface
• Verifying TE Tunnels
• Show mpls traffic-eng tunnels tunnel <num>
• Show mpls traffic-eng forwarding (XR)
• Show mpls traffic-eng forwarding-adjacency
• Verifying FRR Database

• Show mpls traffic-eng fast-reroute database
RSVP Troubleshooting
RP/0/0/0:R1#sh rsvp counters messages summary
All RSVP Interfaces Recv Xmit Recv Xmit
Path 0 25 Resv 30 0
PathError 0 0 ResvError 0 1
PathTear 0 30 ResvTear 12 0
ResvConfirm 0 0 Ack 24 37
Bundle 0 Hello 0 5099
SRefresh 8974 9012 OutOfOrder 0
Retransmit 20 Rate Limited 0
IOS - Show ip rsvp counters summary
IP proto 0x2e – Can use this for performing packet capture
Verify Basic TE Tunnel Forwarding
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels 400
Name: tunnel-te400 Destination: 192.168.4.4 Ifhandle:0x580
Status:
Admin: up Oper: up Path: valid Signalling: connected
path option 1, type dynamic (Basis for Setup, path weight 1)
G-PID: 0x0800 (derived from egress interface properties)
Bandwidth Requested: 0 kbps CT0
Creation Time: Thu Jun 15 19:22:40 2017 (00:15:46 ago)
Config Parameters:
Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff
Metric Type: TE (global)
Path Selection:
<snip>
Fast Reroute: Disabled, Protection Desired: None
Path Protection: Not Enabled BFD Fast Detection: Disabled
Reoptimization after affinity failure: Enabled
Soft Preemption: Disabled
History: Tunnel has been up for: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)
Current LSP: Uptime: 00:15:46 (since Thu Jun 15 19:22:40 UTC 2017)
Path info (OSPF 100 area 0):
Node hop count: 1
Hop0: 10.24.1.4
Hop1: 192.168.4.4
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnels brief

TUNNEL NAME DESTINATION STATUS STATE
tunnel-te400 192.168.4.4 up up
PE1_t100 192.168.2.2 up up
PE4_t100 192.168.2.2 up up
PE1_t101 192.168.2.2 up up
Re-optimization Configs
• Configuration
• Logging
• Logging events lsp-status reoptimize (XR TE Tunnel interface
config)
• Logging events lsp-status reroute (XR TE Tunnel interface config)
Troubleshooting : TE Tunnel does not come up
RP/0/RP0/CPU0:PE2# show mpls traffic-eng tunnel 400 detail
Wed May 29 14:07:50.428 UTC
Name: tunnel-te 400 Destination: 0.0.0.0
Status:
Admin: up Oper: down Path: not valid Signalling: Down
ospf 100 area 0
G-PID: 0x0800 (internally specified)
Config Parameters:
Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff Very verbose
Metric Type: TE (default) reason given here
AutoRoute: disabled LockDown: disabled on this line for
Loadshare: 0 equal loadshares config errors
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 0
Direction: unidirectional
Endpoint switching capability: unknown, encoding type: unassigned
Transit switching capability: unknown, encoding type: unassigned
Reason for the tunnel being down: No destination is configured
History:
Prior LSP:
ID: path option 10 [13]
Removal Trigger: signalling shutdown No Destination
configured under
Tunnel interface
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detail
Name: tunnel-te400 Destination: 192.168.4.4
Status:
ospf 100 area 0
Config Parameters:
Metric Type: TE (default)
AutoRoute: enabled LockDown: disabled
Loadshare: 0 equal loadshares
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 1
Direction: unidirectional
Endpoint switching capability: unknown, encoding type: unassigned
Transit switching capability: unknown, encoding type: unassigned Insufficient RSVP b/w.
History: Bandwidth command not
Prior LSP: configured under rsvp.
ID: path option 1 [21] or
Removal Trigger: path verification failed is misconfigured
Last Error:
PCALC:: No path to destination, 192.168.4.4(bw)
RP/0/RP0/CPU0:PE2#show mpls traffic-eng tunnel 400 detail
Name: tunnel-te400 Destination: 192.168.4.4
Status:
ospf 100 area 0
Config Parameters:
Metric Type: TE (default)
AutoRoute: enabled LockDown: disabled
Loadshare: 0 equal loadshares
Auto-bw: disabled(0/0) 0 Bandwidth Requested: 1 Tunnel has no
Direction: unidirectional alternative path
Endpoint switching capability: unknown, encoding type: unassigned Or
Transit switching capability: unknown, encoding type: unassigned Explicit path is
History: misconfigured.
Prior LSP:
ID: path option 1 [21]
Removal Trigger: path verification failed
Last Error:
PCALC:: No path to destination, 192.168.4.4(reverselink or exclude-link)
TE Tunnel not up (Summary)
• RSVP Signaling in progress
• Show rsvp sessions dst-port
• No path available
• Show mpls traffic-eng igp-area
• Show mpls traffic-eng topology model-type rdm|mam
(Russian Dolls / Maximum allocation)
• Show mpls traffic-eng link-management interface x/y
• Cannot reach dst x.x.x.x from y.y.y.y

• Show rsvp interface
• Or check TE topology database
Class-Based Tunnel Selection – CBTS
Destination NH: PE2 EXP: 4
PE2
Destination NH: PE2 EXP: Default

PE1 Destination NH: PE3 EXP: 3,4 PE3

PE4
• EXP-based selection between multiple tunnels to same destination

• Local Tunnels (Head-end) configured with allowable EXP values
• Tunnels may be configured as default
• No IGP extensions, VRF aware
• Simplifies use of DS-TE tunnels & similar to PVC Bundling
Russian Dolls Model (RDM)
• BW pool applies to one
or more classes BC0
All
Classes
• Global BW pool (BC0) (Class0 Maximum

Reservable
+
equals MRB BC1 Class1 Bandwidth
Class1 +
+ Class2) (MRB)
• BC0..BCn used for Class2
Class2
BC2
computing unreserved
BW for class n
• Current implementation supports BC0 and BC1
• BC0 – Global Pool
• BC1 – Sub Pool
• Supported by Traditional and IETF implementation
Maximum Allocation Model (MAM)
• BW pool applies
to one class
BC0
Class0
• Sum of BW pools may exceed MRB
Maximum
• Sum of total BC1 Class1 All Reservable
Classes Bandwidth
reserved BW may (MRB)
not exceed MRB BC2 Class2
• Current implementation supports BC0

and BC1
• Supported by IETF Implementation
only
CBTS – Configuration Example
Both tunnels to same
interface Tunnel65
destination but different QoS
ip numbered loopback0
tunnel destination 192.168.2.2
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng bandwidth sub-pool 30000
tunnel mpls traffic-eng exp 5
interface Tunnel66
ip numbered loopback0
tunnel destination 192.168.2.2
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng bandwidth 50000
tunnel mpls traffic-eng exp default
Traces to collect on IOS XR
Module Trace commands

MPLS TE Control Show tech support mpls traffic-eng
RSVP Show tech support rsvp
CEF (forwarding) Show cef mpls trace location <line card location>
Show cef platform trace all all location <line card location>
Show cef trace location <line card location>
Show mpls forwarding tunnel detail
Show mpls forwarding labels hardware ingress/egress detail loc
Show cef mpls adj tunnel-te <> hardware ingress/egress detail loc <>
SONET Show sonet-local trace location <line card location>
Bundles Show tech-support bundles
Interface Manager Show tech-support pfi
Tunnel Protection
• Mechanism to mitigate packet loss during a failure
• Pre-provisioned protection tunnels that carry traffic when a
protected link or node goes down
• MPLS TE protection also known as FAST REROUTE
• Protects against LINK FAILURE
• For example, Fibre cut, Carrier Loss, ADM failure
• Protects against NODE FAILURE
• For example, power failure, hardware crash, maintenance
Categories of FRR
• Local Protection
• Link Protection
• Node Protection
• Protects a segment of the tunnel (Node or Link)
• 1:N Scalability
• Faster failure recovery
• Path Protection
• Protects individual tunnels
• 1:1 Scalability
• More resource consumption
Link Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4
TE Label
Troubleshooting MPLS TE PLR
FRR kicks in…
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4
TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4
TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4
TE Label
Node Protection
P-3
pop
PE1 P-2 P-5 PE2
VPN Label P-4
TE Label
MPLS Traffic-
Engineering Demo
SP SDN –
Segment Routing
Segment Routing
Path towards Segment Routing
• LDP had its own challenges
• Extra process required (LDP) + It creates complicated interaction with IGP
(LDP-IGP Sync)
• RSVP-TE – Deployment and scalability issues (Only 10% SP space
uses RSVP-TE and that too with FRR use-case)
• Always-on Feature, even when TE is not required in the network
• Need network that could understand application requirements
Segment Routing
Overview
• SR originally meant “Strade Romane” – network of roads which were built by Roman
Empire
• The name was later changed to Segment Routing
• SR is nothing but Application Engineered Routing, where application makes request
to the network (controller) to provide it a path that would serve the needs of the
application
• SR is a source based routing, where the source chooses a path based on the
application requirements
• The chosen path is encoded in the packet header as an ordered list of segments
• Segment – ID for any type of instruction

• Forwarding or service
Segment Routing – Forwarding Plane
• MPLS: an ordered list of segments is represented as a stack of labels
• IPv6: an ordered list of segments is encoded in a routing extension header
• This presentation: MPLS data plane
• Segment → Label
• Basic building blocks distributed by the IGP or BGP
IGP segments
• Two basic building blocks distributed by IGP
• Prefix Segments
• Adjacency Segments
IGP Prefix Segment 16005
• Shortest-path to the IGP prefix 1.1.1.5/32
• Equal Cost MultiPath (ECMP)-aware

1 2 16005
• Global Segment 16005

16005
• Label = 16000 + Index
5
• Advertised as index 16005
16005
• Distributed by ISIS/OSPF
3 4
16005
All nodes use default SRGB

16,000 – 23,999
IGP Prefix Segment
16004
• Shortest-path to the IGP prefix
• Equal Cost MultiPath (ECMP)-aware
• Global Segment
1 2
• Label = 16000 + Index 16004
16004 16004
• Advertised as index
5
• Distributed by ISIS/OSPF 16004
16004
3 4
1.1.1.4/32
16004
16,000 – 23,999
IGP Adjacency Segment
• Forward on the IGP adjacency
• Local Segment
1 2
• Advertised as label value
• Distributed by ISIS/OSPF
Adj to 2
5
24042
Adj to 5
3 4 24045
24043
Adj to 3
16,000 – 23,999
Combining IGP Segments 16004

24045
• Steer traffic on any path through the network Packet to 5
• Path is specified by a stack of labels

1 2
• No path is signaled
• No per-flow state is created
• Single protocol: IS-IS or OSPF 5

24045
Packet to 5
3 4
16004 24045
Segment Routing – 3 Segments Example
PHP
• Source routing – ordered list of segments
3000 • Stack of MPLS labels
1900 1900 • IPv6 Routing Extension
1700 1700 • MPLS labels are advertised by the IGP
Global label • Simplicity
3000
A B C D
1700
segment 1 Global label
segment 2 I
1700
E F G H
Adjacency
1700 segment 3
label 1900
Segment Routing - Control Plane & Data
Plane
MPLS Control and Forwarding Operation with Segment
Routing
Services
MP-BGP
No changes to
IPv4 IPv6
IPv4 IPv6 VPWS VPLS control or
PE1 PE2 VPN VPN
forwarding plane
Packet
Transport LDP RSVP Static BGP IS-IS OSPF IGP or BGP label
distribution for
PE1 IGP PE2
IPv4 and IPv6.
MPLS Forwarding
Forwarding plane
remains the same
SR enabled node
SID Encoding
SRGB = [ 16,000 – 23,999 ] – Advertised as base = 16,000, range = 8,000
Prefix SID = 16,001 – Advertised as Prefix SID Index = 1
Adjacency SID = 24000 – Advertised as Adjacency SID = 24000
• Prefix SID
• Label form SR Global Block (SRGB)
• SRGB advertised within IGP via TLV
• In the configuration, Prefix-SID can be configured as an absolute value or an index
• In the protocol advertisement, Prefix-SID is always encoded as a globally unique index
Index represents an offset from SRGB base, zero-based numbering, i.e. 0 is 1st index
E.g. index 1  SID is 16,000 + 1 = 16,001
• Adjacency SID
• Locally significant
• Automatically allocated by the IGP for each adjacency
• Always encoded as an absolute (i.e. not indexed) value
SR IS-IS Control Plane Summary
• IPv4 and IPv6 control plane
• Level 1, level 2 and multi-level routing
• Prefix Segment ID (Prefix-SID) for host prefixes on loopback
interfaces
• Adjacency Segment IDs (Adj-SIDs) for adjacencies
• Prefix-to-SID mapping advertisements (mapping server)
• MPLS penultimate hop popping (PHP) and explicit-null signaling
SID index 1
1.1.1.2 1.1.1.1
IS-IS Configuration – Example

1.1.1.4 1.1.1.6
router isis 1 DIS
address-family ipv4 unicast Wide metrics
metric-style wide
enable SR IPv4 control plane and
segment-routing mpls
SR MPLS data plane on all ipv4
!
interfaces in this IS-IS instance
metric-style wide Wide metrics
! enable SR IPv6 control plane and
interface Loopback0 SR MPLS data plane on all ipv6
passive interfaces in this IS-IS instance
prefix-sid absolute 16001
Ipv4 Prefix-SID value for loopback0
!
Ipv6 Prefix-SID value for loopback0
!
!
SR OSPF Control Plane Summary
• OSPFv2 control plane

• Multi-area
• IPv4 Prefix Segment ID (Prefix-SID) for host prefixes on loopback
interfaces
• Adjacency Segment ID (Adj-SIDs) for adjacencies
• Prefix-to-SID mapping advertisements (mapping server)
• MPLS penultimate hop popping (PHP) and explicit-null signaling
SID index 1
1.1.1.2 1.1.1.1 1.1.1.4
OSPF Configuration Example

router ospf 1 1.1.1.5 1.1.1.3
DR
router-id 1.1.1.1 Enable SR on all areas
area 0
interface Loopback0
passive enable
prefix-sid absolute 16001 Prefix-SID for loopback0
!
!
!
MPLS Data Plane Operation (labeled)
Prefix SID Adjacency SID
SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ]
Adjacency
SID = X
Swap Pop
X
X X Y Y
Payload Payload Payload Payload
• Packet forwarded along IGP shortest path (ECMP)  Packet forwarded along IGP adjacency
• Swap operation performed on input label  Pop operation performed on input label
• Same top label if same/similar SRGB
 Top labels will likely differ
• PHP if signaled by egress LSR
 Penultimate hop always pops last adjacency SID
MPLS Data Plane Operation (Prefix SID)
SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ]
A B C D Loopback X.X.X.X
Prefix SID Index = 41
Push Swap Pop Pop

Push
16041 16041
VPN Label VPN Label VPN Label
Payload Payload Payload Payload Payload
MPLS Data Plane Operation (Adjacency SIDs)
SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ] SRGB [16,000 – 23,999 ]
A B X D Loopback X.X.X.X
Adjacency Prefix SID Index = 41
SID = 30206
Push Pop Pop Pop
Push
Push
30206
16041 16041
VPN Label VPN Label VPN Label
Payload Payload Payload Payload Payload
LDP-SR Migration
Assumptions:
• all the nodes can be upgraded to SR
Simplest migration LDP to SR • all the services can be upgraded to SR
• Initial state: All nodes run LDP, not SR

LDP LDP
3 4
LDP LDP
1 LDP 2
5 6
LDP LDP
LDP Domain
Assumptions:

SR+LDP SR+LDP
• Step1: All nodes are upgraded to SR
• In no particular order 3 4
SR+LDP SR+LDP
• leave default LDP label imposition preference
1 LDP 2
5 6
SR+LDP SR+LDP
SR+LDP Domain
Assumptions:

SR+LDP SR+LDP
• Step1: All nodes are upgraded to SR
SR+LDP SR+LDP
• leave default LDP label imposition preference
1 SR 2
• Step2: All PEs are configured to prefer SR
label imposition
sr-prefer
SR+LDP SR+LDP
SR+LDP Domain
Assumptions:

• Step1: All nodes are upgraded to SR SR SR
• leave default LDP label imposition preference SR SR
• Step2: All PEs are configured to prefer SR 1 SR 2

label imposition
• Step3: LDP is removed from the nodes in SR SR
the network
SR Domain
• In no particular order
• Final state: All nodes run SR, not LDP
Node X
SR Topology Lo0 – 1.1.1.x/32
Link XY – 99.X.Y.0/24; X < Y
Prefix SID – 16000 + X
SRGB: 16000-23999
Prefix SID Prefix SID
16001 16003 16005 16007

XR-1 XR-3 XR-5 XE-7
1.1.1.1 3.3.3.3 24010 24012 5.5.5.5 7.7.7.7
16008 16009
XE-8 XR-9
PeerAdj SID* 8.8.8.8 9.9.9.9
16002 16004 16006 16010
24011 24008
XE-2 XR-4 XR-6 XR-10
4.4.4.4 6.6.6.6 10.10.10.10
2.2.2.2
ISIS SR ISIS SR ISIS SR

ISIS Level-2 ISIS Level-2 AS64002 ISIS Level-1
AS 64001
Note (*) = PeerAdj SID values are dynamically allocated
Enabling Segment Routing – XR and XE
IOS-XR
segment-routing
!
router isis SR-AS-1
!
interface Loopback0
!
commit
IOS-XE
XE-2(config)#segment-routing mpls
XE-2(config-srmpls)#connected-prefix-sid-map
XE-2(config-srmpls-conn)#address-family ipv4
XE-2(config-srmpls-conn-af)#2.2.2.2/32 absolute 16002 range 1
XE-2(config-srmpls-conn-af)#exit
XE-2(config-srmpls-conn)#exit
XE-2(config-srmpls)#exit
XE-2(config)#router isis SR-AS-1
XE-2(config-router)#segment-routing mpls
Segment Routing Migration Demo
ODN
SR-PCE
• SR-PCE is an IOS XR multi-domain stateful SR Path Computation
Element (PCE)
• IOS XR: XTC functionality is available on any physical or virtual IOS XR
node, activated with a single configuration command
• Multi-domain: Real-time reactive feed via BGP-LS/ISIS/OSPF from
multiple domains; computes inter-area/domain/AS paths
• Stateful: takes control of SRTE Policies, updates them when required
• SR PCE: native SR-optimized computation algorithms
• SR-PCE is fundamentally distributed

• Not a single all-overseeing entity (“god box”), but distributed across the
network; RR-alike deployment
SR-PCE Building Blocks
WAE Custom app
REST API
Native SR
Multi-Domain algorithms
Topology
Topo
Compute
DB
SR-PCE runs on
virtual or physical
IOS-XR node
Collect Deploy
IGP PCEP
BGP-LS
BGP
ODN Workflow
BGP VPNv4
BGP RR BGP RR
BGP VPNv4
• Routes tagged with a user-defined COLOR BGP VPNv4
BGP color comm.
“gold”
to convey SLA requirements
• VPN routes propagated via BGP
BGP color comm.
BGP “gold” Y/24
XR-1 XR-3 XR-5 XE-7
1.1.1.1 3.3.3.3 5.5.5.5 7.7.7.7
XE-8 XR-9
8.8.8.8 9.9.9.9

2.2.2.2 4.4.4.4 BGP 6.6.6.6 10.10.10.1
0

AS 64001
ODN Workflow
SRTE • Ingress PE matches on user-specified BGP “color” community
XTC-A
On-demand color • Ingress PE enforces a “template” associated with the color community
SR PCE
“gold”
contact PCE
request path to
BGP NH
minimize TE metric Need a path to node (9)?
Minimizing TE metric
PCReq
BGP XE-7
XR-1 XR-3 XR-5
1.1.1.1 3.3.3.3 5.5.5.5 7.7.7.7
XE-8 XR-9
8.8.8.8 9.9.9.9

2.2.2.2 4.4.4.4 BGP 6.6.6.6 10.10.10.1
0

AS 64001
PCEP
XTC-A XTC-B
SR PCE SR PCE
PCEP
PCEP
BGP XE-7
XR-1 XR-3 XR-5
1.1.1.1 3.3.3.3 5.5.5.5 7.7.7.7
XE-8 XR-9
8.8.8.8 9.9.9.9

2.2.2.2 4.4.4.4 BGP 6.6.6.6 10.10.10.1
0

AS 64001
Demo - ODN
Complete your
online session • Please complete your session survey
evaluation after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live water bottle.
• All surveys can be taken in the Cisco Live
Mobile App or by logging in to the Session
Catalog on ciscolive.cisco.com/us.
Cisco Live sessions will be available for viewing
on demand after the event at ciscolive.cisco.com.
Thank you
#CLUS
#CLUS
Reference Slides
Troubleshooting
MPLS with NX-OS
Troubleshooting MPLS with NX-OS
Software Architecture
IM/OIM/
MPLS-Mgr L3VM
CLI URIB
SNMP ULIB
System
LDP IGP
Manager
Feature
Netstack
Manager
License
Manager PSS
MTS
Shared Memory
Message Queue
Component Functions
• Interact with ULIB
• Allocate local label for prefixes
• Interact with URIB

• Learn routes
• Program outgoing labels
• Interact with Netstack

• UDP socket (Hello messages)
• TCP sockets (Other LDP messages)
• Interact with IM/OIM/MPLS_mgr

• Learn interface status
• Learn interface address
• Enable MPLS forwarding on interface
Component Functions (contd…)
• Interact with L3VM
• VRF table id
• Interact with IGP

• LDP-IGP sync
• LDP auto-configuration
• Interact with platform services

• Enabling LDP feature enables multiple services:
• LDP service
• ULIB service
• mpls_mgr service
• mpls_oam service
MPLS Packet Flow
L2FT FIB
LDB ILM ADJ ELM RIT
(DMAC) TCAM
• LDB – L2 Features; Perform LDP lookup to derive LIF / BD for ingress packet
• L2FT – Perform SMAC and DMAC lookup; DMAC should be router MAC
• ILM – Lookup ingress LIF MAP table and identify feature enabled, i.e. MPLS
• FIB – Deals with both PI and PD programming
• ADJ – FIB results provides adjacency points to egress LIF
• ELM – Egress LIF has the DI for egress interface
• RIT – Generate the rewrite (SMAC, DMAC and Label rewrite [push, pop, swap])
LDB – Check if the router BD is set in the LDP entry
module-1# show hardware internal forwarding interface e1/1
Software Tables:
Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002
State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)
Number of Member Ports(0x0)
LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)
Hardware Tables:
Instance: 0x1
L2-LIF-MAP entry with index = 0x422
ldb_base = 0xc801 add_vlan = 0
Instance: 0x1
L2-LIF entry with index = 0xc801
pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0
ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002
<snip>
Check if the router BD is set in the LDP entry
module-1# show hardware internal forwarding interface e1/1
Software Tables:
Interface = Ethernet1/1 LTL Index = 0x422 LIF = 0x4002
State(up) Layer(L3) Mode(0x0) VDC(1) Local Port(yes)
Number of Member Ports(0x0)
LDB Sharing(no) LDB Base(0xc801) LDB Port Features(no)
Hardware Tables:
Instance: 0x1
L2-LIF-MAP entry with index = 0x422
ldb_base = 0xc801 add_vlan = 0
Instance: 0x1
L2-LIF entry with index = 0xc801
pt_cam_en = 0 ipv4_igmp_snoop = 0 ipv4_pim_snoop = 0 ipv6_mld_snoop = 0
ipv6_pim_snoop = 0 bd = 0x2 l2v4 = 0 ingr_lif = 0x4002
<snip>
Verify L2FT and ILM
L2FT
show hardware mac address-table
FE | Valid| PI| BD | MAC | Index| Stat| SW | Modi| Age| Tmr| GM| Sec| TR| NT| RM| RMA| Cap| Fld|Always
| | | | | | ic | | fied|Byte| Sel| | ure| AP| FY| | |TURE| | Learn
---+------+---+------+---------------+-------+-----+-----+-----+----+----+---+----+---+---+---+----+----+----+------
0 1 1 2 0022.557a.32c1 0x00400 1 0x000 0 6 0 1 0 0 0 0 0 0 0 0
0 1 0 1 0100.0cff.fffe 0x00421 1 0x001 0 6 0 0 0 0 0 0 0 0 0 0
ILM
NX-OS# show hardware internal forwarding interface Ethernet 1/1 module 10 | inc mpls_en
l2l3_lkup_cfg = 0 mpls_en = 1 sm_en = 0 red_ids_chk_fail_en = 1 v4_rpfv3_en = 0
ipv4_en = 1 eompls_en = 0 mpls_en = 1
N7k-1# show hardware internal forwarding interface e1/2 module 1 | in mpls_en

mpls_vpn_sel : 0x0 l2_tunnel_type : 0x0 mpls_en : 0x1
Verifying FIB - PI
N7k-1# show forwarding route module 1
----------------+----------------------------------------+----------------------+-----------------
Prefix | Next-hop | Interface | Labels
----------------+----------------------------------------+----------------------+-----------------
<snip>
192.168.2.2/32 nxthop 10.12.1.2 Ethernet1/2 NO-OP
192.168.3.3/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 21
192.168.4.4/32 nxthop 10.12.1.2 Ethernet1/2 PUSH 22
N7k-1# show forwarding route detail

Prefix 192.168.2.2/32,
No of paths : 1 Update time: Wed Jun 14 08:46:21 2017
nxthop 10.12.1.2 Ethernet1/2 NO-OP DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Prefix 192.168.3.3/32,
nxthop 10.12.1.2 Ethernet1/2 PUSH 21 DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Prefix 192.168.4.4/32,
nxthop 10.12.1.2 Ethernet1/2 PUSH 22 DMAC: 001b.54c2.3342
packets: 0 bytes: 0
Verifying FIB – PI – Forwarding and Adjacency Info
N7k-1# show forwarding mpls module 1
--------+-----------+-------------------+----------------+-------------+-------
Local |Prefix |FEC |Next-Hop |Interface |Out
Label |Table Id |(Prefix/Tunnel id) | | |Label
--------+-----------+-------------------+----------------+-------------+-------
18 |0x1 |192.168.2.2/32 |10.12.1.2 |Ethernet1/2 |Pop Label
19 |0x1 |192.168.3.3/32 |10.12.1.2 |Ethernet1/2 |21
20 |0x1 |192.168.4.4/32 |10.12.1.2 |Ethernet1/2 |22
N7k-1# show forwarding adjacency

IPv4 adjacency information
next-hop rewrite info interface
-------------- --------------- -------------
10.1.12.2 001b.54c2.3342 Ethernet1/2
N7k-1# show forwarding adjacency mpls

IPv4 adjacency information, adjacency count 1
next-hop rewrite info interface
-------------- --------------- -------------
10.1.12.2 Ethernet1/2 001b.54c2.3342 NO-OP 3
10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 21
10.1.12.2 Ethernet1/2 001b.54c2.3342 PUSH 22
Verifying FIB – PD – MPLS Programming
N7k-1# show system internal forwarding mpls detail
Table id = 0x1
------------------
----+--------+--------+------------+----------+----------+-----------+--------+
Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op
----+--------+--------+------------+----------+----------+-----------+--------+
0 0x5624 0x23c2 16 0x5c 0x1fe0 0 POP ONE
0 0x5625 0x23c3 17 0x5c 0x1fe0 0 POP ONE
0 0x5224 0x23c4 18 0x62 0x2 3 POP ONE
0 0x5225 0x23c5 19 0x60 0x2 21 SWAP ONE
0 0x5c24 0x23c6 20 0x64 0x2 22 SWAP ONE
0 0x5c25 0x23c7 21 0x65 0x3 0 POP ONE
Table id = 0x2a
------------------
----+--------+--------+------------+----------+----------+-----------+--------+
Dev | Index |Priority| In-label | AdjIndex | LIF | Out-label | Op
----+--------+--------+------------+----------+----------+-----------+--------+
No labels in table
Aggregate Table id = 0x2a
------------------
--------+--------+
label | vpn_id
--------+--------+
0 492287 0x2a
Verify Label Information in Hardware
pe1# show system internal forwarding mpls label
show system internal forwarding mpls
Table id = 1
------------------
----+--------+------------+----------+----------+-----------+--------+
Dev | Index | In-label | AdjIndex | LIF | Out-label | Op
----+--------+------------+----------+----------+-----------+--------+
0 0x1ffa9 18 0x62 0x2 3 POP ONE
0 0x5225 19 0x60 0x2 21 SWAP ONE
0 0x5c24 20 0x64 0x2 20 SWAP ONE
FIB DRAM
FIB TCAM Egress LIF
Adjacency
Index (LTL)
Index
Route Update PD Verification
• Use the following command to check the route in FIB PD
• Show system internal forwarding route
• Use the following command to check the adjacency in FIB PD

• Show system internal forwarding adjacency
• Use the following command to check the MPLS adjacency in LFIB PD

• Show system internal forwarding mpls adjacency
• Use the following command to check the hardware adjacency to verify if

the packet is getting forwarding out correct interface
• Show system internal forwarding adjacency entry <adj> detail
Troubleshooting L3VPN VRF Issues
• Check for L3VM process for the event-traces to verify the events that
occurrent for the VRF
N7k-1# show system internal l3vm event-history vrf
VRF events for L3VM Process - Bufsize 1000 KB2017
2017 Jun 14 09:10:02.139925 l3vm [5710]: [5830]: Updated interface Ethernet1/1 cmd <vrf member TEST>
2017 Jun 14 09:10:02.139757 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) changing from VRF default to VRF TEST - Count 1
2017 Jun 14 09:10:02.139728 l3vm [5710]: [5830]: Interface Ethernet1/1 (IOD 37) will be down, VRF default UP-IF count 1
2017 Jun 14 09:10:02.139680 l3vm [5710]: [5830]: Moving Ethernet1/1 (ifindex: 0x1a000000 iod: 37) from VRF default to VRF TEST
2017 Jun 14 09:10:02.139522 l3vm [5710]: [5830]: Deleting all L3VM_PSS_IF_KEY config for interface Ethernet1/1
2017 Jun 14 09:10:02.137418 l3vm [5710]: [5830]: [VSH] Process interface Eth1/1 cmd <vrf member TEST>
2017 Jun 14 09:06:24.460917 l3vm [5710]: [5830]: Updated vrf TEST cmd <address-family ipv4 unicast>
2017 Jun 14 09:06:24.460771 l3vm [5710]: [5830]: [VSH] Process vrf TEST cmd <address-family ipv4 unicast>
2017 Jun 14 09:06:24.426293 l3vm [5710]: [5830]: l3vm_pd_process_l3vm_mts_msg_from_ctrl: Received l3vm notification (mtype: 4)
2017 Jun 14 09:06:24.424511 l3vm [5710]: [5829]: VRF TEST:ipv4:base table (Up:--) sending: Table create
2017 Jun 14 09:06:24.424372 l3vm [5710]: [5829]: VRF TEST:ipv6:base table (Up:--) sending: Table create
2017 Jun 14 09:06:24.424256 l3vm [5710]: [5829]: VRF TEST (Up:--) sending: VRF create
2017 Jun 14 09:06:24.424006 l3vm [5710]: [5829]: VRF TEST - Created
2017 Jun 14 09:06:24.424002 l3vm [5710]: [5829]: VRF TEST (Up:--) sdb ack
2017 Jun 14 09:06:24.423008 l3vm [5710]: [5829]: gsdb_op_callback() - gsdb context 0x0003ce86
2017 Jun 14 09:06:24.421933 l3vm [5710]: [5830]: Updated cmd <vrf context TEST>
Inter-AS MPLS VPN
Inter-AS MPLS VPNs
Flavors
• Previous section – VPNs within Single-AS boundary
• Inter-AS MPLS VPN – VPNs spanning across multiple AS
boundaries
• Types:
• Option A – Back to Back VRF
• Option B – Inter-Provider VPNs using ASBR-to-ASBR approach
1. Next-Hop-Self Method
2. Redistribute Connected Method
3. Multi-hop EBGP between ASBRs
• Option C – MP-EBGP between RR and EBGP between ASBR
Inter-AS VPN Topology RR
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
PE
R5 XR6 PE
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public
Inter-AS VPN Option A: Back to Back VRFs RR
RR
R3 ASBR XR4
ASBR VRF VPN01 VRF VPN02 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1
VPN02 XR2 VPN02
• Terminate VRFs on ASBRs

PE
PE
Advertise Peering Link to VRF/BGP
R5 XR6
•
• Exchange routes across peering link

• Simple
CE
R7 • Doesn’t Scale Well R8 CE
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
R3 router bgp 100 ASBR XR4

ASBR 192.168.2.2
192.168.1.1
AS100 neighbor 192.168.3.3 remote-as 100
ASBR AS200
VPN02 R1 neighbor 192.168.3.3 update-sourceXR2Loopback0
VPN02
!
PE exit-address-family
R5 XR6 PE
!
address-family ipv4 vrf VPN01
redistribute connected
CE exit-address-family
R7 R8 CE
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

R1
VPN02 bgp 200
router XR2 VPN02
vrf VPN02
rd 200:1
PE
R5 ! XR6 PE
remote-as 100
route-policy PASSALL in
route-policy PASSALL out
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
AS100 ASBR AS200

CE7#trace
VPN02 100.64.8.8
R1 XR2 VPN02
1 172.16.57.5 3 msec 3 msec 3 msec
PE 2 172.16.12.1 [AS 100] [MPLS: Label 204 Exp 0] 4 msec 4 msec 5 msec
R5 XR6 PE
3 172.16.12.2 [AS 100] 5 msec 5 msec 4 msec
4 10.26.1.6 [MPLS: Label 60003 Exp 0] 36 msec 10 msec 10 msec
5 172.32.68.8 [AS 200] 11 msec * 11 msec
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS © 2019 Cisco and/or its affiliates. All rights 100.64.8.8
244 reserved. Cisco Public
Inter-AS VPN Option B1: ASBR to ASBR w/ Next-Hop-Self RR
RR
Next-Hop-Self Next-Hop-Self
R3 ASBR XR4
ASBR MP-EBGP 192.168.2.2
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
No LDP or IGP required on the link between the two

ASBRs.
Configure no bgp default route-target filter on ASBRs
CE
R7 ASBRs advertise to RRs with Next-Hop-Self R8 CE
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS MPLS VPNs
Problems with Route Filtering
• Enabled by default by all non-RRs.
R1#debug bgp vpnv4 unicast updates
BGP updates debugging is on for address family: VPNv4 Unicast
R1#clear bgp vpnv4 unicast * sof
*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1) rcvd
Refresh Start-of-RIB
*Jun 20 19:35:50.710: BGP: nbr_topo global 192.168.3.3 VPNv4 Unicast:base (0x110FC570:1)

refresh_epoch is 3
*Jun 20 19:35:50.711: BGP(4): 192.168.3.3 rcvd UPDATE w/ attr: nexthop 192.168.5.5, origin ?,
localpref 100, metric 0, originator 192.168.5.5, clusterlist 192.168.3.3, merged path 700, AS_PATH
, extended community RT:100:1
*Jun 20 19:35:50.714: BGP(4): 192.168.3.3 rcvd 100:1:100.64.7.0/24, label 5003 - DENIED due to:
extended community not supported;
RR interface GigabitEthernet0/2
ip address 172.16.12.1 255.255.255.0
R3 mpls bgp forwarding ASBR XR4
ASBR ! MP-EBGP 192.168.2.2
192.168.1.1
router bgp 100
AS100 bgp log-neighbor-changes ASBR AS200
VPN02 R1 no bgp default ipv4-unicast XR2 VPN02
no bgp default route-target filter
PE !
R5 XR6 PE
CE neighbor 192.168.3.3 next-hop-self
R7 R8 CE
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR

bgp ASBR
router-id 192.168.2.2 MP-EBGP 192.168.2.2
192.168.1.1
retain route-target all
AS100! ASBR AS200
R1
VPN02neighbor 172.16.12.1
XR2 VPN02
remote-as 100
PE
R5 ! XR6 PE
!
remote-as 200
CE next-hop-self CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS MPLS VPNs
Problems with Route Installation at Remote PEs
• Routes will not install on remote Pes if they have different RTs
• AS 100 was using 100:1
• AS 200 was using 200:1
• Check to see if the routes make it on ASBRs or RRs
Inter-AS MPLS VPNs
Problems with Route Installation: Checking on the RRs
R3#show bgp vpnv4 unicast all | b Netw
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:1
*>i 100.64.7.0/24 192.168.5.5 0 100 0 700 ?
*>i 172.16.57.0/24 192.168.5.5 0 100 0 ?
*>i 100.64.8.8/32 192.168.1.1 0 100 0 200 700 ?
*>i 172.32.68.0/24 192.168.1.1 0 100 0 200 ?
RP/0/0/CPU0:XR4#show bgp vpnv4 unicast | b Netw

Network Next Hop Metric LocPrf Weight Path
*>i100.64.7.0/24 192.168.2.2 100 0 100 700 ?
*>i172.16.57.0/24 192.168.2.2 100 0 100 ?
*>i100.64.8.8/32 192.168.6.6 0 100 0 700 ?
*>i172.32.68.0/24 192.168.6.6 0 100 0 ?
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 1 – Additional Import Statements
Simple Solution, but does it scale?
R3 (IOS PEs) XR4 (IOS XR PEs)
vrf definition VPN01 vrf VPN02
rd 100:1 address-family ipv4 unicast
route-target export 100:1 import route-target
!
export route-target
200:1
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 2 – Route Target ReWrite on ASBRs
IOS ASBRs (R1)
ip extcommunity-list 1 permit rt 200:1
route-map REWRITE permit 10
match extcommunity 1
set extcomm-list 1 delete
set extcommunity rt 100:1 additive
!
router bgp 100
neighbor 172.16.12.2 route-map REWRITE in
Inter-AS MPLS VPNs
Problems with Route Installation: Solution 2 – Route Target Re-Write on ASBRs
IOS XR ASBRs (XR2)

route-policy REWRITE
if extcommunity rt matches-any AS100VPN01
then
set extcommunity rt AS200VPN02
endif
pass
end-policy
!
extcommunity-set rt AS100VPN01
100:1 router bgp 200
end-set neighbor 172.16.12.1
! remote-as 100
extcommunity-set rt AS200VPN01 address-family vpnv4 unicast
200:1 route-policy REWRITE in
end-set route-policy PASSALL out
RR

bgp ASBR
router-id 192.168.2.2 MP-EBGP 192.168.2.2
192.168.1.1
AS100! ASBR AS200
R1
VPN02neighbor 172.16.12.1
XR2 VPN02
remote-as 100
PE
R5 ! XR6 PE
!
remote-as 200
CE next-hop-self CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
RP/0/0/CPU0:XR2#show mpls forwarding

Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------
PE ----------- ------------------ ------------ --------------- ------------
24000 R5Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 796 XR6 PE
24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12010
24003 60003 200:1:100.64.8.8/32 \
192.168.6.6 0
24004 60004 200:1:172.32.68.0/24 \
192.168.6.6 208
24005 Aggregate 172.16.12.0/24 default 0
CE
24006 R7206 100:1:100.64.7.0/24 \ CE
R8
172.16.12.1 0
AS 700
24007 207 100:1:172.16.57.0/24 \ AS 700
100.64.7.7 #CLUS 172.16.12.1 0 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
router static
172.16.12.1/32 GigabitEthernet0/0/0/1
PE
R5 XR6 PE
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02

------
PE ----------- ------------------ ------------ --------------- ------------
24000 R5Pop 192.168.6.6/32 Gi0/0/0/2 10.26.1.6 1070 XR6 PE
24001 Pop 192.168.4.4/32 Gi0/0/0/0 10.24.1.4 12383
24003 60003 200:1:100.64.8.8/32 \
192.168.6.6 0
24004 60004 200:1:172.32.68.0/24 \
192.168.6.6 20176
24006 206 100:1:100.64.7.0/24 \
CE Gi0/0/0/1 172.16.12.1 0 CE
R7 R8
24007 207 100:1:172.16.57.0/24 \
AS 700 Gi0/0/0/1 172.16.12.1 0 AS 700
100.64.7.7
24008 Pop 172.16.12.1/32 Gi0/0/0/1#CLUS 172.16.12.1 0 100.64.8.8
RR
R3 ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
CE7#trace 100.64.8.8
PE Tracing the route to 100.64.8.8
R5 VRF info: (vrf in name/id, vrf out name/id) XR6 PE
1 172.16.57.5 3 msec 2 msec 3 msec
5 172.32.68.8 [AS 200] 16 msec * 18 msec
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS VPN Option B2: Advertise Peering Link RR
RR
Redistribute Redistribute
R3
Connected Static Route ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
No LDP or IGP required on the link between the two ASBRs.

CE Configure no bgp default route-target filter on ASBRs CE
R7 R8
AS 700 ASBRs redistribute/advertise peering link into IGP AS 700
100.64.7.7 #CLUS 100.64.8.8
RR interface GigabitEthernet0/2
ip address 172.16.12.1 255.255.255.0
R3 mpls bgp forwarding ASBR XR4
ASBR ! MP-EBGP 192.168.2.2
192.168.1.1
router ospf 1
redistribute connected subnets
AS100
R1 network 10.0.0.0 0.255.255.255 ASBR area
XR2 100
AS200
VPN02 VPN02
network 192.168.0.0 0.0.255.255 area 100
!
router bgp 100
PE no bgp default route-target filter
R5 neighbor 172.16.12.2 remote-as 200 XR6 PE
!
CE neighbor 172.16.12.2 send-community extended R8 CE
R7
AS 700 AS 700
100.64.7.7 neighbor #CLUS
192.168.3.3 send-community extended
© 2019 Cisco and/or its affiliates. All rights reserved.100.64.8.8
Cisco Public
RR router ospf 1
redistribute static
R3 area 200 ASBR XR4
ASBR
interface Loopback0 MP-EBGP 192.168.2.2
192.168.1.1
interface GigabitEthernet0/0/0/0
AS100
! ASBR AS200
R1
VPN02 XR2 VPN02
router bgp 200
!
PE neighbor 172.16.12.1
R5 remote-as 100 XR6 PE
route-policy REWRITE in
!
CE remote-as 200 CE
R7 R8
AS 700 AS 700
100.64.7.7 address-family vpnv4 unicast#CLUS 100.64.8.8
Inter-AS VPN Option B3: Multi-hop RR
RR
Redistribute Redistribute
R3
Connected Connected ASBR XR4
192.168.1.1
AS100 ASBR AS200

R1
VPN02 XR2 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
Static route on ASBRs to reach remote ASBR loopback

ASBRs peer with each other via Loopback interface. Requires
EBGP Multi-Hop
CE CE
R7 Configure no bgp default route-target filter on ASBRs R8
AS 700 ASBRs advertise remote loopback into BGP AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS VPN Option B3: MultiHop RR
RR ip route 192.168.2.2 255.255.255.255 172.16.12.2
interface GigabitEthernet0/2
R3 ip address 172.16.12.1 255.255.255.0 ASBR XR4
ASBR mpls ip MP-EBGP 192.168.2.2
192.168.1.1
!
router ospf 1
AS100 redistribute static subnets ASBR AS200
R1
VPN02 XR2
network 10.0.0.0 0.255.255.255 area 100 VPN02
network 192.168.0.0 0.0.255.255 area 100
!
router bgp 100
PE no bgp default ipv4-unicast
R5 no bgp default route-target filter XR6 PE
neighbor 192.168.2.2 ebgp-multihop 255
!
CE neighbor 192.168.2.2 activate CE
R7 R8
AS 700 AS 700
100.64.7.7 neighbor 192.168.2.2
#CLUS
route-map REWRITE in Cisco Public
© 2019 Cisco and/or its affiliates. All rights reserved.100.64.8.8
Inter-AS VPN Option B3: Multihop RR
RR router ospf 1
redistribute static
R3 area 200 ASBR XR4
.. ASBR MP-EBGP 192.168.2.2
192.168.1.1
!
router bgp 200
AS100address-family vpnv4 unicast ASBR AS200
R1
VPN02 XR2 VPN02
!
remote-as 100
PE ebgp-multihop 255
R5 update-source Loopback0 XR6 PE
!
CE remote-as 200 CE
R7 R8
AS 700 AS 700
100.64.7.7 address-family vpnv4 unicast#CLUS 100.64.8.8
Inter-AS VPN Option C: MBGP between RRs RR
RR MP-EBGP
VPNv4 Routes/Labels
R3 ASBR XR4
ASBR 192.168.2.2
192.168.1.1
MP-EBGP
PE & RR Labels
AS100 ASBR AS200
R1
VPN02 XR2 VPN02
v1 172.16.1.1
PE
R5 XR6 PE
VPNv4 session is established between RRs

RRs use Next-Hop-Unchanged
CE
R7 ASBRs exchange RRs and PE loopbacks as labeled R8 CE
AS 700 routes AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS VPN Option C: RRs Peer Direct RR
RR
router ospf 1
R3 redistribute bgp 100 subnets ASBR XR4
ASBR network MP-EBGP 192.168.2.2
10.0.0.0 0.255.255.255 area 100
192.168.1.1
network 192.168.0.0 0.0.255.255 area 100
!
AS100 ASBR AS200
VPN02 R1 router bgp 100 XR2 VPN02
!
PE address-family ipv4
R5 network 192.168.1.1 mask 255.255.255.255 XR6 PE
network 192.168.3.3 mask 255.255.255.255
network 192.168.5.5 mask 255.255.255.255
neighbor 172.16.12.2 send-label
CE CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
router ospf 1
redistribute bgp 200
R3 areaASBR
200 ASBR XR4
MP-EBGP 192.168.2.2
interface
192.168.1.1 Loopback0
AS100 interface GigabitEthernet0/0/0/2 ASBR AS200
R1
VPN02
! XR2 VPN02
router bgp 200
network 192.168.2.2/32
PE network 192.168.4.4/32
R5 XR6 PE
network 192.168.6.6/32
allocate-label all
!
remote-as 100
CE address-family ipv4 labeled-unicast CE
R7 R8
AS 700 route-policy PASSALL out AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
router bgp 100
R3 ASBR XR4
ASBR bgp log-neighbor-changes
MP-EBGP 192.168.2.2
192.168.1.1
AS100 neighbor 192.168.4.4 ebgp-multihop 255 ASBR AS200
R1
VPN02 XR2
neighbor 192.168.4.4 update-source Loopback0 VPN02
!
PE neighbor 192.168.4.4 activate
R5 XR6 PE
neighbor 192.168.4.4 next-hop-unchanged
neighbor 192.168.4.4 route-map REWRITE in
CE neighbor 192.168.5.5 route-reflector-client CE
R7 R8
AS 700 AS 700
100.64.7.7 #CLUS 100.64.8.8
RR
router bgp 200
R3 ASBR XR4
ASBRbgp router-id 192.168.4.4
MP-EBGP 192.168.2.2
192.168.1.1
!
AS100 neighbor 192.168.3.3 ASBR AS200
R1
VPN02 remote-as 100 XR2 VPN02
ebgp-multihop 255
PE
R5 route-policy PASSALL out XR6 PE
next-hop-unchanged
!
!
remote-as 200
CE update-source Loopback0 CE
R7 R8
AS 700 route-reflector-client AS 700
100.64.7.7 #CLUS 100.64.8.8
Inter-AS VPN: Examining the VPNv4 Routes RR
RR
Check Routes
R3 Check Routes ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1
VPN02 XR2 VPN02
Check Routes
Check Routes
PE
R5 XR6 PE
Verify the source/destination routes at

entry/remote PE and local/remote ASBR
AS 700 AS 700
CE 100.64.7.7 100.64.8.8 CE
R7 R8
Inter-AS VPN: Examining the MPLS FECs RR
RR Check MPLS Check MPLS
Forwarding Forwarding
R3 Table Table ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1
VPN02 XR2 VPN02
Check MPLS Check MPLS
Forwarding Forwarding
Table Table
PE
R5 XR6 PE
Verify the source/destination is label switched towards

destination with numbered labeles or ASBRs have POP
Remember IOS XR needs a /32 entry for the FEC to
AS 700 populate. AS 700
CE 100.64.7.7 CE
R7 100.64.8.8 R8
Troubleshooting Inter-AS VPN: Tip RR
RR
Add a VRF Add a VRF
R3
Check Here Check Here ASBR XR4
ASBR 192.168.2.2
192.168.1.1
IPv4 + IGP/BGP
AS100 ASBR AS200
R1
VPN02 XR2 VPN02
PE
R5 XR6 PE
Sometimes the issue may not appear directly.
Add a loopback interface on ASBR, and place into a VRF.
Provides a method of checking connectivity across the
CE ASBR link. AS 700
R7 AS 700 CE
100.64.7.7 100.64.8.8 R8
MPLS Carrier
Supporting Carrier
(CSC)
Carrier Supporting Carrier (CSC)
• CSC allows MPLS services across discontiguous areas. Typically
when MPLS services cannot be provided end-to-end because of
geography reasons.
Service Provider 1
Service Provider 2 Service Provider 2
CE CE
Customer Customer
Carrier Supporting Carrier (CSC) Roles
CSC-PE CSC-PE
R1 CSC-CE
CSC-CE XR2
R3 XR4
Backbone
Carrier
Customer Customer
Carrier Carrier
R5 PE PE XR6
CE CE
R7 R8
AS 700 AS 800
#CLUS © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public
CSC is not running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
Labeled BGP LDP + IGP XR2 BGP
R3 XR4
IBGP + RR Client
IBGP
IBGP
Customer
Carrier
R5 XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
MP-IBGP
LDP + IGP
R1
Labeled
LDP + IGP XR2 BGP
R3 description to R3 XR4
vrf forwarding CORE
IBGP + RR Client
IBGP
IBGP
ip address 172.16.13.1 255.255.255.0

mpls ip
! Customer
router ospf Carrier
10 vrf CORE
R5 redistribute bgp 100 subnets XR6
network 172.16.0.0 0.0.255.255 area 200
!
EBGP EBGP
router bgp 100
CE7 address-family ipv4 vrf CORE CE7
AS 700 redistribute ospf 10
AS 800
MP-IBGP
Labeled BGP Labeled

R1
LDP + IGP XR2 BGP
R3 description to R3 XR4
vrf forwarding CORE
IBGP + RR Client
IBGP
IBGP
ip address 172.16.13.1 255.255.255.0

mpls bgp forwarding
! Customer
router bgp 100
Carrier
R5 address-family ipv4 vrf CORE XR6
network 172.16.13.0 mask 255.255.255.0
EBGP EBGP
CE7 neighbor 172.16.13.3 as-override CE7
AS 700 neighbor 172.16.13.3 send-label
AS 800
MP-IBGP
LDP + IGP
or Labeled
R1
router bgp 100 XR4

R3
vrf CORE
rd 100:1 IBGP + RR Client
IBGP
IBGP

allocate-label all Customer
Carrier
!
R5 XR6
remote-as 200
address-family
EBGP ipv4 labeled-unicast EBGP
CE7 CE7
AS 700
as-override AS 800
Where is MP-IBGP
172.32.24.4/32
LDP + IGP
or Labeled
R1
R3 mpls forwarding XR4

RP/0/0/CPU0:XR2#show
Local Outgoing Prefix Outgoing Next Hop
IBGP + RR Client Bytes
IBGP
IBGP

------ ----------- ------------------ ------------ --------------- ------------
24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 1271
Customer
24002 100 172.16.13.0/24[V] 192.168.1.1 0
24003 104 172.16.35.0/24[V]
Carrier192.168.1.1 0
24004 Aggregate R5
CORE: Per-VRF Aggr[V] \ XR6
CORE 4280
24005 103
EBGP 192.168.3.3/32[V] 192.168.1.1 0
EBGP
24006 106 192.168.5.5/32[V] 192.168.1.1 1022
24007
CE7 Pop 172.32.46.0/24[V] 172.32.24.4 0 CE7
24008 Pop 192.168.4.4/32[V] 172.32.24.4 49920
AS 700
24009 44005 192.168.6.6/32[V] 172.32.24.4 8312796 AS 800
router static MP-IBGP
vrf CORE
LDPipv4
address-family + IGP
unicast
or
172.32.24.4/32 GigabitEthernet0/0/0/0 Labeled
R1

R3 XR4
Label Label or ID Interface
IBGP + RR Client Switched
IBGP
IBGP
------ ----------- ------------------ ------------ --------------- ------------

24000 Pop 192.168.1.1/32 Gi0/0/0/1 10.12.1.1 296
24001 Pop 172.32.24.4/32[V] Gi0/0/0/0 172.32.24.4 1210
Customer
24002 100 172.16.13.0/24[V] 192.168.1.1 0
24003 104 172.16.35.0/24[V]
Carrier 192.168.1.1 0
24004 Aggregate R5 CORE: Per-VRF Aggr[V] \ XR6
CORE 4280
24005 103
EBGP 192.168.3.3/32[V] 192.168.1.1 0
EBGP
24006 106 192.168.5.5/32[V] 192.168.1.1 252
24007
CE7 Pop 172.32.46.0/24[V] Gi0/0/0/0 172.32.24.4 0 CE7
24008 Pop 192.168.4.4/32[V] Gi0/0/0/0 172.32.24.4 48880
AS 70044005
24009 192.168.6.6/32[V] Gi0/0/0/0 172.32.24.4 8092044 AS 800
MP-IBGP
LDP + IGP
or Labeled
R1
CE7#trace 100.64.8.8
Tracing the route R3
to 100.64.8.8 XR4
1 172.16.57.5 3 msec 3 msec 3 msec IBGP + RR Client
IBGP
IBGP
2 172.16.35.3 [AS 200] [MPLS: Label 3005 Exp 0] 16 msec 18 msec 15 msec
4 10.12.1.2 [MPLS: Label 24009 Exp 0] 14 Customer
msec 16 msec 16 msec
5 172.32.24.4 [MPLS: Label 44005 Exp 0] 14Carrier
msec 14 msec 16 msec
6 172.32.46.6 [AS
R5 200] 15 msec 15 msec 16 msec
XR6
7 172.32.68.8 [AS 200] 16 msec * 19 msec
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
R3
Customer XR4
Carrier
IGP
LDP
IGP
LDP
R5 IBGP XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS inside its POP Sites
MP-IBGP
LDP + IGP
or Labeled
R1
CE7#trace 100.64.8.8
Tracing the route R3
to 100.64.8.8 Customer XR4
VRF info: (vrf in name/id, vrf out name/id)Carrier
1 172.16.57.5 3 msec 3 msec 3 msec
IGP
LDP
IGP
LDP
2 172.16.35.3 [AS 200] [MPLS: Label 3005 Exp 0] 16 msec 18 msec 15 msec
6 172.32.46.6 [AS
R5 200] 15 msec 15 msec 16 IBGP
msec
XR6
7 172.32.68.8 [AS 200] 16 msec * 19 msec
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS VPN inside its POP Sites
MP-IBGP
LDP + IGP
or R1
Labeled
R3
Customer XR4
Carrier
IGP
LDP
IGP
LDP
R5 MP-IBGP XR6
EBGP EBGP
CE7 CE7
AS 700 AS 800
CSC is running MPLS VPN inside its POP Sites
CE7# trace 100.64.8.8 so lo0
Type escape sequence to abort. MP-IBGP
LDP + IGP
VRF info: (vrf in name/id, vrf out name/id) Labeled
or R1 4 msec
1 172.16.57.5 5 msec 3 msec LDP + IGP XR2 BGP
Labeled BGP
2 172.16.35.3
Backbone Carrier [MPLS: Labels 3005/60005 Exp 0] 18 msec 14 msec 24 msec
3Forwarding
172.16.13.1
Label [MPLS: Labels 112/60005 Exp 0] 15 msec 15 msec 16 msec
Customer XR4msec
4 10.12.1.2 [MPLS:
R3 Labels 24009/60005 Exp 0] 14 msec 12 msec 18
Carrier
5 172.32.24.4 [MPLS: Labels 44005/60005 Exp 0] 21 msec 23 msec 22 msec
IGP
LDP
IGP
LDP

7 172.32.68.8 [AS 800] 26 msec * 19 msec
R5 MP-IBGP XR6
EBGP EBGP
CE7 Customer Carrier CE7
VPN Label
AS 700 AS 800
Complete your
online session • Please complete your session survey
evaluation after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live water bottle.
• All surveys can be taken in the Cisco Live
Mobile App or by logging in to the Session
Catalog on ciscolive.cisco.com/us.
Cisco Live sessions will be available for viewing
on demand after the event at ciscolive.cisco.com.
Continue your education
Demos in the
Walk-in labs
Cisco campus
Meet the engineer

Related sessions
1:1 meetings
Thank you
#CLUS
#CLUS

Mpls l3vpn Tecmpl-3201

Uploaded by

Copyright:

Available Formats

Mpls l3vpn Tecmpl-3201

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mpls l3vpn Tecmpl-3201

Uploaded by

Copyright:

Available Formats

#CLUS

Webex Teams will be moderated cs.co/ciscolivebot#TECMPL-3201

R1#ping 192.168.5.5 so 192.168.1.1

install feature-set mpls

In Out In Out In Out

In Out In Out In Out

PE1#show tcp brief| i 646

PE1#telnet 192.168.11.11 646 /source-interface lo0

• Following avoids both shortcomings

PE1#show mpls ldp discovery

PE1#show mpls ldp discovery P1#show mpls ldp discovery

R1#debug mpls ldp transport connections

PE1(config-if)#mpls label protocol ldp

PE1#show mpls ldp discovery P1#show mpls ldp discovery

PE1#show ip route 192.168.11.11

Problem: Default route towards the peering router

PE1#show mpls ldp discovery PE2#show mpls ldp discovery

0/0/CPU0 t1 [PEER]:581: VRF(0x60000000): Peer(192.168.11.11:0): DOWN - reason 'TCP connection closed'

router (config-isis-if-af) # mpls ldp sync [ level <1-2> ]

router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if)

router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar)

 LDP IGP Sync delays are configured under LDP

router (config-ldp) # igp sync delay on-session-up <sec>

router (config-ldp) # igp sync delay on-proc-restart <sec>

router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> | infinite } ]

router (config-ldp) # log session-protection

• Lo0 defined as the LDP 10.12.1.0/24

router-id on both routers R2

• LDP adjacency is formed

CE1 PE1 P1 PE2 CE2

• Ping mode: Connectivity check of an LSP

• Traceroute mode: Hop by Hop fault localization

• Verify data plane against the control plane

Message Type Reply Mode Return Code Return Subcode

Timestamp Sent (seconds)

Timestamp Sent (microseconds)

Timestamp Received (seconds)

Timestamp Received (microseconds)

1 Malformed Echo Request Received

2 One Or More of the TLVs Was Not Understood

3 Replying Router Is an Egress for the FEC

4 Replying Router Has No Mapping for the FEC

5 Replying Router Is Not One of the “Downstream Routers”

- 172.16.2.2/32 Y 19 24008 24008 172.16.2.2/32 Y -

In Label Prefix Output Out Label

CE1 22 192.168.1.1/32 X pop CE2

CE1 PE1 P1 PE2 CE2

172.16.1.1 TTL=252 Label 16, TTL=253 Label 16

CE1#traceroute 172.16.2.2 source 172.16.1.1 (no mpls ip propagate-ttl forwarded)

CE1 PE1 P1 PE2 CE2

172.16.1.1 TTL=254, 172.16.1.1 TTL=254,

192.168.6.6/32 192.168.1.1/32 192.168.4.4/32

192.168.6.6/32 3 192.168.1.1/32 192.168.4.4/32

192.168.6.6/32 192.168.1.1/32 192.168.4.4/32

PE1#show mpls forwarding-table 192.168.2.2

• Outgoing label also conveys what treatment the packet is going

 So, depending upon the case, we need to check:

14/18 means that the L2 header is of 14 bytes, but

• Need to support transport for all services from all locations