SDR and NFV Extensions in The Ns-3 LTE Module For 5G Rapid Prototyping
SDR and NFV Extensions in The Ns-3 LTE Module For 5G Rapid Prototyping
SDR and NFV Extensions in The Ns-3 LTE Module For 5G Rapid Prototyping
Marco Miozzo, Nikolaos Bartzoudis, Manuel Requena, Oriol Font-Bach, Pavel Harbanau, David López-Bueno,
Miquel Payaró, Josep Mangues
Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA), Spain
{mmiozzo, nbartzoudis, mrequena, ofont, pharbanau, dlopez, mpayaro, jmangues}@cttc.es
Abstract— The virtualization of mobile network functions As for the mobile network architecture, a variety of
constitutes one of the main blocks for addressing the high functional splits have been proposed and deployed in recent
flexibility requirements of fifth generation (5G) communication years. For instance, Cloud Radio Access Network (CRAN)
systems. Reconfigurable hotspots are expected to be massively developments [1][2] have been quite common; in CRAN only
deployed to enable on-demand services and dynamically adapt some physical layer processing is left next to the antenna and
the network capacity according to traffic requirements. In this signal samples are carried to data centers for processing. Other
paper, we present the extensions and modifications of the long options with less stringent data rate and timing requirements
term evolution (LTE) module of the ns-3 simulator (LENA) to have also been proposed (e.g., [3]), providing a new type of
include a software defined radio (SDR) physical layer
virtualization by splitting MAC and PHY layers. More
implementation. These extensions combine the native flexibility
of the simulator with the SDR features of a real-time prototype.
recently, various splits at PHY, MAC, RLC and PDCP layers
Moreover, the framework was designed to distribute the have been considered for relaxing the stringent requirements of
communication functions across different elements of the CRAN while maintaining the benefits of centralized processing
network with the possibility of adjusting several transmission [4]. Such MAC-PHY splits imply structural modifications and
parameters as in a network function virtualization (NFV) extensions to existing research tools in order to enable or
paradigm. Thanks to an emulated full network protocol stack, accommodate SDN/NFV features. For instance, prototypes for
the prototype allows the experimentation of novel 5G solutions 5G should enable the reconfiguration of the network according
and the evaluation of relevant key performance indicators (KPIs) to the actual system conditions.
from the lower layer protocols up to application level. To this
aim, we present the experimental evaluation of the KPIs of
In this paper, we present a novel work to combine the
energy, latency, throughput and reconfiguration time in relevant flexibility of the well-known and widespread open source ns-3
scenarios. LENA long term evolution (LTE) simulator/emulator [5] with
a real-time field programmable gate array (FPGA)
Keywords—5G prototyping, SDR, SDN, NFV, testbed implementation of the PHY-layer [6]. Towards this end, we
implemented an interface that enables the real-time
communication between the FPGA and LENA modules, which
I. INTRODUCTION
in turn allows the FPGA to implement the channel resource
The fifth generation (5G) of mobile communications will allocation defined by LENA. Moreover, we enabled the
need to support a wide range of services, including enhanced emulation of different flexible functional splits for moving the
mobile broadband, ultra-reliable low latency, and massive communication functions across different network nodes with
machine-type communications. According to this, flexibility processing capabilities for mimicking the virtual small cells
will play a crucial role in fulfilling the relevant 5G key (vSC) paradigm [4]. This approach allows maintaining the
performance indicators (KPIs). Examples of such KPIs are: typical advantages of a simulator (e.g., scalability, replicability,
1000x increase in area capacity, reduction of service creation flexibility and low computational complexity), and
time from hours to minutes, zero perceived downtime, or 90% simultaneously move toward a rapid prototyping approach for
energy consumption reduction. Due to the flexibility 5G networks. In fact, the SDR PHY layer extension for LENA
requirements and the increasing capacity of general-purpose enables the real-time over-the-air transmission of actual RF
processing units, software is acquiring a more prominent role signals, contemplating likewise real-world wireless channels.
in network architecture design and deployment. To this respect, Thus, real-time hardware-based experiments can be conducted
Software Defined Radio (SDR), Software Defined Networking to more closely represent the behavior of real-world wireless
(SDN) and Network Functions Virtualization (NFV) have been networks. This will allow to realistically evaluate the
recognized as main building blocks for 5G not just for performance of next generation 5G wireless networks, through
flexibility, but also for energy efficiency and network the analysis of KPIs not only in an end-to-end basis, but also
programmability. when focusing on the individual modules. Moreover, the
integration of the underlying software (SW) and hardware
accelerated (HWA) modules was designed to allow validating The design of the system took particular attention in
scenarios with different system bandwidths (BWs), modulation providing a high level of flexibility with respect to function
and coding schemes (MCSs), resource block (RB) loads, partitioning. HWA and SW functions can be placed and
transmitter output power levels, waveforms (e.g., LTE vs. 5G executed in different processing elements of the network,
candidates), antenna schemes and transmitted output power which may be seen as an enabler for NFV, and each of the
levels. Thanks to this design, the framework maintains the software building blocks, as a virtual network function (VNF).
typical scalability of network simulators and can be used to Currently, LENA when combined with the SDR and NFV
evaluate new solutions in wide scenarios with deterministic extensions allows the emulation of three function splits (FS).
environmental conditions, while allowing to emulate realistic The first one is the classical CRAN, where the entire eNB
over-the-air transmissions. This twofold nature represents an protocol stack is placed in the Cloud together with the EPC; the
important and unique feature since it enables moving rapidly eNB site hosts only the remote radio head (RRH). In the
from the preliminary simulation-based analysis to the proof of second one, the L1 is placed locally at the eNB site, whereas
concept with prototypes. the higher layers starting from upper MAC are in the Cloud, as
in the split MAC architecture [4]. Finally, the third FS is
II. SYSTEM ARCHITECTURE similar to the previous one, except for the higher layers
functions of the eNB are placed in a processing node close to
A. Overall System Architecture the L1, which is inspired by the multi-access edge computing
(MEC) approach.
The main blocks comprising the architecture of the system
are based on the SW and HWA parts shown in Fig. 1, which Thanks to the HWA and SW blocks, the system supports
according to the specific setup can target different 5G most of the LTE functionalities and is therefore able to emulate
virtualization configurations. Considering that the 5G the whole end-to-end LTE network. Table I provides the list of
standardization is under definition, we utilized the 4G LTE the main features. At the time of writing, the system only
technology, since the basic architectural concepts still apply. supports over-the-air transmissions in the DL. The UL is under
Thanks to the flexibility of the implementation, any extensions development and therefore the L1 UL is bypassed, which
required to include new 5G techniques can be contemplated in translates in having an ideal error-free UL channel.
the future. The stack is composed by a SW part based on the
LENA open source network simulator and emulator. LENA B. L2 and above
has been mainly developed at CTTC, which nowadays L2 and above protocols rely on the SW implementation
maintains the module in the official ns-3 distribution. LENA
based on the LENA module. LENA has been originally
has been designed to simulate a full LTE protocol stack,
designed as a simulator. Therefore, it does not include a
including the evolved nodeBs (eNBs), the user equipment (UE)
and the evolved packet core (EPC). The HWA includes the complete L1 implementation, but relies on link-to-system
LTE-based downlink (DL) physical-layer (L1) developed at techniques to build an abstract model to estimate the channel
CTTC [7] that targets FPGA-based system-on-chip (SoC) impact on the higher-layer protocol data units. In order to be
devices. As it will be detailed in this section, the previous two integrated with the HWA system, new features have been
building blocks were significantly modified and extended in introduced. The main one is implementing the functionalities
order to be integrated in a single platform. The SDR-based of the L2-L1 interface that enable the interconnection between
LENA testbed guarantees that computationally-intensive real- LENA and the HWA L1, which in turn interfaces with an
time functions can run without performance degradation, SDR-based sub-6 GHz RF front-end. With this setup, a fully
allowing to validate 5G KPIs at run-time. real-time, end-to-end, over-the-air DL communication link can
be utilized instead of relying on an emulated wireless channel.
In detail, two ns-3 functionalities have been integrated. Since
LENA is natively working in simulation mode that uses an
internal simulated clock, the RealTime ns-3 event scheduler is
used to synchronize the protocols with the real local clock of
the hardware in which the process is running. This allows
generating packets in real-time. Additionally, modifications
were made to allow the interaction with external SW-HWA
modules. The simulator maintains all the data within its
simulated scenario, i.e., inside the simulation process. By using
the FileDescriptor NetDevice functionality of ns-3, LENA can
exchange real IP packets with external HWA and SW
components. Thanks to these features, LENA is able to split
different network functionalities and make them interact. In
detail, the function comprising the EPC, the UE and the eNB
can be placed in different processes running in different
machines. Moreover, LENA can also interact with real
applications (e.g., voice and video-streaming clients) and with
Fig. 1. Overview of the system. real hardware, as done with the L1 HWA.
2018 IEEE Wireless Communications and Networking Conference (WCNC)
TABLE I. SYSTEM FEATURES The L2-L1 interface has been implemented starting from
LTE Protocol Properties the scheduler application programming interface (API) of
DL transmission with 20 MHz BW in FDD mode LENA, which is based on the standard defined by the Small
LTE-compliant channel coding Cell Forum, to mimic a MAC split virtualization [4]. Thanks
Data channels (PDSCH, PUSCH) and a to this API, the common control messages have been
eNB PHY simplified control channel (PDCCH) serialized in time-stamped frames to be exchanged with the
Achievable data rates up to 75 Mbps L1. For instance, the DL control information (DCI) allows
Reference symbols: CRS, UERS, PSS, SRS
Reusable FPGA implementation
managing the MCS and RB allocation profile of the UEs on a
IQ and DC offset correction millisecond basis, or the transmission time interval (TTI) in
Automatic gain control (AGC) LTE nomenclature. The original LTE L1 modules of LENA
Time and frequency synchronization for both the eNB and UE have been replaced by new ones in
Cell-specific and UE-specific channel estimation charge of encapsulating the control and data plane in each TTI
UE PHY
Channel equalization of data symbols
to be transmitted through the L2-L1 interface (“L2 Int” in Fig.
Turbo decoding (data channels) and Viterbi
decoding (control channel) 1). The control plane includes the main API primitives for
Reusable FPGA implementation allowing the DL communications. The data plane serializes
MAC Error correction through CRC the transport blocks (TBs) per logical channel (LC) basis
(Medium Logical Channel multiplexing enabling their multiplexing and includes the error detection
Access Dynamic resource scheduling with Round Robin through cyclic redundancy error check (CRC). The CRC has
Control) Random Access Procedure
Fragmentation, Concatenation and packet
been adopted since HARQ is not implemented at this stage. In
RLC reordering LTE, one subframe is generated every millisecond and the
(Radio Link Transmission modes: Transparent mode (TM), MAC and PHY operations are conditioned by this stringent
Control) Unacknowledged mode (UM), Acknowledge requirement. Fig. 2 shows the jitter between subframes
mode (AM) generated at the MAC layer for a 20 MHz BW configuration.
PDCP (Packet
Real headers following 3GPP specs As it can be seen in the figure, this difference is smaller than
Data
Maintenance of PDCP sequence numbers (SN) 100 microseconds for most of the subframes (i.e. lower and
Convergence
Transfer of SN status (for handover)
Protocol) upper quartiles). In the boxplots of the figure, lower and upper
RRC quartiles are at 1 % and 99 %, respectively. Only a few
Connection management (establish and
(Radio
Resource
reconfiguration) subframes have a bigger difference when the MCS is
System information (MIB, SIB, etc.) increased. This dispersion is compensated in the PHY layer by
Control)
Focus on NAS Active state buffering a few subframes. For smaller BWs, there is no such
NAS UE Attachment outlier.
(Non-access Evolved Packet System (EPS) Bearer activation
Stratum) Multiplexing of data onto active EPS Bearers At the MAC layer, the Round Robin (RR) scheduler has
(based on Traffic Flow Templates) also been updated to include the constraints of the HWA L1. In
S1-U and S1-C (user data and control plane) fact, natively, LENA does not consider any limitation in terms
realistic model including GTP-U of the number of UEs that can be allocated simultaneously due
EPC (Evolved X2-U over GTP/UDP/IP packets
to physical DL control channel (PDCCH) resource limitations.
Packet Core) X2-C over UDP packets (no standard encoding)
S11 interface abstract model (no GTP-C PDUs This requires extending the RR scheduler to allow the
exchanged) simultaneous transmission of a limited number of UEs as a
function of the specific BW configuration. Similarly, the
amount of data that can be transmitted in subframe 0 has been
reduced in order to enable the L1 to fit the physical broadcast
channel (PBCH) within the PDSCH.
A sketch of the main elements modified in the LENA
simulator is depicted in Fig. 1 highlighting the modules and
the logical connections that have been extended in bold red
lines. For debugging purposes, the loopback mode
(highlighted with a red dashed line) allows emulating the
network without the HWA L1 (i.e., the L1 will be a cable
connection, which translates to an ideal error-free channel). It
has to be underlined that, all these extensions have been
implemented in a transparent mode with respect to LENA,
which enables working in two different operative modes: the
standard simulation and the SDR emulation.
C. Real-time FPGA-based PHY-layer (DL)
The digital signal processing (DSP) blocks of the L1 were
Fig. 2. Difference time between consecutive L2-L1 frames generated by the implemented as real-time FPGA-based HWA functions (“L1
SW.
2018 IEEE Wireless Communications and Networking Conference (WCNC)
HWA” in Fig. 1). The register transfer level (RTL) design switch) is reported in Fig. 3. The variance in latency is caused
made use of Xilinx intellectual property cores (i.e., by the network connection and the CPU processing of
precompiled synthesizable DSP functions) which along with incoming packets. The L2-L1 interfacing frames are designed
custom-designed DSP blocks and control units ensure a with a custom format based on the Small Cell Forum API,
flexible operation of the logic. A concise representation of the which has been adapted to efficiently exchange the parameters
L1 processing blocks is shown in Fig. 4. The HWA L1 can be of both control and data plane. Table II provides the list of the
adapted in a subframe basis to the requirements of the L2, the main parameters exchanged for the DL transmission.
configured FS and the selected BW. The L2 defines the
configuration of the different blocks comprising the PHY-
layer according to the instantaneous operative requirements of
the eNB (e.g., channel coding parameters). The control
information propagates from the L2 via a processing system
(PS) embedded in the FPGA device; a series of custom L2-L1
interfacing frames are generated in the PS. The HWA L1
disposes a central state machine that parses this information
and places the user data in the DL shared channel (DLSCH),
programs the required parameters for the turbo encoding stage,
generates the contents of the PDCCH and the PBCH and
finally programs the parameters of the convolutional encoding
stage. This state machine is also responsible for handling
missing or wrongly decoded control information (e.g., errors
in the L2-L1 communication). In that case, the eNB discards
incoming data until receiving valid one from the L2 (i.e., new Fig. 3. Input data latency for BW=20MHz, MCS=10 system configuration
frame generation starting from subframe 0). The error
occurrences are signaled to an embedded memory buffer, TABLE II. L2-L1 INTERFACE DL MAIN PARAMETERS
which resides in the HWA part of the L2-L1 interface; the
latter issues an interrupt to the PS guaranteeing that exceptions Frame Description
will be appropriately handled from all the involved building Radio Network Temporary Identifier
(RNTI)
blocks. A second state machine allocates the contents to each DL-DCI MCS
resource element (RE) in the DL signal, which is then used as TB size
an input to the inverse fast Fourier transform (iFFT) and the RB bitmask
cyclic prefix (CP) insertion DSP blocks. The result of different RNTI
L2-L1 communication errors is also handled from this state MCS
UL-DCI TB size
machine; if for example the required DLSCH control is not
RB start
available when needed, a request to the central state machine RB length
halts the eNB transmission. MIB (Master DL BW
Information Block) System Frame Number
III. L2-L1 INTERFACE Cell identity
SIB1 (System
The L2-L1 interface enables the time-constrained Information Block) Public Land Mobile Network (PLMN)
identifier
interaction between the SW and HWA modules (“L2L1 Int” in Random Access RNTI
Fig. 1). It has been designed to be modular facilitating the RAR (Random RAR Num
interconnectivity and virtualization of the system. The SW and Access Response) Random Access Procedure (RAP) id.
the HWA can be treated as separate entities that communicate RAR UL grant
in this case via UDP, guaranteeing a real-time and over-the-air
communication between the eNB and UE PHY layers. The IV. HARDWARE SETUP
real-time L2-L1 interface is placed in the ARM-based PS of In this section, we present the hardware boards and
the Xilinx Zynq FPGA. In order to satisfy the stringent equipment that have been used to validate the SDR extension
requirements of latency, it is executed on a customized Linux of LENA. The L1 HWA processing blocks and the L2-L1 SW
distribution with a fully preemptive kernel, which enables interface of the eNB prototype are hosted in the Xilinx ZC706
real-time tasks. Apart from this, different queuing and board, which features the Xilinx Zynq XC7Z045 SoC device.
buffering techniques have been applied. This ensures that the The Analog Devices AD-FMCOMMS3 RF transceiver board
strict 1 millisecond timing requirement for the combined low is plugged to the Xilinx ZC706 board. A suitable power
MAC/PHY operations is met, including the exchange of amplifier (PA) unit, RF band-pass filter and antenna were also
control and user plane data between the two layers. An interfaced with the AD-FMCOMMS3 board. A Linux kernel
example of the latency measured over 1 minute in the UDP space application that runs at the PS side of the Zynq
communication between a laptop hosting the SW part of the XC7Z045 device is used to tune and program the AD9361 RF
system and the board hosting the HWA part (via an Ethernet transceiver IC (RFIC) at the AD-FMCOMMS3 board. The
2018 IEEE Wireless Communications and Networking Conference (WCNC)
Xilinx ZC706 board is interfaced with CTTC's EXTREME (ALAT) plus the processing time counting from the first L2-
Testbed [8] using a GigE connection. EXTREME comprises input packet to the first output sample of the L1 inserted to the
generic purpose servers configured either as a datacenter or digital analog converter (DAC) (BLAT); the calculated latencies
distributed throughout the network and hosts LENA's SW- are based on deterministic measurements of the L1 digital
based eNB and UE stack (i.e., L2 and above), as well as the design:
EPC. The setup can also include a real-time multi-channel
L1LAT = ALAT + BLAT (2)
emulator (Elektrobit Propsim C8) able to realistically
reproduce the effect of standard or custom designed mobile
channels. A diagram of the overall hardware setup is provided
in Fig. 4.
5a. Power consumption of the eNB baseband processor featuring the L1.
B. Latency KPI BLAT considers the encoding of the most demanding UE data
sequence (i.e., up to 25 UEs, MCS up to 26, DL BW of 20
The latency has been calculated for both L1 and L2-L1
MHz) for the HWA L1 design (i.e., deterministic latency of
interface for all four supported BW configurations assuming
the RTL architecture). Taking into account (1) and (2), the
the FSs where the L1 is placed in the eNB site. The total
LATTOTAL for each of the supported BW configurations was
latency for the baseband processing and L2-L1 interface is
calculated as follows:
given by:
LATTOTAL = L1LAT (BB processing) + L2L1LAT (interface) = ALAT +
LATTOTAL = L1LAT (BB processing) + L2L1LAT (interface) (1)
BLAT + L2L1LAT (interface), thus:
The L2-L1 interface latency has a fixed value (i.e., ring buffer
LATTOTAL(1.4MHz) = 2.74 + 1.62 + 10 = 14.36 ms
solution described in section III), hence L2L1LAT = 10
milliseconds. The L1LAT is equal to the initialization time LATTOTAL(5MHz) = 1.37 + 1.37 + 10 = 12.74 ms
2018 IEEE Wireless Communications and Networking Conference (WCNC)