ServiceProtector Student Guide
ServiceProtector Student Guide
ServiceProtector Student Guide
Student Guide
Operators & Administrators Training
Introduction
Service Protector Operators Training
Introduction 1-2
Service Protector Operators Training
Introduction 1-3
Service Protector Operators Training
Service Provider networks are under constant threat of attack. Note that our
focus is on threats to the service provider’s key asset – its network and
infrastructure. Threats to individual subscriber computers – viruses for example,
are not covered here, as they are, by their very nature, of less concern to the
provider. We will concentrate on 5 security threats: Worms, Zero day attacks,
DDos attacks, Spam and Zombies & Botnets
We will now examine each of these threats in turn. As we proceed, you will notice
how each of the threats are inter-related.
inter related.
Introduction 1-4
Service Protector Operators Training
Introduction 1-5
Service Protector Operators Training
Introduction 1-6
Service Protector Operators Training
Introduction 1-7
Service Protector Operators Training
During Zero Day or hour attacks, there are huge volumes of flow records that
overload flow collection and flow analysis devices.
This overload renders security detection approaches inoperative. Billing and
reporting data is polluted by the worm traffic and delays in delivery are
experienced.
Introduction 1-8
Service Protector Operators Training
A well known example of a Zero Day attack that you may have heard about was
the SQL Slammer worm. It was first detected back in January 2003 when several
service providers noted significant bandwidth consumption at their peering points.
At the height of the zero day attack, average packet loss ran at 20%. ATMs cash
machines were affected, several airline ticketing systems were completely
overwhelmed – indeed, the country of South Korea lost almost all of its Internet
service for period of time.
Introduction 1-9
Service Protector Operators Training
1-
Introduction 10
Service Protector Operators Training
1-
Introduction 11
Service Protector Operators Training
In late April 2007, a wave of coordinated DDoS attacks struck web sites and
resources associated with the government of Estonia.
Within hours, the site of Estonia's leading national newspaper and a leading
national bank had been brought down. Web sites belonging to the Estonian
government including that of the nation's prime minister were crippled.
The attack, was widely linked to a very public diplomatic dispute between nations
and it was the first time that a DDoS attack threatened the national security of an
entire nation
nation.
1-
Introduction 12
Service Protector Operators Training
What about spam? We are all familiar with the annoying issue of INCOMING
spam – the bombardment of a subscriber’s mailbox with endless unsolicited
mails. But what concerns the Service Provider more is OUTGOING spam.
Sending lots and lots of spam from a service provider’s network might cause an
ISP to be blacklisted as a mail relay spammer.
If an ISP’s domain is blacklisted, this can have dire consequences. Spamming
blacklists are commonly circulated and used by mail servers. If an ISPs domain
has been added to a blacklist, then mail from the ISP will be rejected at each
server which has imported this blacklist.
1-
Introduction 13
Service Protector Operators Training
1-
Introduction 14
Service Protector Operators Training
To recruit more zombies, worms are particularly effective because they do not
require human intervention to spread. They simply need a network connection to
a potential candidate. They scan IP addresses for potential candidates. This
activity can create very high volumes of traffic.
If the zombie is sending high volumes of DDoS or worm traffic, then the
subscriber may experience a slow Internet connection (particularly if they are
uploading data). This typically leads to numerous and lengthy calls to the call
center.
Zombies are often used to deliver SPAM mail themselves – hence the name
SpamBots. As its relatively easy for an ISP to detect abuse of its own mail server,
bots set themselves up as mail relays themselves. If the ISP’s domain is
blacklisted, customers will soon start complaining that their email is not being
delivered.
In addition, worm and DDoS traffic can create high volumes of international
traffic. This is often usually highly undesirable from a traffic engineering and
commercial perspective.
1-
Introduction 15
Service Protector Operators Training
In summary, service providers face the following pain points from the different
security threats that we’ve examined here.
First and foremost they are experiencing Increased Operational expenditure. This
can be put down to 4 separate security-related reasons: They see elevated
international bandwidth costs due to worm and DDoS traffic. They see escalating
call center costs due to rising helpdesk complaints. They face SLA penalties
when their network doesn’t perform, and they also need to invest in their
infrastructure by increasing bandwidth or upgrading routers to cope with
anomalous traffic
Secondly, they find that their customers are increasingly demanding Denial of
Service protection services
Thirdly, not only do they fail on SLAs opposite their customers, but they also fail
to deliver on internal network performance SLA’s
And finally,
y, theyy face email blacklisting
g as a result of outbound spam
p coming
g from
their subscribers.
1-
Introduction 16
Service Protector Operators Training
1-
Introduction 17
Service Protector Operators Training
1-
Introduction 19
Service Protector Operators Training
1-
Introduction 20
Service Protector Operators Training
1-
Introduction 21
Service Protector Operators Training
1-
Introduction 22
Service Protector Operators Training
To see how the ServiceProtector works in practice, lets look at an example where
we identify and mitigate zombie activity.
The sensor first detects the abnormal activity, communicating the traffic patterns
to the controller. The controller then sends a message to the SMP switching the
service plan of the subscriber in question to a quarantined service plan. The new
service plan is then enforced on the NetEnforcer or Service Gateway. Typically, it
blocks or limits the zombie traffic or perhaps redirects the subscriber to a captive
portal for further inspection and cleansing.
1-
Introduction 23
Service Protector Operators Training
1-
Introduction 24
Service Protector Operators Training
Now lets briefly review the technology upon which the ServiceProtector solution
is based.
1-
Introduction 25
Service Protector Operators Training
1-
Introduction 26
Service Protector Operators Training
Floods are detected by modeling and analyzing the monitored network traffic
according to the fundamentals of TCP/IP and detecting anomalous behavior
when the observed traffic breaches the expected model. Once a flood is
detected, the SPS will do a packet capture and run an analysis on it. If the flood
is long enough, subsequent captures occur every 3 minutes to the max tracking
time of one hour.
The analysis results in a pattern that can be imported into common filtering
devices to block this attack. The NBAD system notes the anomalous behavior
and logs it as a flood event.
Note: Entropy in information technology is a measure of randomness in a
sample. Low entropy represents low level of randomness. In network attacks, the
attack packets are typically identical (as it’s faster to generate packets this way),
and due to their volume, the sample of packets taken during the anomaly shows
low entropy. When correlating packets and deciding out of the sample, how many
packets show the most commonality
commonality, this is essentially determining which is the
largest set of packets that is showing the lowest amount of randomness and
identifying the exact elements that are identical across the sample. These
features contribute to Allot’s Deep Packet Signature and this information is used
to create surgical filtering policies.
1-
Introduction 27
Service Protector Operators Training
1-
Introduction 28
Service Protector Operators Training
1-
Introduction 29
Service Protector Operators Training
The service protector monitors all of the traffic on the network. In a typical SP
deployment, the administrator will create groups of network prefixes in such a
way that the network is presented in a logical manner. Subscribers will be
typically grouped together by their prefixes. The result is one or more groups with
the prefixes of your subscribers. Further grouping may be configured, for
example according to service level – Gold/Silver, or any other means that the
administrator feels is logical. Once groups have been setup, the ‘quarantine’
option is turned on for that group, effectively enabling HBAD detection.
From this traffic, flows are created by the internal flows generator and sent to the
HBAD detection engine. Flows for every host are analyzed and the behavioral
profile of the host is inspected. If suspicious behavior is detected, a capture of
1000 flows is initiated for that host. Once the capture is complete the flows are
analyzed and if suspicious behavior is found, the results are stored and (if the
administrator wishes) the subscriber may be mitigated. The following four activity
types are classified: flow bomb, address scan, mass-smtp
mass smtp or port scan.
The system has a backoff period of 1 hour and detection for that host resumes
again once the hour is over. Detection is once an hour regardless of if the host is
infected or not. This ensures that if the host becomes active at some stage
(previously inactive), they will be detected at the next cycle.
Operators receive alerts and can open the GUI to view further information before
making a decision to mitigate or not. In addition, the GUI has a HBAD activity
page, similar to the flood activity page where the operator can interactively query
the database for events of interest, or view events matching notification filters.
1-
Introduction 30
Service Protector Operators Training
1-
Introduction 31
Service Protector Operators Training
1-
Introduction 32
Service Protector Operators Training
1-
Introduction 33
Service Protector Operators Training
Upon login, you are presented with a preset status screen displaying the last
hour of traffic, the most significant groups, and the active & inactive NBAD and
HBAD events at this time. This is by no means a detailed analysis, and is more of
a snapshot of what’s going on at this moment. The screen auto refreshes every
few seconds.
The main menu allows you instant access to the following options:
-This Start page
-The Traffic
ff pages (covered
( in the following
f slides))
- The NBAD/Flood pages (covered in module 2)
- The HBAD/Quarantine pages (covered in module 3)
- The Ad-Hoc Sampling page (covered in the following slides)
- Help
- Logout
1-
Introduction 34
Service Protector Operators Training
Several filters are found in the upper portion of each page or report. These may
vary depending on the specific filters required
required, but the most common filters to
appear are:
• Sensors – devices physically connected to the network and monitoring traffic
• Groups – logical collections of network prefixes
• Time range – preset or user selected times
• Timezone – displays the report in the selected timezone.
NOTE: Timezone refers to the client side. It is stored in the browser as a cookie
and charts are displayed in the user’s
user s timezone.
timezone Operators in different timezones
can each view the GUI in their own timezone with no conflicts.
• Report-specific controls. Other controls specific to that report (e.g: Protocols)
Each filter has the following two checkboxes:
• The All button (a star). This selects all the items in the list directly below.
• The Join button (a sigma). This collects the information from all the marked
items and joins their values into a single trace. For example, if all protocols in the
traffic detail’
‘traffic detail report were selected and the Join button was not selected
selected, each
protocol would be charted as a separate line. If Join were selected, only one
trace would be displayed, the aggregate of all the selected protocols.
Clicking the Update button will generate the new report. In some cases an option
to restore previous values will appear on the chart. This display alerts the user to
a difference between the current chart information and the selection made by the
user. Most reports are either auto refreshing or have a countdown timer to auto
refresh.
1-
Introduction 35
Service Protector Operators Training
The traffic detail graph displays the traffic for the defined period and displays it
according to the defined filters. The interactive traffic chart contains four controls:
•‘+’ enables zoom in. Mark the area requested and the chart updates to match.
Double clicking this button zooms in 3x.
•‘-‘ enables zoom out. Click and hold, then move the mouse to one side to zoom
out. Double clicking this button zooms in 3x.
•<-> pan mode. Click and drag the chart to pan left and right.
•‘*’ inspect mode. Click to select points or mark areas. NBAD and HBAD events
within the marked areas will be listed below the chart. More than one area can be
selected.
1-
Introduction 36
Service Protector Operators Training
Traffic - Trend provides visibility into traffic trends over a selected period of time.
This report uses the same traffic data as Traffic - Detail, therefore the data
available is the same as for the Traffic report. The time selection area on the top
allows different time ranges or preset values to be selected.
The major select boxes are identical (Sensors, Groups, Protocols, Directions,
and the output charts). Once you've selected the data to be charted, select
duration and choose the output format (Units and Display). You can receive
Bytes, Packets or packet size charts, and display them in one of four formats
(Bar chart stacked, normal bar chart, Pie chart or Plain text).
1-
Introduction 37
Service Protector Operators Training
Ad hoc sampling unlocks the powerful sniffer capabilities of the service protector.
SP sensors deployed in your network have 100% visibility of the traffic they
monitor. Traffic capture enables system administrators to conduct network
diagnosis and troubleshooting directly from a device already present on the
network.
The adhoc sampling menu contains 3 options: request, history and report. Using
the request form you can run a sampling request. It consists of 3 main sections:
preset items including sensors
sensors, groups
groups, directions,
directions and protocols
protocols. To the right
right,
user configured ranges including host, port, pattern, timeout, packet limit and
custom filter. The lower section displays custom filter settings. Ad hoc sampling
utilizes the linux tcpdump program to capture traffic. tcpdump software is present
on all SPS units and depending upon the SPS units selected in the sensors
select box, captures will be run on those boxes. When the operator enters
capture settings, the system automatically compiles the arguments to tcpdump
before running the command
command. the sample will end when the timeout limit has
been reached or the packet limit is reached, whichever happens first. if no
packets were sampled you will receive a message notifying you of this.
1-
Introduction 38
Service Protector Operators Training
A customer has 8 full duplex 1Gbps links which are accessible for monitoring.
Which controller should be proposed?
1-
Introduction 39
Service Protector Operators Training
Match the two threats on the left side with the damage they typically cause to
Service Providers
1-
Introduction 40
ServiceProtector Operators Training
Network Behavioral
Anomaly Detection
(NBAD)
Service Protector Operators Training
NBAD 2-2
Service Protector Operators Training
NBAD attacks target your network. They may be launched from outside
your network or from inside it. A common for of NBAD attack is a
distributed denial-of-service (DDOS) attack. In such cases a multitude of
hosts, typically compromised systems known as zombies, attack a single
target. The flood of incoming messages to the targeted system essentially
forces it to shut down, thereby denying service to legitimate users.
Another common type of NBAD event is known as scanning activity. This
takes place where a single host connects to multiple hosts.
NBAD 2-3
Service Protector Operators Training
NBAD 2-4
Service Protector Operators Training
NBAD 2-5
Service Protector Operators Training
NBAD 2-6
Service Protector Operators Training
We now see how to get an overview of all major NBAD events from the
“activity” item in the NBAD/Floods menu.
NBAD 2-7
Service Protector Operators Training
NBAD 2-8
Service Protector Operators Training
Using the various filters at the top of the screen, you can choose to
display only those floods detected by a particular sensor, or affecting a
particular group. You can choose to display only floods of a particular type
or direction. If you have defined filters in the Service Protector GUI
(covered in the Administrators Course), you can view only those floods
which meet your predefined filter conditions.
In the upper right part of the screen, you can choose to view only those
floods that are above a defined bit rate, packet rate, duration or shape
severity.
In this example, we’ve chosen to view only those floods with a minimum
shape severity of 4 (strong).
NBAD 2-9
Service Protector Operators Training
Lets now see how we can zoom in to learn more about a specific flood.
2-
NBAD 10
Service Protector Operators Training
You can view report for a particular flood by choosing “Event Report” from
the NBAD/Flood menu, and then entering the flood ID in the top right
corner.
Most often though, you will reach this page after clicking on the link of an
NBAD event from the start page, the traffic detail page or the NBAD
activity page.
Here we see a sample event report for flood number 2011.
The event report consists off several sections:
• Flood summary: contains basic identification information about the flood
• Statistics: displays model detection information
• Charts: displays the tracked statistic within the model
• Patterns: lists the various patterns that were detected during the flood
• Packet Captures: lists the packet samples taken during the flood
flood.
We will now examine each of these in turn
2-
NBAD 11
Service Protector Operators Training
Here we see two examples of the Flood Summary section of the event
report. The flood summary lists the following information (mostly what is
listed in the flood activity table):
Type. The type of flood detected
Status. Is the flood still active or has it cooled down?
Sensor: The sensor which detected the flood
Group:
p The g
group
p affected by
y the flood
Detected: When the flood was detected
Duration: The duration of the flood
Shape Severity: From 0 to 4 (strong). The higher the shape severity, the
more likely it is that this event resembles a true attack.
Captures: The number of samples taken
Patterns: The number of patterns detected
Alerts: M (malformed packets), D (DDoS Attack), S (scanning) or B
(Bogon)
Matched Filters: Does the flood match the conditions of any of the
notification filters that you have setup?
2-
NBAD 12
Service Protector Operators Training
2-
NBAD 13
Service Protector Operators Training
The Charts windows to the right display the tracked statistic within the
model. The upper chart displays the packet rate for that statistic while the
lower chart displays the tracked ratio which was breached.
In the first example, the upper chart shows outgoing UNR that differs from
the expected norm.
In the second example, the tracked ratio is “incoming TCP syn” to
“outgoing tcp fin”. We observed an increase in “incoming tcp syn” packets,
so the upper chart displays this statistic.
statistic
2-
NBAD 14
Service Protector Operators Training
The Patterns section displays the various patterns that were detected
d i th
during the fl
flood.
d Th
The system
t automatically
t ti ll orders
d patterns
tt b
by relevance
l
and places the most relevant patterns on the top. For each pattern you will
see the following:
Pattern ID. ID number for each pattern. Summary. The pattern format
appears as follows: Source IP : source port -> destination IP : destination
port protocol signature length. Header & Payload. Here are displayed the
Header & Payload signature length. These are the number of consistent
bytes in the header and payload (the “blue”
blue bytes).
bytes) Relevance. The
pattern relevance displays how relevant the pattern is to the overall flood.
When calculating the relevance, he algorithm checks if the pattern
accounts for the “deviation” but not the “expected” part of the flood. After
all, the deviation is the anomalous portion of the flood and should be
depicted by the pattern. In addition, we calculate this for the entire length
of the flood, and a more relevant pattern should cover a larger portion, if
not all of the flood.
Hosts. The hosts count displays the number of source hosts talking to the
count of destination hosts. A large number of sources to a low number of
destinations resembles a DDOS, whereas a single host talking to multiple
destinations seems like scanning activity. The cutoff point is usually 20
although in some cases it may rise to higher; a flood may have physically
more than 20 hosts involved as source or destination, but to keep the
display neat, 20 is the maximum number that is usually displayed.
2-
NBAD 15
Service Protector Operators Training
The last portion of the screen shows the Packet Captures that occurred
during the flood. Packet captures are taken once every three minutes, to
the maximum life of a flood of one hour. Shorter floods will have fewer
samples.
Captures are taken every 3 minutes. The captures are taken for the life of
the flood or for 60 minutes - whichever happens first.
Each packet capture displays a Capture ID, the Capture Timestamp, the
Deviation from the model in percent
percent, the number of packets,
packets their size,
size
and finally a link to download the capture.
The All Packets link at the very bottom includes all packets from the
entire life of the flood.
2-
NBAD 16
Service Protector Operators Training
Once we have isolated the NBAD event that we are interested in, we can
drill down to view any of the detected patterns.
2-
NBAD 17
Service Protector Operators Training
2-
NBAD 18
Service Protector Operators Training
In the top right corner of the pattern page you will see the pattern
summary. This consists of the following information:
Pattern Identifier: A number for every unique pattern. If patterns reoccur
the ID will be used again. Search Floods links to the flood activity page,
with the pattern id field already filled in for you. This enables you to search
if this pattern has appeared before.
Summary - repeats the main pattern information line from the event report
screen.
screen
Header octets / Payload octets displays the number of consistent bytes
(blue colour) in the header and the payload.
Hosts shows the ratio of source hosts to destination hosts
Captures displays the number of packet taken in the life of the flood. if the
flood is currently active, this number will still be growing
Quality
Q lit shows
h th
the percentage
t off packets
k t matched
t h d from
f the
th capture
t by
b
this pattern n
Alerts will display M, S, D or B if any alerts were triggered by this pattern
2-
NBAD 19
Service Protector Operators Training
If the operator decides to block this pattern, they may do so with any
packet filtering device (router, IPS, firewall, etc). A conversion of the
pattern information from the Allot Deep Packet Signature to a variety of
devices is displayed here. Use the pulldown box to select a format.
2-
NBAD 20
Service Protector Operators Training
Here we see the full pattern information. Consistent bytes are colored
blue; these are bytes that are present throughout the packets for this
particular signature. Inconsistent or randomized bytes are colored pink.
NOTE: If an entire section of the header or payload is inconsistent, with no
consistent areas at all, it will not be displayed in the pattern.
The payload is displayed in hex (default) or ASCII.
In the IPv4 header, if a consistent IP address is detected it becomes a link
and can be clicked to open the flood
f activity page. The IP address is
entered as a search parameter. This is a useful feature to search for
recurring attacks to a particular host, or if a source host is a repeat
offender.
2-
NBAD 21
Service Protector Operators Training
The pattern chart displays the pattern (in pink) overlaid over the entire
traffic (in green). This chart is a measure of how accurate the pattern is.
Accurate patterns have a consistent pink section throughout the traffic,
whereas less accurate patterns have inconsistent or broken “pink bits”.
As usual, the packet samples are overlaid as yellow vertical bars and the
expected traffic as per the model is the dotted blue line.
2-
NBAD 22
Service Protector Operators Training
At the bottom of the pattern page you will see further details about the
pattern.
The packet captures is the same as that displayed in the flood report
page, with the addition of “Match” – the packet count that matches this
particular pattern from the packet capture.
The top TX hosts and top RX hosts are displayed in the window below.
The cutoff point is 20 hosts.
Each off the IP addresses is clickable to the flood
f activity page, where it
fills the “address” field.
If DNS is configured, the hostname section will be filled in with the
reverse DNS lookups of this host if the query succeeded.
Next to every IP address is a “?”. This opens up a ‘whois’ query on that IP
address.
2-
NBAD 23
Service Protector Operators Training
From the pattern page, you can click on any of the captures to analyze
them further. Packet captures have a life of two weeks. During this time
they may be analyzed using the built in tools or downloaded for archive or
analysis with third party tools. Once the two weeks expire, samples are
automatically deleted to save database space.
ServiceProtector has a number of built in tools for sample analysis:
tcpdump & tcpdump hex: open the sample using this viewer
Snort: runs the packet sample via snort and queries the snort database iff
S
the packet sample is known. If no signatures are present, you will only see
“run time for packet processing was xxx seconds”.
Analyze flow: views the samples as flows, as seen on the slide here. Use
the ‘join directions’ button to join the tx and rx directions. In this view, flow
counts per flow are displayed, along with the packet count per flow
Analyze port – display the sample according to port
port. Both source and
destination port are analyzed and flow data displayed. ‘join direction’ is
also available.
In addition, you can choose to use Analyze host, analyze protocol,
analyze tuple or analyze length
2-
NBAD 24
Service Protector Operators Training
Finally, Ethereal, as seen on this slide & ethereal detail, open the sample
using ‘wireshark’
2-
NBAD 25
Service Protector Operators Training
Several other graphs are available from the NBAD/Floods main menu and
provide the operator with a longer term view of flood activity.
2-
NBAD 26
Service Protector Operators Training
The NBAD trends report displays flood trends as a factor of time. This way
you can visually display the count of specific events in a predefined time
range. The “filters” select box allows you to display events matching that
particular filter. This option is also useful for searching for trends in
recurring events. This is best achieved by using a filter that matches
events of interest (for example, events that have characteristics of large
DDOS). As usual the report has the standard set of built-in GUI filters. For
maximum control over events to be reported, custom notification filters
may be created in the CLI. For GUI reporting purposes, these events don’t
need a post recipient. The report supports various charting options, and is
auto- refreshing.
2-
NBAD 27
Service Protector Operators Training
2-
NBAD 28
Service Protector Operators Training
2-
NBAD 29
Service Protector Operators Training
The NBAD top sources graph takes the sources of NBAD events that
have been most detected over the period of time selected. For each
source, the graph displays the number of times it has been detected, the
average bit rate and average packet rate.
In the case of this example, we see the top sources over the past month.
The number one source was detected 24 different times. Clicking on the
target link takes us to the flood activity page where we see all the flood
events emanating from this address.
2-
NBAD 30
Service Protector Operators Training
The NBAD top targets graph gives us a similar output but presents the
most popular targets for NBAD attacks.
2-
NBAD 31
Service Protector Operators Training
Now lets look at one or two examples that illustrate what we’ve seen
2-
NBAD 32
Service Protector Operators Training
As our first example, lets look at event 529, tracked in the GUI.
Our filter is set only to pick up events with a minimum bit rate of 3000kbps.
We see straight away that this flood has a strong shape severity, and we
can learn from the “D” pattern that initial impressions suggest a DDoS
attack. Lets zoom into event 529 for a full report.
2-
NBAD 33
Service Protector Operators Training
Here we see that the flood is an incoming syn attack which is still active.
Its been detected by the sp1.allot.com sensor. It seems to have the profile
of a DDoS attack, and 100% of the activity matches the DDoS profile. It
has triggered one of our pre-defined filters (flood 1)
We see from the pattern summary that it has a relatively clear signature
(with few xxxs). This shows us that the attack is very consistent. From the
top right graph, we can see how much bandwidth is being eaten up by the
attack.
2-
NBAD 34
Service Protector Operators Training
The flood has now been active for 24 minutes. More captures have been
taken and more patterns have been identified. Over that time there have
been 9 captures (these can be seen in the charts in the upper right area
as vertical orange lines) and 5 different patterns.
2-
NBAD 35
Service Protector Operators Training
Here we see that the flood has now ceased. This can be seen in the
status reading and also in the chart tracking the TX TCP SYN packet rate
statistic.
Lets now zoom into the pattern which has the highest relevance.
2-
NBAD 36
Service Protector Operators Training
When analyzing the full pattern, we look for the blue items – these are
consistent throughout the pattern.
The consistent Source MAC address tells us the single point of entry to
our network. The attack is clearly coming from a single point – this will
help us later when we come to mitigate the attack. The packet length is
consistent at 89 bytes – this will also help mitigation. In this example we
see that every attack is 128 hops. That’s not particularly characteristic of a
true Ddos attack where we would normally expect different distances from
each source. It suggests that the attack is being spoofed. We see that the
protocol is TCP, and that makes mitigation easier. There is a consistent
destination address & port and a consistent source port. The Syn flag is
turned on. Finally we see that there is a consistent payload – another
factor that will help us when it comes to mitigation.
2-
NBAD 37
Service Protector Operators Training
The Pattern chart shows some noise (green). Remember that the NBAD
event itself is shown in pink.
We see that the packet rate here is very low indeed, which is why a
sample of 500 packets takes several seconds.
2-
NBAD 38
Service Protector Operators Training
2-
NBAD 39
Service Protector Operators Training
2-
NBAD 40
Service Protector Operators Training
Zooming into event 540, we now see 3 captures have been taken, and the
event is still active. 30% of the capture matches the scanning profile. The
chart shows that the packet rate for the tracked (RX TCP SYN) statistic
clearly exceeded the expected model. We will zoom into pattern 2672,
which is a very clear pattern.
2-
NBAD 41
Service Protector Operators Training
Here we see pattern 2672. Once again, we focus on the blue elements of
the screen.
This time we see a very small payload. There is a single source address
with multiple destination addresses on a single destination port. This is
typical pattern for scanning. Again, the Syn flag is turned on as this is a
syn scanning attack.
2-
NBAD 42
Service Protector Operators Training
2-
NBAD 43
Service Protector Operators Training
Were there any suspected DDoS attacks on the SILVER group over the
displayed time period?
2-
NBAD 44
Service Protector Operators Training
Which elements of the attack below are common in the attack pattern?
2-
NBAD 45
Service Protector Operators Training
2-
NBAD 46
ServiceProtector Operators Training
HBAD 3-2
Service Protector Operators Training
HBAD attacks are launched from within your network. Typically, the
attacks come from subscribers infected with botnet software who are
not aware that their machines are being used to launch DoS attacks,
send spam or search for other machines to infect.
HBAD 3-3
Service Protector Operators Training
HBAD 3-4
Service Protector Operators Training
The GUI has an HBAD activity page, similar to the NBAD/Flood activity
page where the operator can interactively query the database for events
of interest, or view events matching notification filters. The HBAD Activity
page is accessed from the HBAD / Quarantine item on the main menu.
HBAD 3-5
Service Protector Operators Training
Just like the NBAD activity page, the HBAD activity page is divided into
two, with the top half including time range and time-zone settings, followed
by lists of the sensors, groups, event types and predefined filters. All of
these can be used to filter the HBAD events listed in the HBAD events
table.
Likewise, the events listed can be further limited by the options in the top
right corner, to events emanating from specific subscribers, events above
a specified bit rate, packet rate or connection rate, and events targeting
particular hosts.
The table HBAD Events lists events that match the search criteria.
Clicking on the Capture ID on the left opens the HBAD Event Report page
for this event. Here the individual flows are displayed and further analysis
can be conducted. The Subscriber name will appear alongside the IP if
ServiceProtector is integrated with an SMP (see HBAD mitigation
module) Clicking Subscriber IP or name,
module). name or target,
target enters that field in
the filters. Other fields in the report include the sensor that detected this
event , the group the infected host belongs to, the time of detection, the
type of activity, the bitrate and packet rate for the host, and finally the
number of flows per second for the host.
It is possible for the same HBAD event to include more than one kind of
activity. In this case, statistics for each kind of activity are shown on a line
on their own, and a summary field is shown below.
HBAD 3-6
Service Protector Operators Training
Now lets see what details are made available when we zoom into an
individual HBAD event.
HBAD 3-7
Service Protector Operators Training
HBAD 3-8
Service Protector Operators Training
HBAD 3-9
Service Protector Operators Training
The HBAD Events table can be found in the top right corner of the HBAD
events page. This shows the various activities found during an analysis of
1000 flow captures for this subscriber. For each type of event found, the
target IP address is presented, along with the bit rate, packet rate and
connection rate. The figures in brackets show the event as a percentage
of overall traffic sampled from this subscriber. This window also has a link
to the ad hoc sampling window.
3-
HBAD 10
Service Protector Operators Training
The activity sample table can be found in the bottom part of the HBAD
event report. All 1000 flows are displayed here – 100 per page. Each flow
has a timestamp along with the following information:
Age: The duration of the flow in seconds. This is the time difference
between the first packet of the flow and the last packet of the flow (time
[seconds] = Last Packet – First Packet).
If the flow consists of only one packet, it has an age of zero.
Protocol: displayed next, followed
f by the source IP and source port.
The destination IP:destination port is in the next column followed by the
count of packets per flow.
Byte count for the flow is displayed last. In our example each packet is
198 bytes.
3-
HBAD 11
Service Protector Operators Training
Lets now look at some of the other graphs which are available from the
HBAD/Quarantine item on the main menu.
3-
HBAD 12
Service Protector Operators Training
The HBAD Trends graph is similar to the flood trend report. It displays a
count of selected events over a defined period of time. The report
selection includes the familiar sensors, groups, time range and time zone,
as well as HBAD-specific filters such as event types and user-defined
filters.
The report can chart items such as count,
bitrate/packetrate/connectionrate, and display them in a variety of charts.
Further filtering can be achieved via the minimum bit rate/packet
rate/connection rate text boxes.
In the example on the screen we see a daily distribution over the past
week of HBAD attacks with colors representing the event types detected
and which group they affected.
3-
HBAD 13
Service Protector Operators Training
3-
HBAD 14
Service Protector Operators Training
The report in this example shows the distribution of HBAD events over the
past week. Each bar represents the
total event count of HBAD events for each day in the past week. The
colors represent the HBAD event types and the groups affected.
3-
HBAD 15
Service Protector Operators Training
The report in this example shows the packet rate distribution of HBAD
events over the past week. Each bar represents a different bit rate, with
colors representing the HBAD event types and the groups affected.
The Y axis shows the event count, while the X axis shows the bit rate.
3-
HBAD 16
Service Protector Operators Training
The HBAD (Quarantine) top sources report shows, for the chosen time
period, the top sources of HBAD attacks. For each source, the report
displays the number of times it has been detected, and the average bit,
packet and connection rate of each HBAD event.
3-
HBAD 17
Service Protector Operators Training
The HBAD (Quarantine) top targets report shows, for the chosen time
period, the top targets of HBAD attacks. For each source, the report
displays the number of times it has been detected, and the average bit,
packet and connection rate of each HBAD event.
Clicking on any of the HBAD targets takes you to the HBAD activity page
filtered to that particular target. This gives you a history of the HBAD
events targeting that particular host.
3-
HBAD 18
Service Protector Operators Training
3-
HBAD 19
Service Protector Operators Training
3-
HBAD 20
Service Protector Operators Training
Here we see details of a typical HBAD event on the HBAD activity page.
We see the subscriber IP that was the source of the event. We also see
that it is a Mass SMTP (SPAM) attack and that 100% of that host’s activity
is SPAM. The attack targets TCP Port 25 – the classic SMTP port. The
packet rate is around 200 packets per second, which is a fairly typical DSL
user rate. Lets zoom in further to capture 14 for more details.
3-
HBAD 21
Service Protector Operators Training
Here we see the flows captured in the HBAD event report in the activity
sample table. We see the constant source IP, the varying destination and
the constant count of packets per flow and bytes per packet.
3-
HBAD 22
Service Protector Operators Training
Our example has all the ingredients of a classic outgoing SPAM attack.
3-
HBAD 23
Service Protector Operators Training
For the three types of HBAD events, state whether they identify attacks on
For or
single themultiple
three types of HBAD
destination portsevents, state whether
and destination hosts they identify
attacks on single or multiple destination ports and destination hosts
3-
HBAD 24
Service Protector Operators Training
Were there any address scans over 30kbps during the period selected?
Were there any address scans over 30kbps during the period
selected?
3-
HBAD 25
Service Protector Operators Training
3-
HBAD 26
ServiceProtector Operators Training
4-2
Host Based Mitigation
ServiceProtector Administrators Training
What are the Allot network elements required to enable HBAD mitigation?
HBAD mitigation requires Allot’s Subscriber Management Platform (SMP).
The SMP consists of three separate elements, each of which performs a
distinct role.
The NetXplorer server is used to configure subscriber service plans, to
manage the different network elements and to view reports.
The SMP server is integrated into the Service Provider’s IP allocation
system (be it RADIUS or DHCP based) and will sometimes also be
integrated with the provider’s OSS. The server matches between each
subscriber’s name, its allocated IP address and the service plan
purchased.
The Service Gateway or NetEnforcer is the DPI device in the network. It is
responsible for collecting statistics and enforcing the service plans that are
defined in the NetXplorer.
NOTE: A fuller explanation of Allot’s SMP can be found in the advanced
ACPP training course.
4-3
Host Based Mitigation
ServiceProtector Administrators Training
In addition to the NetXplorer Server, the SMP Server and the DPI device,
the ServiceProtector Sensor is responsible for collecting statistics from the
network, and the ServiceProtector Controller serves to compile the
information from the sensors, to identify NBAD and HBAD events and
finally to enable HBAD mitigation by sending specific action requests to
the SMP server. For this purpose the SPC needs to be integrated with the
SMP server, and the two communicate over the SMPs SOAP interface.
4-4
Host Based Mitigation
ServiceProtector Administrators Training
Lets now see a typical data flow where we employ automatic HBAD
mitigation.
To prepare the system for HBAD mitigation, the administrator must first of
all create the quarantine service plans in the NetXplorer server, and
secondly create an HBAD notification filter in the ServiceProtector
Controller.
4-5
Host Based Mitigation
ServiceProtector Administrators Training
Subscriber traffic begins to flow through the system. Lets see what
happens when the ServiceProtector Sensor detects an HBAD anomaly.
The Anomaly will be reported to the ServiceProtector Controller, and the
subscriber’s name will be resolved from the SMP. If the conditions of the
automatic filter are met, this filter will be triggered and the
ServiceProtector Controller will send a SOAP message to the SMP
informing it to change the service plan of the offending subscriber. The
details are updated in the SMP Server and passed down to the DPI device
which then enforces the new “quarantined” service plan for the offending
subscriber.
4-6
Host Based Mitigation
ServiceProtector Administrators Training
Lets now see how the integration between the ServiceProtector Controller
and the SMP server is enabled.
4-7
Host Based Mitigation
ServiceProtector Administrators Training
4-8
Host Based Mitigation
ServiceProtector Administrators Training
The first stage is to create service plans in the NetXplorer. The exact
procedure for creating service plans is covered in the ACPP advanced
training course and also in the SMP User Guide. The process can be
summarized with the 4 steps below:
1) Create condition catalogs (e.g: time, service, VLAN)
2) Create action catalogs (e.g: QoS, connection control)
3) Create a Pipe or VC service plan, choosing the appropriate condition
and action catalogs you have created
4) Insert the service plan into the NetXplorer policy table.
4-9
Host Based Mitigation
ServiceProtector Administrators Training
4-
10
Host Based Mitigation
ServiceProtector Administrators Training
Rather than block specific traffic, a different way of dealing with infected
subscribers responsible for HBAD anomalies is to redirect them to a web
page which informs them that there is a suspicion that they have been
infected with malware that is responsible for spamming activity, and
instructing them, for example, to contact customer services.
4-
11
Host Based Mitigation
ServiceProtector Administrators Training
4-
12
Host Based Mitigation
ServiceProtector Administrators Training
The third stage of integration is to inform the Service Protector of all the
Service Plans that have been set up via the NX GUI. This is done by using
the command:
smp plan name <CUSTOM NAME> <SERVICE PLAN>. “Service Plan”
is the exact name of the service plan as defined in the SMP. “Custom
Name” is the name which you give to this service plan in the Service
Protector. In many cases it may be easier to use the same custom name
as the name of the service plan. In which case, simply use the command:
smp plan <Service Plan> This tells the Service Protector of the Service
Plan that exists in the SMP, and creates an identical custom name within
the Service Protector.
Note the following additional commands:
name <custom name> changes the custom name of the plan
plan <plan
p p name> is used to change
g the p
plan name
timeout <timeout> <units> is used to define the length of time that a
subscriber will be assigned to this plan, before returning to the plan he
was originally allocated. If no timeout is set, the user remains quarantined
indefinitely until manual removal, or the ‘clear’ command is used
4-
13
Host Based Mitigation
ServiceProtector Administrators Training
show smp config shows you the configuration of the SMP integration
show smp plan gives you a summary of the smp plans that have been
integrated
show smp plan <PLAN NAME> shows you details of a specific plan
4-
14
Host Based Mitigation
ServiceProtector Administrators Training
4-
15
Host Based Mitigation
ServiceProtector Administrators Training
4-
16
Host Based Mitigation
ServiceProtector Administrators Training
Here we have filtered HBAD events over the past day by the subscriber
name itai1 and we see two Address Scanning HBAD events.
4-
17
Host Based Mitigation
ServiceProtector Administrators Training
We can of course, drill down into each event for further details, as shown
here.
4-
18
Host Based Mitigation
ServiceProtector Administrators Training
4-
19
Host Based Mitigation
ServiceProtector Administrators Training
In the pipe distribution graph on the screen we see the effect, as the
switch is made from sp4 to sp5. By choosing “subscriber status” from the
“tools” menu of the NetXplorer GUI and searching for “itai1” we see now
that while the configured service plan was sp4, the actual service plan is
sp5.
4-
20
Host Based Mitigation
ServiceProtector Administrators Training
The same information can be seen if we return to the SMP Admin GUI and
run “get subscriber” for “itai1”. The “Permanent Host Group” will show the
service plan that was initially assigned to itai1 from the NetXplorer GUI.
The “Temporary Host Group” shows the current temporary service plan
while the subscriber is in quarantine.
4-
21
Host Based Mitigation
ServiceProtector Administrators Training
4-
22
Host Based Mitigation
ServiceProtector Administrators Training
4-
23
Host Based Mitigation
ServiceProtector Administrators Training
Which command should you use to check all the SMP service plans which
have been configured in the ServiceProtector?
4-
24
Host Based Mitigation
ServiceProtector Administrators Training
Fill in the gaps in the sentence below using the words at the bottom
Fill iin th
the gaps iin th
the sentence
t b
below
l using
i ththe words
d att th
the b
bottom
tt
4-
25
Host Based Mitigation
ServiceProtector Administrators Training
4-
26
Host Based Mitigation
ServiceProtector Administrators Training
Installation
Service Protector Administrators Training
In this module we will learn how to install the Allot ServiceProtector and perform initial
configuration of network parameters. We will begin with an overview of what you need in
order to install the product
Installation 5-2
Service Protector Administrators Training
The Service Protector appliance is based on an IBM server. The server is shipped from
Allot with the SP software pre-installed, and the software license already entered. No
further configuration settings are made in the factory beyond the software installation
and license. Even if the network settings are known in advance, the ServiceProtector
cannot be configured before deployment. Configuration must be carried out in the real
network due to checks performed by the software.
If you have purchased a controller-sensor appliance, you will receive a single server.
Thiss iss used for
o what
at iss also
a so known
o as a “standalone
sta da o e installation”.
sta at o If you
your network
et o
requirements warrant a “distributed installation” you will have purchased a single
controller, plus several sensors and will receive multiple servers.
NOTE: While standalone servers are not scalable, distributed systems are scalable to
the limit of the controller (An SPC-80 can handle up to about 6-10 sensors depending
upon groups setup, while an SPC-200 can handle up to 16 sensors). It is important to
spec the controller correctly taking into account future growth. There is no simple
upgrade path from an SPC SPC-8080 to an SPC
SPC-200
200 without losing historic data
data.
In addition to the servers you will receive a software CD to reinstall the software if
required, a letter detailing the license (to be re-entered in case of reinstallation) and a
documentation CD including the Operations Guide and the Installation & Admin Guide.
These guides are also available from the main menu help.
You will also need the following (not supplied): A VGA monitor and USB keyboard (or
serial console with null-modem cable), ), plus
p port SPAN or fiber optic
p p taps.p Two monitoring g
ports are required for every full-duplex link. Aggregating directions is possible as long as
there is no data clipping (<1Gbps).
Installation 5-3
Service Protector Administrators Training
Now we will examine how to connect to the ServiceProtector and configure initial
network parameters on the controller, sensor and the controller-sensor products.
Installation 5-4
Service Protector Administrators Training
You can connect the VGA Monitor and keyboard to the front of the ServiceProtector or to
the rear as seen in this picture. In addition there is a console port for serial connectivity
at the rear of the product.
Installation 5-5
Service Protector Administrators Training
Login to the Service Protector as root. ServiceProtectors are shipped from Allot with the
default password root / Allot$001 . Remember to change the default root password as
soon as possible. Use the ‘passwd’ command from the top level CLI. Do not change the
password from the LINUX shell.
After a fresh software installation however, there is no default root password. So if you
are re-installing the ServiceProtector software yourself, you will first need to specify a
password.. You will need to choose this and enter it twice. Note the stringent password
ccriteria.
te a The e pass
password
o d ca
can be aanyyoof tthe
e opt
options
o s be
below,
o , where
e e a family
a y iss de
defined
ed as
small letters, capital letters (not including the first character), numbers or other symbols:
- 8 characters, 2 families
- 7 characters 3 families OR
- 6 characters 4 families
After you have logged in, Enter cli and then Enter configure (Note that when working with
CLI <TAB> activates auto-complete and <?> displays a short help screen.)
CLI, screen ) After
entering CLI you will see failure messages (e.g: No Configuration, No NTP or No License
if you have reinstalled the software). You are now ready to begin configuring the network
parameters.
Installation 5-6
Service Protector Administrators Training
The Command Line Interface (CLI) is the administrative portal for the SP appliance. It
has three main uses: (1) Initial setup (network settings, connectivity) – as covered in this
module, (2) Advanced administration and configuration (notifications, user management,
sensor management, groups and prefixes) and (3) Troubleshooting.
Administrative rights are required to access the CLI. The superuser (root) receives login
at shell level, and users with administrative rights receive login directly to the CLI. Even
though LINUX shell access is supplied, Allot strongly recommends that the command
prompt
p o pt be used only
o y by experienced
e pe e ced LINUX U administrators,
ad st ato s, a
and
doonly
y when
e instructed
st ucted by
an Allot support engineer. Available CLI commands depend upon the device being
controlled. Sensors serve only basic connectivity and troubleshooting settings, whereas
Controllers provide a much wider array of commands. Standalone systems are a single
combined Controller-Sensor appliance and serve the menus of both components from a
single box.
The CLI consists of the main menu, the configure menu, and a series of sub menus.
Logging into the CLI is necessary for root users who receive a shell prompt
prompt. Type <cli>
to enter the CLI. The CLI supports the following navigational aids:
‘?’ brings up a context-relevant help screen. <TAB> enables auto-complete. It completes
the command and shows you all possibilities of commands with the prefix you already
typed in. ‘exit’ takes you up one level. ‘help’ gives a quick help screen at the top level.
‘do’ executes top-level commands. Arrow up/down browses the command history.
NOTE: long outputs are piped through the ‘less’ pager and appear bottom first - you
have to scroll up to read the first lines.
Installation 5-7
Service Protector Administrators Training
Here we see the 8 stage configuration process. The order of configuration here is
strongly recommended. This is firstly because some items (e.g timezone) cannot be
configured without other (e.g: NTP), and secondly, the suggested order means that no
reboots are required in the middle of the process - just a single reboot command is
required at the end.
Installation 5-8
Service Protector Administrators Training
Let’s examine the first five stages of the network parameters configuration:
1) Enter a hostname:
• hostname <HOST NAME>
2) Enter the IP domain name:
• ip domain-name <DOMAIN NAME>
3) Enter the IP address and length, followed by 4) the default route:
• ip
i address
dd <IP ADDRESS>/<LENGTH> default-route
d f lt t <DEFAULT ROUTE>
• E.g: ip address 192.168.1.2/24 default-route 192.168.1.254
NOTE: The default route setting can be made separately - For Example: ip default-route
192.168.1.254
NOTE: Once entered, the default route is immediately sanity tested. This is why the
network configuration must be done
onsite, on the real network. If it is not done on the real network, this will fail.
5) If you have reinstalled the software yourself or if for some other reason, the license
needs to be installed, this can be done as follows:
• license <License>
The license string is generated by Allot and entered before shipping the device. If you
lose your license you can do ‘show license’ from the top level cli and send the entire
output to Allot Customer Support, who will generate a new license for you.
Installation 5-9
Service Protector Administrators Training
Installation 5-10
Service Protector Administrators Training
Before the reboot, there are a couple of additional steps that may be added if you wish to
add a static route or an ACL.
To add a static route, enter ip route <prefix or IP> This may be for the destination host
or network. show ip route displays the routes to this particular appliance, be it a
controller or sensor
To add an ACL, enter ip acl <parameters>
There are two possible options for the IP Route format:
Option 1 (route to a host): ip route 192.168.1.1 172.10.1.254
Option 2 (route to a network): ip route 160.4/16 172.10.1.254
show ip acl displays the acls to this particular appliance, be it a controller or sensor –
there are none in the example on the screen.
Installation 5-11
Service Protector Administrators Training
Installation 5-12
Service Protector Administrators Training
Installation 5-13
Service Protector Administrators Training
NTP is of the highest importance to the integrity of the system. Database records and
time synchronization across a distributed system depend upon an accurate system
clock. Indeed, a system cannot be configured until NTP is working. Until NTP is
configured the system will display error messages and will not allow you to make any
configuration beyond basic IP connectivity
Initial NTP settings must be configured at install time
A minimum of one NTP server is required, and if multiple NTP servers are configured
this results in more accurate timekeeping
timekeeping. If you use pool
pool.ntp.org
ntp org (default and tried
automatically) the SP polls it 4 times and enters 4 NTP servers.
When NTP is not configured a single message is displayed upon each login to the CLI.
The message is not displayed once NTP is fully functional.
Installation 5-14
Service Protector Administrators Training
You can run “test ntp” at any time to test the ntp settings. It requests the time from the
NTP server. You can see here the details of the server you have polled
Installation 5-15
Service Protector Administrators Training
Most of these network settings are done at install time, but may be amended at any time
• To change the system hostname enter ‘hostname <HOSTNAME>’
• To set an IP and a default route in a single command, enter ‘ip address
<IP>/<LENGTH> [default-route <ROUTE>]’
• To set the default route, enter ‘ip default-route <ROUTE>’
• To set the DNS server, enter ‘ip dns-server <DNS>’
• To set the domain name,
name enter ‘ip
ip domain-name <DOMAIN>
<DOMAIN>’
• To set a route, enter ‘ip route <PREFIX|IP>
Installation 5-16
Service Protector Administrators Training
Additional NTP servers can be added by using the command ‘ntp-server <NAME|IP>’
NTP servers can be removed by entering ‘no ntp-server <NAME|IP>’
NOTE: You cannot remove the last NTP server (At least one server needs to remain
configured). If you want to change the last NTP server, or it is the only one configured,
add the new server first, then remove the old one.
Installation 5-17
Service Protector Administrators Training
There is no ‘no’ timezone function. To change the timezone simply configure a new one
by entering ‘timezone <TIMEZONE>. Valid settings for the timezone are obtained by
pressing the <TAB> at the timezone parameter.
A reboot is mandatory to ensure that all the processes are updated with the new
timezone setting.
Installation 5-18
Service Protector Administrators Training
Installation 5-19
Service Protector Administrators Training
Installation 5-20
Service Protector Administrators Training
As a root user you can also set the access rights for different users by adding
administrators and GUI-Only users. We will now see how to do this.
Installation 5-21
Service Protector Administrators Training
There are three levels of users in the system – root, administrators and operators.
The root user logs in to linux shell, and runs ‘cli’ to get the CLI. This user has full access
to everything:
system administration, shell access, maintenance and troubleshooting, adding and
removing users (other admins and operators).
The admin user logs in directly to the CLI. This user can gain LINUX shell access via
the ‘bash’ command, and has the same privileges as the root user (system
administration,
d i i i shell
h ll access, maintenance
i and
d troubleshooting,
bl h i adding
ddi andd removing
i
users)
The operator user has access to the GUI and no shell access. This user’s access can
be further limited to sensor and group.
Limited privileges users can be controlled with fine grain detail
Installation 5-22
Service Protector Administrators Training
Here we see how to create a new user. This is only possible on the SPC.
Make sure you enter the suffix for the type of user requested
To create an administrator user, enter: “user <NAME> superuser”
To create an operator with limited privileges, enter: “user <NAME> gui-access”
You will automatically enter the “user <NAME>” submenu. From this menu you can set
the public key for this user by entering ‘authorized-key <KEY>’ and can set the password
for this user byy entering
g ‘passwd
p <PASSWORD>
NOTE: Never change password from the LINUX shell, only from the CLI
Installation 5-23
Service Protector Administrators Training
Installation 5-24
Service Protector Administrators Training
Three different “show” commands give us information about the users configured and
can be useful for troubleshooting user access problems.
‘show user <NAME>’ This displays information about user <NAME> together with the
GUI ACLs assigned to that user
‘show user summary’ This displays only summary information on all users
‘show user config’ This displays the entire user configuration
Installation 5-25
Service Protector Administrators Training
Installation 5-26
Service Protector Administrators Training
The SSH host key used by a ServiceProtector appliance can be changed using the 'ssh
host-key' command.
The key is entered as a string following this command. Administrators may generate the
key pair using
a tool of their choice, one way of doing this from LINUX is using a command similar to
'ssh-keygen –t dsa' . The key pair can be located in the target directory the keygen
specifies in the output. The private key is then pasted into the CLI.
T change
To h the
h kkey SSH uses ffor encryption,
i enter: ‘ssh
‘ h host-key
h k <KEY>’
KEY
Some security engineers may consider root login a security threat since a variety of
attacks on the root user have been known. Disabling root access only affects the user
'root', other superusers are not affected. By default, root user login is permitted. To
enable “root” to login remotely via SSH (this is the default configuration) enter: ‘ssh
permit-root-login’
To disable remote root login
login, enter: ‘no
no ssh permit-root-login
permit-root-login’
NOTE: Disabling remote root login is used to protect against attacks on the root user.
Other superusers defined in the system can still login remotely though.
Installation 5-27
Service Protector Administrators Training
Installation 5-28
Service Protector Administrators Training
Installation 5-29
ServiceProtector Administrators Training
In this module we will examine how to setup IPSec communication between the SPC and
the SPS in 3 different scenarios.
In the first and simplest case, there is no firewall and no NAT impeding the traffic between
the SP-Controller and SP-Sensor. Typically (but not necessarily) in this scenario, they will
be situated on the same network.
The procedure in this case is simple. In order to add the sensor to the controller and
automatically
t ti ll open up an IPIPsec tunnel,
t l from
f the
th SP-Controller
SP C t ll enter t either
ith ‘Sensor
‘S
<HOSTNAME>’ or ‘Sensor <HOSTNAME> ip-address <IP>’. Each command adds this
Sensor to the system. You need to use the second command which explicitly enters the
sensor IP address if the sensor hostname is not added to the DNS server. Note that the
hostname must match either the Sensor hostname or the Sensor FQDN. For example – if
a Sensor is called Sensor1.foo.com, you should use either “Sensor1” or “Sensor1.foo.com”
for the hostname. The sensor is added and an IPsec tunnel is opened. Communication is
via ESP and UDP/500 only. NOTE: TCP/22 is required to setup and initiate the IPsec
tunnel. After the tunnel is initiated, TCP/22 is not used any more.
Alternatively, you can use ‘Sensor <HOSTNAME> no-groups’ or ‘Sensor <HOSTNAME>
ip-address <IP> no-groups’. These both add this Sensor to the Controller (via either of
the methods above) without any groups assigned to it. If the ‘no-groups’ were omitted, the
controller would try to assign all the available groups to the Sensor. This might not
necessarily be what you want if (a) there are more than 30 groups in the system (30 groups
is the limit per Sensor), or (b) you don’t want the existing groups added to this Sensor. This
is discussed in more detail in the groups and prefixes module of this training course
course.
NOTE: You can remove groups from a Sensor at a later time if you accidentally add a
Sensor with groups assigned (Go to the group configuration ‘group <NAME>’ then remove
the Sensor ‘no Sensor <Sensor>’). This can also be done from the sensor menu by
entering ‘sensor <NAME>’ then ‘no group <GROUP>’. To remove all the groups from a
particular Sensor, enter the Sensor configuration menu ‘sensor <NAME>’ then type ‘no
group all’
In the second case scenario, there is a firewall between the SP-Controller and the SP-
Sensor.
In this case you will need to open UDP/4500 on the firewall and perform the following
procedure.
The procedure consists of 3 stages – you must follow the order shown on the screen. Note
that you may wish to use raw ESP or UDP. If using UDP, then open UDP/4500 both ways.
If using raw ESP, open udp/500 and ip protocol 50 (esp). There is no need to open
udp/4500 in this scenario. Note that whichever choice you make also affects whether you
use “no udp-encap” in step 2 below.
1)On the SP-Controller, enter sensor <NAME> shutdown. You will be given a new IPsec
key. Now enter the IP address of the sensor using the command ip-address <IP>. Finally
enter
e te tthe
e co
command
a dg group
oup all
a ((note:
ote tthis
s co
command
a d is
s not
ot mandatory)
a dato y)
2) On the SP-Sensor, enter controller <Controller IP> shutdown. Now enter the IPsec
key received above using ipsec-key <KEY>. IPsec speaks either UDP or Raw ESP. The
network policy may require one or the other. If you wish to communicate over raw ESP
rather than UDP, enter the command: no udp-encap. Finally, enter the command no
shutdown
3) On the SP-Controller, enter no shutdown. When prompted, enter the password of the
S S
SP-Sensor
That completes the process
The final scenario involves network address translation. The procedure required differs,
depending on whether it is the sensor, the controller or both that sit behind the NAT device.
The procedure for adding a sensor when the sensor sits behind a NAT device is as follows.
Again, you must follow the order on the screen. Don’t forget of course, to configure NAT on
your NAT device.
1) On the SP-Controller, enter sensor <NAME> shutdown. You will be given a new IPsec
key. Now enter the IP address of the sensor using the command ip-address <Sensor IP>.
Now enter the IP address of the NAT device using the command: tunnel via <NAT IP>.
Finally enter the command group all
2) On the SP
SP-Sensor,
Sensor enter controller <Controller IP> shutdown
shutdown. Now enter the IPsec
key received above using ipsec-key <KEY>. Now enter the command: tunnel. Finally,
enter the command no shutdown
3) On the SP-Controller, enter no shutdown. When prompted, enter the password of the
SP-Sensor
That completes the process
A second variation of this scenario is where it is the controller that is situated behind the
NAT device.
The procedure for adding a sensor when the controller sits behind a NAT device is as
follows. In stage two you will need to choose a port for this procedure - don’t forget to
forward/translate a port of your choice to UDP/4500 and the IP translation Again, you must
follow the order on the screen:
1)On the SP-Controller, enter sensor <NAME> shutdown. You will be given a new IPsec
key. Now enter the IP address of the sensor using the command ip-address <Sensor IP>.
Now enter the command: tunnel. Finally enter the command group all
(note: this command is not mandatory)
2) On the SP-Sensor, enter controller <Controller IP> shutdown. Now enter the IPsec
key received above using ipsec-key <KEY>. Now enter the IP address of the NAT device
using the command: tunnel via <NAT IP>. Now enter the command udp-encap <port
number> (entering a port of your choice). Finally , enter the command no shutdown
3) On the SP-Controller, enter no shutdown. When prompted, enter the password of the
SP-Sensor.
That completes the process
The final variation is where both the controller and sensor are behind separate NAT
devices (or behind a single firewall with NAT configured in both directions). In this case
follow the procedure on the next slide:
The procedure for adding a sensor when both controller and sensor sit behind a NAT
device is as follows. Again, you will need to choose a port for this procedure. Remember to
forward/translate the port of your choice as well as setup the IP translation. This scenario is
very tricky to get right and any error in setup of the NAT device may seem like the
ServiceProtector isn’t working. Again, you must follow the order on the screen:
1) On the SP-Controller, enter sensor <NAME> shutdown. You will be given a new IPsec
key. Now enter the IP address of the sensor using the command ip-address <Sensor IP>.
Now
o e enter
te tthe
e IP add
address
ess o
of tthe
e NAT de
device
ce us
using
g tthe
e co
command:
a d tu
tunnel
e via
a <NAT IP>.
Finally enter the command group all (note: this command is not mandatory)
2) On the SP-Sensor, enter controller <Controller IP> shutdown. Now enter the IPsec
key received above using ipsec-key <KEY>. Now enter the IP address of the NAT device
using the command: tunnel via <NAT IP>. Now enter the command udp-encap <PORT
Number>. Again choose a port number here. Finally, enter the command no shutdown
3) On the SP-Controller, enter no shutdown. When prompted, enter the password of the
SP-Sensor
S S
That completes the process
We will end by looking at a few commands which can help you to configure the sensor and
view its current configuration of the SP-Sensor.
Here we see a summary of the different available commands for an existing sensor.
Enter ‘Sensor <HOSTNAME>’ to enter the configuration menu for this Sensor. This is only
possible for Sensors already installed on this Controller. Let’s now see the available
commands:
‘description <TEXT>’ This is used to define a text description for this Sensor. ‘do
<COMMAND>’. This is used to run top level commands on the CLI. ‘exit’. This will take
you to the ‘configure’ menu. ‘group <GROUPNAME>’. This assigns a group to this Sensor.
ip-address
‘ip address <IP>
<IP>’ You can use this command to manually change the IP address of this
Sensor. It’s useful if the Sensor needs to be moved. Note that the sensor needs to be in
“shutdown” mode to change it’s IP address. ‘name <NAME>’. This is used to manually
change the Sensor name. Again note: The sensor needs to be in “shutdown” mode to
change its name. ‘no <COMMAND>’ This is used to move a configuration item (see next
slide). ‘public-key <TEXT>’ Enter the public key for this Sensor. ‘shutdown‘ This is used
when you want the Sensor to enter shutdown mode. The sensor is still attached to the
controller but ceases all communication
communication. This is necessary for example if you are replacing
a defective sensor, changing the IP address of the controller or sensor, or changing the
sensor name. It is also required during the IPsec configuration procedure.
NOTE: Once the changes have been made, issue the ‘no shutdown’ command to revive
the Sensor. See the Troubleshooting slides for the Sensor replacement procedure
Here we see a list of the available “no” commands, which are used to remove different
configuration items.
‘no group <GROUP>’ for example removes a GROUP from this Sensor
‘no ip-address’ Removes the IP address setting for this Sensor
‘no public-key’ Removes the public key for this Sensor
‘no shutdown’ Removes this Sensor from “shutdown” mode. This is used in the procedure
p g an Sensor ((See the System
for replacing y Maintenance module of this training
g course for
the Sensor replacement procedure). When an Sensor is removed from ‘shutdown’ mode,
depending upon how long it was down, it may require up to 2 hours to rebuild the traffic
models before anomaly detection resumes.
In Show Sensor <name>, we see the sensor name and description, its IP address.
If any special IPsec configurations have been defined they will appear here. You also see
here all the assigned groups
On the SP-Controller you can run the command: show sensor summary to reveal all of
the sensors currently configured to work with that controller. For each one we see the
sensor name and its domain name, how many groups are assigned to the sensor, its
communication status (shutdown - yes or no) and whether its alive or not.
(Shutdown communications is a special maintenance mode for the sensor – it should be
turned off)
Running show sensor subsystems on the controller, helps you identify if the subsystems of
a specified sensor are running.
- detect, fast and hints-fast are all NBAD related services
- informer is the service responsible for notifications
- quarantine is responsible for HBAD detection
- smpd is the daemon which talks to SMP
- threshold is used for threshold based detection
Running the “show controller” command from a specific SP-Sensor displays the
Controller IP address that this sensor is communicating with.
Which port is used only while an IPSec Tunnel is being setup and is not used thereafter?
In this module we will learn about Groups and Prefixes and how to define, assign, modify
and delete them using the Allot ServiceProtector CLI. Let’s start with a quick overview.
7-2
Groups and Prefixes
ServiceProtector Administators Training
When working with Allot’s Service Protector, a consistent system of prefix management is
employed.
l d Thi
This is
i called
ll d grouping.
i Groups
G are an efficient
ffi i t way off b
breaking
ki d down th
the
customer’s network into a set of logical units. The idea is to create groups of prefixes that
perform similar tasks in your network. For example, the servers in your DMZ might span
several prefixes, but they all perform a similar task, thus making sense to put them into a
manageable unit. Likewise, if your organization has a large number of business
customers, their IPs would similarly be grouped into one unit. This system also eases
management from the operators perspective, because, now machines spread over large
IP ranges can be collectively managed as a single entity. An attack originates at a source
and targets a destination. From a systems point of view, the attack is originating from a
particular group, and is targeting another group. A DDOS attack by several sources will
appear as coming from a particular group, making the task of identification and mitigation
much easier. At the highest level, the operator will see a source group and a destination
group in the NBAD/Floods page. Drilling down deeper into the pattern analysis will reveal
the finer details - the true source and destination IP addresses. This layering significantly
eases the operators management, allowing them to focus on only significant attacks.
Setting up groups is therefore probably the most significant and important step of a
Service Protector deployment. Although the system will function quite well with an
average level of grouping, accurate grouping will provide the operator with more detailed
reports - attacks will be identified to the precise source and destination, saving the
operator the effort of looking them up manually. The better the grouping, the more
accurate the system will be in anomaly detection, analysis, and reporting.
7-3
Groups and Prefixes
ServiceProtector Administators Training
Lets begin by defining some key terms. A prefix is a range of IP addresses in the form of
IP address/Length.
dd /L th
A group can consist of one or more prefixes. There is no limit on the number of prefixes
that can be included in a group. Groups are assigned to Sensors. The same group can be
assigned to several or indeed all sensors.
7-4
Groups and Prefixes
ServiceProtector Administators Training
7-5
Groups and Prefixes
ServiceProtector Administators Training
Lets now review a typical step by step procedure for creating groups.
7-6
Groups and Prefixes
ServiceProtector Administators Training
7-7
Groups and Prefixes
ServiceProtector Administators Training
The first step is to break down the architecture of the service provider’s network into
logical units which will be called groups. For example, you may divide subscribers into
Gold, Silver etc according to how they are divided up by the service provider
A group is a collection of prefixes. A Prefix is a range of IP addresses in the form of
IP/Length
/32 is a single IP address
/24 is Class C
/16 is Class B
/8 is Class A
/28 is 16 IP addresses
7-8
Groups and Prefixes
ServiceProtector Administators Training
Groups are created on the SP Controller (The ‘group’ submenu is not available on
sensors)
If you enter group dan, it creates a group called dan. Note that by default, this group will
be assigned to ALL SENSORS. If you enter group dan no-sensors it creates a group
called dan and assigns to NO SENSORS. This is useful when you have multiple sensors
and you don’t want every sensor to have knowledge of this particular group. Doing this
also saves on system resources, if a group has no useful function on an sensor, don’t
assign
ass g it.
t In add
addition,
t o , if you wish
s to just assign
ass g a group
g oup to a ssingle
g e se
sensor,
so , you may
ay
optionally specify that the group is not assigned to any sensors, and then add it to one
NOTE: Even if the maximum number of groups have been created, you can still create
the group. If it isn’t assigned to any sensors, then no limits are exceeded. In other words,
while a sensor can have a maximum of 30 groups assigned, the controller can have any
number of groups configured.
7-9
Groups and Prefixes
ServiceProtector Administators Training
Once any group has been created, the configure-group submenu becomes active for that
particular group. You can then use this submenu to add a prefix to this group, enter:
prefix <IP>/<LENGTH> [description <TEXT>]’
A group needs at least one prefix, and can contain any number of prefixes
NOTE: The ‘description <TEXT>’ part is optional but highly recommended. It is required
for specific notification filter settings such as ‘destination’ filter under the ‘subfilter pattern’
setting
7-10
Groups and Prefixes
ServiceProtector Administators Training
The fourth step is to decide if the groups you have created are inside groups or outside
groups. To determine this, you should ask whether or not the Service provider owns the
prefixes which have been used to create the groups. If the answer is YES, then the group
is an inside group. If the answer is NO, then the group is an outside group. e.g: your
peering partners.
NOTE: There may be circumstances where a ServiceProvider wishes to define groups
belonging to peering partners as inside groups too.
By default,
default groups are created as inside groups
7-11
Groups and Prefixes
ServiceProtector Administators Training
You can define a group as inside or outside from the configure-group submenu. Note
that this is a sub-menu of a named group. configure-group only exists if a group exists.
configure-group
inside defines this group as “inside”
configure-group
outside defines this group as “outside”
7-12
Groups and Prefixes
ServiceProtector Administators Training
The fifth step is to decide which functions are to be applied per group: floods (NBAD),
quarantine (HBAD) or both.
For INSIDE GROUPS, Quarantine is turned on by default
For INSIDE & OUTSIDE GROUPS, Floods, inside floods & local floods are turned on by
default.
For OUTSIDE GROUPS, Quarantine can never be turned on.
7-13
Groups and Prefixes
ServiceProtector Administators Training
The following functions may be activated or disactivated in each group, depending on the
type of group (inside or outside) and on your needs:
Quarantine. This enables HBAD functionality. It is enabled by default for Inside groups
but may be turned off. It cannot be enabled for Outside groups, as HBAD events by their
very nature, originate from within the Service Provider’s Network.
(Outside) Floods. This refers to NBAD events whose source is an outside group. These
are enabled by default in both types of groups
IInside
id Floods.
Fl d ThisThi refers
f to NBAD events whose
h source iis one iinside
id group and
d
destination is a different inside group. These are enabled by default in both types of
groups and may be disabled.
Local Floods. This refers to NBAD events that originate from and target addresses within
the same group. These are enabled by default in both internal and external groups and
may be disabled.
To summarize
summarize, when you create an inside groupgroup, all types of floods and quarantine are
enabled by default. You can choose to disable all of these functions except for “outside
floods”. When you create an outside group, all types of floods are enabled by default,
but quarantine is not enabled and cannot be enabled. You can choose to disable inside or
local floods in outside groups if you so wish.
7-14
Groups and Prefixes
ServiceProtector Administators Training
You can turn different functions on and off from the configure-group submenu. Note that
this is a sub-menu of a named group. configure-group only exists if a group exists.
configure-group
floods turns on NBAD detection for this group.
configure-group
local-floods turns on NBAD detection for floods that originate from and
g the very
target y same ggroup
p
configure-group
inside-floods turns on flood detection for floods detected between this
group and other groups marked as outside
configure-group
quarantine turns on HBAD detection for this group
Similarly each of these functions can be turned off by entering
Similarly,
configure-group
no <FUNCTION>
7-15
Groups and Prefixes
ServiceProtector Administators Training
The final step is to assign groups to sensors which can see them. When you create a
group it is automatically assigned to all sensors. You can use the CLI commands to
remove specified groups from a sensor
7-16
Groups and Prefixes
ServiceProtector Administators Training
We have seen how when you create a new group that group is automatically assigned to
all sensors. If you enter
group <NAME> no-sensors the new group is assigned to none of the sensors.
You can then assign a group to an individual sensor from the configure-group submenu.
Note that this is a sub-menu of a named group. configure-group only exists if a group
exists.
configure-group
sensor <SENSOR NAME> assigns this group to a specific sensor
configure-group
sensor all assigns this group to all sensors
Remember that by entering
configure-group
description <TEXT> you can enter text to describe a group if you haven’t
haven t
already done so.
7-17
Groups and Prefixes
ServiceProtector Administators Training
When you create the group, you decide what to do about sensors:
) g to all
a)Assign
b)Assign to none and then add manually
7-18
Groups and Prefixes
ServiceProtector Administators Training
7-19
Groups and Prefixes
ServiceProtector Administators Training
The system always tries to match the smallest prefix (the most accurate setting, the
smallest range of IPs)
If two groups compete for the same anomaly, the group with the smallest prefix wins
The level of detail you receive, and the granularity of anomaly detection depends upon the
size and number of your prefixes
•Try to keep an accurate breakdown of your network by using meaningful groups and
prefixes
•Don’t unnecessarily define groups and prefixes, this affects performance
•Don’t assign groups to sensors that don’t need them
7-20
Groups and Prefixes
ServiceProtector Administators Training
You may often have hundreds of prefixes to be loaded into groups. The most efficient way
to do this is to copy them into a template file (available from the Allot website at:
http://www.allot.com/support/KBItemDetails.aspx?ItemID=8389084) and load them into
the CLI via the command prompt. The procedure is as follows:
1.Open up the template file and copy & paste your prefixes into the file ensuring you keep
the formatting correct. You can insert as many prefixes as you want per group
(remember, there are a maximum of 30 groups per sensor).
Note the following pointers about the file syntax:
-There should be a space before each prefix
-There should be a ‘!’ at the end of the lot
-Prefixes are in the format prefix/length
-Shortcuts are supported (10.1/16 and 10.1.0.0/16 are both ok)
-Descriptions
Descriptions are optional
2. From the LINUX shell prompt, load the prefixes file you created
-‘cli –c ‘configure’ - < GROUPS_FILE’
Note that group configuration does not require a reboot
7-21
Groups and Prefixes
ServiceProtector Administators Training
The show group <NAME> command displays the group configuration for group <NAME>
7-22
Groups and Prefixes
ServiceProtector Administators Training
The show group config command begins by showing the default settings when we
create a new group. Then we see group by group, the entire configuration for each group.
7-23
Groups and Prefixes
ServiceProtector Administators Training
The show group summary command shows a list of groups, how many prefixes per
group and how many sensors to which each group has been assigned.
7-24
Groups and Prefixes
ServiceProtector Administators Training
The ‘show prefixes’ command displays all prefixes sorted by prefix (ascending). The
‘show prefixes <PREFIX/LENGTH>’ command displays this particular prefix information
7-25
Groups and Prefixes
ServiceProtector Administators Training
The ‘show prefixes <IP>/<LENGTH> subnets’ command shows the prefixes configured
in the system that are encompassed by the stated prefix
7-26
Groups and Prefixes
ServiceProtector Administators Training
7-27
Groups and Prefixes
ServiceProtector Administators Training
Here we see a group called dan1 which was created and assigned to no sensors. By
default all new groups are defined as inside groups. By default all inside groups have all
detection functions turned on.
7-28
Groups and Prefixes
ServiceProtector Administators Training
Here we see what happens when we turn off quarantine and turn it into an outside group
7-29
Groups and Prefixes
ServiceProtector Administators Training
Fill in the gaps in the sentence below using the words at the bottom
Fill iin th
the gaps iin th
the sentence
t b
below
l using
i ththe words
d att th
the b
bottom
tt
7-30
Groups and Prefixes
ServiceProtector Administators Training
7-31
Groups and Prefixes
ServiceProtector Administators Training
7-32
Groups and Prefixes
ServiceProtector Administrators Training
Notification Filters
Service Protector Administrators Training
I this
In thi module
d l we will
ill examine
i ththe filt
filter and
d notification
tifi ti mechanism
h i iin d
detail.
t il WWe b
begin
i
with an overview of how the mechanism works.
Wh use filt
Why filters?
? Fi
Firstt it enables
bl you tto enjoy j hihigh
h quality
lit output
t t iin th
the SP GUI
GUI. IIn mostt
networks, a certain amount of "noise" exists. These are floods that are so tiny as to be
insignificant. ServiceProtector was designed to cater for all levels of organizations, from
enterprises where even the smallest flood may be of interest, through to ISPs and Telcos,
where only floods that are large enough to bring down a network will be dealt with. The
sensitivity level of anomaly detection has been kept high to ensure detection of even very
small anomalies. By default the ServiceProtector GUI shows you all NBAD and HBAD
events
t unless
l you create t filters.
filt If filters
filt are created,
t d then
th events t off interest
i t t are filtered
filt d
in, and the GUI only shows events matching the filter criteria you have chosen. In this
way you can define filters to include only events of interest, resulting in a cleaner gui
display and a high true positive alerting rate.
Secondly, filtering is used to enable notifications. The system administrator configures a
set of predefined conditions. All events detected are matched against these conditions,
and those meeting g the criteria can trigger
gg an alert to the operator.
p This is the notification
system, a means by which the system can proactively alert the operator on NBAD or
HBAD events. The operator then follows the link provided, qualifies the event and can
apply the recommended filter if necessary.
Finally, the filtering mechanism drives the HBAD mitigation when integrated with SMP.
This is covered in the HBAD mitigation module.
A soon as a filt
As filter is
i created
t d it appears in
i the
th GUI and
d can b
be used
d tto filt
filter iin th
the events
t
you wish to see.
If you wish you can add a notification to the filter. This ensures that when the filter
conditions are met a notification message is sent.
N tifi ti
Notifications are configured
fi d on th
the SPC by
b a user with
ith administrator
d i i t t rights.
i ht Th
They can b
be
received by anyone, but only users with GUI access rights may view the floods page on
the GUI. Notifications of both NBAD and HBAD events may be sent via one or more of
the following transport mechanisms: Email, SNMP, Syslog and GUI.
• Email - Information is provided within an email. One or more recipients can receive
notification emails. A URL link to this flood is provided within the body of the mail. Email is
the most user-friendlyy wayy to send with all the fields clearlyy marked and structured.
• SNMP - Traps are sent to an SNMP server. The traps are pushed to the server by the
Controller. For use in deployments where SNMP is used. The traps are based on a private
MIB which can be imported into an SNMP server.
• Syslog - Flood information is sent to a Syslog server as plain text. Flood information is
sent as one long line, and although it is easily machine-parsable, it is not as human-
friendly as email. For use where Syslog servers are deployed.
• GUI – Notification filters also appear in the GUI and can be viewed by users with GUI
rights. Filters are especially useful during the setup and fine tuning stages of notifications
since results are immediately visible with no effort required from the user.
In addition the filter notification mechanism is also used to enable integration with a
Redback BRAS or with Allot’s SMP. In both these cases SOAP messages are sent from
the SP Controller to the BRAS or SMP to enable changes in service plans for the
purposes of HBAD Mitigation. NOTE: Any method(s) of transport can be sent , and to any
combination of recipients
H
Here we see an example.
l
For NBAD events, an administrator will typically create filters which quantify and qualify
the NBAD event and then send a notification in the case of serious attacks.
For HBAD events, the filter and notification system is used to enable automatic mitigation.
A passive server such as syslog or SNMP will typically be notified.
In this case, the administrator has set up a notification filter which sends an email to the
administrator warning
arning whenever
hene er there is a large scale DDoS attack (e (e.g:
g 200,000
200 000 packets
per second of malicious traffic). In addition, a syslog message is sent to the syslog server
with every outgoing SPAM attack (an HBAD event).
N
Now llets
t examine
i th
the parameters t th
thatt need
d tto be
b sett before
b f you begin
b i sending
di
notifications via notification filters.
B f
Before you can sendd notifications
tifi ti b
by email,
il snmp trap
t or syslog,
l there
th are severall iinitial
iti l
settings that need to be defined.
B f
Before you can send
d emailil notifications,
tifi ti you fifirstt need
d tto d
define
fi ththe ffollowing:
ll i
- The email server name or address
- The person who will be sending the emails (and who an email reply will be sent to)
- The person who will receive the mails (although this can also be done while entering the
actual filter)
NOTE: The sender is the controller so it can be set to something like
serviceprotector@customer.com
If it is conceivable that someone might reply to this email, ensure that either the mailbox
exists or the reply-to is set to the email address of the ServiceProtector administrator.
Once you’ve defined these settings, it is very important to perform a test email by running:
do test email <your email address>. Check that the test email arrives ok.
Y create
You t the
th SNMP server byb entering
t i snmp manager <name>. TheTh other
th SNMP
fields (such as SNMP community, port number and protocol (UDP or TCP) can then be
configured. The available options appear on the screen.
N
Now llets
t examine
i h how tto create
t bbasic
i notification
tifi ti filt
filters for
f NBAD events
t (floods)
(fl d ) and
d ffor
HBAD events (quarantine).
W will
We ill show
h how
h to
t create
t notification
tifi ti filt
filters step
t b by step.
t
Step 1 is to create a filter name. Simply creating the filter name alone is enough to view it
in the GUI.
Step 2 is to add conditions (known as filters). You can enter multiple conditions, and all of
them must be true to trigger a notification (logical “AND” conditions). Different filter options
are available for NBAD and HBAD events.
NOTE These are “filter in” conditions
NOTE: conditions.
Step 3 is to add actions (known as notifications). This is an optional step. You can add
one or more of the available actions if you wish.
Step 4 is to add other information if necessary. Certain other filter parameters (such as
importance, template length and timeout) can be configured here.
Step 5 (another optional step) is to add subfilters. For every subfilter, conditions are once
again required and actions can be added tootoo. We will discuss subfilters in the section on
advanced filters.
Fi t create
First t a filter
filt name, by
b using
i the
th command:
d
notification filter flood <name> for NBAD filters or
notification filter quarantine <name> for HBAD filters
Once the name has been created, it appears in the GUI filter window.
Here is a brief explanation of the list of HBAD(quarantine) filters you see on the screen
screen. NOTE: All the
values are based upon the 1000 flows capture performed for all hosts. The 1000 captured flows are
analyzed and the results of the analysis are used for the notification filters various filter values
sensor / group. Name of the SPS or Group reporting this event. The operator may only wish the filter to be
activated if the event is reported by a particular sensor or is included in a predefined group.
time/wday/day/month/year. Time of the day (24 hour), days of the week (0, Sunday – 6, Saturday), days of
the month (1 – 31), months of the year (1-12) or year (4 digits)
activity can be categorized into one of 4 types of activities: flow bomb, port scan, address scan and mass-
SMTP as explained in the HBAD chapter.
SMTP, chapter
bps /cps / pps. Bits per sec (bps), flows per second (cps) or packets per second (pps) for that host.
bps percent / cps percent / pps percent. Given that the same host can perform several activities at the
same time, some legitimate and some illegitimate, this % value displays the % of the overall host bitrate
(bps percent), flow rate (cps percent) or packet rate (pps percent) for the selected activity
connsize_avg / pktsize_avg. The average size of the flows in packets or the average size of the packets
pin bytes. Scanning attacks tend to have few or one packet per flow, whereas spam tends to be several
packets per flow. DOS attacks usually have one packet per flow
IP . The target address
address. Useful for detecting attacks on known targets for example your DNS or mail servers
servers.
Port. Any single target port. Proto / Proto Name. The protocol number (proto) or name (proto name) used
for the activity.
Spread. Every type of HBAD event has a variable aspect. In Addr-Scans and Mass-SMTPs the destination
IP varies. In Port-Scan the destination port varies. In Flow-Bomb the source port varies. Spread represents
the percentage of unique values for the variable aspect among all connections involved in the event
(excluding background, seemingly legitimate connections). For all events except Mass-SMTP, higher spread
means higher likelihood of misconduct. With Mass-SMTP, big mail hubs that concentrate many domains
often skew the spread down.
Subscriber When SMP is configured
Subscriber. configured, the SP retrieves subscriber names from the SMP and these can be
used in the filter. Suspect_IP. The IP address of the host demonstrating suspicious behavior
Filters can be
Filt b made
d more efficient
ffi i t by
b choosing
h i the
th correctt operator
t for
f the
th task.
t k Some
S
operators accept
only one value, some accept a list, and others accept regexes. The available operands
will be defined by the selected field. For example – if the field is of type ‘sensor’, the
operand will be one of the associated sensors. If the field is of type ‘type’, the operand will
be one of the flood types.
H
Here we see an example
l off an NBAD filter
filt called
ll d ““critical”
iti l”
An NBAD event will be “filtered in” if the packet rate deviates from the expected model by
at least 100,000 packets per second, if the NBAD event lasts for at least 20 seconds and
if the severity of the event (measured by the shape vector) is greater than 2 (on a scale of
0-4 where 4 is the most severe). This filter is also limited to incoming traffic only.
N
Now we can add
dd an action,
ti b
be it an email,
il snmp ttrap or syslog.
l Thi
This iis d
done b
by entering
t i
action from within the filter
W can also
We l adddd extra
t iinformation
f ti tto the
th filter.
filt Importance
I t is
i a value
l (a( signed
i d 32 bit
integer) which is used to resolve conflicts when two or more filters match. In such a case
the filter with the highest importance receives precedence and will trigger the notification
(where defined)
A th example
Another l off extra
t information
i f ti which
hi h can b
be iincluded
l d d iin th
the filt
filter iis “k
“keepalive”.
li ”
Normally you will not need to use the keepalive field in the filter. If you don't enter a
keepalive, you receive a message at the start of the anomaly, a message at the end and a
message every time there is a significant deviation from the model (more than 30%). Use
keepalive if it’s important to continue sending notifications.
<TIME> is the number of seconds between updates. Updates are sent via the selected
post method for the current filter
p
U i th
Using the ttemplate
l t command d ffor notification
tifi ti about
b t NBAD events,
t you can also
l determine
d t i
whether you want the NBAD notification to contain concise information (default) or
detailed information (long). The long NBAD template includes pattern information in
addition to the normal flood data. HBAD (quarantine) templates can only be default
(short).
H
Here we see a summary example l off th
the b
basic
i NBAD filt
filter we h
have b
built.
ilt WWe see th
the
name of the filter (critical), the conditions, the actions (email and syslog) and the extra
information (importance and template).
N
Now llets
t llook
k att h
how tto create
t advanced
d d ((nested)
t d) filt
filters.
Th basic
The b i notifications
tifi ti section
ti ddealt
lt with
ith simple,
i l one llevell filt
filters. Th
They, without
ith t ddoubt
bt are
powerful tools in fine tuning the content of alerts being sent to operators, but they can be
controlled even further. This section introduces subfilters, a concept where filters can be
nested within other filters, creating really powerful filtering tools.
Each filter can include as many nested subfilters as you wish. Nested subfilters are
reached only if higher levels match too. There are 3 different types of subfilters: flood
subfilters and p
pattern subfilters ((for NBAD events)) and q
quarantine subfilters ((for HBAD
events).
L t’ llook
Let’s k att th
the possible
ibl filt
filter nesting
ti patterns
tt ffor NBAD subfilters.
bfilt
A Flood filter can have either a flood subfilter or a pattern subfilter under it. Flood
subfilters have the same contents as regular flood filters, but pattern subfilters contain
completely different condition options, based on the different fields measured in an NBAD
pattern. Each flood subfilter can have either a flood subfilter or a pattern subfilter under it.
A pattern subfilter can only have another pattern subfilter under it. Within these structural
boundries yyou can include as manyy subfilters as youy wish.
The nesting patterns for HBAD subfilters are much simpler. Beneath a quarantine filter
you can place quarantine subfilters. Beneath a quarantine subfilter you can place another
quarantine subfilter and so on.
A with
As ith regular
l filt
filters, you first
fi t create
t the
th name by
b entering
t i
subfilter flood <NAME> for NBAD flood subfilters
subfilter pattern <NAME> for NBAD pattern subfilters
subfilter quarantine <NAME> for NBAD pattern subfilters
You then proceed to add the relevant subfilter conditions and actions
While subfilter flood and subfilter quarantine contain the same options as filter flood and
filter quarantine respectively, subfilter pattern has its own set of unique options which we
will analyze now.
H
Here are th
the diff
differentt options
ti ffor pattern
tt subfilters.
bfilt
H
Here are th
the remaining
i i options
ti ffor pattern
tt subfilters.
bfilt All
Allott Customer
C t Support
S t have
h
prepared one or two filters that you can use as templates. Feel free to modify or create
new ones if you wish, based on your specific needs.
H
Here we see an example l off an advanced
d d NBAD notification
tifi ti filt
filter. It contains
t i 4 flood
fl d filt
filters
which quantify the NBAD event into critical, warning, minor or warning. Each filter then
contains 3 pattern subfilters that qualify the event into 3 different types of floods – DDoS
attacks, DoS attacks and IPscan attacks.
H
Here we see th
the ““critical”
iti l” fl
flood
d filt
filter with
ith itits 3 pattern
tt subfilters
bfilt as shown
h on the
th previous
i
slide. Note how conditions (filters) and actions are grouped together
The subfilter pattern is constructed to qualify the event as a DDoS attack, IF:
- The number of destinations is <=4, AND
- The source ip count >15, AND
- The l3size >6, AND
- The filter order <5
A low number of destinations and high number of sources means a many to one attack,
also known as a DDoS.
The L3size measures the clarity of the event – a count of consistent bytes in the patters
(shown in blue on the GUI) at the TCP/IP layer 3 is greater than 6 (less than 6 is not very
clear attack)
The filter order measures the relevance of the event. If the filter order is less than 5 this
means that we are filtering one of the top 4 patterns. Patterns are sorted by relevance,
with the better the match, the higher the position.
W finish
We fi i h with
ith a b
brief
i f llook
k att h
how tto ttroubleshoot
bl h t notification
tifi ti filt
filters.
A useful
f l way to
t begin
b i ttroubleshooting
bl h ti notification
tifi ti filt
filters iis tto see which
hi h ones have
h b
been
configured in the system. This is done by entering the command show notification
config.
The output displays all the notifications configured in the system – be aware that if you’ve
setup a lot of notifications, this output can be very long.
Similarly,
Si il l you can display
di l theth details
d t il off a specific
ifi notification
tifi ti filt
filter that
th t you’ve
’ setup
t b be
entering show notification filter flood <name> or show notification filter quarantine
<name>
In the example on the screen, we are displaying a particular NBAD notification filter called
“TestFlood”
Th “test”
The “t t” command d is
i partt off the
th CLI TTop llevell commands.
d All off th
these commandsd are
available on the controller only, and they are used to test functionality of various services
– including the notification filter delivery mechanisms. For example, if the ServiceProtector
has one or more SNMP servers configured, test snmp tests connectivity to the SNMP
server. If the ServiceProtector has one or more syslog servers configured, test syslog
tests connectivity to the server.
On the screen we see an example
p of test mail which tests the email notification
mechanism.
Wh t d
What does thi
this notification
tifi ti filt
filter d
do?
?
Pl
Please refer
f tot the
th ServiceProtector
S i P t t Student
St d t Workbook.
W kb k
Thresholds
ServiceProtector Administrators Training
9-2
Thresholds
ServiceProtector Administrators Training
9-3
Thresholds
ServiceProtector Administrators Training
9-4
Thresholds
ServiceProtector Administrators Training
9-5
Thresholds
ServiceProtector Administrators Training
The threshold based system though has its limitations. Static thresholds
trigger at a preset value, and depending upon where that value is placed,
they might trigger at sub-optimal traffic volumes. So if the trigger value is
too high, you may miss events that are slightly smaller than it, but since
the baseline traffic may be much lower, the event itself may be large. This
can be seen for example in the example at the top of the screen, where
the threshold catches the first anomaly but is too high to catch the second
anomaly.
On the other hand, if the trigger value is too low and the natural traffic is
very close to this value, you may have false alarms by traffic triggering the
threshold. In the example at the bottom of the screen for example, three
events are detected, but the third time the threshold is crossed, the event
is a false alarm.
9-6
Thresholds
ServiceProtector Administrators Training
Lets first of all examine basic (static) thresholds and how to configure
them
9-7
Thresholds
ServiceProtector Administrators Training
9-8
Thresholds
ServiceProtector Administrators Training
9-9
Thresholds
ServiceProtector Administrators Training
9-10
Thresholds
ServiceProtector Administrators Training
9-11
Thresholds
ServiceProtector Administrators Training
There are 5 steps to creating dynamic thresholds. Steps 1-4 are as with
static thresholds:
1.Create a new threshold by entering: threshold <NAME>
2. Enter a description (optional) by using: description <DESCRIPTION>
3. Select an SPS and group, SPS comes first followed by group name. Note
that only one SPS and group must be configured per threshold. location
<SPS NAME> <GROUP NAME>
4. Select a tracked statistic using: watch <INCOMING/OUTGOING>
<STATISTIC> <BITRATE/PACKETRATE> This command accepts 3
parameters : a. Incoming/outgoing – direction of the traffic; b. Tracked
statistic –the actual statistic that we are monitoring for a breach of limits; c.
Bitrate/packetrate – monitor the bitrate or packetrate of the tracked statistic
5. Define the long and short term windows using the following command:
dynamic <SHORTTERM> <LONG TERM> <MINIMUM/MAXIMUM>
PERCENTAGE>
The dynamic threshold uses sliding windows for the long and short term
periods. This command accepts four parameters: a. short term period; b.
long term period; c. minimum/maximum – is the limit breached by exceeding
or falling below; d. How much was the limit breached by. If the % symbol is
added, this value becomes a percentage.
9-12
Thresholds
ServiceProtector Administrators Training
9-13
Thresholds
ServiceProtector Administrators Training
Once they have been defined and triggered for the first time, thresholds
appear in the GUI in the “types” window on the NBAD/Flood Activity page
just like regular built-in floods do. In addition, a threshold can trigger a
flood notification. The notification flood filter is created in the usual way
(see Module 8: Notifications). The filter type field is where the name of the
threshold is entered. Thresholds create entries in the “type” operands.
They
ey are
a e accessible
access b e by selecting
se ect g Filter
te type == {p {press
ess tab}
tab}. ((The
e ==
operator is used as an example. You can use any of the valid operators)
Bear in mind however that the ‘type’ field only updates after the threshold
has been triggered for the first time. A workaround is to artificially make
the trigger value artificially low so its name appears there. Once triggered
you can then change it back to its real value.
Alternatively, you can simply enter the threshold name in the type field.
There is no validity checking so it will notify
f when triggered
9-14
Thresholds
ServiceProtector Administrators Training
Lets see an example of how this works. In our example, we will use the
dynamic threshold created in the previous section. This threshold triggers
an NBAD event whenever the packet rate for the last hour is 20% more
than the packet rate over the last day. We called this threshold “dynamic-
example”
Once the threshold has been triggered for the first time, we can create the
notification filter. The sole condition of the filter is if the filter type equals
dynamic
dy a c example.
e a p e A notification
ot cat o iss sent
se t to use
user@allot.com.
@a ot co
9-15
Thresholds
ServiceProtector Administrators Training
9-16
Thresholds
ServiceProtector Administrators Training
Remember that the names of the thresholds you have defined will not appear in
the “types” window of the ServiceProtector GUI (e.g: on the NBAD/Flood Activity
page). They will only appear here once they have been triggered for the first
time. Likewise, until triggered, your thresholds will not appear as one of the
options of an event that may trigger a notification filter.
9-17
Thresholds
ServiceProtector Administrators Training
You can display all of the thresholds that have been configured by using
the command: show threshold
Alternatively, show threshold <NAME> displays the settings for the
specific named threshold – in this case we are looking at a threshold
called TestThr.
9-18
Thresholds
ServiceProtector Administrators Training
9-19
Thresholds
ServiceProtector Administrators Training
9-20
Thresholds
ServiceProtector Administrators Training
System Maintenance
ServiceProtector Administrators Training
In this module, we will examine several procedures and tools required to ensure the
maintenance of the system. These tools will include how to upgrade software versions,
how to remove groups and sensors, how to relocate elements of a system and replace
an SP-Sensor. We begin by examining how to perform a software upgrade.
Before performing the software upgrade, you may wish to back up your existing
configuration. There are 2 ways to do this. The first is to use the snapshot command. To
take a snapshot run the following from the top level CLI:
system support-info long usb-storage | web-ui
The snapshot command is discussed in more detail in Module 11 – Troubleshooting.
The second option is to run the following from the linux shell prompt:
cli -c ‘show running-config’ > FILENAME
Running the top level command ‘show version’ tells you the current version of the
ServiceProtector software that is running. It can be run on an SPC, and SPS or an
SPCS.
There are two different ways to upgrade the software of the ServiceProtector system.
In any event, it is important to first of all upgrade AND REBOOT the CONTOLLER,
and only then upgrade and reboot each sensor one by one. The two methods are as
follows:
1) You can upgrade each component manually by inserting a CD into each component
that you wish to upgrade, be it the SP-Controller or SP-Sensor, and entering
<system upgrade cd-rom>. Alternatively, each component can be upgraded
individually
d dua y from
o aU URL by eentering
te g <system
syste upg
upgrade
ade uurl> o
on tthe
e local
oca uunit.
t In tthis
s
case, the cdrom should be available at that URL for the command to work.
2) Alternatively, you can enter the CD into the Controller and choose which components
you wish to upgrade (including remote sensors). This is done by entering: <system
upgrade sensor <HOST> >. You must then reboot the sensor immediately after the
upgrade. This is done by entering system reboot sensor <NAME>
To perform the upgrade, you should first check the software upgrade path in the
ServiceProtector release notes or with Allot Customer Support. Download the ISO file
from the Allot FTP site, burn on a CD ROM and insert into SP-Controller. You can then
enter: system upgrade cdrom
Once the upgrade has been successfully completed, enter: system reboot
NOTE: It is mandatory to reboot each box immediately after upgrade
Lets now see how to permanently remove groups and sensors from the
ServiceProtector.
Purge is used to completely remove an SPS or Group from the database. The SPS
reports traffic
ffi statistics
i i ffor these
h groups and d this
hi iinformation
f i iis used d iin the
h various
i
reports such as traffic detail. In addition, floods and thresholds generate events, packet
samples, and analysis. All this information is stored in the database. When a group or
SPS is deleted using the 'no' command, that SPS or group is removed from the statistics
gathering, but is not physically removed from the database. This is important because
historic data for that group or SPS exists and will be included in relevant reports.
Purging a group or SPS means completely removing it from the database. This action is
nott reversible.
ibl All hihistoric
t i d data
t ffor th
thatt particular
ti l ititem iis llost.
t Th
The ititem iis nott available
il bl ffor
configuration in the CLI and does not appear in the reports either. Depending upon the
size of the database, the purging process may take a long time. Make sure you run the
purge command from the controller console so that the process is not interrupted. If you
must run it from a shell, use the ‘screen’ LINUX command to ensure that a network
disconnection does not cause the session to be lost and the database corrupted.
Here we see the difference between no group and purge group. In our example, first of
all the group “dan” is added. By running the command “show group summary” we see
the group with a single prefix and a single sensor. When we run “no group dan” the
group is not deleted, but the service protector stops collecting new data for it. The history
is therefore kept and data will still appear in reports. When we purge the group, it is
removed fully. This will remove the entire group and all references to it. Historic data will
be completely removed. Use this command with caution - no confirmation requests are
presented and there is no way back. The group purge may take a while, depending upon
the quantity of information collected. It is also highly recommended that the purge
command be done from the SPC console and not a remote terminal. If the connection to
the terminal is closed, errors in the database may occur. An alternative is to use the
‘screen’ command on remote terminals to ensure that in case of a disconnection, the
process keeps on running.
Note – the purge command may take a very long time depending upon how much
information is stored,, so do make sure that purge
p g is done on a stable network connection
on from the console itself. By purging a sensor or a group, you lose all of the data
collected, and it can’t be recovered. The database items for that sensor or group are
wiped clean (traffic, samples etc.)
In this section we examine how to relocate a system – how to change the IP address
and network details of the various components.
Lets now look at the procedure for relocating a controller (SPC) in a distributed
deployment The procedure must be followed in the order outlined below
deployment. below.
1.Put the sensors into shutdown mode. Open an SSH session to the SPC. Enter the cli
by typing ‘cli’. Enter the configure menu by typing ‘configure’. Enter the Sensor
configuration menu by typing ‘sensor <NAME>’. Put the Sensor into shutdown mode by
entering ‘shutdown’. This step should be repeated for all attached sensors.
2.Configure IP and Other Information on the controller. Connect via a console to the
controller. This is necessary as you will lose communication if you change the IP
address. Now you can change the networking parameters of the Controller. You may
change IP Address
Address, Hostname,
Hostname Domain namename, Default Route
Route, DNS Server(s)
Server(s), NTP
Server or Timezone. Use the ‘no’ command to remove a configuration item such as ‘no ip
dns-server <DNS_SERVER>’. Note that there is no command ‘no hostname’. Simply
configure a new hostname over the existing one by entering ‘hostname <HOSTNAME>’.
3.Reboot the controller machine. Enter ‘do system reboot’
4.Bring Sensors out of Shutdown Mode. Once the controller has rebooted, bring the
sensor out of shutdown mode. On the controller, enter the cli by typing ‘cli’. Enter the
configure menu by typing ‘configure’. Enter the Sensor configuration by typing ‘sensor
<NAME>’. Bring the Sensor out of shutdown mode by entering ‘no
<NAME> no shutdown
shutdown’. NOTE:
You will see an error message (ssh using old credentials) and then be requested the
Sensor password. The system will start several processes and will be operational within
a few seconds. Again, step 4 should be repeated for all attached sensors.
NOTE: If the controller is relocating to a situation where the IPsec settings change (for
example the controller now is moving behind a NAT) you’ll need to make the necessary
adjustments to the controller and/or sensors. Refer to the Sensors and IPsec Training
Module.
Lets now look at the procedure for relocating a ServiceProtector Sensor (SPS) in a
distributed deployment. The procedure must be followed in the order outline below.
1.Put the sensor into shutdown mode. Open an SSH session to the SPC. Enter the cli
by typing ‘cli’. Enter the configure menu by typing ‘configure’. Enter the Sensor
configuration menu by typing ‘sensor <NAME>’. Put the Sensor into shutdown mode by
entering ‘shutdown’.
2.Configure IP and Other Information on the sensor. Connect via a console to the
sensor This is necessary as you will lose communication if you change the IP address
sensor. address.
Now you can change the networking parameters of the Sensor. You may change IP
Address, Hostname, Domain name, Default Route, DNS Server(s), NTP Server or
Timezone.
3.Reboot the sensor machine. Enter ‘do system reboot’
4.Configure the New Sensor Settings on the Controller. Start an SSH session to the
controller and enter the configure
g menu. Enter the sensor configuration
g menu by
y typing
yp g
‘sensor’ followed by the old name (if it has changed). ‘sensor <NAME>’; Then you can
type the new name by entering ‘name <NEW_NAME>’ and the new IP address by
entering ‘ip-address <NEW_IP>’
5. Bring Sensor out of Shutdown Mode. On the controller, from within the sensor
configuration menu, bring the Sensor out of shutdown mode by entering ‘no
shutdown’. The system will start several processes and will be operational within a
few seconds
seconds. The controller is now connected to the new sensor
sensor.
In rare situations, a Sensor hardware fault may cause the system not to boot. In this
case a replacement
l t sensor may be
b provided.
id d Thi
This procedure
d shows
h h
how tto replace
l th
the
faulty hardware with the new hardware.
1. Install the Same Software. On the SPS, install the same software as that which
exists on the SPC. (Note: this means the same version and build – e.g: SP4.1.0b45).
Note: you must follow the full install process and will need to enter the license manually.
See Module 4: Slides 8-10 for more details.
2. Configure Network Parameters. On the SPS, configure network parameters - IP,
Hostname etc.etc
3. Shutdown Sensor. On the SP Controller, open up the CLI, enter the Sensor
submenu by entering ‘configure’ and then sensor <SENSOR_NAME>. If the Sensor
wasn’t shutdown through the SP Controller type ‘shutdown’
4. Remove the Sensor’s Public Key. Remove the Sensor public key by typing ‘no
public-key’
5. Bring the Sensor out of Shutdown Mode. This is done by entering ‘no shutdown’
6. Add New Key. You will be prompted that the host key verification failed, and if you
want to add the new key. Type ‘yes’ and then type the Sensor password you provided
during install.
Finally lets see how to calculate the disk space required by the ServiceProtector.
You may however, wish to perform a more realistic calculation. Information is stored in
the database on a per-group basis. Floods are detected, categorized, and stored by
group. Database usage on average is around 11Mb per day per Sensor-group with
floods and quarantine enabled. This number is based upon the average usage we have
observed at our customer sites.
What is the estimated daily disk space usage? The estimated daily disk space usage
is calculated using this formula: [(groups on Sensor1) + (groups on Sensor2) + ... +
(groups
(g oups on
o Sensor
Se so N)] )] * 11 Mb.
b
What is the effective storage? 146x2 disks in RAID1 counts as 116Gb effective
storage (this is because formatting causes disk overhead, the database itself has
overhead and the database cycles at 90% diskspace). Meanwhile, 146gbx6 in RAID5
counts as 584GB effective storage.
What is the storage duration? Calculate storage duration by dividing the effective
storage by the Mb/day number from above. Remember that the true disk space used will
be slightly higher, because traffic
ff samples are stored in the database for
f 14 days, so
after the first 14 days of activity, there will always be 14 days worth of samples in the
database too.
Running the df command from the Controller tells you the physical amount of space
used up on the hard disk.
If your SP Controller has 146 effective storage space, and you are working with 3 sensors,
each with 5 groups. How long will the controller database last?
Troubleshooting
ServiceProtector Administrators Training
In this module we will introduce several key troubleshooting tools. We being by examining
the main processes that should be running on the ServiceProtector Controller and Sensor.
.
Troubleshooting 11-2
ServiceProtector Administrators Training
Here we see a list of the processes which should be running on the Controller. The list is
accessed by entering ‘show system status verbose’ on the SP Controller.
Troubleshooting 11-3
ServiceProtector Administrators Training
Here we see a list of the processes which should be running on the Sensor. The list is
accessed by entering ‘show system status verbose’ on the SP Sensor.
Troubleshooting 11-4
ServiceProtector Administrators Training
Now lets see how to avoid problems of communication between the different elements in the
system.
.
Troubleshooting 11-5
ServiceProtector Administrators Training
Here we see the different ports that need to be open on firewalls and/or routers to ensure
communication between the administrator and the sensor or controller, between the sensor
and controller and corporate servers (for the purpose of notification) and between the
controller and its sensors themselves.
User access to the SP-Controller is via the following ports: 22/TCP for SSH communication
and 443/TCP for HTTPS.
User access to the SP-Sensors is via 22/TCP for SSH communication.
The SP
Th SP-Controller
C ll may needd to access corporate servers ((e.g: SNMP server or S
Syslog)
l ) over
the following ports: 25/TCP for SMTP notifications, 514/UDP for Syslog communication,
162/UDP for SNMP and 53/UDP for DNS. NTP synchronization takes places over 123/UDP.
DNS communication from the sensor is also required over 53/UDP.
For details on communication between the SP-Controller and SP-Sensor, refer to the
Sensors & IPSec training module.
Troubleshooting 11-6
ServiceProtector Administrators Training
Various troubleshooting commands exist which give you an overview of what is currently
running on the system. We will examine these in turn now.
.
Troubleshooting 11-7
ServiceProtector Administrators Training
The ‘show system status’ command can be run from either the controller or the sensor. It
lists any errors if detected, or otherwise will record “System status ok” if there are no faults.
A second version of the command, ‘show system status verbose’ , displays the processes
running in addition to the system status.
Troubleshooting 11-8
ServiceProtector Administrators Training
‘show system usage’ is available for both the controller and sensor from the top level
menu. It displays the CPU and network statistics and is a good indication of how the SP is
managing with the traffic load.
Troubleshooting 11-9
ServiceProtector Administrators Training
The command ‘show running-config’ can also be run on either the controller or sensor and
is available from the top level menu. Its display depends upon the type of box. Controllers
have more settings and will therefore display much more detail than sensors.
This command is particularly useful for when you are backing up and restoring
configurations and to look for settings you’ve made.
Troubleshooting 11-10
ServiceProtector Administrators Training
Troubleshooting 11-11
ServiceProtector Administrators Training
Here we see a list of the main ServiceProtector logs and an explanation of their usage. You
also see from the table which of the logs are available on the Controller only, on the Sensor
only and which ones are available on both the controller and the sensor. On both the SPS
and the SPC, the relevant logs are found in the directory: /var/ntais2/log
Troubleshooting 11-12
ServiceProtector Administrators Training
The table here presents a list of useful Linux system logs which are created on the system.
On both the SPS and the SPC, the relevant logs are found in the directory: /var/log
Troubleshooting 11-13
ServiceProtector Administrators Training
You can tracking changes in log files by entering watch -d -n 1 ls –l inside the
/var/ntais2/log directory. Use this command to watch which files are changing or for that
matter to track any change in output. This can be used in troubleshooting the system
because if some log file grows unusually fast it may imply to some fault. Once the file is
found, use the 'tail' command to track it.
Troubleshooting 11-14
ServiceProtector Administrators Training
In case of doubt – use the snapshot command to archive up the logs and email to
support@allot.com
system support-info long web-ui Gives you download from the GUI (NOTE: the web-ui
command can only be run from a controller)
system support-info long usb-storage Writes the logs to a usb drive
The syntax of the ‘system support-info’ command gives you several options:
g Enables yyou to determine if yyou wish a short or long
short/long: g snapshot
p
estimate: Lets you see an estimate of the snapshot size
usb-storage or web-ui: Shows where you wish to store the snapshot.
Troubleshooting 11-15
ServiceProtector Administrators Training
Troubleshooting 11-16
ServiceProtector Administrators Training
Aside from the “show” and “test” commands which have been covered throughout this
course, other useful commands can help you troubleshoot problems with the
ServiceProtector system. We will examine some of these here.
Troubleshooting 11-17
ServiceProtector Administrators Training
Throughout this course we have made use of the show and test commands. When it comes
to troubleshooting, these commands make a good place to start. The show commands
enable you to see exactly what has been configured. The test commands enable you to
check that the transportation mechanism is working.
Troubleshooting 11-18
ServiceProtector Administrators Training
This table presents a list of debugging tools, all of which are accessed from the top level
menu.
The bash prompt gives you access to the shell and is useful for debugging, checking log
files, etc. Type ‘exit’ to return to the CLI. A Root user logs in to the system to a prompt then
can start the cli manually. Administrator users log in to the cli directly and must issue the
‘bash’ command to receive a shell
The passwd prompt changes the password for the logged in user. Note that the password is
checked for strength
strength. Weak passwords will not be accepted
accepted. NOTE: Always change
password from the CLI ‘passwd’ command - never from the shell
‘ping <IP|HOSTNAME>’ serves the same purpose as a ping from the command prompt.
‘trace <IP|HOSTNAME>’ runs a trace route to the IP or host specified.
‘system reboot sensor <sensor name>’ reboots the specified sensor from the controller.
‘system restart <COMPONENT>’ restarts a component of the system.
Troubleshooting 11-19
ServiceProtector Administrators Training
Finally here is a list of useful commands and how they can help with the troubleshooting
process.
Troubleshooting 11-20
ServiceProtector Administrators Training
We end by looking at some common problems faced when re-installing the ServiceProtector
software, and how they can be solved.
Troubleshooting 11-21
ServiceProtector Administrators Training
If the installation fails, you receive an error message and the installer halts, letting you check
the logs and copy the logs to USB. Press CTRL ALT F2 to receive the debugging menu.
Choose from the menu:
L: View logs using ‘less’. This lets you view the detailed logs on the screen
S: Save log to USB flash. This saves the detailed log to a USB flash disk
Z: Drop to shell. This gives you access to the LINUX shell to debug the failure further.
g is stored under /tmp/install.log
NOTE: The detailed log p g
This file is only copied to the hard disk after a successful installation. When the installer CD
is booted, /var is NOT the harddisk, and /var/ntais2 does NOT exist. /var/log/dmesg is only
written when a system boots up, and not updated after that. It probably will not be created
when the installer CD is booting anyway. Use the 'dmesg' command instead. This is
available on the installer CD.
Troubleshooting 11-22
ServiceProtector Administrators Training
Troubleshooting 11-23
ServiceProtector Administrators Training
Which command can be used to view the processes currently running on the
ServiceProtector?
Troubleshooting 11-24
ServiceProtector Administrators Training
Troubleshooting 11-25