Csci Iscw
Csci Iscw
Csci Iscw
(Full Paper)
on
Email: {duju0001, brjo1036, ruma0003, krre0001}@hs-karlsruhe.de
Abstract—In this work we present an approach to derive test described. Here, the item under analysis is also a black-box
cases for security testing by reusing threat modeling artefacts. for the tester who has to gather information about the item he
We suggest to perform a threat modeling in an early stage wants to penetrate. Unfortunately, the phase of information
of the product life cycle and show how these threat models,
gathering and planning for penetration testing is not an easy
iti
originally created for the security design phase, can be reused
for security testing. We further show the practical applicability task. Therefore, we deem it as meaningful to reuse generated
of our approach by describing the creation, execution and artefacts to reduce the effort for penetration testing. In detail,
evaluation of a penetration test for an Electronic Control the contributions of this paper are the following:
Unit (ECU). For the evaluation of our approach we selected Problem: In this work we address the problem that
Ed
a Pyrotechnic Control Unit (PCU) due to its safety critical
functionality, which is commonly referred to as an Airbag-
ECU. This scenario leads to a safety impact as a PCU is
connected to pyrotechnic charges, e.g., airbags, that can lead
to serious injury or death, if deployed in a non-appropriate
scenario. During this penetration test, the selected PCU was
installed on a test bench. We were able to successfully exploit
a found vulnerability, causing the deployment of charges.
Keywords-threat modeling; automotive security; security tes-
actually no common approach exists to derive test cases in
a defined way based on the outcomes of the threat modeling
analysis.
Solution: We provide an approach to derive test cases for
penetration testing on ECU-level in an early stage of the
automotive SDL based on the results of a performed threat
modeling.
Contribution: We contribute a structured procedure star-
ting; CSCI-ISCW
ting with a functional threat analysis of an automotive ECU
s
and how the results of that analysis can be reused to gather
I. I NTRODUCTION
information for penetrating the observed ECU. Furthermore,
Modern automobiles host more than 50 ECUs, which we evaluate our approach by applying it to a safety-critical
or
contain and implement a total of up to 100 million code ECU. As a result of our research, we have uncovered a
lines to control safety-critical functionality [1]. This fact vulnerability, which likely scales over several manufacturers.
and the close interconnectivity of automotive ECUs open up However, for exploiting this vulnerability some conditions
new possibilities to attack these systems which impair the have to be met, which are described in this paper.
safe operation of the vehicle [2][3][4][5]. To counter these The paper is structured as follows: In section II we
th
attacks a holistic approach has to be applied, often referred give a short overview of security testing concepts used
to as security by design, starting with a threat analysis in Information Technology (IT) and their suitability for
in an early development phase and ending with security the automotive domain. Furthermore, we summarize threat
testing [6]. These tasks are typically embedded in a Security analysis methodologies which are known to be applicable for
Development Lifecycle (SDL) for the item to be built. Even automotive systems. Afterwards, we present in section III
Au
though the beginning and the end of the SDL seem quite our approach to integrate threat modeling in penetration
different at first glance, generated artefacts can be reused. tests. To show the practical applicability of our approach we
In particular, artefacts of a performed threat modeling for present an example in section IV. Additionally, we present
security testing. possible countermeasures to mitigate the found vulnerabi-
Threat modeling describes a technique to analyze the lity, and we conclude with an outlook to future work in
security level of an item, determining security risks asso- section V.
ciated with the item and their mitigation. Typically, such
an analysis is done in an early development phase whereby II. BACKGROUND
the item can be seen as black-box due to the fact that no The classical approach in IT to perform a black-box secu-
technical information or implementation exists. Characteris- rity test is the application of a penetration test. In this type of
tically, only a functional description exists. Similar to that testing the testers do not have any detailed information about
the knowledge level for a black-box penetration test can be the target system. The objective of penetration testing is to
analyze the security level of a system, network or application III. A PPROACH
from an attacker’s perspective. This also implicates to use all
Based on the assumption that a threat analysis is per-
tools and methods, which are available for a potential hacker.
formed in the first phase of product development which
The primary goal of penetration testing is mostly finding
means during or directly after requirements engineering.
vulnerabilities, which could be exploited by a potential
We propose the generation of attack trees during threat
attacker.
analysis as they give a general procedure how selected
Nowadays, there are several tools, frameworks and met- misuse cases could be achieved - even for less experienced
hodologies available, which push the level of test automa- testers. To build such a tree, all relevant assets have to
tion for minimizing the costs and higher reproducibility. be collected. An asset is something valuable for an entity
on
Furthermore, penetration testing is often a mandatory part and must be protected from a malicious attacker. From the
of a security audit. However, traditional black-box testing standpoint of the asset, there are certain use cases, which
considers less needs with respect to software developers and must be identified at the beginning of the threat analysis.
business risks, because the execution of penetration tests is Afterwards, the related misuse cases can be defined, which
normally in a late stage in or after the SDL. If any security are represented by the second level of the tree in figure 1.
issues will be uncovered at this point, the costs and efforts of Our approach then utilizes the attack tree to further elaborate
iti
software patches are high in contrast to early development on the misuse case. For describing the different attack
stages. For this purpose, it would be more effective to derive stages (nodes between the root and the leaves) we use the
test cases based on a business and architectural risk analysis STRIDE [13] terminology.
to integrate parts of the penetration test into the life cycle Lastly, we describe the leaves of the attack tree with the
as early as possible.
Ed
In the penetration testing sector, there are some orga-
nizations, which provide manuals and guidelines as met-
hodologies for determining such tests. The best known
are: Open Source Security Testing Methodology Manual
(OSSTMM) [7], Information Systems Security Assessment
Framework (ISSAF) [8], Open Web Application Secu-
rity Project (OWASP) [9], Penetration Testing Execution
terminology of information assets, presented by the Confi-
dentiality, Integrity, and Availability (CIA) triad. This allows
us to quickly derive first protection goals, e.g., ”The integrity
of diagnostic messages has to be ensured.” Additionally, the
information assets can further be used to derive test cases to
check the correct functionality of the demanded protection
mechanisms. To reuse the results of the attack tree in figure 1
Threat Analysis standpoint of the asset, there are certain use cases, e.g., the
Misuse Case
User Requirements Acceptance
deployment of airbags in case of a crash. Contrary to that, a
on
Engineering Testing tester or attacker can define certain misuse cases. We used
Misu
se C
No ase the approach presented in section III to derive two misuse
de
System Requirements in System
an
At
cases which are shown in table I.
N
Engineering ta Testing
od
t
in
h
the occupants of a vehicle. The first misuse case could
an
No
t
ac
e
k
in
Engineering Testing
Pa
iti
an
th
At
Design
ck
Testing
t h
OR
situations have to be taken into account when performing the
s
acceptance testing. Consequently, while system testing the
Spoofing of Tampering of Spoofing
physical impact of a vulnerability can not be determined Diagnostic Services Diagnostic Services Crash Situation
OR OR OR
of an attack path.
Tampering of Tampering of Tampering of
Crash Signals Speed Signals Gyro Signals
th
IV. E VALUATION OR OR OR
on
function correctly, as it could not differentiate between the
signals being authentic or being tampered with. In the following we summarize all issues which were
discovered during the penetration test. We discovered issues
B. Connectivity of Device under Test which were solely based on the software implementation,
After modeling the misuse case with the help of attack e.g., the weak Security Access (SA) and we discovered is-
trees, the assumed connectivity of the target was visualized. sues where the weakness consists of an unused combination
This overview helped us to identify possible attack vectors. of hard- and software, e.g., no ACL.
iti
In figure 4 the connection of the components is either part 1) Security Access: The created attack tree for the se-
of an automotive bus system or an end-to-end connection. lected misuse case (see figure 3) includes the use of vehicle
diagnostics. Diagnostic functionality in modern cars is often
ACL
OBD-II Port implemented according to the Unified Diagnostic Services
Speed Sensor
ECU
CAN_1
Diagnostic
Central
Gateway
CAN
CAN_2
Ed
Crash Signals
PCU
Gyro Signals
(UDS) standard [15]. The UDS standard contains the Secu-
rity Access service, which is typically used for authentication
before safety critical diagnostic functions are performed. The
service uses a challenge-response procedure, providing a
seed to the tester and demanding the correct key afterwards.
In the scenario of a legitimate attempt, both tester and target
possess the secret to perform the calculation of the key.
With the Security Access service having a history of being
exploited, this access was assumed to possess a vulnera-
s
Figure 4. Assumed connectivity of the device under test.
bility. In particular, the algorithms analyzed so far violate
Figure 4 shows our device under test (PCU) connected the Kerckhoff’s Principle by using the algorithm as secret
via CAN to the Central Gateway (CGW). There are two instead of the key. In this case, we analyzed multiple seeds to
or
different internal signal paths for communicating with the check for patterns and the overall amount of different seeds
PCU either over the CGW or directly by physical access to provided by the target. Here, the given target provided new
the CAN 2 wires. In addition, an external communication seeds for different requests and after the target has been
can be established via the On-board diagnostics (OBD)- power cycled.
II-Port by using the Diagnostic CAN, which is connected While studying the UDS standard, we further inves-
th
to the CGW. Furthermore, the PCU has two additional tigated a reference to another International Organization
connections, one for receiving data from crash sensors for Standardization (ISO) standard for End-of-life (EOL)
(Crash Signals) as well as the Additional Communication deployment of pyrotechnic charges [16]. A section in the
Line (ACL). To clarify, both connections are hard-wired in EOL standard describes the implementation of the Security
the physical sense, but we are going to refer to the end-to- Access service to protect against unauthorized deployment.
Au
end connections as hard-wired signals in this work. In case This is followed by an example demonstrating the exchange
of the source being implemented inside one of the parts, it of seed and key. In this example a 2 byte seed is shown,
is included in the appropriate box, e.g., the gyro sensor. The where the value in the MSB is the version number of the
placement of the gyro sensor inside a PCU is a standard implemented standard. The LSB is a random value. For the
implementation in the automotive industry. calculation of the corresponding key, i.e., the secret, one’s
complement of the seed is used. Standards usually provide
C. Test Bench for Penetration Test an example of the process and some suggestions for an
Due to the fact that an unintended deployment of charges algorithm, e.g., the UDS standard [15] clearly states that
could harm our life, we decided to build a test bench for the example in its Security Access section is just one way
the penetration phase. The setup consisted of a PCU which of implementing the secret.
is connected to a Linux machine using an USB to CAN Sending a key calculated by one’s complement of the sup-
adapter and the CAN pins of the PCU. The Linux machine plied seed, the authentication attempt proved to be successful
and unlocked the target. The raw communication between this functionality would not have been possible in the way
tester and target can be seen below in listing 1. described in this work.
3) Lack of Plausibility Checks: The observed PCU with
Listing 1. Security Access successful seed and key exchange the given functionality has access to information from hard-
7F1 [ SF ] ln : 2 data : 27 5F wired signal sources and bus messages. However it does
7F9 [ SF ] ln : 4 data : 67 5F 01 E3 not use all of them to derive if the current state of the
7F1 [ SF ] ln : 4 data : 27 60 FE 1C vehicle is correct. In fact, it only required one CAN message,
7F9 [ SF ] ln : 2 data : 67 60 containing the vehicle speed, and physical requirements to
In the above listing, the seed is 0x01E3 and the corre- enable deployment on the test bench used for this work. The
on
sponding key is 0xFE1C. To visualize one’s complement, it physical requirements were connected pyrotechnic charges,
can be demonstrated by looking at the binary representation in our case the substitution of those by appropriate resistors,
of both seen below (see equation 1): and the ignition being turned on or the engine running
(if installed in a real vehicle). Further attached hard-wired
0000000111100011 = 0x01E3 signals like seat occupancy were present on the connector
(1) of our target but not used.
1111111000011100 = 0xF E1C Combined with the fast and highly automated procedure
iti
However, in our practical evaluation we found that this of the attack, this leads to the possibility of malicious
given example in the standard is used in a PCU series unit deployment while the engine is running and people reside
in-field which contradicts the fact that in the given challenge- in the car, e.g., while waiting at a stoplight. This should be
response approach the algorithm (here: one’s complement) seen as a real threat, as the remote exploitation of passenger
should (at least) be secret. The one’s complement operation
inverts the bits. The zeros in the seed turn to ones in the
vulnerable for exploitation, e.g., a brute force attack. cases by sending or replaying CAN messages on the target’s
To verify the exploitation of this vulnerability we used bus. CAN can be seen as a bus system not designed
the test bench for the selected target and executed all the with security issues in mind. Techniques like threat mo-
necessary steps according to the EOL standard. To ensure deling [13][19] could provide valuable information before
the functionality of the target, the pyrotechnic charges were the actual implementation and address these issues in ad-
th
simulated by attaching resistors to the corresponding pins on vance. Moreover, choosing the optional deployment methods
the connector of the target. To enable the EOL functionality using an ACL during the design phase could have prevented
CAN traffic was replayed, consisting of only one message the exploitation via CAN only.
including a vehicle speed of zero. This CAN bus only 2) Hard-wired Plausibility Checks: The connector of
included the target and a CAN transceiver sending the our target was designed to accommodate over 100 pins.
Au
diagnostic messages and playing back the afore mentioned Although the presence of pins depends on the exact model
vehicle speed message. The exploitation was successfully the target is installed in, this high number is explained by
validated by the oscilloscope displaying the firing impulse. the presence of physical signals being directly connected
The related Common Vulnerabilities and Exposures (CVE)- to the target. Besides the obvious need for hard-wired
ID is listed under [17]. connections, e.g., the connection to the pyrotechnic charges,
2) Lack of an ACL: While the EOL standard allows the it was discovered that signals such as seatbelt status and
detonation using CAN communication only, it gives four seat occupancy were connected as well. While this does not
more implementation possibilities, making use of ACLs. An exclude the possibility of information being transmitted over
ACL provides a direct hardware connection to a PCU and a bus system as well, this would allow for plausibility checks
is exposed on the OBD-II connector of the vehicle. From based on sources that are not easy to be tampered with [20].
the information gathered about the deployment process, 3) Usage of Cryptography: When developers want to
when used with an ACL, it seems like the exploitation of design a new security feature, they should follow the Kerck-
hoff’s Principle. The principle explicitly describes that cryp- checks to ensure that the vehicle state can be determined
tosystems shouldn’t be secure due to hidden details about correctly.
how the algorithm works (security by obscurity). Moreover,
V. C ONCLUSION AND F UTURE W ORK
the whole secret should be based on the confidentiality of the
utilized key. The past has already shown multiple times that In this paper we introduced an approach of reusing
mechanisms which disregard the mentioned principle are threat modeling artefacts for penetration testing during the
very often broken as soon as the algorithm has been reverse- automotive development process. We presented a method
engineered [21]. Our investigation has also confirmed the to derive test-cases based on attack-trees which are created
violation of the mentioned principle by analyzing the sent during the threat analysis phase. Our experimental evalu-
on
seed. A first approach to fix the existing issue could be an ation has shown that misuse cases should be identified as
additional security layer which ensures a correct authentica- early as possible, e.g., after the requirements phase, so
tion. that security engineers and penetration testers are able to
4) Hardening Against Brute-Force Attacks: The use of prioritize and focus on critical scenarios. We deem that with
Negative Response Codes (NRCs), such as exceeded number penetration testing based on our threat modeling approach,
of attempts and required time delay not expired, as described some vulnerabilities in our target could have been discovered
in the UDS standard could be used to slow down brute- and prematurely fixed. Generally, on starting a new system
iti
force attempts. While the former would require a power development, the focus should be on a careful reflection
cycle of the target to continue the brute-force attempt, the which technologies are suitable to prevent known security
latter would slow the approach down. To counter the use of weaknesses.
scripted power cycling of the target in rapid succession, the As next steps, we want to continue by analyzing further
ACKNOWLEDGEMENTS
as the same seeds occur more frequently, making it faster to
iterate the possible keys. All authors contributed in the same way. This work has
5) Stateful Firewalls: The execution of diagnostic functi- been developed in the project SAFE ME ASAP (reference
onality in general should only be allowed in their correspon- number: 03FH011IX5) that is partly funded by the German
ding vehicle states. However, our evaluation has shown, that Federal Ministry of Education and Research (BMBF) within
th
this simple guideline is not always implemented correctly. the research programme ICT 2020.
For instance, in one of our tested vehicles, the gateway did R EFERENCES
not check for the appropriate conditions, before performing
certain diagnostic functions. As a consequence of this, di- [1] R. N. Charette, “This Car Runs on Code,” 2009, accessed
12.02.2016. [Online]. Available: http://spectrum.ieee.org/
agnostic functions are available during vehicle conditions in transportation/systems/this-car-runs-on-code
Au
on
[8] OISSG, “Information systems security asses-
sment framework,” Internet. [Online]. Avai-
lable: http://cuchillac.net/archivos/pre seguridad pymes/2
hakeo etico/lects/metodologia oissg.pdf
[9] OWASP, “Open web application security project - testing
guide v4,” Internet. [Online]. Available: https://www.owasp.
org/images/1/19/OTGv4.pdf
iti
[10] PTES, “Penetration testing execution standard,” Internet.
[Online]. Available: http://www.pentest-standard.org
[11] NIST, “Technical guide to information security testing and
Ed
assessment,” Internet. [Online]. Available: http://nvlpubs.nist.
gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf
[12] SAE, “Cybersecurity Guidebook for Cyber-Physical Vehicle
Systems,” 01.2016, accessed 12.04.2016. [Online]. Available:
http://standards.sae.org/wip/j3061/
[13] “The STRIDE Threat Model,” 2005. [Online]. Avai-
lable: https://msdn.microsoft.com/en-us/library/ee823878(v=
cs.20).aspx
[14] D. Firesmith, “Using V Models for Testing: The Latest
s
Research in Software Engineering and Cybersecurity,”
11.11.2013. [Online]. Available: https://insights.sei.cmu.edu/
sei blog/2013/11/using-v-models-for-testing.html
or
mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-14937
[18] C. Miller and C. Valasek, “Remote exploitation of an unalte-
red passenger vehicle,” Black Hat USA, vol. 2015, 2015.
[19] J. Dürrwang, K. Beckers, and R. Kriesten, “A Lightweight
Au