Why embedded device security is essential to ICS systems

Why embedded device security is essential to ICS systems

Why embedded device security is essential to ICS systems

Protections at the device level are not a replacement for security controls in OT systems and networks. They’re a necessary extension of them.

Embedded devices in industrial control systems (ICS) operate within an increasingly complex array of systems, networks and protocols. The complexity is only increasing as end users require more insight into how ICS operate, and push for more connectivity between controls and individual devices that underpin the systems’ performances. This has had the effect of complicating ICS’ hierarchical communication structure and introducing new cyber threats with the potential to target devices that operate below the control level.

 

As with IT systems that increased connectivity and exposed endpoints (such as PC’s and network infrastructure), ICS have expanded into multiple layers, each of which has security controls designed to protect against cyberattacks that are increasingly common and capable of targeting devices on the operational technology (OT) side.

 

We first witnessed the security evolution in informational technology (IT) systems on the enterprise level. Over the course of two decades, IT systems expanded and incorporated a vast number of new endpoints, and a correspondingly complex new system of connection points, networks, and communication protocols. The proliferation of endpoints and connections led to increased cyber threats and successful intrusions, which ultimately provided incentive to harden and expand security controls throughout the IT environment.  

 

A similar evolution occurred in control rooms in the last 10 years. Devices became more common, more connected, and subsequently more vulnerable. Each year there are more documented attacks targeting endpoints such as SCADA servers and historians, engineering work stations, HMIs, and communications infrastructure. By now, it is accepted that security controls at this ICS level should be as robust as those applied to IT systems.

 

We are now at the point where the next expansion of security must cover the endpoints closest to the ICS physical processes. This includes devices such as actuators, sensors, valves, robotics, and safety equipment, as well as human machine interfaces (HMI), programmable logic controllers (PLCs), fieldbus I/O, and other controllers that operate outside the control room. Here too, some assumptions about the insularity of these endpoints have persisted: They are too hard to reach and exploit; they are not sufficiently valuable as attack targets; controls at higher levels of the ICS technology attack offer sufficient protection.

 

To explain why these assumptions are no longer valid, it is helpful to view the modern ICS system in the context of current ICS cyber threats. It’s also imperative to recognize that cyberattacks can work through even the most robust sequence of security layers or bypass them through exploitation of permissions. Equally important is the recognition that embedded devices at the lowest ICS levels can be accessed via the control room, and that persistent compromise of such devices is not only possible, but not particularly difficult to execute.

Perdue_Model_ICS@2x

Attacks that reach the bottom of the ICS stack are rare, but typically much more damaging that attacks at the higher levels.

Security at all ICS layers will remain essential, and controls at the IT, DMZ and control room levels will continue to deflect the large majority of ICS attack attempts. But the next essential step — which remains far from complete — is to create an truly in-depth ICS security structure by pushing controls onto devices themselves. To understand why, it can be helpful to review of what security at other layers accomplishes — and what it can miss.

ICS security is an evolutionary process

It’s hardly surprising that the concept of on-device security is still gaining purchase, since only a few years ago most of the ICS layers below the enterprise level were thought to be air-gapped, or so hard to access as to endpoint security controls unnecessary, or not worth the disruption they would cause once they were implemented.  

 

The air-gap premise was plausible only as long as the control-room level generally was not connected to the enterprise network, public Internet and wireless networks. However, attacks such as Stuxnet demonstrated that air-gapping was not an impenetrable defense, and the lack of connectivity became an impediment once control room-level endpoints and their collected data became valuable to working groups on the enterprise level.

 

At this point, there were strong arguments for creating or strengthening firewalls, the DMZ, and robust communications protocols. But there was still resistance to putting security directly onto endpoints at the control room level.

 

Five years ago, control room endpoints typically did not run with antivirus, whitelisting or other digital security tools, and they could not receive patches or updates, for fear this would disrupt the plant or system operations. These concerns were not entirely unfounded, as it did requite engineering adjustments to make controls such as antivirus work effectively within SCADA systems and other control assets.

 

But as systems improved, software update signing became the standard, and security controls were calibrated to function without drawing down too much processing power. There also has been a corresponding rise in the number of cyberattacks on the control room level to solidify the case for endpoint protection. Today, in most ICS deployments, control room devices are outfitted with a level of security that equals that of endpoints on the enterprise level.

ICS security, layer on layer

ICS deployments are often proprietary and their security controls may be distinctive. The Perdue Model, first created in the 1990s, is an imperfect representation of the most ICS technology-security stacks; it does not reflect technologies and trends that have made ICS more complex over time, such as increasing use of remote connectivity, more channels between the enterprise and control room layers, and expanding access to external parties and vendors. Despite this, our discussion can benefit from an adapted and simplified version of this model, as seen below:

ICS_Web_New@2x

Endpoints need to be secured at every layer, but the quality of security controls is not consistent layer on layer.[1]

As with the enterprise level in most organizations, endpoints at the ICS control level should be (and typically are) protected by and host a wide range of safeguards around the perimeter, within the network and on the endpoints themselves. The importance of physical, digital, cloud and cybersecurity controls at this level has been a given since criminals began to target this level.

Security controls for endpoints at the enterprise level and the control room level have achieved rough equivalency.

Existing ICS protections are not enough to isolate the embedded device layer

The higher up the stack a security control sits, the more attacks it’s likely to deflect. Each of these security controls takes pressure off lower levels and the monitoring and investigation capabilities of the ICS, leaving resources available for detection of threats directed lower in the technology stack.

 

Attacks that reach embedded devices at the lowest ICS level — the controllers, sensors, safety equipment, manufacturing machinery are just a few examples — are low-frequency events. But given these devices’ mission-critical purpose, or their position in hazardous deployments, such attacks carry the highest potential for catastrophic outcomes.

 

Means by which the end devices can be exposed include:

  • Attacks that exploit permissions. These may originate with a person working below the majority of upper-level security controls, who has access to a network that runs next to or very close to mission-critical devices. This may be an operator or an engineer with malicious intent, or one whose credentials or access has been compromised and unwittingly used to transport an attack via accepted communications protocols.

 

  • Supply chain compromises, in which a device sent out for repair or being maintained on site, or one receiving updates, is corrupted at the firmware level.

 

  • Intrusions such as successful phishing compromises that jump the IT/OT divide and access the control room, and from there access the devices using approved communication channels.

 

  • Intrusions perpetrated by operators who deliberately bypass approved communication paths to access devices directly (either for convenience or malicious purposes).
Potential_ICS_Attack_Vectors@2x

Attacks on embedded devices can have multiple points of entry.

Vulnerabilities devices represent a different category of threat, since they originate in device engineering. Like many other vulnerabilities, they sometimes are used to provide engineers with access to device controls. But as with any other convenience, a vulnerability or access path can be exploited by a bad actor who wants to evade security controls and authorization requirements to upload malicious firmware or send commands.

 

Additionally, some communications must pass through all ICS levels, which can be used for malicious purposes as well as normal ones. Communications and operations that can expose the lowest level of devices include:

  • Updates to antivirus, software and firmware need to travel down the technology stack (starting at the DMZ).

 

  • Operators or vendors who may use remote connections to the ICS. Ideally, these remote comms will terminate in a jump server in the DMZ, but often they connect to PCs in the control network or — even worse — directly to the devices. In a worst-case scenario, the remote access server could be located in the control network or device network, allowing direct access to these levels and devices, which will bypass the security controls of upper levels.

 

  • Operators who may deploy unsigned software due to convenience or time constraints, this software is downloaded from external sources or manually carried into the system including down to the device network. Unsigned or maliciously signed software allows malware to be introduced into the system.

 

  • Operators who may deploy software with signatures that are faked, or stolen.

As with other legitimate operations or engineering features, attackers can exploit approved communications protocols (Modbus, Profitnet, etc.) to travel between the control network to the device network.

The current insecure state of devices on the lowest levels

ICS the security provided by controls around or on the shell of the devices is typically not as robust as that provided by controls higher in the system. When attackers are able to circumvent or defeat higher controls, there typically is little meaningful defense at the lowest critical layers:

This effectively leaves embedded device security at the same level of protection we saw in control room endpoints five to 10 years ago. As it was then, conventional thinking presumes a few conditions:

  • Embedded devices are unreachable. This is refuted by supply chain risks, insider errors or threats, carefully orchestrated attacks that succeed in worming through some or all of the other layers of ICS security.

 

  • Embedded devices are not desirable targets. Since these devices are essential to the operation of electrical grids, manufacturing plants, chemical processing, and other critical and/or hazardous ICS, they should be considered valid targets for cyberattacks.

 

  • Embedded device security is too complicated, or will interfere with the primary function of the device. Red Balloon Security’s work, and that of other leading cybersecurity companies, has demonstrated that these devices, like any other endpoint, can operate with robust security hardening their shells and running concurrently with their essential functionality.
 

Recent examples of ICS attacks that impacted the control network layer

While the layers of security provide necessary and, in most cases, effective defense that justifies investment, there are relevant examples of attacks that simply bypassed entire layers or defeated them through compromise of legitimate communications protocols. Notable examples include:

  • The 2015 Ukraine grid attack, in which attackers first spear-phished and then compromised log-in credentials of workers accessing the SCADA system remotely. They then erased firmware in devices, providing a gateway for communications to the substation, and sent commands to end devices at the control network level or above (UPSs).

 

  • The 2016 CRASHOVERRIDE attack, which mostly likely began with credential capture on the IT system and the re-use of those credentials to log into machines at the ICS level, and ultimately led to access of SCADA-connected protection relays, which were then put into diagnostic mode that disabled their protection algorithms.[2]

 

  • The 2017 Triton attack successfully manipulated the firmware of safety instrumentation systems by targeting the engineering work station, rather than HMI or SCADA.

In each of these attacks, normal communications protocols were exploited, which in turn allowed the attackers to access devices at the lowest levels of the ICS.

 

While in some cases the attackers may have exploited flawed access controls or protocols, each of these events demonstrates the feasibility of a dedicated, deliberate campaign to learn how a system operates and use established communications channels to deliver malicious payloads.

 

As such, they are irrefutable proof that devices on the lower levels of ICS can — and will — be subject to cyberattacks. This in turn demonstrates the need for monitoring and protection at the device level.

The case for on-device security

Loading security controls directly onto embedded devices is not the first step in ICS protection. The protections built into other layers are necessary to deflect the large majority of malicious intrusions and mitigate most human errors.

 

But the subset of legitimate threats that can directly access control networks and communications protocols is large enough to warrant an investment in on-device security, particularly given the increasing connectivity of ICS, the prevalence of remote access, and the potential damages that would result from device-level compromise.

 

Security on or next to these devices can also help prevent lower-impact events, such as an incorrect configuration download, unauthorized device tampering or a well-intentioned but faulty device maintenance. Basic security features around the devices also can mitigate the effects of attacks higher in the technology stack that increase network traffic to the point of an overload, which can lead to device failure.

On-device defense, such as Red Balloon’s Symbiote, is designed to fill the embedded endpoint security gap by protecting the application level, OS system, firmware and secure boot process.

Host-based defenses can include firmware hardening through binary reduction and randomization, and runtime protection that can close the ICS security circuit by blocking on-device attacks related to memory corruption, process spawning or forking, malicious code execution, and firmware corruption or erasure. It brings protection down to the hardware layer and a means to proactively alert operators to any anomalous behavior in real time.

 

On-device security is not meant to eliminate patching; rather, it provides a robust defense during the critical time period between when a vulnerability is discovered (whether by an engineering team, researcher, or attacker) and when the patch can be created and applied. At the firmware level, this is especially critical, since patching typically takes longer than software patching.

 

Adoption of these solutions can greatly reduce the vulnerability of essential devices — and provide device manufacturers with a cutting-edge solution designed for the modern deployments and their evolving risks.

 

The deployment of such protections is not a simple process. It requires careful engineering and an iterative process to properly calibrate the security controls so that there is no interference with the devices’ primary functionality. Security, safety and network engineers will have to work collaboratively to achieve a full integration of this technology into existing embedded systems.

 

The good news is that we have been at this juncture before. The challenge is to provide incentive without experiencing the full effects of attacks at the device level, which could easily lead to destruction of equipment, loss of necessary services or serious harm to operators.

 

Click below to learn more about how RBS’s solutions can work with your ICS system.

Protections at the device level are not a replacement for security controls in OT systems and networks. They’re a necessary extension of them.

Embedded devices in industrial control systems (ICS) operate within an increasingly complex array of systems, networks and protocols. The complexity is only increasing as end users require more insight into how ICS operate, and push for more connectivity between controls and individual devices that underpin the systems’ performances. This has had the effect of complicating ICS’ hierarchical communication structure and introducing new cyber threats with the potential to target devices that operate below the control level.

 

As with IT systems that increased connectivity and exposed endpoints (such as PC’s and network infrastructure), ICS have expanded into multiple layers, each of which has security controls designed to protect against cyberattacks that are increasingly common and capable of targeting devices on the operational technology (OT) side.

 

We first witnessed the security evolution in informational technology (IT) systems on the enterprise level. Over the course of two decades, IT systems expanded and incorporated a vast number of new endpoints, and a correspondingly complex new system of connection points, networks, and communication protocols. The proliferation of endpoints and connections led to increased cyber threats and successful intrusions, which ultimately provided incentive to harden and expand security controls throughout the IT environment.  

 

A similar evolution occurred in control rooms in the last 10 years. Devices became more common, more connected, and subsequently more vulnerable. Each year there are more documented attacks targeting endpoints such as SCADA servers and historians, engineering work stations, HMIs, and communications infrastructure. By now, it is accepted that security controls at this ICS level should be as robust as those applied to IT systems.

 

We are now at the point where the next expansion of security must cover the endpoints closest to the ICS physical processes. This includes devices such as actuators, sensors, valves, robotics, and safety equipment, as well as human machine interfaces (HMI), programmable logic controllers (PLCs), fieldbus I/O, and other controllers that operate outside the control room. Here too, some assumptions about the insularity of these endpoints have persisted: They are too hard to reach and exploit; they are not sufficiently valuable as attack targets; controls at higher levels of the ICS technology attack offer sufficient protection.

 

To explain why these assumptions are no longer valid, it is helpful to view the modern ICS system in the context of current ICS cyber threats. It’s also imperative to recognize that cyberattacks can work through even the most robust sequence of security layers or bypass them through exploitation of permissions. Equally important is the recognition that embedded devices at the lowest ICS levels can be accessed via the control room, and that persistent compromise of such devices is not only possible, but not particularly difficult to execute.

Perdue_Model_ICS@2x

Attacks that reach the bottom of the ICS stack are rare, but typically much more damaging that attacks at the higher levels.

Security at all ICS layers will remain essential, and controls at the IT, DMZ and control room levels will continue to deflect the large majority of ICS attack attempts. But the next essential step — which remains far from complete — is to create an truly in-depth ICS security structure by pushing controls onto devices themselves. To understand why, it can be helpful to review of what security at other layers accomplishes — and what it can miss.

ICS security is an evolutionary process

It’s hardly surprising that the concept of on-device security is still gaining purchase, since only a few years ago most of the ICS layers below the enterprise level were thought to be air-gapped, or so hard to access as to endpoint security controls unnecessary, or not worth the disruption they would cause once they were implemented.  

 

The air-gap premise was plausible only as long as the control-room level generally was not connected to the enterprise network, public Internet and wireless networks. However, attacks such as Stuxnet demonstrated that air-gapping was not an impenetrable defense, and the lack of connectivity became an impediment once control room-level endpoints and their collected data became valuable to working groups on the enterprise level.

 

At this point, there were strong arguments for creating or strengthening firewalls, the DMZ, and robust communications protocols. But there was still resistance to putting security directly onto endpoints at the control room level.

 

Five years ago, control room endpoints typically did not run with antivirus, whitelisting or other digital security tools, and they could not receive patches or updates, for fear this would disrupt the plant or system operations. These concerns were not entirely unfounded, as it did requite engineering adjustments to make controls such as antivirus work effectively within SCADA systems and other control assets.

 

But as systems improved, software update signing became the standard, and security controls were calibrated to function without drawing down too much processing power. There also has been a corresponding rise in the number of cyberattacks on the control room level to solidify the case for endpoint protection. Today, in most ICS deployments, control room devices are outfitted with a level of security that equals that of endpoints on the enterprise level.

ICS security, layer on layer

ICS deployments are often proprietary and their security controls may be distinctive. The Perdue Model, first created in the 1990s, is an imperfect representation of the most ICS technology-security stacks; it does not reflect technologies and trends that have made ICS more complex over time, such as increasing use of remote connectivity, more channels between the enterprise and control room layers, and expanding access to external parties and vendors. Despite this, our discussion can benefit from an adapted and simplified version of this model, as seen below:

ICS_Web_New@2x

Endpoints need to be secured at every layer, but the quality of security controls is not consistent layer on layer.[1]

As with the enterprise level in most organizations, endpoints at the ICS control level should be (and typically are) protected by and host a wide range of safeguards around the perimeter, within the network and on the endpoints themselves. The importance of physical, digital, cloud and cybersecurity controls at this level has been a given since criminals began to target this level.

Security controls for endpoints at the enterprise level and the control room level have achieved rough equivalency.

Existing ICS protections are not enough to isolate the embedded device layer

The higher up the stack a security control sits, the more attacks it’s likely to deflect. Each of these security controls takes pressure off lower levels and the monitoring and investigation capabilities of the ICS, leaving resources available for detection of threats directed lower in the technology stack.

 

Attacks that reach embedded devices at the lowest ICS level — the controllers, sensors, safety equipment, manufacturing machinery are just a few examples — are low-frequency events. But given these devices’ mission-critical purpose, or their position in hazardous deployments, such attacks carry the highest potential for catastrophic outcomes.

 

Means by which the end devices can be exposed include:

  • Attacks that exploit permissions. These may originate with a person working below the majority of upper-level security controls, who has access to a network that runs next to or very close to mission-critical devices. This may be an operator or an engineer with malicious intent, or one whose credentials or access has been compromised and unwittingly used to transport an attack via accepted communications protocols.

 

  • Supply chain compromises, in which a device sent out for repair or being maintained on site, or one receiving updates, is corrupted at the firmware level.

 

  • Intrusions such as successful phishing compromises that jump the IT/OT divide and access the control room, and from there access the devices using approved communication channels.

 

  • Intrusions perpetrated by operators who deliberately bypass approved communication paths to access devices directly (either for convenience or malicious purposes).
Potential_ICS_Attack_Vectors@2x

Attacks on embedded devices can have multiple points of entry.

Vulnerabilities or undocumented access on devices represent a different category of threat, since they originate in device engineering. Like many other vulnerabilities, they sometimes are used to provide engineers with access to device controls. But as with any other convenience, a vulnerability or access path can be exploited by a bad actor who wants to evade security controls and authorization requirements to upload malicious firmware or send commands.

 

Additionally, some communications must pass through all ICS levels, which can be used for malicious purposes as well as normal ones. Communications and operations that can expose the lowest level of devices include:

  • Updates to antivirus, software and firmware need to travel down the technology stack (starting at the DMZ).

 

  • Operators or vendors who may use remote connections to the ICS. Ideally, these remote comms will terminate in a jump server in the DMZ, but often they connect to PCs in the control network or — even worse — directly to the devices. In a worst-case scenario, the remote access server could be located in the control network or device network, allowing direct access to these levels and devices, which will bypass the security controls of upper levels.

 

  • Operators who may deploy unsigned software due to convenience or time constraints, this software is downloaded from external sources or manually carried into the system including down to the device network. Unsigned or maliciously signed software allows malware to be introduced into the system.

 

  • Operators who may deploy software with signatures that are faked, or stolen.

As with other legitimate operations or engineering features, attackers can exploit approved communications protocols (Modbus, Profitnet, etc.) to travel between the control network to the device network.

The current insecure state of devices on the lowest levels

ICS the security provided by controls around or on the shell of the devices is typically not as robust as that provided by controls higher in the system. When attackers are able to circumvent or defeat higher controls, there typically is little meaningful defense at the lowest critical layers:

This effectively leaves embedded device security at the same level of protection we saw in control room endpoints five to 10 years ago. As it was then, conventional thinking presumes a few conditions:

  • Embedded devices are unreachable. This is refuted by supply chain risks, insider errors or threats, carefully orchestrated attacks that succeed in worming through some or all of the other layers of ICS security.

 

  • Embedded devices are not desirable targets. Since these devices are essential to the operation of electrical grids, manufacturing plants, chemical processing, and other critical and/or hazardous ICS, they should be considered valid targets for cyberattacks.

 

  • Embedded device security is too complicated, or will interfere with the primary function of the device. Red Balloon Security’s work, and that of other leading cybersecurity companies, has demonstrated that these devices, like any other endpoint, can operate with robust security hardening their shells and running concurrently with their essential functionality.

Recent examples of ICS attacks that impacted the control network layer

While the layers of security provide necessary and, in most cases, effective defense that justifies investment, there are relevant examples of attacks that simply bypassed entire layers or defeated them through compromise of legitimate communications protocols. Notable examples include:

  • The 2015 Ukraine grid attack, in which attackers first spear-phished and then compromised log-in credentials of workers accessing the SCADA system remotely. They then erased firmware in devices, providing a gateway for communications to the substation, and sent commands to end devices at the control network level or above (UPSs).

 

  • The 2016 CRASHOVERRIDE attack, which mostly likely began with credential capture on the IT system and the re-use of those credentials to log into machines at the ICS level, and ultimately led to access of SCADA-connected protection relays, which were then put into diagnostic mode that disabled their protection algorithms.[2]

 

  • The 2017 Triton attack successfully manipulated the firmware of safety instrumentation systems by targeting the engineering work station, rather than HMI or SCADA.

In each of these attacks, normal communications protocols were exploited, which in turn allowed the attackers to access devices at the lowest levels of the ICS.

 

While in some cases the attackers may have exploited flawed access controls or protocols, each of these events demonstrates the feasibility of a dedicated, deliberate campaign to learn how a system operates and use established communications channels to deliver malicious payloads.

 

As such, they are irrefutable proof that devices on the lower levels of ICS can — and will — be subject to cyberattacks. This in turn demonstrates the need for monitoring and protection at the device level.

The case for on-device security

Loading security controls directly onto embedded devices is not the first step in ICS protection. The protections built into other layers are necessary to deflect the large majority of malicious intrusions and mitigate most human errors.

 

But the subset of legitimate threats that can directly access control networks and communications protocols is large enough to warrant an investment in on-device security, particularly given the increasing connectivity of ICS, the prevalence of remote access, and the potential damages that would result from device-level compromise.

 

Security on or next to these devices can also help prevent lower-impact events, such as an incorrect configuration download, unauthorized device tampering or a well-intentioned but faulty device maintenance. Basic security features around the devices also can mitigate the effects of attacks higher in the technology stack that increase network traffic to the point of an overload, which can lead to device failure.

On-device defense, such as Red Balloon’s Symbiote, is designed to fill the embedded endpoint security gap by protecting the application level, OS system, firmware and secure boot process.

Host-based defenses can include firmware hardening through binary reduction and randomization, and runtime protection that can close the ICS security circuit by blocking on-device attacks related to memory corruption, process spawning or forking, malicious code execution, and firmware corruption or erasure. It brings protection down to the hardware layer and a means to proactively alert operators to any anomalous behavior in real time.

 

On-device security is not meant to eliminate patching; rather, it provides a robust defense during the critical time period between when a vulnerability is discovered (whether by an engineering team, researcher, or attacker) and when the patch can be created and applied. At the firmware level, this is especially critical, since patching typically takes longer than software patching.

 

Adoption of these solutions can greatly reduce the vulnerability of essential devices — and provide device manufacturers with a cutting-edge solution designed for the modern deployments and their evolving risks.

 

The deployment of such protections is not a simple process. It requires careful engineering and an iterative process to properly calibrate the security controls so that there is no interference with the devices’ primary functionality. Security, safety and network engineers will have to work collaboratively to achieve a full integration of this technology into existing embedded systems.

 

The good news is that we have been at this juncture before. The challenge is to provide incentive without experiencing the full effects of attacks at the device level, which could easily lead to destruction of equipment, loss of necessary services or serious harm to operators.

 

Click below to learn more about how RBS’s solutions can work with your ICS system.

Protections at the device level are not a replacement for security controls in OT systems and networks. They’re a necessary extension of them.

Embedded devices in industrial control systems (ICS) operate within an increasingly complex array of systems, networks and protocols. The complexity is only increasing as end users require more insight into how ICS operate, and push for more connectivity between controls and individual devices that underpin the systems’ performances. This has had the effect of complicating ICS’ hierarchical communication structure and introducing new cyber threats with the potential to target devices that operate below the control level.

 

As with IT systems that increased connectivity and exposed endpoints (such as PC’s and network infrastructure), ICS have expanded into multiple layers, each of which has security controls designed to protect against cyberattacks that are increasingly common and capable of targeting devices on the operational technology (OT) side.

 

We first witnessed the security evolution in informational technology (IT) systems on the enterprise level. Over the course of two decades, IT systems expanded and incorporated a vast number of new endpoints, and a correspondingly complex new system of connection points, networks, and communication protocols. The proliferation of endpoints and connections led to increased cyber threats and successful intrusions, which ultimately provided incentive to harden and expand security controls throughout the IT environment.  

 

A similar evolution occurred in control rooms in the last 10 years. Devices became more common, more connected, and subsequently more vulnerable. Each year there are more documented attacks targeting endpoints such as SCADA servers and historians, engineering work stations, HMIs, and communications infrastructure. By now, it is accepted that security controls at this ICS level should be as robust as those applied to IT systems.

 

We are now at the point where the next expansion of security must cover the endpoints closest to the ICS physical processes. This includes devices such as actuators, sensors, valves, robotics, and safety equipment, as well as human machine interfaces (HMI), programmable logic controllers (PLCs), fieldbus I/O, and other controllers that operate outside the control room. Here too, some assumptions about the insularity of these endpoints have persisted: They are too hard to reach and exploit; they are not sufficiently valuable as attack targets; controls at higher levels of the ICS technology attack offer sufficient protection.

 

To explain why these assumptions are no longer valid, it is helpful to view the modern ICS system in the context of current ICS cyber threats. It’s also imperative to recognize that cyberattacks can work through even the most robust sequence of security layers or bypass them through exploitation of permissions. Equally important is the recognition that embedded devices at the lowest ICS levels can be accessed via the control room, and that persistent compromise of such devices is not only possible, but not particularly difficult to execute.

Perdue_Model_ICS@2x

Attacks that reach the bottom of the ICS stack are rare, but typically much more damaging that attacks at the higher levels.

Security at all ICS layers will remain essential, and controls at the IT, DMZ and control room levels will continue to deflect the large majority of ICS attack attempts. But the next essential step — which remains far from complete — is to create an truly in-depth ICS security structure by pushing controls onto devices themselves. To understand why, it can be helpful to review of what security at other layers accomplishes — and what it can miss.

ICS security is an evolutionary process

It’s hardly surprising that the concept of on-device security is still gaining purchase, since only a few years ago most of the ICS layers below the enterprise level were thought to be air-gapped, or so hard to access as to endpoint security controls unnecessary, or not worth the disruption they would cause once they were implemented.  

 

The air-gap premise was plausible only as long as the control-room level generally was not connected to the enterprise network, public Internet and wireless networks. However, attacks such as Stuxnet demonstrated that air-gapping was not an impenetrable defense, and the lack of connectivity became an impediment once control room-level endpoints and their collected data became valuable to working groups on the enterprise level.

 

At this point, there were strong arguments for creating or strengthening firewalls, the DMZ, and robust communications protocols. But there was still resistance to putting security directly onto endpoints at the control room level.

 

Five years ago, control room endpoints typically did not run with antivirus, whitelisting or other digital security tools, and they could not receive patches or updates, for fear this would disrupt the plant or system operations. These concerns were not entirely unfounded, as it did requite engineering adjustments to make controls such as antivirus work effectively within SCADA systems and other control assets.

 

But as systems improved, software update signing became the standard, and security controls were calibrated to function without drawing down too much processing power. There also has been a corresponding rise in the number of cyberattacks on the control room level to solidify the case for endpoint protection. Today, in most ICS deployments, control room devices are outfitted with a level of security that equals that of endpoints on the enterprise level.

ICS security, layer on layer

ICS deployments are often proprietary and their security controls may be distinctive. The Perdue Model, first created in the 1990s, is an imperfect representation of the most ICS technology-security stacks; it does not reflect technologies and trends that have made ICS more complex over time, such as increasing use of remote connectivity, more channels between the enterprise and control room layers, and expanding access to external parties and vendors. Despite this, our discussion can benefit from an adapted and simplified version of this model, as seen below:

ICS_Web_New@2x

Endpoints need to be secured at every layer, but the quality of security controls is not consistent layer on layer.[1]

As with the enterprise level in most organizations, endpoints at the ICS control level should be (and typically are) protected by and host a wide range of safeguards around the perimeter, within the network and on the endpoints themselves. The importance of physical, digital, cloud and cybersecurity controls at this level has been a given since criminals began to target this level.

Security controls for endpoints at the enterprise level and the control room level have achieved rough equivalency.

Existing ICS protections are not enough to isolate the embedded device layer

The higher up the stack a security control sits, the more attacks it’s likely to deflect. Each of these security controls takes pressure off lower levels and the monitoring and investigation capabilities of the ICS, leaving resources available for detection of threats directed lower in the technology stack.

 

Attacks that reach embedded devices at the lowest ICS level — the controllers, sensors, safety equipment, manufacturing machinery are just a few examples — are low-frequency events. But given these devices’ mission-critical purpose, or their position in hazardous deployments, such attacks carry the highest potential for catastrophic outcomes.

 

Means by which the end devices can be exposed include:

  • Attacks that exploit permissions. These may originate with a person working below the majority of upper-level security controls, who has access to a network that runs next to or very close to mission-critical devices. This may be an operator or an engineer with malicious intent, or one whose credentials or access has been compromised and unwittingly used to transport an attack via accepted communications protocols.

 

  • Supply chain compromises, in which a device sent out for repair or being maintained on site, or one receiving updates, is corrupted at the firmware level.

 

  • Intrusions such as successful phishing compromises that jump the IT/OT divide and access the control room, and from there access the devices using approved communication channels.

 

  • Intrusions perpetrated by operators who deliberately bypass approved communication paths to access devices directly (either for convenience or malicious purposes).
Potential_ICS_Attack_Vectors@2x

Attacks on embedded devices can have multiple points of entry.

Vulnerabilities or undocumented access on devices represent a different category of threat, since they originate in device engineering. Like many other vulnerabilities, they sometimes are used to provide engineers with access to device controls. But as with any other convenience, a vulnerability or access path can be exploited by a bad actor who wants to evade security controls and authorization requirements to upload malicious firmware or send commands.

Additionally, some communications must pass through all ICS levels, which can be used for malicious purposes as well as normal ones. Communications and operations that can expose the lowest level of devices include:

  • Updates to antivirus, software and firmware need to travel down the technology stack (starting at the DMZ).

 

  • Operators or vendors who may use remote connections to the ICS. Ideally, these remote comms will terminate in a jump server in the DMZ, but often they connect to PCs in the control network or — even worse — directly to the devices. In a worst-case scenario, the remote access server could be located in the control network or device network, allowing direct access to these levels and devices, which will bypass the security controls of upper levels.

 

  • Operators who may deploy unsigned software due to convenience or time constraints, this software is downloaded from external sources or manually carried into the system including down to the device network. Unsigned or maliciously signed software allows malware to be introduced into the system.

 

  • Operators who may deploy software with signatures that are faked, or stolen.

As with other legitimate operations or engineering features, attackers can exploit approved communications protocols (Modbus, Profitnet, etc.) to travel between the control network to the device network.

The current insecure state of devices on the lowest levels

ICS the security provided by controls around or on the shell of the devices is typically not as robust as that provided by controls higher in the system. When attackers are able to circumvent or defeat higher controls, there typically is little meaningful defense at the lowest critical layers:

This effectively leaves embedded device security at the same level of protection we saw in control room endpoints five to 10 years ago. As it was then, conventional thinking presumes a few conditions:

  • Embedded devices are unreachable. This is refuted by supply chain risks, insider errors or threats, carefully orchestrated attacks that succeed in worming through some or all of the other layers of ICS security.

 

  • Embedded devices are not desirable targets. Since these devices are essential to the operation of electrical grids, manufacturing plants, chemical processing, and other critical and/or hazardous ICS, they should be considered valid targets for cyberattacks.

 

  • Embedded device security is too complicated, or will interfere with the primary function of the device. Red Balloon Security’s work, and that of other leading cybersecurity companies, has demonstrated that these devices, like any other endpoint, can operate with robust security hardening their shells and running concurrently with their essential functionality.

Recent examples of ICS attacks that impacted the control network layer

While the layers of security provide necessary and, in most cases, effective defense that justifies investment, there are relevant examples of attacks that simply bypassed entire layers or defeated them through compromise of legitimate communications protocols. Notable examples include:

  • The 2015 Ukraine grid attack, in which attackers first spear-phished and then compromised log-in credentials of workers accessing the SCADA system remotely. They then erased firmware in devices, providing a gateway for communications to the substation, and sent commands to end devices at the control network level or above (UPSs).

 

  • The 2016 CRASHOVERRIDE attack, which mostly likely began with credential capture on the IT system and the re-use of those credentials to log into machines at the ICS level, and ultimately led to access of SCADA-connected protection relays, which were then put into diagnostic mode that disabled their protection algorithms.[2]

 

  • The 2017 Triton attack successfully manipulated the firmware of safety instrumentation systems by targeting the engineering work station, rather than HMI or SCADA.

In each of these attacks, normal communications protocols were exploited, which in turn allowed the attackers to access devices at the lowest levels of the ICS.

 

While in some cases the attackers may have exploited flawed access controls or protocols, each of these events demonstrates the feasibility of a dedicated, deliberate campaign to learn how a system operates and use established communications channels to deliver malicious payloads.

 

As such, they are irrefutable proof that devices on the lower levels of ICS can — and will — be subject to cyberattacks. This in turn demonstrates the need for monitoring and protection at the device level.

The case for on-device security

Loading security controls directly onto embedded devices is not the first step in ICS protection. The protections built into other layers are necessary to deflect the large majority of malicious intrusions and mitigate most human errors.

 

But the subset of legitimate threats that can directly access control networks and communications protocols is large enough to warrant an investment in on-device security, particularly given the increasing connectivity of ICS, the prevalence of remote access, and the potential damages that would result from device-level compromise.

 

Security on or next to these devices can also help prevent lower-impact events, such as an incorrect configuration download, unauthorized device tampering or a well-intentioned but faulty device maintenance. Basic security features around the devices also can mitigate the effects of attacks higher in the technology stack that increase network traffic to the point of an overload, which can lead to device failure.

On-device defense, such as Red Balloon’s Symbiote, is designed to fill the embedded endpoint security gap by protecting the application level, OS system, firmware and secure boot process.

Host-based defenses can include firmware hardening through binary reduction and randomization, and runtime protection that can close the ICS security circuit by blocking on-device attacks related to memory corruption, process spawning or forking, malicious code execution, and firmware corruption or erasure. It brings protection down to the hardware layer and a means to proactively alert operators to any anomalous behavior in real time.

 

On-device security is not meant to eliminate patching; rather, it provides a robust defense during the critical time period between when a vulnerability is discovered (whether by an engineering team, researcher, or attacker) and when the patch can be created and applied. At the firmware level, this is especially critical, since patching typically takes longer than software patching.

 

Adoption of these solutions can greatly reduce the vulnerability of essential devices — and provide device manufacturers with a cutting-edge solution designed for the modern deployments and their evolving risks.

 

The deployment of such protections is not a simple process. It requires careful engineering and an iterative process to properly calibrate the security controls so that there is no interference with the devices’ primary functionality. Security, safety and network engineers will have to work collaboratively to achieve a full integration of this technology into existing embedded systems.

 

The good news is that we have been at this juncture before. The challenge is to provide incentive without experiencing the full effects of attacks at the device level, which could easily lead to destruction of equipment, loss of necessary services or serious harm to operators.

 

Click below to learn more about how RBS’s solutions can work with your ICS system.

[1] Adapted from Cisco CPwE Architecture schematic.

[2] Dragos, Joe Slowik, “Reassessing the 2016 Ukraine Electric Power Event as a Protection-Focused Attack,” 2019

[1] Adapted from Cisco CPwE Architecture schematic.

[2] Dragos, Joe Slowik, “Reassessing the 2016 Ukraine Electric Power Event as a Protection-Focused Attack,” 2019

[1] Adapted from Cisco CPwE Architecture schematic.

[2] Dragos, Joe Slowik, “Reassessing the 2016 Ukraine Electric Power Event as a Protection-Focused Attack,” 2019

LEVERAGE OUR EXPERTISE FOR YOUR SECURITY NEEDS

Reach out to learn more about our embedded security offering and to schedule a demo.

LEVERAGE OUR EXPERTISE FOR YOUR SECURITY NEEDS

Reach out to learn more about our embedded security offering and to schedule a demo.

LEVERAGE OUR EXPERTISE FOR YOUR SECURITY NEEDS

Reach out to learn more about our embedded security offering and to schedule a demo.

LEVERAGE OUR EXPERTISE FOR YOUR SECURITY NEEDS

Reach out to learn more about our embedded security offering and to schedule a demo.