The other basecamp exploit modules released yesterday are:

1. Modiconstop – Stops a Schneider Modicon Quantum PLC from operating. It is another command that lacks authentication or other security, and its only one packet to send to stop the CPU.

2. Ged20tftpbo – A buffer overflow of the tftp service on the GE D20 PLC. Note that other GE D20 Metasploit modules had been released earlier in Project Basecamp including modules that allow remote control and recover all user credentials.

More information can be found at the Dark Reading site.

While it is nice to have the ability to demonstrate how easy it is to subvert PLCs, RTUs, and embedded control system devices, I struggle with how asset owners will be able to react to this to quickly close the risk associated with releasing this capability into the public domain. It is one thing to show this in a training class or security conference for effect or to raise awareness, but it is another thing to release the capability to anyone with an Internet connection and latest BackTrack release.

One thing that we all agree with is that keeping one's head in the sand is no longer an option for those tasked with securing SCADA and Industrial Control Systems. The spotlight has shifted from IT Enterprise systems to SCADA systems, and new SCADA vulnerabilities are cropping up like weeds in a summer garden. It is going to get worse before it gets better. 

If you cannot answer this question, now is the time to implement the next wave of security solutions that can provide you with the second half of a sound cyber security program. We recently provided a 4-hour workshop on Incident Management for SCADA and Industrial Control Systems for UTC and NERC, and we have uploaded the PDF for this workshop to our website under the "Presentations" link. Once you register with our site, you can access it here. Don't worry, we never send out any emails to our registered users, and we protect the privacy of our users.  We hope this is a helpful briefing, and contact us if you would like more information about assessing the security of your SCADA and Industrial Control Systems.

Thanks,
Jonathan

Now that you have watched the video, we can speak on the same page. The next knee-jerk reaction that some asset owners have is to look at the list of devices that were involved in the project, then compare those devices to their inventory, and then say... "Man... I'm glad we are not running any of those PLCs". Avoid that reaction. We have worked with a large number of embedded automation controllers and RTUs, and we can safely say that all have major cyber security issues.

The security vulnerabilities that are explained in this disclosure are systemic across almost all embedded devices used in SCADA and Industrial Control Systems. The next negative reaction that most have is to blame Digital Bond or blame the control system vendors. Neither of these two groups can really do anything today or tomorrow that can protect your system. Even if you nag your control system vendor for them to patch the problems discovered in the basecamp project, and they listen to you and repair these issues, more vulnerabilities will just crop up later. Yes, the long term strategy is to get the control system vendors to create hardened and secure devices from the factory, but that is a very long term strategy that will not address all of the millions of vulnerable devices already installed today or being installed tomorrow.

If your field controller is connected to an ethernet network using TCP/IP based protocols, just go ahead and assume that any of the vulnerabilities mentioned in the S4 video are potential security issues with your devices as well. We have seen PLC devices from major control system vendors that freeze or reboot just by sending them a flood of ping requests or by simply scanning them with NMAP or NESSUS. There are several documented examples of PLCs failing because a local printer ran out of ink and flooded the local control network with SNMP packets. So if these devices fail under simple network broadcast storms, it is easy to see how much mischief you can cause to these devices if you actually spend some time to send specifc attacks to these systems.

The point of this discussion is that with or without the recent Basecamp project disclosures, if your field devices are connected over an ethernet network, you are and have already been exposed to cyber risks to your automation or control system performance. Attackers do not need to be physically present on your control network for these vulnerabilities to work, since they can be routed into the control system from other networks, or become activated from a script or batch file on a USB stick. Air-gapped networks are not safe either.

So what do you do about this? Knowing is half the battle, and some will say that getting funding to fix the problems once you know about them is the other half. That is an entirely different topic, but assuming that you have the backing and support to actually make changes to your system and secure it, here are some steps to think about.

6 Steps to Securing Access to Ethernet-Connected PLCs and Field Devices

1. Flush out the logical data flow and architecture - Dive into the weeds and understand your control system in-depth. Build an accurate depiction of how the your PLC and field devices are connected back through the telemetry, LAN/WAN infrastructure, and back up through to your control room HMI and I/O servers. Ensure that everything is documented well, including the IP addresses of all major system components. If you need to, setup a laptop, run Wireshark or TCPDUMP, and capture actual SCADA protocol commands on the network to understand the full route from the control room out to the field devices. Try running tracroute to document all of the hops along the way. Knowing the logical data flows from the plant floor or field systems back to the control room is a great first step.

2. Determine what TCP or UDP ports are required to be open on your PLCs and field devices - This is not only a requirement for utilities that must map out their ports and services for NERC CIP compliance, but is also equally important for all critical infrastructure sectors. You can not adequately secure something when you do not completely understand its attack surface. For instance, if you are running a Rockwell Automation controller, and it uses ports 23, 80, 443, and 2222, you probably do not need the lower ports to be open to the controller, since the protocol operates over 2222. You can block all of the other non-essential ports. If you do not have this information, take a spare PLC off the shelf and scan all TCP and UDP ports to determine what ones answer back. You can also sniff the network while it is in use. Knowing how the devices are connected together (obtained from step 1 above), and then armed with what ports need to be open are the first two steps.

3. Next determine what source and destination IP addresses are used for PLC communications - This step may be a bit easier, since there should be only a finite number of computers that are required to poll the PLC network. In most SCADA systems, this number is less than 10. Do your homework to determine what computers really need to communicate with the controller network, and then conversely, what field devices are required to receive and answer back to these commands. Build a model in Excel using one column to show the list of computers that are communicating, then the next column to show the TCP or UDP ports that are used, then lastly a third column showing the devices that are the destination for the SCADA commands. This is not only an excellent troubleshooting tool for how the ethernet-based SCADA system functions at the PLC level, but can also be used to help secure or lock down the network.

4. Enable Access Control Lists (ACLs) between the computers in the SCADA system and the embedded devices in the SCADA system.  The computers in the SCADA system, such as operator consoles, engineering workstations, data historians, and special-use application computers all typically run on modern fat operating systems like windows or *nix systems. These need a large number of TCP and UDP ports to be open for them to function on the network. The embedded devices do not require the same ports to be open, so you can innoculate your field devices from being vulnerable to viruses, worms, and malware that affect your computers by simply blocking all non-essential ports. This way a virus would have to be written specifically for your field device in order for it to affect your control system. Create additional access controls that limit what source addresses can send packets into the control network. This can also help greatly reduce the attack surface by reducing the potential computers that have digital access to the ethernet-connected field devices. After this step, there should only be a limited number of computers that can access your field devices. If you can not introduce ACLs between your SCADA operations computers and the field devices, then at a minimum, back up to the firewall or DMZ that sits between the SCADA environment and the corporate IT or other networks. At a minimum, access controls should be placed at this level.

5. Enable Application Control, HIPS, or Application Whitelisting in those computers that can communicate with the field devices. If you remember, the Stuxnet attack did not directly infect the Siemens PLCs, but rather first infected a computer workstation that already had ethernet access to the PLC network. The engineering workstations were used as a weapon to deliver the Stuxnet malicious PLC logic to the PLCs. Now, because of the above 4 steps, you are aware of the finite number of computers that have the ability to communicate with the field devices using TCP/IP SCADA protocols. These computers should have end-point security on them, and we STRONGLY suggest application whitelisting. Application whitelisting can essentially freeze the computer systems that have access to the PLCs so that only those approved applications can run and execute on those computers. Any new code that is introduced to the system that is not already whitelisted will be blocked and will not execute.

6. Detection compliments active defense techniques - Now if you have gone through all of the above 5 steps, you should feel fairly good about how the system is locked down. The work is not done just yet. You have to assume that active defenses will eventually fail, and there should be a way to detect when you may be under an attack. Implementing both network and host-based IDS products can help alert you when packets are dropped at the perimeter that was intended for your field devices. Now that you know the TCP or UDP ports that your controllers are using, and their destination IP addresses, you can pull in the switch or firewall logs into a SEM product, and have it alert you when any network traffic was intended for any of your field devices or attempted to utilize their protocol ports, especially if it came from a non-approved source address. Parsing logs from the network devices is a great way to obtain situational awareness. To compliment that further, you can also add an IDS solution to the infrastructure and either write or obtain IDS siguatures for known control system vulnerabilities.

There are additional measures for protecting access to ethernet-connected PLCs and field devices, so this is in no way a complete list. It does provide some helpful tips and actionable things that most control system admins can do with their system to prevent accidental or intentional abuse of ethernet-connected embedded devices.

The main things that I took away from the basecamp PLC disclosures are:

- Almost all ethernet-connected field devices are vulnerable
- You can not secure your environment unless you understand how it works down to the device level
- Place as many access controls in front of the PLC devices as possible to reduce the attack surface
- Block all unnecessary devices and TCP/UDP ports so that only essential traffic is allowed
- Implement strong end-point security on the remaining computers that are allowed to communicate with the PLC devices
- Ensure to extract logs and meaningful security context off of network and host devices to detect abnormal or malicious behavior on the PLC network

Hope my comments in this briefing help illuminate the real issues surrounding the Basecamp PLC disclosures, and provide practical tips for PLC administrators to consider when securing or locking down the PLC network.

Thanks,
Jonathan

The presentation and demonstration was a moment of awakening for many in the audiance that were not aware of how prevalent embedded devices are in our society, or how they can be utilized in ways the manufacturer did not intend. 

• There has only been a few infections to-date
• It does not self-propigate or spread like Stuxnet did, so the victums have to be selectively targeted
• Unlike the initial perception, it does not seem to target control system equipment or vendors
• The attackers are using Duku to look for information, such as blueprints, design documents, and any information that can potentially be used to create a future attack on an industrial facility
• Some of the executables used within the Duku framework shares code with Stuxnet and appears to have been compiled after Stuxnet was discovered
• The malware is designed to self-desctruct after 36 days
• Duku may have other variants that are not currently detectable by Antivirus
• Seems to leverage C&C servers based in India
  
  

Steps asset owners should consider to avoid infection:

• update all AV signatures
• tighten up host prevention strategies and consider application whitelisting technology
• review all ports open on perimeter devices such as firewalls and reduce the attack surface by closing off unnecessary ports on network devices upstream of critical SCADA or DCS systems
• isolate control system elements from the corporate IT networks as much as possible
• operate at a higher treat level and review logs on an increased periodic basis
• monitor sensitive SCADA and DCS hosts for new and unusual services that are not traditionally running on those devices
• enable verbose logging on all host systems, and forward the logs to a centralized SEM or SIEM
• monitor hosts for new files added to system directories such as system32, and system32\drivers
• monitor outgoing traffic leaving the network to unknown destination IP address
• place a sniffer on the external side of the firewall and record traffic for 24 hours, then parse the data and cross check the IP addresses in the PCAP files to see if the traffic contains any IP addresses for known C&C servernet and botnets
• also monitor the dirty side of the firewall for spikes in traffic

Hope this briefing provides helpful insight as to how Duku works, and the migitation efforts that can reduce the chance of getting infected.

thanks,
Jonathan

Older legacy SCADA and Process Control Systems used thick client applications installed on servers and workstations that were purpose-built and engineered for the SCADA application suite. As these systems evolve they become more like traditional IT systems. The IT community has long realized the benefits of moving from thick client to thin client architectures, and now most Enterprise IT applciations are browser based. As with most technology evolution in the SCADA market, the adoption of browser-based applications started to become mainstream several years ago.

About ten years ago, many of the SCADA and Control System vendors began to offer browser-based SCADA HMI, alarm, data historization, and data trending applications. They were marketing this capability as a way for Operations to easily access the data and information that they need to make decisions about the operation of their plant, all from the convienience of a simple web browser. The vendors started by implementing web servers in the SCADA servers, database servers, and view nodes. Eventually this adoption of web technology made its way down into the embedded devices, and even as early as 2001 several PLC and RTU vendors were shipping with embedded web servers running in their hardware. They also began using the term "web" in the name of their products, with examples like the "PlantWeb" system from Emerson - just to name one.

Although browser-based computing provide a greater ease of use for operations, this proliferation of web technologies deep in the SCADA environments creates a challenge for security. Many forms of malware and attack exploitation techniques leverage brower weaknesses, so while web-based SCADA systems are attractive to operations, it opens up the system to a much larger range of cyber attacks than when these systems were based on thick client applications.

We often find SCADA view nodes, HMI consoles, and data trending portals running with Internet Explorer 6, and in some cases we have found they have updated to IE 7, but it is extrememly rare to find a fully patched environment with up-to-date browsers. When we bring this issue up in our briefing sessions with our client's technical teams, the engineering and operations team often downplay the risk and say that they are not using the web browsers to go onto the Internet, so the risk is low. However, they miss the point that even if these older browsers are only used in a closed SCADA-Intranet purpose, just having the older browsers installed on the client and server systems leaves them open to malware and other attacks that can leverage the older browser framework to exploit the host.  

I find this a timely issue, and in fact just today Robert Lemos, Contributing Editor from Dark Reading posted an article about how outdated web browsers are still leaving many organizations vulnerable to attack. His article builds on this issue, and you can find it at the following link:

http://www.darkreading.com/vulnerability-management/167901026/security/attacks-breaches/231602264/outdated-browsers-leave-many-enterprises-vulnerable-to-attack.html

I suggest giving his article a quick read, and while you are reading the article, think about how this relates to SCADA and ICS systems. If you are a systems administrator for a SCADA environment that utilizes browser-based applications, you may want to take a quick inventory of your systems. It would be a great time to clean house, update all of the OS and security patches, and update those old vulnerable browsers.

Cheers!

Jonathan

Since we run into this situation in almost every field assessment, I thought it was a good time for a quick primer on private IP address ranges and why SCADA and Industrial Control Systems should never be configured with public IP addresses. For many of you, this briefing will be a review of some basics that you already know... for others, this may be helpfiul, so let's begin...

For internal systems that should never be accessable directly from the Internet, there are only three IP address ranges that are reserved by the RFC 1918 and 4193 as private address spaces. They are classified as private because they are not allocated to any specific organiztion, and IP traffic addressed by these IP address ranges can not be transmitted over the public Internet.

The network ranges shown in the above table are reserved for use for internal private networks, and most of us are familiar with the 192.168 range from configuring our home and small business routers. When designing and implementing private networks, these are the only ranges of IP addresses that should be used. Unfortunately, there are several major SCADA and Industrial Control System vendors that do not ship their systems configured to operate in these IP address ranges, and we find SCADA systems that are publically routable over the Internet in almost every one of our field assessments. When we bring this point up, some SCADA and Control Engineers simply reply that it is how their system came from the vendor, or they will justify it and say that the address range does not matter because they are behind a firewall.

Using public routable IP addresses on the inside of sensitive mission-critical SCADA systems is not a good practice, since the firewall(s) protecting these systems are the only line of defense from malicious packets and payloads being routed from anywhere on the Internet into these environments. Configuring firewalls to protect public routable addresses on the inside is also much more complicated because you can not take advantage of built in features for routing classless routes to the outside interface for Internet-bound traffic. Also, if any component of the system is inadvertantly exposed to the Internet, then the system is exposed to attacks that can be routed into the system from anywhere in the world.

Hopefully SCADA and Industrial Control System vendors can start shipping their systems with private IP addresses as the default, and system integrators and asset owners can start implementing these systems with private IP addreses from the start. If the system is up and running in a live state actively controlling production systems, converting from public to private IP addresses is a challenge that may not be possible unless the system is down for maintenance.

More food to chew on while you enjoy the weekend...

Jonathan

However, if you are using RDP (Remote Desktop Protocol) to jump or hop across network segments, make sure that you have changed the default account names and passwords associated with the RDP logon process because a new worm attack is live right now attempting to crawl through networks around the world, and it is taking a gamble that some have left default accounts and weak passwords in place.

Researchers working on the Morto worm say that it infects Windows workstations and Windows servers, and spreads by uploading a Windows DLL file to a targeted machine. The worm looks for weak administrator passwords in Remote Desktop on an organization's network, attempting everything from "12345" to "admin" and "password."

Like most C&C malware, once Morto compromises one system, it connects to a command and control server to download information and to update its payload. It also terminates processes for locally running security applications in order to ensure its activity continues uninterrupted. Even if it can not load onto the local machine or pull down its C&C malware, Morto still tries to connect to targeted host systems. It tries common default account names like "admin" and "guest." Morto is also known as Trojan horse Generic24.OJQ (AVG); Trojan.DownLoader4.48720 (Dr.Web) ; Win-Trojan/Helpagent.7184 (AhnLab); Troj/Agent-TEE (Sophos); and Backdoor:Win32/Morto.A.

During our assessment process, we always find plant control systems that are still using default Windows accounts, so I can bet that there will be control systems impacted by this worm. We always recommend changing the default account names, and modifying the key service account passwords on a routine basis, but this is not always feasible in operational systems.

The best recommendation is to integrate 2-factor authentication into management and administrative services like RDP. Instead of only needing the username and password to log into remote hosts, the administrator would need a key fob with a OTP (One Time Password) AND the correct username and password to log into the remote host. This extra step for authentication not only prevents automated attacks from leveraging harvested passwords or bruteforce attempts, but also helps prevent the insider threat, since the logon process requires something you know PLUS something that you have in your possession.  Implementing 2-factor authentication is the best practice for authentication to SCADA and Control System resources from the Corporate IT and other external networks.

We hope this advice is helpful for our industrial control systems clients, and please pass this link and message onto others in your workplace and at home.

Cheers,
Jonathan

We were aware of the potential security vulnerabilities of using a public cellular infrastructures several years ago when several of our clients asked us about a class of PLC and RTU COMM cards that plugged directly into the controller backplane or snap into the controller. These communications cards allow the PLC to utilize cellular networks as either their primary or secondary communications channel.  We raised this security concern with AT&T because they assign sudo public IP addresses to these communication modules. The reason I say sudo is that AT&T claims that the IP addresses they assign to the modules are "private" and in a block of IPs assigned to the client. They also claim that their MPLS network has various security features that prevents one client on their cellular network from accessing another client's cellular-connected devices.

When NERC approached us early about this vulnerability, we were able to quickly turn around very specific content, language, and Visio graphics to enhance the messaging to the electric sector community. We created the first diagram below that visually shows how an attacker with an entry point into the carrier networks could route cellular commands to the COMM modules directly connected to the PLC or RTU backplane.

 
The advice that we gave to our clients about a year ago was not to trust the cellular carriers, but treat them like the Internet. Assume that over time, they will become compromised. If they wanted to continue to utilize them as a communications medium, we advised them to not use the cellular communication cards that racked directly into the PLC backplane. That is extremely dangerous because they are placing the core of their PLC or RTU system directly on a foreign network. Instead, we advised them to purchase a separate hardware device that accepts a GSM SIM card. We advised them to place a small firewall between their PLCs and the embedded cellular device, and tunnel over the cellular network as if it was the Internet. We created the next diagram below as a way to visualize using a small inexpensive field firewall to create an encrypted tunnel through the carrier network.

 
We were glad to be able to contribute to the NERC advisory, and continue to support not only our clients, but the larger SCADA Security community at large. I hope briefings like this one can continue to help shine a light in areas of our infrastrucutre that need additional attention. Our goal is not so much to expose areas of weakness, but to offer clear and precise guidance that can help mitigate these security findings.

More to come on this subject...

Jonathan