As we rang in the new year just a few weeks ago and watched the ball drop into a nearly empty Times Square from the relative protection of our family bubbles, I can’t remember a time when we were more ready to ring out the old and ring in the new year, looking optimistically to return to “normal” life in 2021.

So, as I write this, the good news is that I am not writing another article about the pandemic as vaccines are being distributed at an accelerated pace and there is glint of hope we may be finally turning the corner as case rates slowly recede. 2020 was truly a remarkable year in in terms of the huge economic, emotional, and mortal toll the pandemic has taken, and power of the human spirit to adapt, improvise and overcome adversity.  Certainly, that spirit was on full display in manufacturing, where innovations not only helped pharmaceutical companies develop life-saving vaccines in record time, but also enabled virtually all industrial operations to fast-track digital transformation programs that helped keep plants running and their employees safe.

Warning Signs

But in the world of industrial automation and controls, there are also warning signs that 2021 may harbor another viral threat from increased cyber-security attacks.

Just over a month into the new year, we have had two significant cyber-security attacks in the national headlines.  In early January, the U.S Cybersecurity and Infrastructure Security Agency provided details on a Russian sponsored attack that posed a “grave risk” to federal, state, and local and private networks. The widescale attack, attributed to the SVR, a Russian foreign intelligence service, was disguised as a software update on the widely adopted SolarWinds network, used by thousands of IT departments to monitor computer networks. FireEye, a leading cybersecurity solutions company said that that the hackers found a way to exploit the distribution of updates for SolarWinds by adding malware to the download. Once installed the malware provided a “backdoor” for the hackers to gain entry into the unsuspecting users computer networks.

And then last week, the City of Oldsmar, Fl, a small town of 15,000 on the northern rim of Tampa Bay, announced that a hacker had infiltrated the control network in their Reverse Osmosis Water Treatment plant and changed setpoint parameters that could have released high quantities of sodium hydroxide, commonly used to adjust the pH of the treated water.  Luckily, a vigilant operator, spotted the change and prevented it from being executed. In a news conference, Pinellas County, Florida state and Federal officials stated the attack was leveraged through TeamViewer, a 3rd party software platform that enables remote access to control, monitor or repair workstations.

The two incidents along with an increasing wave of ransomware attacks that have idled manufacturing plants, like recent WestRock ransomware attack incident, are a red flag of more cybersecurity threats  to come and an urgent need for companies to assess and address potential risks.  It is also possible that our heroic efforts to keep industrial operations running by rapidly deploying new technology to share control room data may have left more of these systems vulnerable.

Bolster Your Defenses

In the municipal sector, for instance, the threat is compounded by belt-tightening enacted due to revenue shortfalls and expensive COVID-19 response plans. In an interview with the Washington Post, Lesley Carhart of Dragos Security, which specializes in industrial controls, said,  “We have known for a long time that municipal water utilities are extremely underfunded and under-resourced, and that makes them a soft target for cyberattacks.”

While there is no single formula for ramping up your cyber security defenses, there are certain steps you can and should take immediately to lower your risk.  These proven recommendations include:

  • Remote HMI viewing can be done safely, but you will need a plan. Create a plan to restrict remote connections to SCADA systems, specifically those that allow physical control and manipulation of devices within the SCADA network. Last year we saw an explosion of demand for web (HTML5) and mobile clients for SCADA networks. While this technology can provide benefit for remote workers, teams still need to centrally manage the total number and security policy for each. In many hub and spoke SCADA architectures, a well -intended “back-door” client for a supporting remote engineer or system integrator, can be an direct access point to the central SCADA Server. Also, when possible, consider, implementing (read-only) unidirectional monitoring devices to prevent a remote client from making any changes to the SCADA system. If full control is necessary, ideally, VPN access with 2FA firewall Access Control Lists (ACLs) to get to manufacturing VLANs is a preferred method.
  • Assess your IT infrastructure and seek to isolate the control network from the internet and use secure networks. At a basic level, when connection is required, install a firewall software/hardware appliance with logging and ensure it is turned on. The firewall should be secluded and not permitted to communicate with unauthorized sources. Evaluate the use of Virtual LANs (VLANs) to separate network traffic and virtual private network (VPN) to encrypt remote connections. There are a host of ICS Network Threat Monitoring tools available as well. These monitor OT traffic for anomalies and threats by detecting manipulated network packets or unauthorized parameters changes.
  • When sharing data from the plant floor, consider strategies for replicating process data like historians to secure, secondary locations or the cloud to allow users access to the data but the not the control network. Sending data directly to the cloud for analysis (Our product is called Insight) can also be a very secure best practice for democratized access to data without having to give access to the Historians on the controls network. In addition, local historian replication to Tier II Historians and the cloud, can also securely send data through DMZs.
  • Evaluate your current Server and workstation Operating Systems. Make sure that obsolete, unsupported Microsoft Server and workstation Operations systems, like Windows 7 and earlier, are upgraded.
  • Implement an update and patch management cycle. Patch all systems for critical vulnerabilities, prioritizing timely patching of Internet-connected systems for known vulnerabilities and software processing Internet data, such as Web browsers, browser plugins, and document readers.
  • Keep SCADA/industrial control systems (ICS) software, patched and up to date. Municipalities often have an annual service contract with a systems integrator firm for ongoing support of their SCADA sand PLC applications. In addition to this, a software maintenance agreement with the SCADA vendor ensures access to all version upgrades, patches and security patches. This also gives vendors direct contact information if they need to notify the owner of a severe or urgent update.  Clients should understand how the vendor publishes security vulnerabilities and the vendor service level commitment to testing and patching threat vectors.   A good example is the AVEVA Cyber-Security Update
  • Application Security. Owners should ask if best practices are being implemented in their SCADA applications. Strong passwords should be used, along with two-factor authentication and there should be no “common password” used across operators or roles.  Application Security built around standards like Microsoft Active Directory can help enforce roles and policies as part of a unified strategy.
  • Educate your teams on risk from social engineering attacks. In almost all cyber-security attacks, the technology breach was enabled by information from the people that worked around it. Discourage the use of sticky notes at workstation or control room that have a username and password that could be captured by a mobile phone.  Don’t post network architectures in public meeting rooms or offices. Train control teams and operators on common phishing, email and text attacks.   And consider the amount of transparency in publishing budgetary information. For instance, while well-meaning municipalities are looking to reduce spending, details on IT sensitive projects might need to be scrubbed.  IT may be perfectly acceptable to explain six-figure expenditures for replacement of high service pumps, but a line item highlighting the urgent need for the upgrade of a SCADA or server platform, can unintentionally suggest an opportunity for attack.
  • Consider your approach to application development and ongoing maintenance. It has been common practice to allow multiple System Integrator (SI) access to the plant control network remotely for projects.  Newer, cost-effective approaches like AVEVA Integration Studio make it very easy for companies to spin up a development sandbox in the cloud. From there, companies can enforce SCADA standards, govern SI access and perform acceptance testing more securely at a fraction of the cost of maintaining this capability on their own servers.

Resources

In addition to the above, we offer these additional resources for review:

And, as always, if you would like to discuss your situation, please contact us at InSource and we would be happy to schedule a free consultation to help develop a plan.