Lessons for All of Us From the SolarWinds Orion Compromise

/by

All cybersecurity news events, like the recent disclosure of compromise involving SolarWinds Orion by APT 29, aka “Cozy Bear,” cause CISOs to ask the same initial questions:

  • Do I have this problem?
  • Where?
  • What are the consequences?

In this instance, the attack is extremely sophisticated, and quite alarming – it’s a supply chain attack, involving compromise of a widely used and trusted monitoring product.  This adds a lot of pressure to these questions.  As organizations are scrambling to respond, we wanted to publish some suggestions here, as a resource.  In discussions with our customers, many of whom have been impacted by this compromise, we find there is a common playbook, as follows:

  • Step 1: Do I have SolarWinds Orion?
  • Step 2: Where is it, in the context of my network?
  • Step 3: What is it capable of accessing or controlling?
  • Step 4: Fix Orion, or take it offline (if subject to the CISA Emergency Directive)
  • Step 5: Block unwanted access to or from SolarWinds Orion, to the extent possible
  • Step 6: For all assets SolarWinds could reach, reset them to known good state

This is an arduous journey. RedSeal can be one of your supporting resources. It is especially helpful in the middle stages – steps 2, 3, and 5 in the above playbook.

Specifically, for Step 2, a RedSeal network map can help you locate the hostnames or addresses of your SolarWinds Orion software.  One large customer of ours had well over 100 distinct addresses with this software installed. Your total is likely to be lower, but still may be more than just a single location.  Mapping out where they all are is a starting point, before heading in to the deeper stages.

Note also, in Step 2, that RedSeal’s L2 mapping capability may be helpful, since you can locate the nearest switch port to any given endpoint.  This may be helpful if you need to abruptly terminate network access, or decide to monitor span traffic closely.  (If you have not previously set up L2 mapping, we would not recommend this as a tactical step in your response, because the data gathering setup would take some time, but if you already have the data in place, this is a good time to use it, as an aid to shutting down any inappropriate activity.)

In Step 3, it’s important to know what a compromised instance of the monitoring product could reach.  Sadly, because this is a widely trusted product, whose whole purpose is to give you wide visibility, in most networks this turns out to be a large space.  We have had customer reports of a “blast radius” of endpoints well into 6 figures.  Figuring this out by hand is absurdly difficult – far better to automate the search.  In RedSeal, this involves an Access Query, from your SolarWinds Orion instances, out to the wider network.  Just be prepared – the query may be so large that RedSeal will prompt you to make sure you want that much data in one go.  If it’s not manageable, you may prefer to break the query into regions – “What can Orion reach in New York?”, or “in my Amazon fabric”, and so on.

For step 5, blocking unwanted access from SolarWinds Orion to the Internet, RedSeal’s capability to define Zones and Policies may be helpful.  As a first step, a Zone containing your SolarWinds Orion endpoints, and another Zone of Internet, can be used to investigate what access is already possible.  Unfortunately, this may be quite wide, since you may actively be using Orion to monitor cloud fabric and you may want to permit access for software updates (even though, ironically, this was the method originally used in the compromise – but subsequently addressed).  Still, before you can lock this down just to the access you feel is necessary, it can help to review what the current state is, and see what blanket restrictions might be possible, without removing any access pathways you need to keep open.

Hopefully this overview is of use, as a playbook of the common steps we are seeing.  If we can be of any assistance as you work through the cleanup of this incident, please don’t hesitate to get in touch.

Download: A step-by-step guide for using RedSeal to respond

RedSeal customers: Take advantage of our complimentary Sunburst Exposure Assessment.

Not a customer yet? Contact us at info@redseal.net to explore how we can help.