Tag Archive for: network segmentation

Tales from the Trenches: Vol 3 — Security Operations and Network Operations are always at odds. Or are they?

/by

Since 2004, RedSeal has helped our customers See and Secure their entire complex network. And while those customers may have understood the value of understanding their environment, how it was connected and see what’s at risk, there is often an “Aha” moment when the true significance is clear. The stories of these moments are lore within the walls of RedSeal. But these tales so clearly illustrate the value of RedSeal beyond just theory that we think they’re worth sharing. In the words of our team in the field, the ones working directly with our customers, this blog series will share the moments where it all gets real.

In this edition of the series Brad Schwab, Senior Security Solutions Consultant tackles the potential friction between two departments with RedSeal.

Security Operations and Network Operations are always at odds. Or are they?

Empirically, using the greatest technical brevity, you could explain the two areas as:

  • Security Operations (SecOps) is about limiting where network traffic goes. They are also usually responsible for Vulnerability Scanners
  • Network Operations (NetOps) is about the uninterrupted, fast, network traffic flow

As you can see, these departments could easily be at odds – one is the brakes, the other the throttle. So, yes, they usually are at odds. Everything one wants can easily create work for the other, resulting in a back-and-forth pendulum of requests. SecOps to NetOps, “I need these ports shut down, they are creating security exposure…” NetOps to SecOps, “sure, you deal with the backlash…” Outcome, Finance to NetOps, “I can’t print paychecks…” NetOps emails SecOps the Finance department email and goes dark…. Net affect, neither department likes the other…

How does this involve RedSeal you may ask? RedSeal is in the unique position to work with both SecOps and NetOps and help both realize their Operational Goals and allow visibility into outcomes beforehand so that situations like the above don’t happen. This creates a positive working relationship between the teams.

Working with a large Health Organization, we were at the end of a Proof of Concept, and were having a meeting with the CISO, and the heads of SecOps, “Wendy”, and NetOps, “Bill”. We had been having problems with the NetOps people not providing the access required to gather device configuration files across the entire network fabric. NetOps was claiming they didn’t have the time. On the sly, we also heard that they thought providing us with the ability to gather the configs would only make work for them.

During this meeting Wendy was talking about the mountain of scan data she had and that prioritization was key to her work. I demonstrated how RedSeal could prioritized her patching routine(s) based on Network Access. Which, wait for it, requires the network device configuration files. Knowing that SecOps and NetOps were not friends, I decided to see if I could get a dialogue going, and at the same time incent NetOps to get us the access we required to gather the config files. So, I posed the question to Wendy of SecOps, “Wendy, are you scanning the entire network?” She said “yes.” I asked how she knew she could reach every host on the entire network? She said, “Bill told me I could.” I then said to Bill, “Is that what goes on the upcoming Audit, “Bill told me I could scan the entire network…”. Before he could reply I said, wouldn’t you like actual documentation showing that Wendy’s network reach was complete and entire network scans could and were taking place?

That is when the CISO chimed in and said “yes, we need that. How do we get that?” I said that if I had all the config files, I could provide you, Ms. CISO with your required audit documentation, which would eliminate Bills’ manual effort to supply the always asked for “ACL’s of all devices, not just the edge.” And, in addition, I can prioritize Wendy’s mitigation procedures and show the actual trending decrease in your exposure as her team works through the tickets. What followed was over an hour-long discussion of how RedSeal could provide value, focusing of tasks, and reduction of effort to both NetOps and SecOps, plus provide progress reporting to the CISO.

During the rest of the deployment NetOps was always willing to listen and quickly respond quickly to requests. The final outcome was that both teams, SecOps and NetOps, embraced the RedSeal Secure Methodology of Discover, Investigate and Act.

Interested in how RedSeal can help your team? Click here to set up a demo or an introductory call.  

Tales from the Trenches: Vol 2 — They have access to WHAT?!

/by

Since 2004, RedSeal has helped our customers See and Secure their entire complex network. And while those customers may have understood the value of understanding their environment, how it was connected and see what’s at risk, there is often an “Aha” moment when the true significance is clear. The stories of these moments are lore within the walls of RedSeal. But these tales so clearly illustrate the value of RedSeal beyond just theory that we think they’re worth sharing. In the words of our team in the field, the ones working directly with our customers, this blog series will share the moments where it all gets real.

In this edition of the series, Nate Cash, Senior Director, Federal Professional Services/ Director of Information Security at RedSeal looks at a unique application for RedSeal, eliminating potential threats before they could happen.

They have access to WHAT?!

I’m always surprised at the new use-cases we come up with on site with RedSeal. There is a lot of information about a customer’s environment that allows us to answer questions pretty easily, if you know where to look. One Monday morning as I showed up to the office, before I was able to grab coffee, a SOC analyst stopped me at the door to ask me a very simple question, “We have a bunch of site-to-site VPNs with a few business partners, what can they access?”

On the surface this seems like a simple request, “How quickly do you need this information?” I responded. “Last Saturday.” Suddenly the caffeine rush I needed seemed like a long ways off. Turns out, one of the business partners of this customer was breached over the weekend and they notified this customer of the potential. The SOC had been manually mapping out the configs and drawing the paths on the white board, when I came in.

Staring at the board I thought, “There has to be a better way.” When the eureka moment happened. I fired up RedSeal and found the devices which lead to the business partners. In order to map across the VPN tunnels, we needed the config files from the other end. Of course, our business partners were not going to give that up, so I took the configs from our end, and reversed the ACLs changed the IPs based on the tunnel configs and reimported them into RedSeal.

10 min later we were able to answer the question, “what does this business partner have access to?” the answer, 1 server in the DC on 3 ports. But that one server had access to over 20 other servers which increased the downstream risk. This was a fun exercise where we got to see the power of RedSeal and how it can be used to quantify the real risk to the organization and reduce it by putting controls into place.

Once the incident was contained, we decided to go through, and hand jam the rest of the business partners VPN configurations into the model so the SOC would have this information in the event another partner was compromised. After writing up the configs and placing them into the model we found a couple of business partners with configurations that allowed any traffic on any port across.

Firewall engineers know, that sometimes during troubleshooting they’ll configure a device to allow anything across and verify that the firewall is or is not blocking an application. My theory is that these two partners were having issues with traffic going across or getting the VPN tunnel up, and put these rules in as placeholders, probably around two or three A.M. and when things “worked” they were going to come back later to fix them, and never got the time.

But this exercise allowed us to find a major security misconfiguration. If one of those two business partners with the ‘allow any traffic’ were the ones compromised over the weekend this story would have had a much different outcome. In security having a complete picture of your environment is key to being secure. What you don’t know can hurt you.

Interested in how RedSeal can help your team? Click here to set up a demo or an introductory call.  

Tales from the Trenches: Vol 1 — In security, consistency is key.

/by

Since 2004, RedSeal has helped our customers See and Secure their entire complex network. And while those customers may have understood the value of understanding their environment, how it was connected and see what’s at risk, there is often an “Aha” moment when the true significance is clear. The stories of these moments are lore within the walls of RedSeal. But these tales so clearly illustrate the value of RedSeal beyond just theory that we think they’re worth sharing. In the words of our team in the field, the ones working directly with our customers, this blog series will share the moments where it all gets real.

The first in this series is by Nate Cash, Senior Director, Federal Professional Services/ Director of Information Security at RedSeal.

In security, consistency is key.

Once at a customer site while going through the install of RedSeal, we were going over the hardening standards. I clicked on a couple of configurations to start showing how we could go about setting up best practice checks. I had inadvertently pulled up a device which has not been updated in over five years. The customer was shocked, this is one of the many times where I have had to stop mid-sentence while the person I worked with, reached out to someone to “fix the problem.”

The problem is not the fact the device is had not been updated, but somehow their process missed it. This device was just one of many. The first thing we did with RedSeal was develop a set of custom checks to see how many devices passed or failed the latest hardening standard. Once set we started data collections. In 15 min we saw 30% of the devices were not running their latest hardening standards. 

30% of their network devices were not using the latest encryption, had management ACLs set to an old subnet which was now known as their guest subnet, inconsistent SSH versions, telnet still enabled, and some devices pointed to old radius servers, falling back to local accounts. Luckily their firewall blocked the guest subnet from anything internal but, their network management tool still couldn’t access 100% of the devices.

Fixing the underlying problem, not the result of the problem.

With the latest hardening standards in hand, the network engineers got an emergency change request and started logging in to update their configurations. Each device required firmware upgrades, followed by configuration changes. While this was a big undertaking, each day that passed we could grab the numbers and see trending on remediation and report to management.

Once we got the model complete, we found a couple of firewalls that blocked their network management solution from accessing 100% of their network devices. Once the customer opened their firewall, the networking team could start pushing the config to all of the devices. 

As luck would have it, the customer was currently undergoing a review of a new hardening standard. We took the new standard, and I showed them how to create checks for each of the configuration points. At the end of the day, the customer had 75 individual checks for each of the network devices. Upon data collection RedSeal will run those checks against each of the configurations automatically and we could ensure that all network devices passed all of checks required for their baseline configuration. 

This customer had a unique process where devices about to be deployed were plugged into the network and received a static DHCP address. Their network management tool would push the baseline to the config, then the engineers would login the next day to assign the interface Ips per the documentation. The rest of the hardened config was automagically configured by the tool. With this in hand, we were able to add the ‘staging area’ to RedSeal. With every data collection we noticed random inconsistencies where some device would get the whole config and others did not, using those same checks. 

Using RedSeal and custom checks this customer is able to push configs, then double check the config took properly before deploying out into their network. They had better visibility ensuring that all of their devices were hardened, were more confident that automating the work were consistently driving the results they wanted, had a double check with the automation, and essentially reduced their risk significantly, just by checking the configs of their network devices.

Interested in how RedSeal can help your team? Click here to set up a demo or an introductory call.  

IT/OT Convergence

/by

Operational Technology (OT) systems have decades of planning and experience to combat threats like natural disasters – forces of nature that can overwhelm the under-prepared, but which can be countered in advance using well thought out contingency plans. Converging IT with OT brings great efficiencies, but it also sets up a collision between the OT world and the ever-changing threats that are commonplace in the world of Information Technology. 

A Changing Threat Landscape 

The security, reliability, and integrity of the OT systems face a very different kind of threat now – not necessarily more devastating than, say, a flood along the Mississippi, or a hurricane along the coast – but more intelligent and malicious. Bad actors connected over IT infrastructure can start with moves like disabling your backup systems – something a natural disaster wouldn’t set out to do. Bad actors are not more powerful than Mother Nature, but they certainly are more cunning, and constantly create new attack techniques to get around all carefully planned defenses. This is why the traditional strategies have to change; the threat model is different, and the definition of what makes a system “reliable” has changed. 

In the OT world, you used to get the highest reliability using the oldest, most mature equipment that could stay the same, year after year, decade after decade. In the IT world, this is the worst possible situation – out of date electronics are the easiest targets to attack, with the most known vulnerabilities accumulated over time. In the IT world of the device where you are reading this, we have built up an impressive and agile security stack in response to these rapidly evolving threats, but it all depends on being able to install and patch whatever software changes we need as new Tactics, Techniques and Procedures (TTP’s) are invented. That is, in the IT world, rapid change and flexible software is essential to the security paradigm. 

Does this security paradigm translate well to the OT world?

Not really. It creates a perfect storm for those concerned with defending manufacturing, energy, chemical and related OT infrastructure. On the one hand, the OT machinery is built for stability and cannot deliver the “five nines” reliability it was designed for if components are constantly being changed. On the other hand, we have IT threats which can now reach into OT fabric as all the networks blend, but our defense mechanisms against such threats require exactly this rapid pace of updating to block the latest TTP’s! It’s a Catch-22 situation. 

The old answer to this was the air gap – keep OT networks away from IT, and you can evade much of the problem. (Stuxnet showed even this isn’t perfect protection – humans can still propagate a threat across an air gap if you trick them, and it turns out that this isn’t all that hard to do.) Today, the air gap is gone, due to the great economic efficiencies that come from adding modern digital communication pathways to everything we might need to manage remotely – the Internet of Things (IoT).

How do we solve this Catch-22 situation?

So, what can replace the old air gap? In a word, segmentation – it’s possible, even in complex, blended, IT/OT networks to keep data pathways separate, just as it’s essential for the same reason that we keep water pipes and sewer pipes separate when we build houses. The goal is to separate vulnerable and critical OT systems so that they can talk to each other and be managed remotely, but to open only these pathways, and not fall back to “open everything so that we can get the critical traffic through”. Thankfully, this goal is achievable, but the bad news is it’s error prone. Human operators are not good at maintaining complex firewall rules. When mistakes inevitably happen, they fall into two groups:

  1. errors that block something that is needed
  2. errors that leave something open

The first kind of error is immediately noticed, but sadly, the second kind is silent, and, unless you are doing something to automatically detect these errors and gaps, they will accumulate, making your critical OT fabric more and more fragile over time. 

One way to combat this problem is to have a second set of humans – the auditors – review the segmentation regularly. Experience shows, though, that this just propagates the problem – no human beings are good at understanding network interactions and reasoning about complex systems. This is, however, a great job for computers – given stated goals, computers can check all the interactions and complex rules in a converged, multi-vendor, multi-language infrastructure, and make sure only intended communication is allowed, no more and no less.

In summary, IT/OT convergence is inevitable, given the economic benefits, but it creates an ugly Catch-22 scenario for those responsible for security and reliability – it’s not possible to be both super-stable and agile at the same time. The answer is network segmentation, not the old air gapped approach. The trouble with segmentation is it’s hard for humans to manage, maintain and audit without gaps creeping in. Finally, the solution to resolve this Catch-22 is to apply automation – using software such as from RedSeal to automatically verify your segmentation and prevent the inevitable drift, so that OT networks are as prepared for a hacker assault as they are for a natural disaster. 

The Unique Security Solution RedSeal Brings to Multi-Cloud and Hybrid Network Environments

/by

One of the most significant benefits of implementing a multi-cloud strategy is the flexibility to use the right set of services to optimize opportunities and costs.

As public cloud service providers (CSPs) have evolved, they have started to excel in different areas. For example, programmers often prefer to use Azure because of its built-in development tools. However, they often want their apps to run in AWS to leverage the elastic cloud compute capability.

Adopting a multi-cloud strategy enables enterprises to benefit from this differentiation between providers and implement a “best of breed” model for the services that need to consume. They can also realize significant efficiencies, including cost-efficiency, by managing their cloud resources properly.

But multi-cloud solutions also bring their own challenges from administration to security. This can be especially challenging for organizations that don’t have deep experience and knowledge across all platforms and how they interconnect. It can sometimes seem like speaking a different language. For example, AWS has a term called VPC (virtual private cloud). Google Cloud Platform (GCP) uses that term, too but it means something different. In other cases, the reverse is true. The terminology is different but they do the same things.

Cloud provider solutions don’t always address the needs of hybrid multi-cloud deployments. Besides the terminology of AWS, Azure, GCP, Oracle’s OCI, IBM’s cloud, and others have different user interfaces. In a multi-cloud environment or hybrid environment, it can be far more difficult to secure than a single cloud.

Because of these challenges the need for a platform-independent solution that can understand all of the languages of each platform is needed to translate how your multi-cloud solutions are configured, interconnected, and help mitigate the risks.

How RedSeal Manages Multi-Cloud and Hybrid Cloud

At RedSeal, we provide the lingua franca (or bridge) for multi-cloud and on-premise networks. Security operations center (SOC) teams and DevOps get visibility into their entire network across vendors. RedSeal provides the roadmap for how the network looks and interconnects, so they can secure their entire IT infrastructure without having to be experts on every platform.

In most organizations using multi-cloud and hybrid cloud, however, network engineers and SOC teams are being asked to learn every cloud and on-prem resource and make sure they are all configured properly and secured. Many will deploy virtual cloud instances and use virtual firewalls, but as complexity rises, this becomes increasingly difficult to manage.

RedSeal is the only company that can monitor your connectivity across all of your platforms whether they are on-prem or in the cloud. This allows you to see network topology across all of your resources in one centralized platform.

Proactive Security

Proactive security is also complex. Most security offerings monitor in real-time to alert you when there’s an attack underway. That’s an important aspect of your security, but it also has a fundamental flaw. Once you recognize the problem, it’s already underway. It’s like calling 9-1-1 when you discover an emergency. Help is on the way, but the situation has already occurred.

Wouldn’t you like to know your security issues before an incident occurs?

RedSeal helps you identify potential security gaps in your network, so you can address them proactively. And, we can do it across your entire network.

Network Segmentation

Segmenting your network allows you to employ zero trust and application layer identity management to prevent lateral movement within your network. One of the most powerful things about RedSeal is that it provides the visibility you need to manage network segmentation.

It’s a simple concept, but it can also become incredibly complex — especially for larger companies.

If you’re a small business with 100 employees, segmentation may be easy. For example, you segment your CNC machine so employees don’t have admin rights to change configurations. In a mid-size or enterprise-level company, however, you can have an exponential number of connections and end-points. We’ve seen organizations with more than a million endpoints and connections that admins never even knew existed.

It’s only gotten more complex with distributed workforces, remote workers, hybrid work environments, and more third-party providers.

RedSeal can map it all and help you provide micro-segmentation for both east-west and north-south traffic.

Vulnerability Prioritization

Another area where RedSeal excels is by adding context to network vulnerability management. This allows you to perform true risk-based assessments and prioritization from your scanners. RedSeal calculates vulnerability risk scores that account for not only severity and asset value but also downstream risk based on the accessibility of vulnerable downstream assets.

In many cases, RedSeal uncovers downstream assets that organizations didn’t know were connected or vulnerable. These connections provided open threat surfaces, but never showed up in alert logs or only as low-to-medium risks. So, SOC teams already overwhelmed with managing critical and high-risk alerts may never get to these hidden connections. Yet, the potential damage from threat actors exploiting these connections could be even greater than what showed up as high risk.

RedSeal shows you the complete pictures and helps you prioritize vulnerabilities so you can focus on the highest risks in your unique environment.

Play at Your Best

In the late ’90s, world chess champion Garry Kasparov faced off against Deep Blue, an IBM supercomputer, in a six-game exhibition. Kasparov won the first match. Deep Blue won the second and the next three ended in draws. When Deep Blue won the final match and secured the overall victory, Kasparov was asked to concede that the best chess player in the world is now a computer.

Kasparov responded by saying that people were asking the wrong question. The question isn’t about whether the computer is better, but rather how do you play the best game of chess? Kasparov believes he lost not because the computer was better, but because he failed to perform at his best and see all of the gaps in his play.

You can’t afford to make mistakes in your security and beat yourself. By understanding your entire network infrastructure and identifying security gaps, you can take proactive measures to perform at your best.

RedSeal is the best move for a secure environment.

Learn more about how we can help protect your multi-cloud and hybrid cloud environments. Contact RedSeal today.