RedSeal

Last year, the federal government enhanced its cyber-security requirements, moving away from annual assessments and emphasizing “continuous monitoring” of security controls as part of a reformed FISMA.  The Department of State has been a visionary in defining and implementing these new processes, and its iPost system is a pretty compelling solution for continuous monitoring of host-level vulnerabilities and controls.

So what’s next for continuous monitoring?  The State Department’s Office of Inspector General evaluated their information security program.  Their first finding was that while State does continuously monitor host configurations, it isn’t yet monitoring the critical, volatile and inherited network controls that affect the hosts.  You can read the entire report here.

One (of many) things that the new FISMA regs do well is to require IT to consider the security of the information system as a whole–not just the individual hosts or devices.  Firewalls and other network access controls are part of almost every information system.  OIG’s concern was that these weren’t taken into account:

“Additionally, the review team reviewed iPost controls as part of the Configuration Management (CM) review and found that while the controls did provide continuous monitoring, they did not compensate for the lack of annual testing for access and other volatile controls at the system level.”

This report is getting some attention beyond just the State Department.  Here’s Michael Ono discussing it in last week’s Huffington Post.

Doug Dexter, Audit Lead at Cisco Systems, and I recently had some fun presenting our joint work at Cisco’s end user conference (CiscoLive!) in Las Vegas. We had a great audience, full of seasoned professionals, judging from the way they peppered us with practically focused questions. (I suppose seasoned professionals would pepper us …)

We recorded the event, and the “tape” (?!) is available here:

http://www.redseal.net/news-and-events/Cisco-Live-Presentation-6-2010

Check it out!

The presentation shows lots of real-world data – a rarity in this business. It boils down the experiences of Cisco as they applied automation to various manual and near-impossible time consuming tasks.  They’ve seen benefits in both efficiency and effectiveness.

As Doug says in the presentation, “Computers are better at reading phone books than you are – get over it!”

I couldn’t have put it better myself.  Thanks, Doug (and Cisco) for presenting your experiences!

I’ll be presenting at CiscoLive! in Vegas this month – my talk (co-presented with Cisco) is on Tuesday morning, June 29th. I’ve included the abstract below. Do let us know if you’ll be able to attend – this should be a lot of fun.

Title: Automating Network Security Assessment

Abstract:

This presentation will describe how Cisco’s Audit team automates security assessments for over 30,000 devices. The audit team has applied automation to scale up their capabilities, reduce manual labor spent on firewall and device configuration review, and dramatically increase speed and accuracy when reviewing one of the largest and most complex network environments in the world. The presentation shows that it is possible to visualize a complete global environment, understand relationships and dependencies, and uncover major problems and compliance issues before they are uncovered or exploited by others. Powerful results from new technology for identifying zones, breaking a global environment into manageable pieces, and for automated configuration analysis will be shown.

I’ve been struck in the past several weeks about the confusion in the market between vulnerabilities and risk.  This confusion is certainly heightened by the “risk” scores that are displayed by the various vulnerability scanners.  In truth, there is no way a scanner can tell you the risk associated with a vulnerability.  To see why, let’s look at 4 scenarios.  Each of these scenarios has the same vulnerability and a scanner treat them all as equivalent.  However, it’s obvious that they present dramatically different levels of risk to an enterprise:

This is the worst case scenario: a serious vulnerability on an important server that is directly exposed to threats on the internet.  The risk associated with this scenario is very high.

Same threat and vulnerability, but this time we’ve added a firewall to prevent the threat from being able to directly access the vulnerability.  The risk in this scenario is much less.

Same threat and vulnerability, and the firewall is still blocking direct access.  In this scenario, though, the threat is able to exploit a vulnerability in another, less important system and use that as a base to launch an attack on the important server.  This scenario is again high risk.  This technique, called “pivoting”, is very commonly used in breaches since important servers are almost always behind one or more layers of firewalls. One key point here is that the impact of a vulnerability may be to more than just the the value of the vulnerable system.

Finally, network security can be deployed in layers to contain the effect of a breach.  Using ACLs or firewalls can limit the impact of the exposed vulnerability to just the unimportant server, therefore reducing the risk.

Virtually every definition of risk, such as the one in NIST SP 800-30, combines 4 factors: Threat, Vulnerability, Impact and Mitigating Controls.  Evaluating impact and mitigation requires one to understand the entire environment–not just the host containing the vulnerability.  This is the kind of calculation RedSeal does to create true risk scores.  The “risk” scores provided by scanners are really vulnerability scores that simply combine threat and vulnerability information with an estimate of host value.

Why does this matter?  Most organizations discover thousands of vulnerabilities when they scan–more than they could hope to remediate in the short term.  However, the vast majority of these are usually  mitigated by network security so addressing them can be delayed.  True risk scores can identify those few vulnerabilities that need to be dealt with ASAP.

For example, in the scenarios above,  the vulnerability creating the most risk is the one in the unimportant system that can be used to attack multiple high value servers.  By patching, removing, blocking or containing that vulnerability, the IT department can get the greatest benefit for its efforts.

IP addresses are in short supply – the pool is drying up. Some time before hell freezes over, or even cools noticeably, we will all need to move to the wider vistas (so to speak) of IPv6. This will bring its own “interesting times” to the security world. (Try port-scanning a single IPv6 subnet.  Let me know if you finish in your lifetime…)

As the oversight organization IANA gets requests for new IPv4 address space, it hands out space that is currently “unused.”  (I put “unused” in quotes for a reason.)

So why should you care about such administrivia? Because this time, they handed out a space that you are quite likely to be using already! Not that you should be using it – I’m just saying the odds are pretty good that you do, and once it belongs to somebody else you will have problems.

Why would you be using someone else’s space? Because of the way networks are built and maintained. Most corporate networks include some “Wild West” spaces where time-pressured projects get done without thinking about long term consequences. (Some networks are basically all Wild West, end to end.) The problem is that once IT fabric is set up, operators are loath to take it down again. Who knows what might depend on this strange corner? If you know your organization is different, and everything is done with due care and deliberation, and any messy projects get cleaned up promptly, then feel free to stop reading here (and congratulations).

Now that the irrationally optimistic have surfed elsewhere, what exactly has happened? As of January 2010, IANA marked the space of addresses of form “1.*.*.*” (aka 1.0.0.0/8) as allocated. This means it’s been given to an Internet address registry (APNIC in this case), and they in turn will hand it out as requests for new space are filed. Just how rapidly the addresses will be doled out to end users is not clear, but it’s now inevitable that this space will begin to see active use. For details, take a look at their reference list here:

http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.txt

(Look for “001/8″, near the top of the table, and note the minty-fresh date.)

At RedSeal, we have quite a bit of experience mapping out corporate networks, and we’ve seen a great many cases where parts of this address space (1.0.0.0/8) are in use. Why? Well, think Wild West. You need to bring up something temporary, you don’t want to go through all the overhead of formal allocation, and your quick solution isn’t going to last, right? So you just think up something quick that isn’t being used by anyone you know about.  People routinely pick addresses like “1.1.1.1″, or “1.2.3.4″.  (You also see a lot of “10.10.10.10″, especially in labs, but that’s another story.) Why don’t they go for something random, such as 208.47.89.62? Well, because that almost certainly belongs to someone. The example I picked at random happens to belong to Qwest Communications.  If I want to talk to Qwest from those addresses, it’s not going to work–Packets won’t know if they are coming or going!

OK, so network teams under pressure to build quickly pick addresses from known “empty” space–especially 1.0.0.0/8 because it’s easy to remember. Unfortunately, IANA just decided to allow new colonists into that space. Soon, the address “1.2.3.4″ is going to belong to someone, and you won’t be able to talk to them if your network thinks “1.2.3.4″ still belongs to you.

So what’s the easiest way to check if this change applies to you?  If you use RedSeal, it’s really easy – open up the client, go to Maps and Views, set the View to Subnets, and open the Trusted folder.  This folder is sorted, so any subnets from 1.0.0.0/8 will pop right to the top of the list.  You can see how many you have, and you can open any of them to see where in the network the addresses are being used.  If you have any, I recommend you open a ticket saying it should be changed to use truly private (RFC-1918) space as soon as reasonably possible – include the reference to the IANA update for anyone who didn’t hear about the change.

A little over a year ago I wrote an article about the challenge of managing security risk when IT is managed in silos (Security Risk and Overcoming IT Silos in ISSA 09/08).  Branden Williams (then at Verisign and now at EMC/RSA) quickly followed up with an article ( Crushing Cross-Dysfunctionalism in ISSA 10/08) in which he amplified my position. My daily work has also strengthened my perception that silos are a fundamental issue contributing to IT Risk.
There is a very interesting recent discussion initiated by Michael Krigsman including a nice picture of a silo with a bridge on the very top – representing enterprises where only top management interacts. The blog entry and the ensuing reader discussion brings up a few good points:

• IT silos are still being created to organize the communication within an enterprise — if everybody communicates to various groups and sub-groups it will create a rapidly growing communication overhead.
• Mapping IT onto business needs and structures rather than organizing by IT internal structures might naturally erode silos. This point was also recently made by Jonathan Eunice on his blog “Apps meet Opps”.

These two points are worth examining.

It is certainly true that having random communication patterns prohibits efficiency. But so do walled off silos, especially if they are no longer consistent with the overall business objectives. Advances in system management technology now allow different groups to communicate much more efficiently. Management and workflow platforms provide dashboard views that can be customized and consumed by many constituencies to obtain a shared and consistent view of IT. Enterprise 2.0 applications allow sharing of information without a lot of meeting time overhead or e-mail inbox overload.
The blogs reference the second point in terms of applications, which now no longer map neatly onto IT silos, but rather embody the interconnectedness of the IT-based services powering the business. Just think about virtualized applications. Their mapping onto a IT structure can change frequently without human intervention. Similarly, security risk of the business no longer maps into any IT silo. Security risk is the result of the interplay of security controls from all IT silos. The risk posture can easily change based on the state of virtualized data and apps.

In summary, I am hopeful that external forces, such as business-centric applications, virtualization and clouds, enterprise risk assessments, enterprise 2.0 information sharing will eat away at the ugly silos.

Now that the dust has settled on this news story and the facts are bit clearer, it is worthwhile to take a look what happened and what lesson we can learn. Clearly, a lot of facts were collected and known about the perpetrator: His father talked to the US Embassy in Nigeria, his visa to UK was revoked, he booked a one-way ticket with cash. All these indicators were available before the terror suspect boarded the airplane. While these facts were processed by different security analysts, nobody connected them in time before the plane took off.

Similarly, IT security posture is made out of many separate controls and corresponding data clusters such as firewall configurations, host vulnerability scan reports, compliance reports generated by IT-GRC systems, CMDB reports and more. Just like in the terror attack:

(1) All these data clusters are available before any cyber attack occurs

(2) the challenge is not to collect the data – most organization already collect more data than can ever be processed by humans.  It’s how to summarize and share it among multiple IT organizations such as Network Operations, IT Administration, Security Operations, Compliance, and more.

These organization often have separate objectives. Network Operations is mostly concerned about availability of business-critical services, IT administration is focused on server provisioning, Security Operations often is running a SOC focusing on reacting to emerging attacks under way, and Security Oversight has to engage with all these organizations.

For example, a new server might be provisioned by the IT Admin team. If that server is in the same subnet as other Internet servers, it might be immediately visible from the outside, even if it is not fully configured yet. This only becomes apparent after the server change is made known to firewall administrators. Before this, new risk has been created even though nothing changed in the network.

To control risk, a CISO and his team have to create a process where new relevant facts (like provisioning a new server) are shared with, analyzed by and acted on by all affected IT organizations.  Otherwise, they are relying on the same sort of luck that thankfully prevented a disaster on Christmas day.

Doug Dexter,  internal audit team lead for Cisco, gave a great presentation on how Cisco is automating their internal network audits at the National Summit on Planning and Implementing 20 Critical Controls (Consensus Audit Guidelines).

Doug mentioned that the Cisco network contains 27,000 layer 3 devices.  Before any kind of organization, this is what the network map looked like:

Doug jokingly referred to it as the “bug splat”!

If you’re interested in how Cisco makes sure a network of that size is truly secure, you can see the slides from Doug’s presentation here.

I heard a talk by Heartland CEO Robert Carr at SRI in Menlo Park, CA.  He laid out the timeline of the attacks on Heartland that resulted in their huge data breach:

• Sometime in 2007: Hacker executes a SQL injection attack on an externally accessible web page–Installs sniffer software on a database server and begin to monitor traffic
• Early 2008: Pen test discovers the SQL injection vulnerability and mitigates it.  However, the sniffer is not discovered.
• April 2008: Heartland passes its PCI audit
• May 2008: Hackers begin to steal credit card data

So what are lessons here?

1. Audits can give you a false sense of security.  A manual audit won’t discover everything.  Two separate audit activities (the pen test and PCI audit) were performed during the time Heartland was breached and neither discovered it fully.
2. Passing an audit is not the same thing as compliance.  As mentioned in an earlier post, PCI requires firewalls to be configured to prevent the kind of data extrication that occurred at Heartland.  So even though Heartland passed its audit, it was not compliant with PCI.

Manual audits can never be more than a spot check on your security.  By their nature, they only sample  a small percentage of your infrastructure.  Passing an audit shoudn’t make you feel secure.  And when issues are found (like Heartland’s SQL injection vulnerability), you should wonder what else might be out there (like Heartland’s sniffer).

Cisco Webex has just been given the InfoWorld 100 award for their RedSeal-based Operational Security Oversight Dashboard. The dashboard was named one of the top 100 IT projects of 2009.   Mike Machado, Manager of Security Engineering and Operations,  envisioned and led the project.

Congratulations Mike!