RedSeal

IP addresses are in short supply – the pool is drying up. Some time before hell freezes over, or even cools noticeably, we will all need to move to the wider vistas (so to speak) of IPv6. This will bring its own “interesting times” to the security world. (Try port-scanning a single IPv6 subnet.  Let me know if you finish in your lifetime…)

As the oversight organization IANA gets requests for new IPv4 address space, it hands out space that is currently “unused.”  (I put “unused” in quotes for a reason.)

So why should you care about such administrivia? Because this time, they handed out a space that you are quite likely to be using already! Not that you should be using it – I’m just saying the odds are pretty good that you do, and once it belongs to somebody else you will have problems.

Why would you be using someone else’s space? Because of the way networks are built and maintained. Most corporate networks include some “Wild West” spaces where time-pressured projects get done without thinking about long term consequences. (Some networks are basically all Wild West, end to end.) The problem is that once IT fabric is set up, operators are loath to take it down again. Who knows what might depend on this strange corner? If you know your organization is different, and everything is done with due care and deliberation, and any messy projects get cleaned up promptly, then feel free to stop reading here (and congratulations).

Now that the irrationally optimistic have surfed elsewhere, what exactly has happened? As of January 2010, IANA marked the space of addresses of form “1.*.*.*” (aka 1.0.0.0/8) as allocated. This means it’s been given to an Internet address registry (APNIC in this case), and they in turn will hand it out as requests for new space are filed. Just how rapidly the addresses will be doled out to end users is not clear, but it’s now inevitable that this space will begin to see active use. For details, take a look at their reference list here:

http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.txt

(Look for “001/8″, near the top of the table, and note the minty-fresh date.)

At RedSeal, we have quite a bit of experience mapping out corporate networks, and we’ve seen a great many cases where parts of this address space (1.0.0.0/8) are in use. Why? Well, think Wild West. You need to bring up something temporary, you don’t want to go through all the overhead of formal allocation, and your quick solution isn’t going to last, right? So you just think up something quick that isn’t being used by anyone you know about.  People routinely pick addresses like “1.1.1.1″, or “1.2.3.4″.  (You also see a lot of “10.10.10.10″, especially in labs, but that’s another story.) Why don’t they go for something random, such as 208.47.89.62? Well, because that almost certainly belongs to someone. The example I picked at random happens to belong to Qwest Communications.  If I want to talk to Qwest from those addresses, it’s not going to work–Packets won’t know if they are coming or going!

OK, so network teams under pressure to build quickly pick addresses from known “empty” space–especially 1.0.0.0/8 because it’s easy to remember. Unfortunately, IANA just decided to allow new colonists into that space. Soon, the address “1.2.3.4″ is going to belong to someone, and you won’t be able to talk to them if your network thinks “1.2.3.4″ still belongs to you.

So what’s the easiest way to check if this change applies to you?  If you use RedSeal, it’s really easy – open up the client, go to Maps and Views, set the View to Subnets, and open the Trusted folder.  This folder is sorted, so any subnets from 1.0.0.0/8 will pop right to the top of the list.  You can see how many you have, and you can open any of them to see where in the network the addresses are being used.  If you have any, I recommend you open a ticket saying it should be changed to use truly private (RFC-1918) space as soon as reasonably possible – include the reference to the IANA update for anyone who didn’t hear about the change.

A little over a year ago I wrote an article about the challenge of managing security risk when IT is managed in silos (Security Risk and Overcoming IT Silos in ISSA 09/08).  Branden Williams (then at Verisign and now at EMC/RSA) quickly followed up with an article ( Crushing Cross-Dysfunctionalism in ISSA 10/08) in which he amplified my position. My daily work has also strengthened my perception that silos are a fundamental issue contributing to IT Risk.
There is a very interesting recent discussion initiated by Michael Krigsman including a nice picture of a silo with a bridge on the very top – representing enterprises where only top management interacts. The blog entry and the ensuing reader discussion brings up a few good points:

• IT silos are still being created to organize the communication within an enterprise — if everybody communicates to various groups and sub-groups it will create a rapidly growing communication overhead.
• Mapping IT onto business needs and structures rather than organizing by IT internal structures might naturally erode silos. This point was also recently made by Jonathan Eunice on his blog “Apps meet Opps”.

These two points are worth examining.

It is certainly true that having random communication patterns prohibits efficiency. But so do walled off silos, especially if they are no longer consistent with the overall business objectives. Advances in system management technology now allow different groups to communicate much more efficiently. Management and workflow platforms provide dashboard views that can be customized and consumed by many constituencies to obtain a shared and consistent view of IT. Enterprise 2.0 applications allow sharing of information without a lot of meeting time overhead or e-mail inbox overload.
The blogs reference the second point in terms of applications, which now no longer map neatly onto IT silos, but rather embody the interconnectedness of the IT-based services powering the business. Just think about virtualized applications. Their mapping onto a IT structure can change frequently without human intervention. Similarly, security risk of the business no longer maps into any IT silo. Security risk is the result of the interplay of security controls from all IT silos. The risk posture can easily change based on the state of virtualized data and apps.

In summary, I am hopeful that external forces, such as business-centric applications, virtualization and clouds, enterprise risk assessments, enterprise 2.0 information sharing will eat away at the ugly silos.

Now that the dust has settled on this news story and the facts are bit clearer, it is worthwhile to take a look what happened and what lesson we can learn. Clearly, a lot of facts were collected and known about the perpetrator: His father talked to the US Embassy in Nigeria, his visa to UK was revoked, he booked a one-way ticket with cash. All these indicators were available before the terror suspect boarded the airplane. While these facts were processed by different security analysts, nobody connected them in time before the plane took off.

Similarly, IT security posture is made out of many separate controls and corresponding data clusters such as firewall configurations, host vulnerability scan reports, compliance reports generated by IT-GRC systems, CMDB reports and more. Just like in the terror attack:

(1) All these data clusters are available before any cyber attack occurs

(2) the challenge is not to collect the data – most organization already collect more data than can ever be processed by humans.  It’s how to summarize and share it among multiple IT organizations such as Network Operations, IT Administration, Security Operations, Compliance, and more.

These organization often have separate objectives. Network Operations is mostly concerned about availability of business-critical services, IT administration is focused on server provisioning, Security Operations often is running a SOC focusing on reacting to emerging attacks under way, and Security Oversight has to engage with all these organizations.

For example, a new server might be provisioned by the IT Admin team. If that server is in the same subnet as other Internet servers, it might be immediately visible from the outside, even if it is not fully configured yet. This only becomes apparent after the server change is made known to firewall administrators. Before this, new risk has been created even though nothing changed in the network.

To control risk, a CISO and his team have to create a process where new relevant facts (like provisioning a new server) are shared with, analyzed by and acted on by all affected IT organizations.  Otherwise, they are relying on the same sort of luck that thankfully prevented a disaster on Christmas day.

Doug Dexter,  internal audit team lead for Cisco, gave a great presentation on how Cisco is automating their internal network audits at the National Summit on Planning and Implementing 20 Critical Controls (Consensus Audit Guidelines).

Doug mentioned that the Cisco network contains 27,000 layer 3 devices.  Before any kind of organization, this is what the network map looked like:

Doug jokingly referred to it as the “bug splat”!

If you’re interested in how Cisco makes sure a network of that size is truly secure, you can see the slides from Doug’s presentation here.

I heard a talk by Heartland CEO Robert Carr at SRI in Menlo Park, CA.  He laid out the timeline of the attacks on Heartland that resulted in their huge data breach:

• Sometime in 2007: Hacker executes a SQL injection attack on an externally accessible web page–Installs sniffer software on a database server and begin to monitor traffic
• Early 2008: Pen test discovers the SQL injection vulnerability and mitigates it.  However, the sniffer is not discovered.
• April 2008: Heartland passes its PCI audit
• May 2008: Hackers begin to steal credit card data

So what are lessons here?

1. Audits can give you a false sense of security.  A manual audit won’t discover everything.  Two separate audit activities (the pen test and PCI audit) were performed during the time Heartland was breached and neither discovered it fully.
2. Passing an audit is not the same thing as compliance.  As mentioned in an earlier post, PCI requires firewalls to be configured to prevent the kind of data extrication that occurred at Heartland.  So even though Heartland passed its audit, it was not compliant with PCI.

Manual audits can never be more than a spot check on your security.  By their nature, they only sample  a small percentage of your infrastructure.  Passing an audit shoudn’t make you feel secure.  And when issues are found (like Heartland’s SQL injection vulnerability), you should wonder what else might be out there (like Heartland’s sniffer).

Cisco Webex has just been given the InfoWorld 100 award for their RedSeal-based Operational Security Oversight Dashboard. The dashboard was named one of the top 100 IT projects of 2009.   Mike Machado, Manager of Security Engineering and Operations,  envisioned and led the project.

Congratulations Mike!

Another Albert Gonzalez indictment gives details of how he hacked into Heartland Systems.  This breach compromised 100 million records, cost Heartland $12.6 million in Q1 2009, and destroyed $300 million in shareholder value.  Again, the interesting part is how easily it could have been mitigated in retrospect.

The breach began when hackers exploited a  SQL injection vulnerability on an externally accessible server.  This is a well known error and can be prevented using basic programming techniques and detected using widely available web application scanners.

The SQL injection was used to install “back door” malware onto a Heartland server.  This back door allowed the hackers to communicate to and from that server.  This mechanism was used to transmit the credit card data to the external servers.  As with TJX, this communication could and should have been blocked by firewall rules.

The indictment of Albert Gonzalez gives some interesting insights into exactly how TJX was hacked–a breach of 94 million customer records. While the indictment is a bit vague, it’s still striking how basic the attack was and how easily (in retrospect) it could have been prevented.

This breach began when the hackers compromised a wireless access point in a store in Florida. They found this simply by driving around businesses looking for wireless access (“wardriving”). This compromise could have been prevented by using stronger encryption on the access point. Old wireless encryption (WEP) is notoriously easy to break. The indictment points out that some stores (BJ’s Wholesale—not TJX) had no encryption at all on their wireless at all.

The hackers then gained access to servers in Framingham containing card data. They used this to both directly obtain cardholder data and to install a sniffer. While the techniques they used to obtain this access aren’t specified, they should have been able to be prevented by either firewall rules or proper server patching.

The hackers then set up a VPN tunnel from the transaction server to an external server and used this to extract data. This outbound connection should easily have been prevented by firewall rules. In fact, PCI explicitly requires firewalls to block this type of connection.