Data, Data Everywhere, nor Any Time to Think

I remember when I first started trying to solve network security problems, using fancy network analytics.  I applied the classic suspension of disbelief that’s necessary to work on any emerging technology – first, you assume all the hard problems will be easy, and second, you assume the impossible ones will just go away.  Happily, much of this is true – it’s funny how well it works.  Only later do you learn which problems are the truly hard ones.

What’s hard about network security analytics?  Well, not the security, and not the analytics – we’ve found we can do plenty on both of those that pays off really well, given the data.  The pesky data, now that’s a different kettle of enchiladas.

At first, I didn’t want to talk about data gaps – that sounded like a challenge to good analytics.  I was half right.  Eventually, enough CISO’s got it through my skull that uncovering data gaps may be pointing to reasons why analytics will be held back, but it’s also major value, in and of itself.  I was being dense – if we try to analyze security data, and we find it’s got holes in it, well, this means the security team didn’t know what was going on to start with!  Turning up these gaps is one of those inconvenient truths.  These days we’ve gotten pretty good at it.

But then what?  Typical security organizations are drowning in data, so how can I complain about needing more?  Well, facts are just facts; useful information, or better yet, actionable intelligence is something else altogether.  We stockpile data from sensors, but we struggle to find useful signal in there.  We deploy automated signal reduction engines, but they just turn mountains of alerts into hills of alerts, and we still don’t have time or people enough to climb those.  And along come these network security analytics people saying “what you need is more data”.  Hmmm.

Of course, what we need is the RIGHT data, processed the right way, at the right time.