Thursday, March 28, 2024

Harnessing the Flood of Security Data

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

According to the U.S. Congress’ 9/11 Commission, one of the key elements

that allowed the attacks to occur was the FBI’s inability to easily share

information gathered by different offices — to coordinate and analyze

that data.

But remedying this has not proven easy.

After spending as much as $170 million on custom software to address this

issue, the FBI announced on Jan. 13 that it might have to scrap the

software and start over.

”The FBI’s long-anticipated Virtual Case File has been a train wreck in

slow motion,” Sen. Patrick Leahy, (D-Vt.) said in a statement released

that day.

On a smaller scale, IT departments are having trouble managing their own

security data.

”Everything was a mess,” says Jim Patterson, security analyst for the

State of Illinois Legislative Information System (ILIS). ”Even from a

big vendor like Cisco, each device had its own reporting console, and

there was no way to have a central point to manage them.”

To make things worse, each device didn’t just require its own software,

but a dedicated PC in order to avoid running into conflicts.

”We would try to gather data from all these different devices, but there

was no way to correlate the information,” Patterson adds. ”The Cisco

Pix firewalls were generating so much traffic — millions of messages a

day — that the little built-in SQL server couldn’t even handle it.”

At Cisco’s recommendation, ILIS turned to a new type of management

software called Security Information Management (SIM), installing nFX

software from NetForensics, Inc., a company focused on the SIM market and

based in Edison, N.J. This allowed Patterson to aggregate all the

security information into a single database for analysis and alerting.

”The netForensics has a realtime event console with a scrolling display

of what is happening,” says Patterson. ”This reduces the number of

people you need to have monitoring security since everything is on one

central location.”

Message Madness

SIM is an outgrowth of network log management software adapted for use

with security devices and software, including firewalls, Intrusion

Detection Systems (IDS), Intrusion Prevention Systems (IPS),

authorization software and anti-virus. One of the main drivers has been

simply being able to make sense out of the huge amount of data that these

devices spit out on a regular basis. It is impossible to manually go

through the millions of messages and gain a clear concept of what is

happening from a security standpoint.

”Many times, this is driven by a failed IDS project that dumps out too

much data to effectively interpret,” says Paul Proctor, vice president

of Security and Risk Strategies for META Group, an analyst firm based in

Stamford, Conn. ”IDS implementations fail because organizations do not

tune them properly, not because they inherently produce too much data.”

Of course, when you have devices generating that much information, it can

be hard to properly tune the devices, thereby reducing the number of

messages.

Patterson says having a SIM has enabled the state to fine tune its

firewalls and sensors.

META Group’s Proctor advises that organizations shouldn’t start out with

the goal of cutting down on what they have. Instead, they should start by

determining what they need.

”A more effective approach is to start with a detection requirements

list tied to business needs, and then determine which events need to be

collected to support those requirements,” he says. ”If you take this

approach, SIMs can have value.”

A Global View

Installing and configuring a SIM can be a major undertaking. Patterson

says the Illinois system started out running its SIM on a single server,

but found that as the number of security devices increased, he had to

split it up. He is now monitoring 30 devices at three sites.

The netForensics software resides on three low-end Dell dual-processor

servers — one for collecting the data, one for the Oracle database, and

a third for reporting and analysis.
Larger installations require much more.

”When they embark on a large SIM project (with more than 300 audit

sources/nodes) they should put aside at least $50,000 in their services

budget for the vendor or a competent third-party to come in and install,

and tune for appropriate business requirements,” says Proctor.

Deployments with more than 1,000 nodes are usually multi-year efforts, so

set realistic expectations and project goals.”

Unisys Global Infrastructure Services of Blue Bell, Penn., for example,

has three security operations centers. One is in Blue Bell, Amsterdam,

another is in the Netherlands, and the third is in Wellington, New

Zealand. The centers provide the company, which has 200 managed security

clients, services around the globe.

Unisys Global began deploying a SIM from ArcSight, Inc. of Cupertino,

Calif. in June f 2003. The final roll out will be completed this year.

Once this is completed, the security analysis will be performed at three

levels — customer, regional and global. Having the global system in

place lets them spot a problem in one area of the world and take action

to harden security in others areas before they are hit.

”It has proven useful in helping to detect the zero-day threats out

there before there is a signature available for it,” says John Summers,

Global Director for Managed Security Services. ”Our European operations

center, for example, found a particular threat, what the network traffic

looked like, what ports it was talking on, and we wrote a specific

correlation rule to monitor data on those ports.”

Summers says that having a SIM has two main values.

To begin with, it enables them to do complex pattern detection across a

heterogeneous infrastructure. This has been useful in spotting blended

threats which seek to exploit multiple vulnerabilities.

The other benefit is that it is able to reduce the number of false

positives, allowing them to accurately spot the true threats.

”With IDS or any security device, you get way too many messages coming

in, so to handle it, people turn down the gain on their sensors so they

put out less noise, but also put out less signal,” says Summers. ”But

an event correlation platform allows you to turn the gain up again and

gives you a more accurate ability to detect suspicious or bad activity.”

Subscribe to Data Insider

Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more.

Similar articles

Get the Free Newsletter!

Subscribe to Data Insider for top news, trends & analysis

Latest Articles