Third shift at the Network Operations center is a quiet time, well suited for the engineer
types who understand and speak the language of machines.
Let’s take a look at what could easily happen on any given night at any company, in any
industry, around the country.
Traffic in the wee hours of the morning is sparse, and the night thus far has passed
uneventfully. Administrators joke with each other and share the usual gripes about work.
Tonight’s concern focuses on a new software vulnerability found in an email program used by
the company’s worldwide offices. This means that later the technicians will have to
exhaustively test and set another code patch into the system. But for now, all is well and
the skeleton IT crew feels fairly safe.
And for good reason.
The company has invested in IT security heavily. The latest firewalls and intrusion
detection systems are keeping the network’s perimeter secure, and IT managers will discuss
new threats and the email patch roll-out at Monday’s staff meeting. A good plan will have to
be developed to tackle that task. Merely applying an untested patch to the critical email
servers is a risk the IT staff can’t afford. They know because they’ve been burnt too many
Normally, these concerns would pass as the shift wore on, but tonight the issue with the
messaging servers gnaws at them.
The firewalls were programmed to let all email traffic through and that leaves a wide gap in
their defenses. Email is essential to the company’s business and attacking it could, in
theory, reach every server in the company.
Just as the IT crew thinks about the email problem and works on their nightly duties, a
hacker somewhere in Eastern Europe hits ‘Enter’ and releases a new worm into the Wild. The
mass-mailing worm spreads quickly, doubling the number of infected machines every 10
seconds. Unlike Slammer, a worm that simply replicated itself, this worm’s payload is much
A simple, yet elegant, piece of code, the multi-threaded worm is able to execute multiple
tasks in parallel — a devastating double threat that first looks for targets to infect
while, at the same time, examining stored messages on infected machines. This second threat
could identify external email addresses and forward all stored messages from the infected
server to anyone and everyone.
Nothing is safe.
Sensitive intellectual property could be made public and mailed to competitors; product
plans, account information and customer interactions could be exposed; and state and federal
disclosure and securities and privacy statues could be violated. All of the company’s
information is fair game. It could be culled and mailed anywhere with abandon.
It only takes a few minutes before alerts start to ring out on Operations consoles. Servers
across the network suddenly stop responding, and then blank out completely. Grabbing onto
multiple infected outside email servers with address books referencing the company, infected
data packets stream in.
Once inside, the worm’s first thread starts looking for other machines on the network to
infect. The internal targets are easy prey. The company had invested heavily in perimeter
defenses, but the internal soft center was wide open. Production servers, test servers —
all are rapidly compromised.
The second thread rifles through the company’s stored messages. It is late at night so
emails that users hadn’t yet access had been piling up. All sensitive information inside
those messages will soon find itself out in public by first light.
The guys in Operations stand by helplessly while the worm burrows through their network. As
CPU and disk usage in each of the mail servers maxes out, the worm prepares for its finale.
After compromising each server and sending all its data out onto the Internet, a
self-destruct command within the worm activates, deleting all stored information and
mutilating each infected machine as thoroughly as if it had been hit by a grenade.
The attack just described is hypothetical, but it easily could be real. The Slammer virus,
for instance, could have been much worse with just a little extra code added to it. It was,
relatively speaking, a benign beast. We may not be so lucky next time.
Attacks that exploit common technologies, such as web and email servers, won’t be stopped by
firewalls. How can a company react to this class of attack?
Monitor, monitor, and monitor. If Operations only finds out about an attack when an email
server crashes, it’s too late. Use real-time monitoring tools to analyze data from IDS and
firewalls in real-time to give the best warning of a new attack.
Secondly, protect your core assets. Consider modifying processes based on the potential
threat posed by a vulnerability, and make sure that core machines are always at the highest
practical level of protection. That way the potential damage inflicted by an attack that
gets through (and there always will be some attacks that get through) is greatly reduced.
Also remember to isolate infected systems. If the infected system is the Internet instead of
an internal server, figure out when to cut the cord to prevent further damage to the
And don’t throw out your firewall. It can be useful. It’s simply not the be all and end all
of a secured network. Enable operators to shut down outbound or internal traffic on common
ports if that’s what it takes.
Another thing to think about is correlating and suppressing alarms. When a massive worm,
like the one described in the story above, appears, the operations team could get swamped
with thousands of similar alerts. Real-time correlation and security event management
systems will link and escalate similar events, creating a few ‘master’ alerts. This reduces
information overload during crises and clearly identifies the root cause of the attacks. Any
organization with more than five to 10 firewalls and IDS should consider a security event
Phil Hollows is vice president of security products for OpenService, a vendor of network security event management products.