Managing Multiple Networks Without Strain

Gone are the days when all a network administrator had to worry about was learning and supporting a single OS. Now a firm may have Windows desktops, Linux or Apache web servers, a Unix database, PDAs running Palm O/S, and several Macs running in Marketing.

Now consider the predicament of a managed service provider (MSP). An MSP must design and operate a system that requires minimal customized integration no matter how complex its customers’ networks may be.

“Every time you install something on their network, you can blow up the client’s servers,” says Jim Lancaster, president of MBD Network Services, LLC.

MBD is a network management services provider based in Dallas, Texas. The company provides network monitoring and on-site support for 60 small- to medium-sized clients, each with a network ranging from 15 to 150 seats. Lancaster needed to come up with a standardized method of managing that many networks without affecting the internal operations of those systems.

His solution: separate the management from the network operations.

BlackBox

MBD addresses this situation, by installing what it calls a “BlackBox” on each client’s network. These consist of Dell Optiplex workstations, running Windows 2000 and loaded with the suite of management tools that Lancaster and his team has assembled over the years. This software package monitors the health of the network and systems, reporting the data back to MBD’s Network Operations Center (NOC).

Doing it this way provides one consistent set of software that MBD’s network administrators can learn and manage. Client data is reported in a standardized manner into a single console where it can be viewed and analyzed. This makes it simpler and faster to deploy updates since these are sent to boxes with identical configurations.
In selecting tools, Lancaster prefers open source since it allows customization towards exact needs rather than making do with vendor feature sets.

“The open source movement has been a huge benefit to us,” he says. “Fifty percent of our new installations are Linux.”

He also finds that push technology works better than having management tools that pull the data from the remote sites. Instead, his black boxes gather the data from the equipment on each network and then push it through the firewall to the NOC.

“Most of the big frameworks that are in use with an MSP use pull technology and require a virtual private network (VPN) or direct connection to each client,” Lancaster explains. “In small networks that is not a secure way to operate and it is a headache to manage.”

The Toolbox

The BlackBoxes run on Windows 2000 and come with a mySQL database and software for managing event logs, virus protection and firewall events, as well as network management tools.

To centrally manage all the Windows NT/2000 event logs, MBD uses three tools from Adiscon GmbH (Erftstadt, Germany) — EventReporter, WinSysLog and MoniLog.

EventReporter resides on the BlackBoxes on the client sites and forwards all Errors, Warnings and Failure Audits to a WinSysLog server in the NOC which aggregates the data from all the clients. The MSP then uses MoniLog to produce a daily summary of each client’s log entries. The BlackBox also acts as the logging server for firewalls, most typically WatchGuard Technologies, Inc.’s (Seattle, Wash.) Firebox System.

MBD recommends that its clients use Symantec Corp.’s Anti-Virus Corporate Edition. For those who do, the BlackBox includes a copy of Symantec System Center, the control console for the security product.

In addition to the out-of-the-box software, the BlackBoxes also contain several tools written by Lancaster. This includes a suite of Perl scripts, scheduled and run by the BlackBox that generate snapshot reports on Windows NT/2000 servers, NetWare servers, RMON capable switches and American Power Conversion Corp. (West Kingston, R.I.) uninterruptible power supplies.

The boxes also contain a custom-built routine to e-mail the backup logs from all tape drives to the NOC, no matter what backup software the client uses, so MBD’s personnel can verify that all daily backups are completed as scheduled. Although he would like to automate the reporting on backups, that hasn’t been possible yet due to the disparate ways that different software reports the data.

“The only way you can tell if the backup was successful or not is by reading the log,” Lancaster explains. “The subject line on the e-mail may say that it failed, but the only problem is that one file which was open wasn’t backed up.”

Separating Management from Network Operations

Perhaps the key to MDB’s success-through-simplicity approach, though, is keeping management completely separate from network operations. While the above tools provide various management functions, network health is monitored separately.

With thousands of aggregate users to cater to, most would expect MDB to utilize one of the large framework packages for network management. However, Lancaster notes that since each network the company manages is relatively small, these products are much too expensive and cumbersome for his needs.

“We’ve never been in the market for a comprehensive system like Unicenter or Tivoli,” says Lancaster. “They are a massive overkill for a client with two servers and 25 desktops.”

The first tool he settled on was WhatsUp Gold from Ipswitch, Inc. (Lexington, Mass.). WhatsUp Gold scans the network’s router tables and creates hierarchal maps showing all the devices. It then monitors the Simple Network Management Protocol (SNMP) parameters on these devices and sends alerts when administrator-defined thresholds are exceeded.

While WhatsUp Gold gave MBD some of the information that it needed, Lancaster was still looking for a higher-level tool. This led him to Denika Performance Trender from Somix Technologies, Inc. (Sanford, Maine). Denika works with WhatsUp Gold, but automates the generation of daily, weekly, monthly and yearly historical tracking reports on items such as bandwidth consumption, disk space, CPU and memory utilization.

“We follow the International Telecommunications Union’s FCAPS (Fault-management, Configuration, Accounting, Performance, Security) network management model,” says Lancaster. “WhatsUp Gold addresses fault, and Denika is the performance piece, giving us the trends over time.”

As Denika worked well, MDB upgraded to the same vendor’s comprehensive network management suite for service providers — WebNM xSP. Besides including copies of Denika and WhatsUp Gold, this suite contains modules for help desk, hardware and software inventorying, web cam, environmental monitoring and event log management. Although he decided to stick with the Adiscon products for event log management, Lancaster bought the full package so that he could use the inventory and help desk modules, while also gaining an easy way to present complex information about network issues.

“What we were missing was a way to present data to both the clients and our own engineers,” he explains. “WebNM has allowed us to take our service to a whole new level.”

This approach also enabled MBD to set up a personal support portal for each client. Through the portal, clients can view their network documentation and contact information, check the status of any service request and read netlog reports. Only one of MBD’s clients, however, currently views its performance graphs on line. This client has a WAN connecting nine locations in the Eastern U.S. and wants to track the bandwidth utilization.

The typical customer, though, doesn’t need or want to monitor information to that degree. Most are satisfied with being able to use WebNM to inventory hardware/software and know that someone else is monitoring the network so that they don’t have to.

“There is a bewildering amount of management technology available, but they are not necessary for small or medium businesses,” Lancaster says. “It’s the fundamentals that are important, and Somix and Ipswitch address those with cost effective tools that help us provide great service to our clients.”