Case Study: The Stocks Must Go On

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More .

At the theaters on Broadway, they say that “the show must go on.” But if you drive a few miles further south on Broadway, you hit a part of Manhattan where that statement is even more applicable — the Financial District, home to Wall Street and the New York Stock Exchange. The organizations there take extraordinary steps to ensure nothing stops. When last year’s August 14th blackout shut down the rest of the city, trading continued without a glitch.

For any such critical systems, secure backup is essential, of course. Either the backup software does its job or it gets replaced with something better. That’s exactly what occurred at NYFIX, where homegrown systems were replaced with modern backup software.

“Nothing was being backed up properly,” says Sean Lentner, systems analyst for NYFIX, Inc. “It was kludgy and a management headache. We had no choice but to find a better solution”

Error Free Trading

NYFIX is one of the companies that works behind the scenes to ensure error-free financial trading operations. Headquartered in Stamford, CT, it provides brokers with trading floor hardware, electronic trading systems, industry-wide connections and routing, and processing of financial transactions. It bases its services on the FIX (Financial Information eXchange) protocol — hence the name NYFIX. On any given day, traders route orders on hundreds of millions of shares of stock through the NYFIX Exchange.

The company has servers at three sites. Its two primary sites each contain about two hundred servers running a mix of Sun Microsystems, Inc.’s Solaris, Microsoft Corporation’s Windows, and Linux. Oracle is the main database, although there are also some instances of SQL Server. NYFIX also maintains a development site.

Now, one of the basic principles in systems design is to eliminate any single point of failure. So, while companies use RAID, mirroring, and other technologies to provide data redundancy in the event of a disk failure, relying too much on a single employee’s knowledge can also be disastrous.

In the case of NYFIX, the backup system depended on just one person, the one who wrote its code. This consisted mainly of mySQL systems for the Solaris servers and command line UNIX utilities written in PERL.

“The guy who wrote it understood it well and could control it,” says Lentner. “But others didn’t understand it, and it’s never good to have one guy you have to depend on in IT.”

Also, as is true with most homegrown software, it may deal adequately with its one major function, but it lacks the full range of features present in commercial software.

“The holes were mostly in reporting, plus it didn’t scale well,” Lentner continues. “If a new application came out or we developed a new product version, he had to write new code into the scripts.”

Page 2: The “Fix”

Rather than continuing to rely on its custom-built software, NYFIX decided to go with an outside vendor. To begin with, it purchased new LTO (Linear Tape Open standard) tape drives from San Diego, CA.-based Overland Storage, Inc. The tape libraries connect to the network through SCSI interfaces, though NYFIX may bring them into the SAN later.

Next came the selection of backup software. The company already used several Islandia, N.Y.-based Computer Associates products, including Unicenter network and systems management software and Unicenter Service Desk and Software Delivery modules. As a result, the firm decided to stick with the same family of interoperable products.

“Our goal is to protect and quickly recover data for both our LAN and NAS operations across Windows, Solaris, and Linux,” explains Greg Petras, another systems analyst working for NYFIX. “BrightStor Enterprise Backup lets us reliably protect and recover this diverse environment so effectively that we deployed CA’s solution in production mode even before the planned final release of the code.”

Lentner reports that installing and configuring BrightStor is much quicker and easier than continually writing new code for the old software. He describes it as automated and plug-and-play.

NYFIX reports several benefits. To begin with, there is the increased backup speed. The servers need to operate around the clock. Although the New York Stock Exchange closes at four, the company still has to service after-hours traders as well as the international markets.

The faster backup permits it to perform the functions within the short time windows available. But there are also other functions that endear the product to its users.

“It has good reporting features, something that was lacking previously,” says Lentner. “Overall, this new backup program does its job far more reliably than we had previously been able to achieve.”

»

See All Articles by Drew Robb