Vender benchmarks have long been thought a waste of time — largely because of the inherent bias a vendor has in testing its own stuff. When only one outcome is acceptable, it should be no surprise that the vendor’s product always wins a vendor’s competitive test. I often wonder why they bother.
However, customer benchmarks are generally valid — largely because a customer is going to fund the result, and a biased benchmark will result in a bad decision. That will likely result in the poor sap who did the benchmark being reassigned, probably to unemployment.
This is why the Tencent benchmark of IBM OpenPOWER servers was important. Not only did it showcase what appears to be a significant OpenPOWER advantage, but because it is from a customer and not IBM, this advantage should be reliable.
Let’s talk about why this Tencent benchmark bodes wells for IBM.
Cloud Loads Are Unique
One of the fascinating things we don’t talk about in the current market is that servers, like all technology, have been evolving primarily for on-promise workloads. While there certainly have been hosting companies for a long time, they never reached the kind of scale and influence that cloud providers now have. Cloud providers not only now consume massive numbers of servers, their needs are actually very different because they make even the largest and most dense traditional deployment look tiny by comparison.
And this isn’t the only difference. The level of flexibility that cloud providers must have is unprecedented. They have to be able to move workloads of all sizes, from individual loads that might typically run on workstations and PCs, to massive analytics projects that could overwhelm most mid-market and some enterprise data centers.
Finally, they have to achieve massive economies of scale because, unlike on-premise deployments, cloud customers can generally move their projects from vendor to vendor far more quickly and easily. This creates a near laser-like focus on cost because price is king. If cost gets out of line, the cloud provider will either lose the customer as they try to maintain margin or drop into the red as they try to keep the customer. Neither path is acceptable.
General Purpose Servers Can’t Do It
A couple of cloud providers concluded that their flexibility requirements would largely drive them towards generic servers that they could define themselves. A couple of the largest providers went that direction. This made sense as long as loads were mostly individual. Once enterprises started to place massive analytics loads on these same services, it became clear to some of the providers that a generic approach to servers, while still valid for many of their high volume small loads, didn’t scale up cost effectively.
As large loads scaled, a need for more specialized servers emerged. This opened the door for IBM OpenPOWER to apply their unique technology and for partners like Mellanox to develop solutions that were capable of outperforming their generic counterparts on targeted loads.
Much like you need different equipment and tools for different jobs, so too does the modern cloud provider need different kinds of servers to cost effectively handle their massive load diversity. This shouldn’t have been a surprise, and it suggests we will be getting more, not less, diversity within cloud providers going forward.
Often, as a market requirement matures, the products needed to address that requirement have to evolve and diversify. The cloud is no different.
While a generic approach initially both made sense and was viable, as this market has matured, it has become clear that more specialized tools were needed — particularly at scale. IBM’s OpenPOWER has stepped into that opportunity and, based on these benchmarks, appears to have found a unique opportunity. The big story is the emergence of both the need and the related solutions for technology diversity in the cloud.
Photo courtesy of Shutterstock.