What's the best way to measure the efficiency of a data center? No one really knows. But there's no shortage of IT minds trying to figure it out.
Last month, in partnership with Intel, data-center management software vendor iTRACS took up the challenge of developing a metric -- a single number -- that will allow data-center operators to assess the efficiency of their facilities relative to their bottom line, and the company is only the latest in a string of organizations trying to develop similar metrics. Industry coalition The Green Grid has been working on the problem for four years.
"A lot of folks have been interested in this issue for some time, and there's never really been a good consensus in the industry on how to tackle it," says John Pflueger, a Green Grid board member and principal environmental strategist for Dell. "And we quickly came to the realization that there were some really good reasons behind that."
Settling on a metric is only a step toward solving the even bigger challenge of calculating a data center's value to the business it serves. In the Internet age, as more and more of the world's data move online, the importance of this task will only grow. But some question whether developing a data-center business-value metric is even possible. As you move up the chain from power systems to servers to applications to balance sheets, the variables increase and the values are harder to define.
"The holy grail of efficiency metrics in the IT and datacenter industries is finding a standard way of measuring useful IT work per unit of energy consumed," says John Stanley, an analyst with market research firm The 451 Group. "If Intel, iTRACS, or anyone else can do that, then more power to them."
PUE to PAR4
The practice of measuring data center efficiency got off to a promising start when the Green Grid came out with Power Usage Effectiveness (PUE) in 2007. PUE is a ratio of the amount of energy a data center consumes to the amount it's IT equipment uses. If a data center's power systems were 100 percent efficient and all of the electricity that entered the data center went to its servers, storage systems and networks, the data center's PUE would be 1.0. Data center operators today generally claim PUEs of 1.1 to 1.6.
PUE has become the touchstone for measuring data center energy efficiency. According to Rhonda Ascierto, a senior analyst for energy and sustainability technology at market research firm Ovum, it has succeeded because it was the industry's first metric and was easy to understand. "PUE embodies data center efficiency to the degree that some CIOs’ salary bonuses rely on their PUE ratio," said Ascierto. "It has mattered little that PUE only measures infrastructure efficiency."
But the industry quickly recognized that PUE only scratched the surface, and numerous alternative metrics have sprung up in recent years. Among the more widely recognized are:
- Corporate Average Datacenter Efficiency (CADE) is a set of four metrics developed by the Uptime Institute and McKinsey & Company that measure facility asset utilization, facility energy efficiency, IT asset utilization and IT energy efficiency.
- Power to Performance Effectiveness (PPE), developed by market research firm Gartner, measures server performance per kilowatt with the goal of helping IT raise server utilization levels.
- PAR4, from Underwriters Laboratories and Power Assure, calculates transactions per second per watt.
These metrics address aspects of IT equipment efficiency, but they aren't sufficient to give the whole picture of a data center's efficiency. What iTRACS is aiming for is a metric that weighs asset work capacity, asset utilization and asset power utilization. "Once you have that IT efficiency metric you can start to understand the overall business effectiveness of the data center," says Elizabeth Given, the CEO and president of iTRACs. "What does it cost to output a service versus the business output gained from that service?"
The company is developing the top step of a stepladder of efficiency metrics. The first step is PUE, the second step is asset energy efficiency, the third step is total IT efficiency, and the top step is business value.
The Green Grid his working in the same direction, but there's a lot of work to do to get there. "We're converging on a point sometime in the future where the folks on the IT side who want to understand the useful work out of the data center and how it relates to resource consumption are going to meet up with folks on the business side who want to understand business value and how we're extracting [it] out of the resources that we're putting into these facilities," says Pflueger.
The Green Grid is studying ways of measuring IT equipment to find proxies for productivity. The organization is working with a national laboratory in the Northwest to test these measurements, but they don't have results to report yet. "We're starting to get to some interesting places."
Another approach the Green Grid is looking at is measuring efficiency at the application level. Many applications have counters to track how much work they're doing -- for example, the number of e-mails processed or number of users supported. This approach doesn't give you a comprehensive view of a data center's efficiency, but it can tell you how well an architecture or technology implementation delivers a particular service. Aggregate a bunch of these application-specific metrics, and you can get a useful higher-level view. "It's a matter of figuring out how to pull all of the information together and getting it to one digestible form," says Pflueger.
Uh, What's a Data Center?
The rub is that no one can agree on what a data center is. "Some datacenters are converted parking garages, some are on ships and still others are in shipping containers," says Katherine Broderick, a senior research analyst at IDC. "Finding a metric that will work in all of those disparate environments is difficult."
Plus, the goal itself is different depending on who's looking at it. What's business value for Google is not the same as business value for eBay or Dell or Verizon. "Think American Airlines reservations versus Facebook," says Jim Smith, the chief technology officer of Digital Realty Trust, one of the largest data center operators in the world. "Those are two completely different systems -- different scale, different functions, different costs, different reliability. It's going to be very hard to find something that connects the dots between those two different applications and the tens of thousands of others that are out there."
According to ITRACS CEO Given, business value is unique to every organization. But it should still be possible to develop an equation for calculating business value that different businesses can use.
Factories vs. enterprises
Looking from an applications perspective, you can divide the world of data centers into two camps, one that produces services for the market and another that supports businesses. "You've got one class of data centers where the data center is a factory and it's producing something that's being sold," says Pflueger.
These "factories" include the big Internet companies, social media players, and telecommunications carriers. "For those organizations where the data center is actually a factory, then the useful IT work and the business value becomes a lot easier to connect," he says.
Some of the bigger companies -- including Verizon, Google, AT&T and Akamai -- are even beginning to discuss their internal metrics publicly. And smaller companies, including cloud services providers, are also beginning to make the connection between useful IT work and business value.
The other camp -- enterprise data centers -- is a different story. Where the "factory" data centers run relatively few applications, enterprise data centers can run hundreds or even thousands "of all shapes and sizes that are doing vastly different tasks," says Pflueger. "And those are the folks that today really don't have a lot of guidance on how to measure the useful work of their IT equipment."
Mission impossible?
It's an understatement to say that developing IT productivity and business value metrics for enterprise data centers is a challenge. The number of applications alone is a huge hurdle for enterprise data center operators. "Should I be spending my time trying to figure out the connection between business value and IT work for this specific application, which sits on three servers or four servers in my data center, knowing that I have to replicate this analysis a thousand times to cover everything?" says Pflueger.
There are also institutional barriers. Typically, a facilities manager pays the electricity bill and not the IT manager. And for IT, the bottom line is uptime. "If they increase efficiency or save money on the power bill then they get a pat on the back," says IDC's Broderick. "If the datacenter or a key workload goes down, then they lose their job."
And it may be that enterprise IT departments are simply overmatched for the task. Efficiency is one of many priorities, including service performance and security. "If you spend any time with enterprise customers, you realize very quickly that they're overwhelmed," says Digital Realty's Smith. "Their jobs are so difficult."
"I think maybe we could get to a combined 'useful IT work per unit of energy' metric," says The 451 Group's Stanley. "But it still wouldn’t factor in cost, reliability, carbon, security, or any of the million other things datacenter operators need to care about," he said. "We’ll always need multiple metrics, even if we manage to improve the ones we’ve got."
Organizations need a dashboard of metrics to know which lever to pull in response to changes in business demand, market forces, cost, IT and energy capacity, and local and global regulations and requirements.
When Digital Realty's Jim Smith looks at the thorny problem of measuring the efficiency of servers, switches, and storage systems, he does so from a comfortable distance. Digital Realty doesn't have to worry about the efficiency of the IT equipment other people use in its facilities. But he does think about it, and from his vantage point, an IT productivity metric could be an unachievable goal. At best, it's one or two generations of computer scientists away.
"If you really are after the holy grail of cost savings, the most important thing you can do is lower the demand of power that your applications consume," he says. "Do you really need to serve up a webpage in a certain way, do transactions need to be verified 42 times, do you need 42 backup centers?"
