Transcript
Introduction to Data Center Energy Consumption
Good afternoon, everyone. This is Chris Mullins here with Joe Fackler with today’s White Paper Webinar. Today, we have an interesting topic. We’re talking about energy consumption from data centers. As Joe will talk about, data centers, the server farms are a huge consumer of power for utilities, and it’s a very different kind of load for utilities. The load profile is very different, and it’s growing so fast that it’s very difficult for utilities to even keep up, even in terms of getting transformers to these huge data centers. Joe’s gonna talk more about how this is happening, the details of the growth, and why it’s so different for utilities.
Rapid Growth of Data Centers
Data centers, which are giant buildings housing many hundreds and thousands of servers and networking equipment, are growing very rapidly. We’ve got a chart in here that shows you energy forecast for data centers. You can see data centers down there in green taking up a massive amount of energy, and you can see the rapid growth there as these centers are exploding.
As we were researching this paper, the number of data centers in each year’s new estimates of data centers in the United States, data centers in China and Europe growing dramatically every year. Just a decade ago, there were less than 8,000 data centers in the whole world. Today, the United States alone has over 5,000, and rapid expansion’s planned for those numbers.
Data Centers in Virginia
If we can take a look at the second page, we have here in Power Monitor’s home state of Virginia the amount of data center space we currently have, and the planned expansion is about four times what we have now. Also in our home state, right now, data centers consume 25% of all the generated power, which is a truly astronomical number. We’re potentially looking at, by in the next decade, half of all power in Virginia being consumed by data centers. One industry consuming half of the state’s available power.
Now, of course, we know that it takes quite a while to plan and actually get facilities put together to expand the grid and get more power available in these data centers. Certainly, it takes a lot longer time to get power supplied to these data centers than it takes to build a data center.
I actually grew up in Ashburn, Virginia, right in the heart of this data center alley in Northern Virginia, and truly amazing. I go home to see my family for the Fourth of July, and by Thanksgiving, there’s two new data centers already built, already up and running. Truly astronomical growth. If anything, projections seem like they’re undershooting the eventual growth.
And that’s a real challenge for utilities because the lead times on transformers and especially running new transmission lines or even medium voltage lines with different rights of ways can take much, much longer, as Joe mentioned, than it takes to actually build the data center in itself. And so utilities are struggling to keep up with this demand, just in terms of getting the materials together and getting the stuff built at the same rapid pace as they did.
Consumer Power Trends and Data Center Efficiency
This is really an amazing increase here. So on this graph, we’re about here at 2024, and you can see the consumer power consumption is actually shrinking a little bit as I guess as we get more efficient appliances, more LED lights. But that’s, of course, greatly offset by data centers here and networks.
That’s right. We have in there a chart of what they call PUE, which is the data center efficiency. So looking at the total facility energy of the data center versus the IT equipment. Now, the IT equipment includes both the servers and the networking equipment to join them all together. But you can see that there are numbers on average 1.8.
So you can see a typical graph there, the PUE, truly a tremendous amount of power being consumed, about a third of it to half of it in very efficient, more modern systems. Data centers, you’re looking at half of that load being taken up by the IT equipment. But that giant, massive side over there for the HVAC systems to keep these data centers from overheating and keep them in the correct not just temperature bands, but humidity bands. A truly shocking amount of power.
HVAC Systems and Power Quality Vulnerability
And that’s an interesting mix also because the servers themselves may be fairly well-behaved loads from a utility standpoint and are likely backed up with UPSs, but the HVAC system is often more sensitive to power quality issues with variable frequency drives driving the chillers and the traditional types of loads utilities have challenges with in terms of voltage sags and other types of power quality issues that the servers themselves may be insulated from with power conditioning and battery-backed UPSs. So from a power quality standpoint, the most vulnerable aspects of the data center is likely to be the HVAC system rather than the servers themselves.
LLM Power Consumption
Also, some of these projections we have from various sources here were before the rise of these LLM technologies. We saw some estimates. I had trouble finding a good scholarly source for power consumption of LLM queries against more traditional online searches. Some were saying they estimate somewhere three to four times as much power consumption per LLM query versus a traditional Google search. But obviously, those companies are keeping those numbers internal. These are estimates, which is why they didn’t go into the paper.
But still, at even triple the power usage is tremendous. You look at how much consumption we’re already anticipating, no small part of that being training these new, very complicated models. We did find something in there saying estimated 1.3 gigawatt hours consumed training the GPT-3 model. And of course, as we know, that one’s obsolete now. That’s obsolete now, and a much, much larger model has taken its place, and a much, much larger model will be following it, I’m sure. And that’s just one company providing these services.
It’s really amazing how much power these large language models consume in the training and just their operation, and this is really getting started. We are just beginning to see this. Only a few companies have really kicked off their LLMs, and a number of them, if you spend any time looking at startups, the number that are suddenly saying, “Hey, we’re an AI LLM model backed” startup, it’s truly incredible.
The CEBEM Curve
Let me transition a little bit to some power quality issues as when it comes to data centers that Joe’s talked of. You hear about the energy consumption of these, and that’s really the main thrust of the paper, the challenge dealing with that for utilities, but there are also some power quality aspects. As Joe said, there are really two types of loads in these data centers. There’s the servers themselves, which are often kinda insulated from the utility through UPSs, and so from the utility standpoint, the UPS may be the load. And then there’s the HVAC system, the cooling system and lighting.
Now if we look at the power quality standards and the types of consumption these systems are concerned with, let’s switch over to a separate series of slides I have. First on the CEBEM curve. The CEBEM curve is the standard used by electronic equipment manufacturers, in particular PC server power supply manufacturers, for guidelines for how rugged their power supplies need to be.
This was originally developed in the 1970s, and it’s a type of standard that quantifies a voltage disturbance in terms of how long it lasted, its duration, and its magnitude. And it divided the space where these events could fall on into three regions. The no interruption function region, where the device should work no matter what. In this region, the region where that’s not prohibited, but you might have an interruption function. The computer may reset, or in this area is something where you might have equipment damage.
The ITIC Curve
Now this was replaced a little bit later by the ITIC curve. They renamed themselves from CEBEM to ITI, Information Technology Institute, and they replaced these continuous curves with straight lines with discrete break points to make this a bit easier for equipment manufacturers to understand.
So the modern curve, again, has the concept that a voltage event is defined by its duration, ranging from microseconds to seconds to all the way to steady state, and the percent voltage change. 100% is a normal voltage, the nominal, and then we can go down to 0% or 500%. So if a voltage event falls in this region, that’s acceptable. The PC or the server should be able to take that without any interruption in function. If you have a voltage sag in this region, it shouldn’t damage the equipment, but it may not operate correctly. It might have a computer that resets. If you have an overvoltage in this region, that could damage something.
So this gives guidelines for these server manufacturers as to how much surge suppression to put in their power supplies, how rugged they need to be on the top end here, and how much energy storage they need in terms of their DC buses, how much capacitance, how much ride-through they need to ride through voltage sags. And this is not a requirement. It’s kind of minimum recommended practice.
UPS Types and Double Conversion
Here is actually a link to a very nice paper that describes in more detail how this works. But again, in many cases in large data centers, the UPS is what the utility will see, but the UPS passes the utility voltage along unless it’s the double conversion type. A double conversion UPS takes the utility voltage, charges its own battery, and supplies a synthetic AC voltage at all times to the server itself. And the better designed UPSs are double conversion, but there’s some fine print with that.
Some UPSs, in an attempt to maximize efficiency, have a bypass mode where it’s not truly double conversion unless it senses a disturbance in the voltage. And if we go back to the paper, there’s some significant energy savings to be had here. Here, this 9% of the consumption is losses in the UPS. That’s usually due to the double conversion UPS. It’s not quite as efficient for it to always be synthesizing the AC voltage and having the power flow through that.
So to save some of this 9%, some UPSs have that bypass mode, so it’s not truly in double conversion mode even though it may be a double conversion UPS. So there’s that footnote there that even though in theory a double conversion UPS should insulate the servers from power quality disturbances, that may not be the case depending on the mode of the UPS itself.
ITIC Curve Compatibility with Utility Systems
Now one problem with this ITIC curve is the fact that it’s difficult or impossible for utilities to actually adhere to these guidelines. If we have a voltage sag say down to zero volts, that’s allowed up to 20 milliseconds. But that’s not a very long time. Utilities often have faults that are based on recloser times that are in the several cycle range. For example, three cycles is gonna put you over here.
So if you have three-phase or single-phase reclosers on your system, you’re often gonna have events clustered right around here or up here on the higher side with voltage swells from single-phase reclosers and single-phase faults. So unfortunately, this curve isn’t really compatible with most distribution systems. If the curve had been just a little bit higher here and a little bit longer here, that would have greatly increased compatibility, but that’s not the case. So that’s gonna be a challenge for utilities to technically meet this curve just given the faults that happen on most systems and how the reclosers are programmed.
Cryptocurrency Mining as a Server Load Example
Now here’s an example of a server type load. Here we have a cryptocurrency mining rig. This is actually a residential customer that has a three-phase 150 kVA transformer installed, and the only thing running here are banks of servers. Now these servers are generally ENERGY STAR compliant, which means that from a power consumption standpoint, they’re fairly friendly electronic loads. There are strict limits on maximum power factor and current distortion for server power supplies.
So you have waveforms like this where the current is in blue, voltage in red. A low current THD, 3%, very good power factor. So from a waveform standpoint, these servers are fairly well-behaved. They look like resistive loads.
Flat Power Profile and Transformer Sizing
The challenge for utilities is the flat power profile. This is 150 kVA transformer, and we can see on the graph here, it worked about 145 kilowatts 24 hours a day. And as Joe’s paper mentioned, that flat power consumption is very common for these large data centers. There’s no real daily profile like there is for pretty much all traditional utility loads.
So if you’re sizing a transformer for one of these, you need to really size that for that different load pattern. The fact that it’s gonna be basically at capacity or full load or the nominal load 24 hours a day for decades. So it’s not gonna have the normal lifetime, which is based on a traditional 24-hour power profile. So you wanna definitely take that in mind when designing a transformer.
And also from an energy planning standpoint, if you’re looking at loading an entire feeder, that’s going to be, in some respects, a nice load in terms of good power factor and low distortion, but it’s not gonna have the same daily profile all the other loads are gonna have.
Efficiency Gains Versus Demand Growth
And as we mentioned in the paper here, just because equipment is getting more efficient and you’re able to do more computing with less power, that would be driving down the load, but the expansion and demand for that computing is escalating significantly faster than the gains in efficiency or the gains in processing power per electrical power consumed.
That’s a real effect. You can bet that if you have a new data center going up, before that transformer’s life is expired, all those servers will have been replaced multiple times with even more computing power in the same footprint. That’s right. Many, many times more computing power.
Questions and Answers
Transit Characteristics of Data Center Loads
Good question. Have we done any research on the transit characteristics of these loads? Not specifically on that. From my understanding at least, there aren’t, in terms of the energy consumption, really that many energy or current spikes in these types of loads. They would be more associated with the HVAC system.
THD and Mitigation Techniques
Do we have any examples of a typical THD associated with large-scale data center loads, 100-plus megawatts, and mitigation techniques by the utility or the customer? And surprisingly, the current distortion from these loads is not as bad as you might think, mostly due to those ENERGY STAR compliant PCs and power supplies. It’s mostly the HVAC system that creates that kind of current distortion, and that really depends on the nature of the chillers. You’ll have the typical variable frequency drive, fifth and seventh harmonics present there. And then for those, the traditional mitigation techniques of line reactors and series with the VFDs or harmonic filters come into play.
Power Factor
Is there an average power factor that these data centers have? The power factor is usually not bad. Again, the servers themselves generally have a near perfect unity power factor, and power factor usually isn’t a problem for VFD-driven chillers either.
Conditions That May Cause Loads to Trip Offline
Your question, types of conditions that may cause these loads to trip offline. Well, that really depends a lot on the strategy behind the UPSs that are driving these systems. That’s something the utility really should work with these data centers before they go in, looking at the sag profile of that feeder in the planning stage so that the UPSs can be configured correctly and have enough ride through to mitigate that.
And you definitely want a power quality profile or power quality recording or two of that circuit during the planning stage to see what the sag profile looks like, maybe look at your SARFI index to determine how frequent sags are on that circuit.
Cooling Load and Energy Density
And another good comment, cooling load is too excessive. Does that seem reasonable? Well, that’s because these servers are somewhat unreasonable in terms of energy density per cubic inch or compute power per cubic inch. It’s amazing how many CPUs they can fit in a 1U rack space these days. And so the energy, the compute density is incredibly high on these servers. So one 19-inch rack is many, many kilowatts and just getting worse and worse. There’s a huge amount of power consumed on a small footprint, which requires an enormous amount of cooling to get that heat out of the building.
Good question. Do we have any information on the energy density? For example, how much square footage or space would a 100-megawatt demand require? We don’t really have any specifics on that. This introduction paper is kind of an overview of the issue, so we don’t really have a lot of detail on those kind of numbers. Although we can tell you for these few states we have in the actual chart here, we sourced from SP Global. You can see it looks like Virginia has 49 and a half million square feet of data centers consuming 4.9 gigawatts. Yeah, so that gives you a rough idea of watts per square inch.
Why Data Centers Are Concentrated in Virginia
A good question. Are there any insights as to why data centers are going crazy in Virginia? A lot of that is because Northern Virginia is home to a lot of government agencies. It’s very close to Washington, D.C. So a lot of intelligence agencies, a lot of military stuff up there. A lot of early network infrastructure went in up there. Tremendously good fiber backbone through the area.
And that’s a good point that Joe mentioned in his paper. The location of these data centers is not kind of evenly distributed around the country. They tend to cluster in areas of the country where there’s good access to power and also good networking. They tend to draw each other together. And that’s one reason why that Northern Virginia corridor is so rich in data centers. There’s a really strong networking backbone there. That’s probably the heart of the internet with all the military up there, the Pentagon and all the intelligence agencies up there driving a lot of that.
Roughly a decade ago, they claimed, when they were opening, cutting the ribbon on a data center nearby, they claimed that half of the world’s internet traffic was going through Ashburn, Virginia. Now, it’s of course different now, a decade later, as significantly more of Asia and Africa has come online, but what an astronomical amount of data moving through. It’s no longer half, but I’m sure it’s a greater quantity.
Flicker Issues
Another good question. Any examples of flicker issues associated with a typical data center cryptocurrency mining? Not really, at least not from the server standpoint. The servers tend to be fairly well-behaved loads from a utility standpoint. Any sort of flicker would be, again, from the HVAC system, especially if there’s compressors that have a bad starting capacitor or other types of issues.
So from a load standpoint, the HVAC system is where you might run into trouble, or blower motors for fans, if you’ve got hard start motors for those blower motors or a bad starting capacitor or other issues like that because as you can see, even just the fans, that’s a lot of blower motors. 16% of many tens or a hundred megawatts is a lot of air movement.
Now, my parents would agree with you. The air conditioning is excessive from their house. About a half mile away, there’s a data center, and every time the air conditioner kicks on, you can hear it from inside the house.
Closing
Well, thanks everyone for those great questions. And again, if you like to chat about power quality later, just give us a call anytime at 1-800-296-4120 or send an email to support@powermonitors.com. Thanks for attending everyone. Have a great afternoon.