Key Takeaways:
Maintaining strict environmental control has been critical to data center performance and reliability since the 1950s, when early mainframe computers required tight temperature limits to prevent failures, especially in defense applications where downtime impacted critical operations.
By the 1960s and early 1970s, mainframes evolved from classified military systems into commercial platforms. Corporations adopted them to improve functions, including banking, finance and human resources, driving efficiency, cost savings and improved data processing.
As demand increased, advances in hardware, operating systems and solid-state electronics enabled smaller, denser systems with greater performance. But this miniaturization also raised heat density, creating a key constraint on further performance gains.
To address rising thermal challenges, early liquid cooling methods using water and refrigerants were introduced in specialized computing systems. As a leading manufacturer, IBM evaluated hybrid air and liquid cooling approaches and determined that liquid cooling significantly improved thermal management, enabling higher power densities, better performance and greater system reliability. Some legacy IBM mainframe installations even incorporated integrated water storage systems located next to the mainframe.
Today, while computing power has increased by orders of magnitude, the core challenge remains unchanged — how to cool high-density systems effectively, economically and reliably.
Integrated computer cabinets isolate cooling air from the data center by drawing it from an underfloor plenum, directing it across components and exhausting hot air into an overhead return plenum. This approach enables precise, efficient cooling with minimal air recirculation, but air-cooling capacity has practical limits.
At very high densities, on-board fans must move large air volumes, increasing power use and adding heat to the system. Lowering supply air temperature can help, but it raises compressor energy use in central cooling systems. Also, air cooling often leads to uneven airflow distribution, creating hot spots and potential component damage.
AI data centers primarily support training and inference workloads, with far higher power and cooling demands than traditional facilities. During training, systems run at sustained full capacity. During inference, loads fluctuate. Designs also must handle instant-on behavior, where power demand ramps to peak within seconds.
One key distinction is close collaboration with manufacturers of graphics processing units and tensor processing units, whose guidance shapes liquid cooling system design and provides uncommon access to technology roadmaps.
AI-driven compute densities require a shift in cooling strategy. Traditional facilities mainly rely on air cooling with hot-aisle containment. They typically operate at around 10kW per rack in enterprise environments and up to roughly 30kW per rack in hyperscale and cryptocurrency settings. But AI environments exceed these levels and demand hybrid air- and liquid-cooling approaches.
Air cooling remains necessary for some components — e.g., storage and networking — but integrating both methods adds complexity to data hall design, central plants and cooling distribution.
While legacy supercomputing designs do not directly translate to modern AI facilities, they established the foundation for liquid cooling. Early systems commonly used hybrid approaches — e.g., approximately 80 percent liquid and 20 percent air — with closed-loop heat exchange tied to chilled water plants.
AI data centers differ in one important way: They are modular and rapidly scalable. Power and cooling systems must follow a just-in-time deployment model to control capital costs and align with phased growth.
The rise of AI has driven purpose-built liquid cooling solutions, most notably cooling distribution units (CDU). These systems transfer heat between facility water and server-level coolant — e.g., cold plates — while integrating pumps, controls and redundancy into modular packages. CDU capacities now scale from approximately 100kW to multi-megawatt levels, matching AI cluster demands and enabling repeatable, scalable design.
This shift from custom, site-built cooling to standardized, modular, liquid cooling systems sets the stage for emerging industry guidance and best practices.
Over the past two decades, data centers have grown significantly in scale and power to support cloud and internet applications. While early guidance emphasized uptime and reliability, detailed HVAC and cooling design standards initially were limited.
This limitation shifted in the early 2000s as ASHRAE introduced more technical guidance, evolving to emphasize energy efficiency and sustainability. ASHRAE Standard 90.4, released in 2013, established code-ready requirements for energy use and cooling performance and is incorporated into broader code frameworks.
Other organizations also have shaped best practices. The Green Grid introduced metrics like PUE, the Open Compute Project advanced open infrastructure standards, and the U.S. Green Building Council adapted LEED frameworks for data centers.
With the rise of AI, these groups are expanding guidance to address ultra-high densities and liquid cooling. Standards continue to evolve, requiring ongoing collaboration to keep pace with rapidly changing demands.
Large AI data centers — so-called AI factories — are purpose-built for dedicated workloads, while smaller deployments often are integrated into multi-tenant, hyperscale facilities where base building systems are provided by the landlord. Air-cooled designs have well-defined handoffs, but liquid cooling introduces added complexity.
Start-up and commissioning are critical, particularly for the technical coolant system (TCS). Fluid cleanliness must meet strict particulate limits far beyond typical hydronic systems because debris can clog cold-plate microchannels and degrade heat transfer. Clear testing and commissioning protocols are essential to ensure performance.
In multi-tenant environments, the boundaries of responsibility are less defined. The CDU often is used as the demarcation point, with tenants responsible for TCS distribution. Regardless of ownership, TCS piping and connections must be rigorously tested during commissioning. While guidance exists, standardized industry protocols are evolving.
Liquid cooling offers clear advantages for high-density and AI-driven applications, but realizing its efficiency and sustainability benefits depends on the way systems are designed, integrated and operated. Key considerations include heat rejection, parasitic energy — e.g. pumping and fan power — heat recovery and alignment with performance metrics. These areas are increasingly addressed through evolving industry guidance.
At the same time, AI data center power and cooling requirements are developing, particularly for liquid cooling. Hybrid approaches that combine air and liquid systems will remain common, while growing availability of AI-specific equipment is enabling more modular and scalable designs. Continued industry collaboration and standards development will be essential to establish consistent practices for design, start-up and operation.
Bill Kosik, P.E., CEM, LEED AP, is mission critical sector leader at HED, an architectural and engineering firm. Kosik has over 20 years of experience in optimizing HVAC systems and liquid cooling for data centers. He has authored over 50 articles, presented at more than 50 industry events and contributed to initiatives with the U.S. Department of Energy, the U.S. Environmental Protection Agency, ASHRAE and The Green Grid to advance energy efficiency in the data center industry.
