As artificial intelligence workloads push rack densities beyond 40 kW and approaching 130 kW for systems like NVIDIA’s GB200 NVL72, traditional perimeter cooling fails to provide adequate thermal management. Modern AI data centers require targeted cooling strategies that position heat removal equipment closer to heat sources, creating more efficient and effective thermal management systems.
The shift toward high-density computing has fundamentally changed data center cooling requirements, demanding solutions that can handle concentrated heat loads while maintaining energy efficiency and equipment reliability.
What Is In-Row Cooling for Data Centers?
In-row cooling is a targeted thermal management approach that places cooling units directly within server rack rows, typically between every few racks or at row ends. These systems draw hot air from nearby equipment and deliver conditioned air precisely where needed, eliminating the inefficiencies of room-based cooling systems that must condition entire spaces.
Unlike traditional perimeter cooling that relies on long air paths and room-level temperature management, in-row systems create localized cooling zones. This targeted approach offers 30-40% higher energy efficiency compared to traditional perimeter cooling systems, making it essential for managing the thermal demands of modern AI workloads.
In-row systems integrate seamlessly with hot aisle and cold aisle containment strategies, creating controlled airflow patterns that prevent hot and cold air mixing while ensuring consistent temperatures across all equipment.
How Does In-Rack Cooling Differ from In-Row Systems?
In-rack cooling takes thermal management one step further by integrating cooling components directly within individual server racks or mounting cooling equipment on rack doors. While in-row systems serve multiple racks from strategic positions within the row, in-rack solutions provide dedicated cooling capacity for single racks or specific high-heat equipment.
The capacity differences are substantial. Air-cooled in-row systems can support densities up to 12 kW per rack, while in-rack liquid hybrid solutions can manage 100 kW or more per rack. This makes in-rack solutions particularly valuable for AI training systems where individual racks may consume 40-100+ kW or more.
Rear door heat exchangers represent a popular in-rack approach, mounting directly on rack doors to capture hot exhaust air before it enters the room environment. These systems can handle 30-75 kW per door, with active units reaching 100 kW capacity.
When Should You Choose In-Row Cooling Over Traditional Systems?
In-row cooling becomes the preferred solution when rack densities exceed 15-20 kW per rack or when equipment creates significant thermal variations across the data center floor. The global in-row air conditioner market reached USD 1,843 million in 2024, reflecting growing adoption driven by AI workload demands.
Traditional server racks typically consume 5-15 kW, well within the capacity of properly designed room-based cooling. However, AI-optimized server racks require 40-100+ kW, with cutting-edge AI training facilities pushing individual racks beyond 100 kW. NVIDIA’s GB200 NVL72 rack consumes approximately 130 kW per rack, according to Schneider Electric research from 2025.
Nearly 70% of new AI deployment plans incorporate in-row cooling solutions, recognizing that traditional cooling approaches cannot efficiently manage these concentrated heat loads. The decision often hinges on whether existing infrastructure can handle increased heat densities or requires targeted cooling intervention.
Key Decision Factors
- Rack density: Above 15-20 kW per rack favors in-row solutions
- Equipment type: AI accelerators and high-performance computing benefit most
- Facility constraints: Limited floor space or ceiling height may require in-row approaches
- Energy efficiency goals: In-row systems typically achieve better PUE metrics
- Scalability requirements: Modular in-row systems support phased expansion
What Is Liquid Cooling in Data Center Applications?
Liquid cooling in data centers involves circulating coolant directly to heat sources, either through closed-loop systems that connect to server components or through immersion systems that submerge equipment in dielectric fluids. Goldman Sachs estimates that 76% of AI servers deployed by the end of 2026 will be liquid-cooled.
Liquid cooling offers superior heat dissipation compared to air-based systems, reducing energy consumption by up to 40% compared to air-cooled systems. The thermal conductivity of liquid coolants exceeds air by orders of magnitude, allowing direct heat removal from high-temperature components like GPUs used in AI workloads.
In-row coolant distribution units (CDUs) serve as the interface between facility chilled water systems and rack-level liquid cooling loops. These CDUs offer up to 40% better energy efficiency compared to traditional air cooling systems while supporting the extreme heat densities generated by modern AI accelerators.
Liquid Cooling Implementation Approaches
- Direct-to-chip: Coolant loops connect directly to CPU and GPU cold plates
- Immersion cooling: Servers operate submerged in dielectric coolant
- Hybrid systems: Combine liquid cooling for high-heat components with air cooling for ancillary equipment
- Rear door heat exchangers: Liquid-cooled heat exchangers mounted on rack doors
Energy Efficiency Benefits of Targeted Cooling Systems
In-row and in-rack cooling systems significantly improve data center energy efficiency by reducing the parasitic power consumption associated with cooling infrastructure. The industry’s weighted average Power Usage Effectiveness (PUE) remained stable at approximately 1.54 for the sixth consecutive year in 2025, according to the Uptime Institute.
Targeted cooling approaches can achieve PUE values between 1.05 and 1.15, particularly when combined with liquid cooling systems. This improvement stems from eliminating the energy waste associated with conditioning entire rooms when only specific areas require cooling.
The efficiency gains become more pronounced as rack densities increase. While cooling systems typically consume 30-40% of total data center power, well-designed in-row systems can reduce this percentage by focusing cooling energy precisely where needed rather than maintaining room-wide environmental conditions.
Implementation Challenges and Design Considerations
Implementing in-row and in-rack cooling requires careful planning around space constraints, power distribution, and integration with existing infrastructure. Unlike room-based systems that operate independently of IT equipment layout, targeted cooling systems must coordinate closely with server placement and rack configuration.
Refrigerant selection has become increasingly important due to EPA regulations under the AIM Act, which mandates an 85% phasedown of HFC production and consumption in the U.S. by 2036. Many facilities are transitioning from R-410A (GWP of 2088) to lower-impact alternatives like R-454B (GWP of 466).
ASHRAE guidelines provide thermal targets for different equipment classes. Most data center equipment (Classes A1-A4) operates effectively with inlet temperatures between 18°C and 27°C (64.4°F to 80.6°F). However, high-density computing systems like AI accelerators (Class H1) require tighter temperature control between 18°C and 22°C (64.4°F to 71.6°F).
Critical Design Elements
- Airflow management: Proper containment prevents bypass and recirculation
- Redundancy planning: N+1 or N+2 cooling unit configuration for high availability
- Monitoring integration: Real-time temperature and humidity tracking
- Maintenance access: Adequate space for service and component replacement
- Emergency protocols: Backup cooling during maintenance or failure events
Integration with Modular Data Center Designs
In-row and in-rack cooling systems align perfectly with modular edge data center concepts, offering scalable cooling that matches computing deployment patterns. These systems can be deployed incrementally as computing loads increase, avoiding the upfront capital investment required for full room-based cooling infrastructure.
Modular cooling approaches support distributed computing architectures where processing power must be positioned close to data sources or end users. Edge computing applications particularly benefit from the precision and efficiency of targeted cooling systems, which can operate effectively in space-constrained environments.
The projected growth of the in-row cooling market to USD 2,646 million by 2032, representing a 5.3% CAGR, reflects increasing adoption across both enterprise and edge computing segments. This growth is driven by the recognition that traditional cooling approaches cannot scale to meet the thermal demands of modern AI and high-performance computing workloads.
Frequently Asked Questions
What power densities can in-row cooling handle effectively?
Air-cooled in-row systems typically handle up to 12 kW per rack, while liquid-assisted in-row solutions can manage 30-50 kW per rack depending on configuration and environmental conditions.
How much energy does in-row cooling save compared to traditional systems?
In-row cooling systems offer 30-40% higher energy efficiency compared to perimeter cooling by eliminating conditioning of unused space and reducing air transport energy requirements.
Can existing data centers retrofit in-row cooling without major infrastructure changes?
Most existing facilities can accommodate in-row cooling units with moderate modifications to power distribution, chilled water piping, and floor layouts, though specific requirements vary by facility design.
What maintenance requirements do in-row cooling systems have?
In-row units require regular filter replacement, coil cleaning, and refrigerant system maintenance similar to traditional HVAC equipment, but positioned within the data center environment for easier access.
Do in-row cooling systems require special refrigerants due to environmental regulations?
Many facilities are transitioning to low-GWP refrigerants like R-454B to comply with EPA AIM Act requirements phasing down high-GWP substances like R-410A.
How does in-row cooling integrate with hot aisle containment systems?
In-row cooling units work optimally with containment systems, drawing hot air from contained hot aisles and delivering conditioned air to cold aisles, preventing air mixing and improving efficiency.
What backup cooling options exist if in-row units fail?
Redundant in-row unit deployment (N+1 or N+2), temporary portable cooling, or emergency room-based cooling systems can provide backup thermal management during equipment maintenance or failure events.
Are liquid-cooled in-row systems more complex than air-cooled versions?
Liquid-cooled systems require additional infrastructure for coolant distribution and monitoring but offer significantly higher cooling capacity, making them necessary for AI workloads exceeding 50 kW per rack.