Essential Cooling System Requirements
Introduction
Despite revolutionary changes in IT technology and products over the past decades, the design of cooling infrastructure for Datacenters has changed very little since 1965. Although IT equipment has always required cooling, the requirements of today’s IT systems, combined with the way that those IT systems are deployed, has created new cooling-related problems which were not foreseen when the cooling principles for the modern Datacenter were developed over 30 years ago. In this paper, a systematic approach of identifying and classifying user problems provides insight regarding the nature and characteristics of cooling systems in next-generation mission critical installations.
This paper focuses on the problem of removing power in the form of heat from the mission-critical installation. A related APC white paper #4, “Essential Power System Requirements for Next Generation Datacenters,” addresses the related problems of providing power.
Survey
A survey of management personnel relating to mission critical installations was conducted, interviewing corporate CIOs, facility managers, and IT managers. Over 90 people were interviewed from over 50 different organizations including Fortune 1000 corporations and companies in the government, education and service-provider sectors. Approximately 50% of customers interviewed were from North America, 20% from Europe, and 30% from the Japan, Pacific, Australia and Asia (JPAA) region.
The nine-month survey utilized “Voice of the Customer” techniques, which relies on data collection of verbal and/or written responses to open-ended questions. This provides extremely unstructured responses, with the advantage that the responses are not limited or constricted by preconceptions within the question. During the course of the survey, some of the questions were expanded and/or changed in order to clarify ambiguous responses.
Results: Cooling-system challenges in mission-critical
installations
Survey responses were grouped according to common concepts, and for each group a solution requirement, corresponding to a challenge for mission-critical installation design, was derived. This process identified 23 core challenges. These core challenges were then further grouped according to theme into the following five key theme areas:
- Adaptability / Scalability
- Availability
- Lifecycle Costs
- Maintenance / Serviceability
- Manageability
For each theme area, the challenge, underlying problem and power system requirements are presented in tabular form. The highest priority problems are listed first under each theme, and were identified by priority as determined by number of mentions combined priority expressed by the respondents.
The survey found adaptability challenges were the most important requirement, and were particularly focused on problems involving the cooling of high-density rack systems and uncertainty as to the quantity, timing and location of high-density racks. This is complicated by IT refreshes in the Datacenter or network room that typically occur every 1.5 to 2.5 years. This is discussed in detail in APC White Paper #29, “Rack Powering Options for Datacenters and Network Rooms.”
The survey showed that customers are often unable to predict if their cooling system will supply a future load, even when the characteristics of the load are known in advance.
Survey respondents universally identified frustration with the ability to assure required input temperature and airflow to all IT equipment in the Datacenter or network room, even when the load is not changing. Respondents showed very low confidence in the ability of any redundant cooling features in their Datacenters to perform.
The survey found the lifecycle cost challenges were of less concern than adaptability and availability requirements.
The cooling system requirements to meet the lifecycle cost challenges share many features in common with the solution requirements for adaptability. In particular, pre-engineered, standardized and modular solutions are needed.
A common theme among the serviceability challenges is a belief by the respondents that cooling equipment could be designed to make it much easier to service.
The manageability solution requirements are expensive to design, install and test in uniquely engineered systems. These challenges clearly suggest the need for pre-engineered, pre-tested and standardized management tools.
Survey respondents showed a lack of awareness regarding the time-varying power consumption of the newest generation of IT equipment, a characteristic which will give rise to time-varying heat outputs. Therefore managing this issue did not emerge as a challenge. Nevertheless, this issue is expected to emerge as a key manageability challenge in the near future and is discussed in detail in APC White Paper #43, “Dynamic Power Variations in the Datacenter and Network Room.”
Contrast between power and cooling challenges
A related study of power challenges in mission critical installations reveals that 13 of the cooling challenges are in common with powering challenges, four are closely related, and six of the cooling challenges are completely distinct from the powering challenges.
The common power and cooling challenges share the theme of adaptability, particularly to the unpredictable and changing requirements of IT equipment that is added or swapped out during the life of the Datacenter. Modular, scalable, pre-engineered, and standardized systems are the common solution for these problems. A number of concerns regarding service were also common between power and cooling needs.
The needs relating to system management were similar.
The biggest differences between power and cooling challenges relate to performance and cost concerns. The respondents clearly indicated that they were very concerned with the performance and availability of cooling systems. There was a clear pattern: customers did not believe their existing cooling systems were operating as intended, and most were uncertain as to whether their planned cooling redundancy, if any, would actually function during a fault condition. Performance issues were a greater concern than lifecycle cost for cooling users. This is a striking contrast to the power survey, in which lifecycle cost issues were prioritized higher suggesting a greater satisfaction with the performance of power systems.
Cooling systems for mission-critical installations
To satisfy the mission-critical installation cooling challenges identified in this survey, there are a number of changes required from current design practice. Many of these changes will require changes in the technology and design of cooling equipment, and how it is specified. Integration of the components of the cooling subsystem, particularly the air distribution and return systems, must move away from the current practice of unique system designs and toward pre-engineered and even pre-manufactured solutions. Such onscreen solutions would ideally be modular and standardized, expandable at will, and would ship complete but in parts that would rapidly plug together on-site. Standardization will facilitate the learning process. By spreading the cost of developing high performance management systems across large numbers of standardized installations, coordinated management would be affordable to all customers.
One suggested solution to the problem of cooling for high density racks is direct water cooling to the rack. IT equipment itself may be water cooled, or heat exchangers in the rack may be used. Advances in air distribution systems will affect the power level at which this approach is preferred. For isolated racks above 10kW or large groups of racks above 6kW, water-cooling currently appears to have benefits. However, this represents a very small fraction of racks today. Therefore, direct water-cooling to the rack is not expected to become a mainstream solution for the foreseeable future, and few references to it were found in the survey data.
Conclusions
A systematic analysis of customer problems relating to Datacenter and network room cooling systems provides a clear statement of direction for mission critical installations. The most pressing problems that are not solved by current design practices and equipment have the common theme of the inability of the Datacenter to adapt to change. Datacenter cooling systems must be more adaptable to changing requirements, in order to improve both availability and cost-effectiveness.
Cooling users are not confident that their current and planned systems will be able to cool high-density racks. This is primarily a problem relating to air distribution and mixing. Datacenter cooling systems must provide the capability for greater control of airflow at the rack.
In many industries, a maturity level is reached where new advances in reliability, cycle time and cost require standardization, pre-engineering and modularization. Designers of mission-critical installations, designers of the cooling equipment used in them, and owners should consider whether this point has been reached. The results of the survey in this paper suggest the need for a new generation of adaptable cooling systems for mission-critical installations.
References
-
1) FIPS Pub. #94, “Guideline for Computer Power for ADP Installations.”
National Technical Information Service
