Data center

The Telecommunications Industry Association's Telecommunications Infrastructure Standard for Data Centers[17] specifies the minimum requirements for telecommunications infrastructure of data centers and computer rooms including single tenant enterprise data centers and multi-tenant Internet hosting data centers. The topology proposed in this document is intended to be applicable to any size data center.[18]

Telcordia GR-3160, NEBS Requirements for Telecommunications Data Center Equipment and Spaces,[19] provides guidelines for data center spaces within telecommunications networks, and environmental requirements for the equipment intended for installation in those spaces. These criteria were developed jointly by Telcordia and industry representatives. They may be applied to data center spaces housing data processing or Information Technology (IT) equipment. The equipment may be used to:

• Operate and manage a carrier's telecommunication network
• Provide data center based applications directly to the carrier's customers
• Provide hosted applications for a third party to provide services to their customers
• Provide a combination of these and similar data center applications

Data center transformation takes a step-by-step approach through integrated projects carried out over time. This differs from a traditional method of data center upgrades that takes a serial and siloed approach.[20] The typical projects within a data center transformation initiative include standardization/consolidation, virtualization, automation and security.

• Standardization/consolidation: Reducing the number of data centers[21][22] and avoiding server sprawl[23] (both physical and virtual)[24] often includes replacing aging data center equipment,[25] and is aided by standardization.[26]
• Virtualization: Lowers capital and operational expenses,[27] reduces energy consumption.[28] Virtualized desktops can be hosted in data centers and rented out on a subscription basis.[29] Investment bank Lazard Capital Markets estimated in 2008 that 48 percent of enterprise operations will be virtualized by 2012. Gartner views virtualization as a catalyst for modernization.[30]
• Automating: Automating tasks such as provisioning, configuration, patching, release management, and compliance is needed, not just when facing fewer skilled IT workers.[26]
• Securing: Protection of virtual systems is integrated with the existing security of physical infrastructures.[31]

Perforated cooling floor tile.

A raised floor standards guide named GR-2930 was developed by Telcordia Technologies, a subsidiary of Ericsson.[32]

Although the first raised floor computer room was made by IBM in 1956,[33] and they've "been around since the 1960s",[34] it was the 1970s that made it more common for computer centers to thereby allow cool air to circulate more efficiently.[35][36]

The first purpose of the raised floor was to allow access for wiring.[33]

The "lights-out"[37] data center, also known as a darkened or a dark data center, is a data center that, ideally, has all but eliminated the need for direct access by personnel, except under extraordinary circumstances. Because of the lack of need for staff to enter the data center, it can be operated without lighting. All of the devices are accessed and managed by remote systems, with automation programs used to perform unattended operations. In addition to the energy savings, reduction in staffing costs and the ability to locate the site further from population centers, implementing a lights-out data center reduces the threat of malicious attacks upon the infrastructure.[38][39]

• Class 1 single path solution
• Class 2 single path with redundancy solution
• Class 3 multiple paths providing a concurrent repair/operate solution
• Class 4 multiple paths providing a fault tolerant solution (except during maintenance)

The Telecommunications Industry Association's TIA-942 standard for data centers, published in 2005 and updated four times since, defined four infrastructure levels.[40]

• Rated-1 - basically a server room, following basic guidelines
• Rated-2 - Redundant component, key components are redundant
• Rated-3 - Concurrently maintainabile, able to handle maintenance on any part of the distribution path or any single piece of equipment without causing an interruption to the data center operations
• Rated-4 - Fault tolerant, able to handle one single fault at a time on any part of the distribution path or any single piece of equipment without causing interruption to the data center operations

Four Tiers are defined by the Uptime Institute standard:

• Tier I - BASIC CAPACITY and must include a UPS (uninterruptible power source)
• Tier II - REDUNDANT CAPACITY and adds redundant power and cooling
• Tier III - CONCURRENTLY MAINTAINABLE and ensures that ANY component can be taken out of service without affecting production
• Tier IV - FAULT TOLERANT allowing any production capacity to be insulated from ANY type of failure.

A fifth tier has been Trademarked by Switch (company), who have used this tier to define The Citadel, the largest data center in the world. [41][42]

• Availability expectations: The costs of avoiding downtime should not exceed the cost of the downtime itself[48]
• Site selection: Location factors include proximity to power grids, telecommunications infrastructure, networking services, transportation lines and emergency services. Other considerations should include flight paths, neighboring power drains, geological risks, and climate (associated with cooling costs).[49]
  • Often, power availability is the hardest to change.

Various metrics exist for measuring the data-availability that results from data-center availability beyond 95% uptime, with the top of the scale counting how many "nines" can be placed after "99%".[50]

Modularity and flexibility are key elements in allowing for a data center to grow and change over time. Data center modules are pre-engineered, standardized building blocks that can be easily configured and moved as needed.[51]

A modular data center may consist of data center equipment contained within shipping containers or similar portable containers.[52] Components of the data center can be prefabricated and standardized which facilitates moving if needed.[53]

Temperature[note 7] and humidity are controlled via:

• Air conditioning
• indirect cooling, such as using outside air,[54][55][note 8] Indirect Evaporative Cooling (IDEC) units, and also using sea water.

It is important that computers do not get humid or overheat, as high humidity can lead to dust clogging the fans, which leads to overheat, or can cause components to malfunction, ruining the board and running a fire hazard. Overheat can cause components, usually the silicon or copper of the wires or circuits to melt, causing connections to loosen, causing fire hazards.

A bank of batteries in a large data center, used to provide power until diesel generators can start

Backup power consists of one or more uninterruptible power supplies, battery banks, and/or diesel / gas turbine generators.[56]

To prevent single points of failure, all elements of the electrical systems, including backup systems, are typically given redundant copies , and critical servers are connected to both the "A-side" and "B-side" power feeds. This arrangement is often made to achieve N+1 redundancy in the systems. Static transfer switches are sometimes used to ensure instantaneous switchover from one supply to the other in the event of a power failure.

Options include:

• Data cabling can be routed through overhead cable trays[57]
• Raised floor cabling, both for security reasons and to avoid the extra cost of cooling systems over the racks.
• Smaller/less expensive data centers may use anti-static tiles instead for a flooring surface.

Air flow management addresses the need to improve data center computer cooling efficiency by preventing the recirculation of hot air exhausted from IT equipment and reducing bypass airflow. There are several methods of separating hot and cold airstreams, such as hot/cold aisle containment and in-row cooling units.[58]

Cold aisle containment is done by exposing the rear of equipment racks, while the fronts of the servers are enclosed with doors and covers. This is similar to how large-scale food companies refrigerate and store their products.

Typical cold aisle configuration with server rack fronts facing each other and cold air distributed through the raised floor.

Computer cabinets/Server farms are often organized for containment of hot/cold aisles. Proper air duct placement prevents the cold and hot air from mixing. Rows of cabinets are paired to face each other so that the cool and hot air intakes and exhausts don't mix air, which would severely reduce cooling efficiency.

Alternatively, a range of underfloor panels can create efficient cold air pathways directed to the raised floor vented tiles. Either the cold aisle or the hot aisle can be contained.[59]

Another option is fitting cabinets with vertical exhaust ducts Chimney[60] Hot exhaust pipes/vents/ducts can direct the air into a Plenum space above a Dropped ceiling and back to the cooling units or to outside vents. With this configuration, traditional hot/cold aisle configuration is not a requirement.[61]

FM200 Fire Suppression Tanks

Data centers feature fire protection systems, including passive and Active Design elements, as well as implementation of fire prevention programs in operations. Smoke detectors are usually installed to provide early warning of a fire at its incipient stage.

Although the main room usually does not allow Wet Pipe-based Systems due to the fragile nature of Circuit-boards, there still exist systems that can be used in the rest of the facility or in cold/hot aisle air circulation systems that are closed systems, such as:[62]

• Sprinkler systems
• Misting, using high pressure to create extremely small water droplets, which can be used in sensitive rooms due to the nature of the droplets.

However, there also exist other means to put out fires, especially in Sensitive areas, usually using Gaseous fire suppression, of which Halon gas was the most popular, until the negative effects of producing and using it were discovered.

Physical access is usually restricted. Layered security often starts with fencing, bollards and mantraps.[63] Video camera surveillance and permanent security guards are almost always present if the data center is large or contains sensitive information. Fingerprint recognition mantraps is starting to be commonplace.

Logging access is required by some data protection regulations; some organizations tightly link this to access control systems. Multiple log entries can occur at the main entrance, entrances to internal rooms, and at equipment cabinets. Access control at cabinets can be integrated with intelligent power distribution units, so that locks are networked through the same appliance.[64]

In 2020 data centers (excluding cryptocurrency mining) and data transmission each used about 1% of world electricity.[67] Although some of this electricity was low carbon, the IEA called for more "government and industry efforts on energy efficiency, renewables procurement and RD&D",[67] as some data centers still use electricity generated by fossil fuels.[68] They also said that lifecycle emissions should be considered, that is including "embodied" emissions, such as in buildings.[67] Data centers are estimated to have been responsible for 0.5% of US greenhouse gas emissions in 2018.[69] Some Chinese companies, such as Tencent, have pledged to be carbon neutral by 2030, while others such as Alibaba have been criticized by Greenpeace for not committing to become carbon neutral.[70]

The most commonly used energy efficiency metric of data center energy efficiency is power usage effectiveness (PUE), calculated as the ratio of total power entering the data center divided by the power used by IT equipment.

${\displaystyle \mathrm {PUE} ={{\mbox{Total Facility Power}} \over {\mbox{IT Equipment Power}}}}$

It measures the percentage of power used by overhead (cooling, lighting, etc.). The average USA data center has a PUE of 2.0,[71] meaning two watts of total power (overhead + IT equipment) for every watt delivered to IT equipment. State-of-the-art is estimated to be roughly 1.2.[72] Google publishes quarterly efficiency from data centers in operation.[73]

The U.S. Environmental Protection Agency has an Energy Star rating for standalone or large data centers. To qualify for the ecolabel, a data center must be within the top quartile of energy efficiency of all reported facilities.[74] The Energy Efficiency Improvement Act of 2015 (United States) requires federal facilities — including data centers — to operate more efficiently. California's title 24 (2014) of the California Code of Regulations mandates that every newly constructed data center must have some form of airflow containment in place to optimize energy efficiency.

European Union also has a similar initiative: EU Code of Conduct for Data Centres.[75]

The focus of measuring and analyzing energy use goes beyond what's used by IT equipment; facility support hardware such as chillers and fans also use energy.[76]

In 2011 server racks in data centers were designed for more than 25 kW and the typical server was estimated to waste about 30% of the electricity it consumed. The energy demand for information storage systems was also rising. A high availability data center was estimated to have a 1 mega watt (MW) demand and consume $20,000,000 in electricity over its lifetime, with cooling representing 35% to 45% of the data center's total cost of ownership. Calculations showed that in two years the cost of powering and cooling a server could be equal to the cost of purchasing the server hardware.[77] Research in 2018 has shown that substantial amount of energy could still be conserved by optimizing IT refresh rates and increasing server utilization.[78]

In 2011 Facebook, Rackspace and others founded the Open Compute Project (OCP) to develop and publish open standards for greener data center computing technologies. As part of the project Facebook published the designs of its server, which it had built for its first dedicated data center in Prineville. Making servers taller left space for more effective heat sinks and enabled the use of fans that moved more air with less energy. By not buying commercial off-the-shelf servers, energy consumption due to unnecessary expansion slots on the motherboard and unneeded components, such as a graphics card, was also saved.[79] In 2016 Google joined the project and published the designs of its 48V DC shallow data center rack. This design had long been part of Google data centers. By eliminating the multiple transformers usually deployed in data centers, Google had achieved a 30% increase in energy efficiency.[80] In 2017 sales for data center hardware built to OCP designs topped $1.2 billion and are expected to reach $6 billion by 2021.[79]

Data center at CERN (2010)

Power is the largest recurring cost to the user of a data center.[81] Cooling it at or below 70 °F (21 °C) wastes money and energy.[81] Furthermore, overcooling equipment in environments with a high relative humidity can expose equipment to a high amount of moisture that facilitates the growth of salt deposits on conductive filaments in the circuitry.[82]

A power and cooling analysis, also referred to as a thermal assessment, measures the relative temperatures in specific areas as well as the capacity of the cooling systems to handle specific ambient temperatures.[83] A power and cooling analysis can help to identify hot spots, over-cooled areas that can handle greater power use density, the breakpoint of equipment loading, the effectiveness of a raised-floor strategy, and optimal equipment positioning (such as AC units) to balance temperatures across the data center. Power cooling density is a measure of how much square footage the center can cool at maximum capacity.[84] The cooling of data centers is the second largest power consumer after servers. The cooling energy varies from 10% of the total energy consumption in the most efficient data centers and goes up to 45% in standard air-cooled data centers.

An energy efficiency analysis measures the energy use of data center IT and facilities equipment. A typical energy efficiency analysis measures factors such as a data center's power use effectiveness (PUE) against industry standards, identifies mechanical and electrical sources of inefficiency, and identifies air-management metrics.[85] However, the limitation of most current metrics and approaches is that they do not include IT in the analysis. Case studies have shown that by addressing energy efficiency holistically in a data center, major efficiencies can be achieved that are not possible otherwise.[86]

This type of analysis uses sophisticated tools and techniques to understand the unique thermal conditions present in each data center—predicting the temperature, airflow, and pressure behavior of a data center to assess performance and energy consumption, using numerical modeling.[87] By predicting the effects of these environmental conditions, CFD analysis in the data center can be used to predict the impact of high-density racks mixed with low-density racks[88] and the onward impact on cooling resources, poor infrastructure management practices and AC failure or AC shutdown for scheduled maintenance.

Thermal zone mapping uses sensors and computer modeling to create a three-dimensional image of the hot and cool zones in a data center.[89]

This information can help to identify optimal positioning of data center equipment. For example, critical servers might be placed in a cool zone that is serviced by redundant AC units.

This water-cooled data center in the Port of Strasbourg, France claims the attribute green.

Data centers use a lot of power, consumed by two main usages: the power required to run the actual equipment and then the power required to cool the equipment. Power-efficiency reduces the first category.

Cooling cost reduction from natural ways includes location decisions: When the focus is not being near good fiber connectivity, power grid connections and people-concentrations to manage the equipment, a data center can be miles away from the users. 'Mass' data centers like Google or Facebook don't need to be near population centers. Arctic locations can use outside air, which provides cooling, are getting more popular.[90]

Renewable electricity sources are another plus. Thus countries with favorable conditions, such as: Canada,[91] Finland,[92] Sweden,[93] Norway,[94] and Switzerland,[95] are trying to attract cloud computing data centers.

Bitcoin mining is increasingly being seen as a potential way to build data centers at the site of renewable energy production. Curtailed and clipped energy can be used to secure transactions on the Bitcoin blockchain providing another revenue stream to renewable energy producers.[96]

It is very difficult to reuse the heat which comes from air cooled data centers. For this reason, data center infrastructures are more often equipped with heat pumps.[97] An alternative to heat pumps is the adoption of liquid cooling throughout a data center. Different liquid cooling techniques are mixed and matched to allow for a fully liquid cooled infrastructure which captures all heat in water. Different liquid technologies are categorized in 3 main groups, Indirect liquid cooling (water cooled racks), Direct liquid cooling (direct-to-chip cooling) and Total liquid cooling (complete immersion in liquid, see Server immersion cooling). This combination of technologies allows the creation of a thermal cascade as part of temperature chaining scenarios to create high temperature water outputs from the data center.

Data backup techniques include having an encrypted copy of the data offsite. Methods used for transporting data are:[105]

• having the customer write the data to a physical medium, such as magnetic tape, and then transporting the tape elsewhere.[106]
• directly transferring the data to another site during the backup, using appropriate links
• uploading the data "into the cloud"[107]