Mission-Critical Facilities: The Next Generation
By Raj Gupta September 2002 - Data Centers
The severe downturn of the stock market and the lack of capital available to fund technology-related buildings have temporarily slowed the fast-paced development of data centers, telecom hotels and other mission-critical facilities. During this lull, it is important for building owners to examine what was learned when speed to market and system reliability requirements were the mantras and construction budgets held a secondary position. Because the next generation of mission-critical facilities will require even more tenant flexibility and have to clear higher economic hurdles, the buildings must have increasingly flexible, cost-effective and energy-saving support systems.
The owners and developers of the next generation of mission-critical buildings will face many new obstacles. In addition to being more reluctant to take the risk of increasing their equity participation or finding financing more difficult, they will have to tackle a poorer quality of credit for a typical tenant because of the stock market’s demise and deteriorating business fundamentals. What’s more, most electric utilities are not equipped to extend transmission and distribution networks to accommodate the power requirements of a telecom hotel or more energy-intensive mission-critical buildings. Finally, utilities have begun adding significant surcharges to deliver power to facilities where electric demand exceeds that of a typical office building, 5 to 10 watts per square foot.
Add to these factors the need to be close to fiber and the requirement for building flexibility to accommodate multiple uses, and the building owner faces a very different array of obstacles than in the past. The most important is the need to insist that engineers completely re-evaluate proposed mechanical and electrical systems to ensure that the design provides reliable and flexible systems while minimizing the financial impact. The starting point is to debunk the myth that all mission-critical facilities need 200 watts per square foot from day one.
Mission-critical Building Basics Defined
Many terms describe mission-critical facilities, including telecom hotels, carrier hotels, carrier hubs, co-location centers, co-hotels, data centers, cyber centers or server farms. Equally important, a traditional building may require space to act like a mission-critical facility as more and more businesses take on the characteristics of these facilities. Characteristics include housing computer servers for internet service providers (ISPs) and the switching devices required to route voice and data traffic between local and long-distance carriers.
Because the servers and routers housed in the mission-critical building may connect thousands of customers, a momentary service disruption can result in the loss of millions of dollars of revenue, for which the owner could be liable. Thus, building owners and occupants of such buildings must spend hundreds of dollars per square foot to ensure the reliability of the mechanical and electrical infrastructure. But how much is enough?
The power from electric utilities is generally very good, and it is not unusual to have an average availability of 99.98 percent across the utility’s transmission and distribution network. However, this high level of availability still means that power is not available for nearly two hours per year. Although this level of interruption is acceptable for most residences and commercial customers, it is unacceptable for the operations of a high-tech building.
The mechanical and electrical infrastructure of such facilities is often designed to provide six sigma of availability (99.9999 percent). Although this level of availability limits downtime to about thirty seconds per year, it is very costly to achieve. The mechanical and electrical infrastructure price tag can exceed $400 per square foot.
The typical electrical infrastructure for mission-critical facilities will contain two separate feeds from the electric utility, which are backed up by several diesel generators. The generators are commonly found in a parallel, redundant configuration where one extra module is provided to allow for planned maintenance and prevent a single point of failure in the system (N+1). For example, if the critical load for a facility requires 4 million volt-amps (MVA) of power, the N+1 redundancy can be achieved with three 2-MVA diesel generators.
The owner should require a complete engineering analysis to ensure there is no single point of failure with the mechanical and electrical infrastructure. To ensure the six sigma of availability, it is not unusual to see generators, chillers, pumps and other equipment configured for N+1 or N+2 redundancy. Also, the electrical distribution may be configured in a 2N arrangement for the UPS, PDUs and branch panels and circuiting that are closer to the actual critical load, the servers and switches.
This hardened infrastructure costs millions of dollars and can consume a considerable amount of space. In fact, it is not unusual to have a facility where the space for the mechanical and electrical infrastructure is nearly equal to the space for the servers and switches. Thus, it is important to encourage engineers to think critically to minimize the cost to provide the desired level of availability and maximize revenue-generating space.
The enormous cost to harden the number of units needed to support a load plus one additional unit facility is compounded by the charges from electric utilities to bring the large load densities to the site. Generally, the local utility is obligated to provide the transmission and distribution infrastructure to a building, and the cost for the infrastructure is included in the utility’s electric rate. For a typical commercial building, this translates into the electric utility bringing about 5 to 10 watts per square foot to the building.
How Many Watts Are Really Needed?
In truth, electric utilities were caught somewhat off guard by the telecom hotel boom and the request for 200 watts per square foot of power. Most transmission and distribution networks do not have the spare capacity to handle 200 watts per square foot. Moreover, many utilities require prior approval from the local regulating authority before system upgrades can occur. Thus, utilities are hard pressed to upgrade systems in a time frame acceptable to developers, owners and tenants of mission-critical facilities.
System upgrades, coupled with the costs to harden the facility and the costs from the local utility and potential schedule delays, place the owner in a difficult position. How can reliability be ensured, costs contained and schedules met?
The first generation of mission-critical facilities demonstrated that the buildings may require 200 watts per square foot, but that it would take a minimum of several months for the load to materialize. That is the time that owners must “buy” from the utility. In fact, it is because the need isn’t required immediately that an opportunity exists for the electrical and mechanical infrastructure to ramp-up over a period of time.
Making Deals, Debunking Myths
One recent project where this problem was confronted involved having the infrastructure built to handle high-volume electronic transactions. It became important to demonstrate the difference between the tenant’s current data centers with what the new facility would require in terms of power to the utility. The utility wanted to charge $800,000 in upfront fees to the tenant to supply the power requested. This upfront fee was actually a shift in utility positioning from the past, and was basically made by the utility because it did not believe that 200 watts would be needed when the facility first opened. They rightly believed that they should not bear the costs to bring that much power to a location if the location was not going to really use it.
The tenant wanted an infrastructure capable of handling 200 watts per square foot and was not happy to hear about the $800,000 upgrade charge and the impact on the project schedule. An engineering review of the current data center loads was conducted to reach an agreement with the utility regarding what to base the standard service on in terms of load. The analysis indicated the tenant would consume a minimum of 92 watts per square foot on the day the facility opened — a far cry from 200. It was agreed that the utility would upgrade its service over a period of several months to accommodate the new high-density servers the tenant intended to deploy for the 200-watt density. This compromise resulted in a 90 percent reduction in the fees charged by the utilities and a project that remained on its fast-track schedule.
Many owners with a knowledge of utility rates can accomplish the same results. A degree of rationality must take place when dealing with power requests, and the building owner must insist on it with engineers.
As development and occupancy phase in, usage grows incrementally. A considerable margin of the electrical capacity reserved upfront will not be used until later. Consider, then, negotiating a time frame for increasing power usage.
On-site power generation is another strategy that building owners should consider for the next generation of mission-critical buildings. In some locations, the electrical transmission and distribution network can be too costly to upgrade even if the ramp-up strategy is utilized.
In the first generation of mission-critical facilities, it was not unusual for each tenant to demand its own dedicated mechanical and electrical infrastructure. However, since these tenants no longer have the available capital, they will now consider shared support systems.
A modular cogeneration plant can satisfy the flexibility requirements for the next generation of mission-critical buildings. A gas turbine drives an electric generator. A heat recovery steam generator utilizes waste heat from the turbine to produce steam. This high-pressure steam can be used to produce electricity or chilled water. This approach has several advantages:
- Reduced initial capital outlays. That helps financing. The capital outlay from the owner and tenant can be increased over time as the actual load materializes. Moreover, the central plant can usually be smaller because it can take advantage of the collective diversity of each tenant.
- Increased reliability. A centralized system with the proper redundancy can be more reliable than having a dedicated system for each tenant. What’s more, if an engine generator is utilized as the prime mover, it can be equipped with dual-fuel capabilities (gas or oil); that not only makes for a more reliable system but also helps reduce operating costs because of gyrating fuel costs.
- Improved flexibility. The modular design allows for easy expansion.
- Faster delivery. The cogeneration plant may help accelerate the project schedule because the project does not need to wait for the electric utility.
In an era where change happens daily, owners must have their engineering firms implement the most efficient and flexible infrastructure solutions available. At the same time, building owners must work to control the complexity, costs and solutions associated with today’s mission-critical buildings. The challenge is for owners to encourage innovation and service while allowing engineering solutions that work for today’s high-tech tenants, while providing flexibility to support shifting, unpredictable future tenant needs. The first step is developing an awareness of the true needs surrounding the new generation of mission-critical facilities.
Raj Gupta is president of Environmental Systems Design, Inc., a mechanical, electrical, plumbing, fire protection and communication technology engineering firm.
Understanding the language of redundancy
“N” — The number of components needed to support a load
“N+1” — The number of components needed to support a load plus one additional unit
“N+2” — The number of components needed to support a load plus two additional units
“2N” — Two complete systems, each capable of supporting the load, with a dual-path distribution system
Fundamentals of a typical Internet facility
- 14- to 16-foot ceiling heights
- 200- to 300-pounds-per-square-foot floor loading capabilities to accommodate racks, equipment and cable trays
- Proximity and connectivity to multiple providers of fiber optic cables
- Extensive electrical power densities (up to 200 watts per square foot) with full redundancy on the distribution system
- Backup power (critical and essential loads)
- Emergency power (life safety loads)
- Fuel oil tanks to supply 24 to 72 hours of backup power
- Flexible, continuous HVAC (24/7) with built-in redundancies (N+1)
- High level of security
- Large, unobstructed and contiguous vertical and horizontal pathways for fiber, copper, electrical and mechanical conduit and piping
- A stable indoor environment with respect to temperature (72 degrees), humidity (50 percent relative humidity) and good indoor-air quality
- Clean-agent fire suppression and pre-action sprinkler systems
- Backup domestic water supply