4 FM quick reads on data centers
1. How To Avoid A "Frankenstein" UPS
To reduce the risk of getting a "Frankenstein" system of mismatched components in large UPS systems for data centers, it's important for facility managers to understand the importance of selecting a vendor with a substantial and experienced U.S. sales and service organization.
These concerns with buying complex systems from marginal players in the marketplace are well-understood among industry veterans. These concerns with UPS systems are not much different than with other complex data center support equipment technologies such as redundant standby generator systems and HVAC systems, especially central plants or HVAC systems with economizing features.
One additional key factor is not always well-understood. When an order is placed with a vendor for an integrated system of components, it is up to the vendor to pull together all the correct and compatible components and see that they arrive at the jobsite, at the same time, or are otherwise sequenced as required by the construction team. A large "single module" UPS system can require the main UPS box, boxes full of batteries, sometimes a separate battery disconnect box and often a separate maintenance bypass box, sometimes all shipping from different factories and often from different sub-vendors.
Without a strong technical sales application team, the system may not get properly represented and applied with appropriate accessories or it may not get delivered correctly in accordance with customer needs. Without a strong regional service organization, routine preventive maintenance and minor issues can lead to big problems, such as excessive planned or unplanned downtime or excessive repair time.
Imported UPS units are often matched with domestically designed and assembled battery and bypass packages. Sourcing these components is often left to the domestic sales organization, which is usually different in each marketplace. Often, a buyer thinks (or hopes) all of this equipment will be integrated together or even tested as a complete system at the "main factory or assembly plant." This is rarely the case, as it would add significant cost. Getting these different components to show up correctly at the jobsite is where the marginal players often fall down on the job.
Conduct Site Analysis Before Starting Construction On Data Centers
Critical facilities — including data centers, network operations centers, communications centers, command and control centers, emergency response sites, and public safety and law enforcement facilities — need to be sufficiently robust to remain in operation and survive under stress, whether caused by natural or human agents. Carefully developed parameters should be determined at project initiation to identify optimal site characteristics or vulnerabilities that cannot be fully mitigated.
More importantly, sites not meeting minimum criteria thresholds for disaster resilience in critical facilities use should be identified and eliminated from consideration as early as possible. After-the-fact measures to address deficiencies are most often extremely costly and still subject to failure under duress. A planning professional can guide the site evaluation team, including facilities planning and risk management personnel, in determining specific evaluation criteria based on operational mission and optimal site characteristics while identifying vulnerabilities for specific sites.
Public safety officials often speak of "incidents on top of incidents" as events that create unpredictable challenges to continuity of operations. If the facility is properly sited, and external risks are fully considered and, if necessary, mitigated, the facility will be in a better position to respond to such challenges.
A site risk assessment should not be considered a static event, but an ongoing process of continual assessment and reassessment of systems and measures in place to reflect the changing nature of asset characterization, operational mission, and threat and risk environment. From initiation of the planning process, project teams often embrace the concept of "last building standing;" that is, the concept that a critical facility must be rigorously planned, designed and maintained to resist external and internal threats, survive with complete functionality, and remain fully operational. Chances of success are greatly enhanced with selection of a secure site based both on thoroughly developed operational and security criteria and on a location where the facility can survive threats and effectively fulfill its mission.
Hurricane Sandy Shows Value of Co-Location, Redundancy, Cloud Services
In the wake of Hurricane Sandy, many date centers are looking at new options to ensure resiliency. Facilities are considering the pros and cons of cloud services or co-location vs. the use of add-on "containers" to the data center. While using co-location or cloud sites is less expensive than owning and operating a data center, the downside is that, in the event of a catastrophe, an outside provider does not take responsibility for the costs of business interruptions if the site becomes inoperable. Co-location sites also present security risks, where data can be exposed or stolen.
A solution that has worked for some businesses is to have redundant facilities. One option is for organizations to have their own redundant facilities. A customer can also use multiple multitenant data providers or one provider with multiple facilities in different locations.
It is possible to deploy IT in two facilities where data is being constantly copied, every hour or so. It's not uncommon to have an East Coast/West Coast configuration, so that if the data center goes offline, data moves to the secondary site.
Cloud technology allows for virtualization, which means that multiple operating systems can be used on one server and applications can be moved to another server anywhere in the world if it becomes known that a storm is on the way. But not every application can be virtualized.
Other steps can also be taken to prevent future loss of services. Buildings could be made more resilient, for example, and equipment could be placed higher up in the building. Other data centers are talking about moving equipment above the flood plain or relocating.
Hurricane Sandy Prompts New Data Center Thinking
Hurricane Sandy, the second costliest hurricane in United States history, is an object lesson in how the best-laid contingency plans can be shredded in no time when an extreme weather event hits. The storm has spurred a new way of thinking about data center resiliency.
One success story involved a company that took prior action to hold the water back. People welded doors on the lower level so that water couldn't get in, and built a wall in front of the fuel tank. But these success stories were relatively few and far between. In many cases, the flooding caused serious problems in data centers.
During the storm, water rose 13 feet above sea level, which caused New York City to expand its flood plain area and redraw its flood maps. A lot of basements in multitenant data centers were flooded, and that is where much of the critical infrastructure was located.
New York City code requires fuel to be stored at the bottom of buildings, which is why fuel pumps, tanks, and generators, were in the basement. While some facilities have generators on the roof, fuel pumps and fuel tanks got knocked over, and they couldn't bring fuel from the basement to the roof. UPS and transfer systems at higher levels didn't do much good when basements had floor-to-ceiling flooding.
There were similar problems in smaller data centers in high-rise buildings downtown, where pumps wouldn't function and power feeds weren't watertight. When water filled up basements, the oil tanks lifted off their carriages and piping connections to fuel oil were broken. Trucks came in with package pumping systems that were physically connected to the fuel oil risers in buildings to run the generators.