4 FM quick reads on data centers
1. Decommissioned Data Centers: What's Next?
For some organizations, thinking about a new data center occurs in tandem with planning for what will happen to the old data center. What happens when a data center has reached the end of its life? It depends on what's going to happen with the building after the data center is decommissioned.
One option is to re-purpose the building for another use. Many data centers are part of a larger building that also includes office space. Depending on the age and condition of generators, chillers, and other MEP/fire protection systems, there is an after-market for used equipment. There is also a huge after-market to salvage electrical equipment and power cabling due to the high cost of copper.
If the data center has underground fuel tanks, they need to be removed and decommissioned in accordance with EPA requirements. Batteries in UPS systems need to be removed and recycled.
If you are moving out of your data center because you have outgrown the available power and space, one option is to re-sell the property to another user. There is a robust market in the United States for customers looking for existing data center space.
In some cases, the land may be more valuable for an alternate use, which would require decommissioning and demolition on the entire facility. Again, it's worthwhile to consider reselling heavy equipment and salvaging other valuable assets to maximize the value that can be obtained from the old data center.
It should be obvious that no one size fits all. Each type of data center is unique in its mission and life cycle. The up-front planning and total cost of ownership analysis of options are critical steps to a successful project.
Staff Training, Maintenance Schedule Help Keep UPS Running
Ongoing staff training and a comprehensive maintenance schedule are both critical elements of ensuring your data center UPS can handle the load.
Data center facility staff wear multiple hats, and day-to-day operations could go on for many months or even years without UPS interaction. Therefore, facility staff must be properly trained on the use of the UPS system on an ongoing basis, preferably annually, as the personnel who were present during the initial UPS training may not be there 10 years later. But, even when facility personnel remain the same for a decade, annual training provides a much-needed refresher in recognizing what the UPS system dependencies are and what should happen when something goes wrong. The operators responsible for maintaining the UPS should understand how the system's redundancy is intended to operate.
Also, be proactive in implementing a routine maintenance schedule. Here are some guidelines broken down by UPS component: Batteries: Replace every 5 years and do yearly testing each year in between.
Air conditioning: Service it properly, including replacement of filters, etc.
Control boards: Once the UPS starts aging, batteries aside, the next thing to fail are the control boards. Inspect them every year at a minimum. Replace every 10 years or less. Look at consulting with the UPS manufacturer for their suggested replacement schedule.
Cooling Fans: While most manufacturers build redundancy into their UPS cooling fans, fans are a moving mechanical component and will wear out. Replace on an as-needed basis, but be proactive, as multiple fan replacement is recommended also in the 5 to 10 year range.
Power filtering capacitors: Also recommended to be replaced in the 5 to 10 year range, but varies from manufacturer to manufacturer.
The initial UPS design should both provide the data center owner with peace of mind that the mission critical environment has been executed as designed and put the facility personnel in the best possible position to easily maintain the building, through installation and daily operations.
Commission, Test UPS To Ensure Proper Operation
A data center UPS plays a critical role in keeping the data center operating the way it's supposed to, and it's up to FMs to make sure it can play that role. Among the ways to do that are to commission the system thoroughly and test it under real-world, worst-case scenarios. Failure to do either of these things due to budget concerns can lead to much greater expense in the long run.
Because data centers are often designed years before the facility is actually built out and populated with equipment, commissioning the UPS system can be especially crucial to confirming that it can actually hold the data center's load. Additionally, while most UPS manufacturers are diligent in their quality assurance, there's no guarantee that the contractor installed it properly or that it didn't get damaged in transit. As a UPS system design becomes more complex with additional levels of redundancy, it becomes more important to go beyond generic factory startup services and properly commission the system. An artificial load equal to ultimate data center design load is used to simulate the conditions under which the UPS will be expected to operate (called the load bank test). Because the load may vary from moment to moment (i.e., ramping up from idle first thing on a Monday morning, etc.), the tests should include step load testing for various scenarios that the UPS may be expected to accommodate without failure.
A UPS can fail in many ways: overload, battery failure, fan failure, overheating, EPO activation, etc. These are common failure modes that the UPS manufacturer has theoretically anticipated in its design, so don't be afraid to incorporate tests for worst-case scenarios. It's better to understand how a system will perform ahead of time in a controlled environment rather than guess what will happen when it's supporting a critical load. With redundant N+1 and 2(N+1) configurations, it is expected that if there is a fault in the system, the system will be able to automatically compensate for that failure. But with that additional redundancy also comes additional complexity, and associated test scripts should be expanded to account for additional possible failure scenarios. Having facilities personnel witness these tests is also an ideal way to get a head start on training for when things go wrong.
Site-Specific Data Center Commissioning Can Avoid Issues
A site-specific, customized data center commissioning process lets trades and vendors do their jobs, with direction, oversight, management and documentation detailing demonstration of component, subsystem, and overall integrated system performance in accordance with owner requirements.
Commissioning technicians tend to focus heavily on familiar equipment they understand while sometimes glossing over or even ignoring unfamiliar equipment they don't understand. This lack of understanding often extends to original equipment manufacturer (OEM) vendor service technicians, and even more so to third party service technicians. It is not uncommon for OEM vendor service technicians to be unfamiliar with standard features built in to their own equipment. Furthermore, commissioning reports often tell an incomplete story, failing to articulate in summary and in organized detail what exactly was tested, how it was tested, when it was tested, and how it performed.
Commissioning efforts and results for data centers and critical facilities vary widely, from glorified vendor startups billed as complete commissioning to a small army of technicians checking off boilerplate "one size fits all" forms, with little understanding about how the equipment they are inspecting and commissioning actually operates, let alone how it is most likely to fail. In the real world of data center commissioning, this approach results in some all-too-common problems.
Interfaces between different equipment sets and vendor subsystems are the areas where many problems occur. Vendors generally do a satisfactory job of getting their own equipment working, and also performing operator training for their own equipment and sub-systems. After all, they have a warranty and brand name to protect. However, many field technicians are too focused on "inside the box" individual component performance, to the point of attempting to prove specifications that are not relevant to the project (i.e., extreme overload conditions).
Many hours and precious resources can be wasted on minutia, such as simulating every minor alarm point. However, too often no one is taking a look at the big picture: how all the sub-systems need to work together and where the realistic failure points exist. Then, when the available commissioning time or budget are nearly exhausted, important overall performance tests are rushed, cut short, or otherwise compromised.