Image Credit: Data HoldingsKey Takeaways:
Data centers are key pieces of infrastructure for the digital age. With the services they support, these facilities have to be kept operating smoothly around the clock with little to no interruption. The facility’s uptime measures how reliable it is based on how much of the time it remains in operation.
At the data center for Data Holdings in Milwaukee, maintaining 100 percent uptime for power, providing cooling and internet are core responsibilities for director of operations Kurt Brzeski and his team. But the center’s reliability does not depend on a single piece of equipment or system.
Brzeski and his team accomplish that reliable uptime because redundancy is not just a backup strategy but a philosophy that extends throughout the building.
Put another way, each building system has a fallback in place should services stop.
All the equipment inside the data center is powered by uninterruptable power supplies (UPS), Brzeski says. Every server rack has a dual power feed, which is on the UPS system. Both feeds are active at the same time through the facility’s active-active power configuration.
“We have the dual power feeds coming in from the electrical and gas utility company WE Energies,” Brzeski says. “We are one of the few data centers in Wisconsin that has an active-active power system, meaning that both power feeds are active at all times, supplying the A and the B side power going to all of our equipment here.”
In the event of a power failure, two 2.5-megawatt redundant generators provide the necessary backup power. If one of the generators fails, the other automatically takes the load.
“Redundancy is key here,” Brzeski says. “In the event that we do have a failure on one of the systems, we have another system that can handle all the load.”
The cooling plant also incorporates redundancy at multiple levels. A 1,600-ton chiller plant supplies chilled water through a glycol system. Each of the plant’s two 800-ton chillers has two 400-ton cooling skids, allowing the remaining skid to take over if one side fails.
Computer room air handlers distribute this cooling to the data halls, which are configured to use hot aisles and cold aisles. Each data hall has two air handlers, and each air handler can support 100 percent of the room load if the other goes down.
These multiple layers of redundancy allow the facility to continue operating when individual components are unavailable. This strategy only works if the equipment is maintained effectively.
“Preventive maintenance is key to keeping the equipment up and working,” Brzeski says. “Everything that we have here is performed at least quarterly. Some of it has biannual maintenance, but every piece of equipment is touched at least every six months.”
His team uses a scheduling system to ensure all preventive maintenance is completed. For the team, completing maintenance is not the endpoint, they also track problems discovered during maintenance so the issues can either be resolved or mitigated.
The data center’s equipment is commissioned according to best practices, Brzeski says, adding that cooling equipment is configured for the facility’s operating environment and efficiency requirements. While redundancy provides a safety net, preventive maintenance is the force keeping that safety net from becoming the primary operating strategy.
The facility has layers of backup, but Brzeski says those backups are not an excuse for deferring maintenance.
Weathering storms of many kinds is crucial for any facility type, and Data Holdings’ data center is designed with severe weather in mind.
In the event of severe weather, Brzeski says all the staff moves to an inner workspace in the building away from windows on the front of the building.
“The building was designed to withstand even the most powerful tornadoes,” Brzeski says. “With that being said, and with our diverse active-active power, redundant UPSs, generators, chillers and multi-homed Internet, our building is the place to be during severe weather.”
The building envelope, emergency procedures and critical infrastructure all contribute to operational continuity, he says, adding that the power infrastructure was built to handle service disruptions.
For preventative maintenance and testing, Brzeski says the facility has detailed methods of procedure to facilitate the transfer to backup power. These procedures are executed by three people: one to read the step, one to perform the action and one to verify completion. The procedures are also documented in the facility’s runbook.
Documentation is also a critical aspect of Brzeski’s operations.
“A lot of people focus on cooling and power, which is granted to be important,” he says. “Probably the most important thing that I learned here is documentation. Everything that you have needs to be documented.”
When Brzeski started working at Data Holdings, he found that the active documentation was lacking. Since then, he has ensured that the center’s electrical one-line diagrams were updated and documented. The same approach also applies to the cooling plant.
“We have a runbook, which documents everything that we have in the data center and details the entire operating instructions of it,” he says. “It also includes the required preventive maintenance if an issue arises. This is basically the bible of our data center. You need to make sure that everybody on your staff understands it and has access to it.”
This same idea also applies to routine operations and emergency procedures, Brzeski says, adding that sophisticated infrastructure does not eliminate the need for basic operational discipline.
That idea is echoed by Brzeski’s core management philosophy: Reliability comes from multiple layers of physical design, redundant infrastructure, preventive maintenance, testing, documentation and staff familiarity with operating procedures. Ultimately, the systems are only part of the equation, and the data center’s reliable uptime is more than the sum of its parts.
Jeff Wardon, Jr., is the assistant editor for the facilities market. With more than three years of experience, he covers topics including technology, wellness, sustainability and emerging industry trends. Connect with him on LinkedIn.
