23
Mar
Learning from Disaster
What Can the IT Community Learn from the Disaster in Japan?
We all hope that we never have to experience the devastation that is being felt in Japan. They are trying to cope with numerous challenges relating to basic human needs like food, water, and shelter let alone basic infrastructure like power and communications. Would you be prepared if your data center was in the path of
such destruction?
As a data center manager, you are responsible to account for all scenarios that could impact data center availability. Let’s look at the challenges you would face if your data center was located in or near the areas of devastation. The areas of concern are power, people, structure, and communications.
Power
Odds are that you will not have power. This is a basic concern that every data center manager accounts for. The problem with this situation is the power will probably be out for an extended period of time. Normally this would not be too much of an issue because you have negotiated those diesel fuel delivery contracts. The only
problem is that there are no usable roads between you and the fuel source. Even if you are using natural gas, there is no guarantee the pipeline would survive the earthquake.
People
What is the status of your staff and can they get to the facility? Even if they can, would they? In a disaster, your staff is going to be more concerned with their own safety and the well being of their family. Your data center will be pretty far down the list.
Structure
An obvious question is: What is the status of the structure? The building may be damaged to the point of being unusable or dangerous.
Communications
If your data center has survived, you have personnel and power, you may still have issues if you don’t have your communication lines. Most organization will employ diverse, redundant connectivity. However, during a regional disaster, there is a very good probably that all of your connections will be impacted.
Key to Survival: Reundancy
So what does this mean? Because of the multiple catastrophic events, there is not a single data center that would be able to maintain operations. Your only solution is to fail over to a redundant data center. Many (but not all) organizations are using redundant data centers to account for catastrophic failures of their
primary facilities. However, many are keeping the data centers within close proximity to limit the amount of communication latency so they can maintain synchronous storage replication. This is around 30 miles or so. Again, using Japan as an example, the destruction radii would be large enough that there would be a good
chance of losing both data centers.
The bottom line is understand your risks and the probabilities. Make sure management is in agreement. Don’t get caught surprised; be prepared to handle the worse case scenarios. Remember, if you think “it can’t happen here”, so did every Japanese data center manager right before the earthquake hit.
