After the Disaster: How 3 Banks Survived
Calamities can range from the mundane—such as a server crash—to the catastrophic, like a devastating hurricane tearing through your headquarters. During such crises, a robust disaster recovery (DR) plan for your hardware and IT infrastructure can make the difference between chaos and resilience. Over the past decade, numerous community financial institutions have faced such trials, each demanding a unique response. We share three stories of real-life disasters faced by our customers, each demonstrating how powerful solutions can alleviate distress and ensure a speedy return to business as usual.
Story 1: Twister Trouble
In our first disaster, a tornado left a community bank in ruins, rendering the building unusable for several months. Luckily, the servers were untouched. After consulting with Safe Systems, it was decided that the simplest solution was to move the servers and routers to another location. Once communications and the core were in place, the bank’s operations resumed quickly from the new site. When the primary building was finally renovated, Safe Systems returned the servers and routers over a weekend and the bank was fully functional in its original location once again.
This story illustrates that even though the servers were operable after the disaster, the conditions around them made it important to evaluate all the recovery options. Having a trusted managed services partner who isn’t in the “eye of the storm” can help you objectively evaluate the circumstances to make the best decision—even if it diverges from your original DR plan.
Story 2: Silent Disaster
Not all disasters announce themselves as loudly as a tornado. Some, like this one, can be subtle without all the surrounding clatter. After business hours, Safe Systems received a distress call about a failed core router. We were able to quickly establish a site-to-site VPN tunnel to the institution’s DR router which was hosted by us. The issue was resolved within a few hours and most of the bank employees were unaware of the incident. The bank quickly returned to normal operations, never missing a beat in customer service.
Despite the nature or the timing of an unexpected business interruption, your DR plan must ensure business-critical data and applications are available. Having a fully managed provider with after-hours emergency protocols and a high-availability system for fast recovery of critical servers via the Cloud allowed this bank to recover as quickly and as quietly as the incident occurred.
Story 3: Lightning Strike
A lightning strike caused extensive damage to a bank’s switches and the physical server hosting most of their virtual servers. With the switches destroyed and no local backup of the virtual servers, the bank had to resort to a mobile hotspot. Safe Systems set up a VPN from their DR router to the Cloud where the DR servers were housed. The bank managed to operate Wi-Fi-accessible devices for over a week until a new switch and server were installed.
When physical damage is extensive and can take weeks versus hours to repair, it is critical to have a partner that can establish connectivity to your locations and key vendors through various connection types—mobile hotspots, satellite internet, internet lines at another location—all of which should be critical aspects of your recovery plan.
Our Approach to Disaster Recovery
Safe Systems has a comprehensive approach to disaster recovery that encompasses data, server, and communication needs in times of crisis. Typically, Safe Systems hosts a backup disaster router at our Tier 4 data center, while each server is mirrored as a virtual server in a secure cloud. Annually, these servers are brought up in test failover mode and core communication is rerouted during a DR test. This helps us to provide a detailed report on the results and readiness for disaster. These servers and routers stand by, primed to leap into action at a moment’s notice, facilitated by our dedicated DR team.
Each of these stories underscores the importance of having a robust and flexible DR solution in place. Regardless of the disaster’s type or scale, having a reliable partner like Safe Systems helps ensure business continuity and secure access to critical systems and data.