Tag Archive: DR


I’ll admit that I worry about my data.  I have a plethora of backups for personal and business reasons, but I’m still concerned that everything may not be backed up and organized as it should be to ensure a speedy recovery of critical data in the wake of a true disaster.   Furthermore, in order to deter would-be hackers from accessing it, the data needs to be stored in an encrypted format.

In the past three decades, hard drive capacities have managed to grow exponentially, while somehow becoming smaller than ever.  Believe it or not, ten megabyte (mb) drives used to approach the size of cinder blocks and cost $4,500.  Today, you can store 32 gigabytes (gb) on something the size of a quarter for $45.  The data stored on systems, both personally and professionally, has mushroomed, making it all too easy to lose track of what is truly important.

The lazy way

Just because your latest PC has room for over 1 terabyte (tb) of data doesn’t mean it all needs to be backed up.  In real-life scenarios, it is surprising just how little data is absolutely critical.  Lazy backup methodologies that capture everything can make recovery more time consuming and potentially problematic. 

In our first-hand experience with clients faced with recovering from a disaster, the only files immediately required are those used in day-to-day operations.  Though a company may regularly back up a few hundred gigabytes of data daily, it likely needs a small fraction of that data to function.

For example, imagine losing everything – including your portfolio accounting system data – and trying to service your clients.  Axys, one of the most critical systems for many investment advisors, is an excellent example because the files typically don’t take that much space. Given our experience with many Axys users, I suspect that most clients’ entire Axys system takes less than 2 gb.

Now imagine losing all of your data, except your portfolio accounting system data and your access to your email.  It would be inconvenient, but you would be able to get by until your other systems could be fully recovered.

Don’t be cheap

Cost should be a non-issue here.  Assuming you have 8 gb of data, an online backup could cost as little as $4 per month or as much as $100 per month, depending on the vendor.  We prefer to use VaultLogix for many of our clients due to the sophistication of their software, which has versatile features such as Exchange message level backup and restore capabilities. 

Even if you feel that you already have a reliable backup in place, we recommend multiple methods.  Based on our experience with disaster recovery, I prefer to see no less than three methods performed by independent parties, but managed by senior IT staff or a consultant who can attest to their veracity:

  1. Local onsite physical backup via NAS, tapes or external hard drives.
  2. An offsite backup server where vital data is synchronized in real-time or regularly restored.
  3. An online backup through a third party.

Local backups require proper management to make sure that data is being backed up and the resulting backups are usable.  Even after all the years I have spent dealing with these issues, I am still surprised when I find out that somebody isn’t managing this process properly.

To err is human

I have run into companies that cycled through media every day without checking the resulting logs, only to find out when they need the backup that it hasn’t been working for years.  And I have seen clients independently decide to insert the same tape over and over again, eventually destroying the tape in the process … along with the only recent copy of their data. 

Read the SLA

The what?  An SLA, a.k.a. Service Level Agreement, is the fine print. It is probably detailed in 7-point font, where most data backup vendors tell you that they won’t reimburse you for more than you pay them on a monthly basis, even if you lose data because of a system failure.  Your data is priceless – you couldn’t sue the vendor for enough to make up for the loss of your data, so even if you contract with a vendor for data backup, your firm needs to own the process. Perform quality checks and have a contingency plan in case your vendor fails to do the job.

In the past, I have preached about the necessity of backing up data through a variety of methods.  For most RIAs, the data is so utterly vital that they cannot afford to take any chances.  Investment advisors cannot afford to forgo insurance against the unexpected; there is simply no excuse not to have multiple backups of your most important data. However, these backups should be encrypted, whether they are stored on a USB drive or through on online backup service.

Your portfolio management accounting system obviously isn’t the only essential data at your office.  Think about what else would be extremely difficult to do without and take proper precautions.  You will likely come to realize that your firm’s most important data doesn’t take anywhere near the amount of space required for your regular backup.  Once you identify your most valuable data, you can look into additional backup methods to protect it.

Our most valuable data is the source code for the programs we have created, our client database, Quickbooks, marketing materials, and Exchange data, in that order.  This data, excluding Exchange, easily fits on a hardware-encrypted 8 gb USB drive.  We have a lot of other information stored on our server, including data test beds, program files, installation files, documentation, and media, but this data is not crucial.

By definition, non-critical information can be downloaded again, acquired easily enough or won’t be missed if it cannot be recovered.  On the other hand, our most critical information would be very difficult and time-consuming to recreate if not impossible.  These same principles should be applied to backing up your personal files.  What is most valuable to you personally?  My personal file priorities include source code, financial records, important documentation, photos, videos, and music, in that order.

At my home, I have a large hardware-encrypted NAS (Seagate NAS 440) that backs up these files locally and has RAID 5 fault-tolerance.  This device is ideal protection against everything but fire or theft.  Using the Acronis software included with the NAS, I can do a full recovery of any of our PCs.  If the NAS is stolen, none of the data can be accessed without the USB drive “key,” which contains the necessary encryption key to unlock access to the drive’s data.

My self-imposed personal needs for file synchronization and a redundant, independent online backup, as well as the sheer quantity of data I back up, demand that I use a combination of low-cost online backup providers such as Dropbox, Mozy or Carbonite.  Users of these systems can and should take extra precautions to ensure that their data is encrypted with keys they manage.  Doing so requires an extra step.  For example, Dropbox data is encrypted in transit, but your Dropbox folder is probably stored on an unencrypted drive.   If your notebook hard drive isn’t encrypted and gets stolen, your data can be easily accessed.  Placing the Dropbox folder on an encrypted drive is a best practice.

The amount of time required to get the initial image of your files into the cloud can be daunting, but the cost is reasonable.  If you have hundreds of gigabytes of data, it could quite literally take months to perform your initial backup, depending on the speed of your Internet connection.  Given this reality, the sooner you start, the better.  Subsequent daily backups typically take minutes, not hours or days.  Mozy, a subsidiary of VMWare, has recently introduced a data shuttle option that eliminates the need to initially back up over the Internet, allowing users to simply send their encrypted data via a Mozy-provided hard drive.  However, the service is only available to Mozy Pro users with physical addresses in the United States. 

Planning for a real disaster

Like many of the investment firms we service, we have a contingency plan in place that we update regularly, but I am a realist regarding disasters and recovery.  A real disaster is unprecedented: an earthquake that spawns a tsunami that causes a meltdown in Japan, tornadoes in Massachusetts and hurricanes in Vermont.  It isn’t one hard drive failing, it’s two or more, and it can happen over the course of a few weeks or a few seconds.  It can be water where it shouldn’t be, or just plain stupidity.

Contingency planning is intended to lower your overall risk, not anticipate every possible disaster.  So even with redundant, independent backups, I worry about the details of restoring my most critical data, and you should worry about yours too.  Get started by identifying your most critical data today.  Make sure you have it backed up three different ways and stored in three different locations.  Don’t assume that one well-managed backup method is sufficient.  Anticipate problems with what you count on most, and plan for a real disaster – one you cannot predict or imagine.

About the Author:
Kevin Shea is President of InfoSystems Integrated, Inc. (ISI); ISI provides a wide variety of outsourced IT solutions to investment advisors nationwide. For details, please visit isitc.com or contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@isitc.com.

I started setting up contingency plans and disaster recovery (DR) sites for financial services firms ten years ago. Initially, the goal was to reduce the expected recovery effort from an insane week to a few difficult days, but in relatively short time the new standard became less than 24 hours. Given the SEC’s increased scrutiny of business continuity (BC) plans most investment management firms should now have a detailed plan that includes access to offsite servers, and a level of routine data updates to an offsite facility.

To ensure that your BC plan is successful follow these simple rules:

1. Plan to fail
Most plans are too optimistic. When things go awry, they typically go from bad to worse — rapidly.  Planning to fail means envisioning and detailing potential failure scenarios and documenting your contingency plan for each situation.

For example, we recommend three forms of backup (with the assumption that one or two of the backups may not be sufficient). While many of the firms we work with could probably get away with a single backup method, each method that we implement lowers the overall risk of losing any data.

2. Don’t set it and forget it
Remember, there is no silver bullet. The day you don’t check your systems is the day you should expect them not to work. Systems are ultimately managed by people and even the most competent people sometimes make mistakes. Beware of vendors that tell you, “It’s automatic. You don’t have to do anything.”

If you have a metered Internet backup service and you are getting billed monthly, the invoice amount should never be the same. If it is, it may indicate that the data being backed up isn’t changing.

Absolute vigilance is required to be successful at planning for a contingency.

3. Establish strict and coherent responsibilities
Who is the steward of your firm’s plan? How do they validate the plan? How does the plan work? Our real-world experience indicates that multiple parties need to understand and check the plan for problems on an ongoing basis.  When new systems are implemented a disciplined approach to adding and updating the contingency plan needs to be executed.

4. Institute operational checks and balances
We recommend a multi-faceted approach designed to ensure that multiple parties independently share ultimate responsibility for backing up your company’s data and validating that the contingency plan works. Your firm cannot afford to make assumptions about whether those responsibilities are being met.

If you think you’re ready, test it. Ask your IT folks to throw the switch with little or no warning to see how well your plan really works. You may want to think about this carefully since some plans are like having a gun that can only shoot one bullet. In order to test the system again you may need to rebuild the system.

5. Continuously improve and refine
Contingency plans fall into five basic categories: non-existent, poor, okay, good and excellent. As a decision maker and responsible party at your firm, do you know how your firm’s plan would rate? Moreover, IT systems are in a nearly constant state of change. If your plan was “good” last year, is it as “good” today?

A company that has an excellent contingency plan for a catastrophic event may not have a good plan for the more likely event of losing Internet access at their office tomorrow.

There is always room to improve your plan.

About the Author:
Kevin Shea is President of InfoSystems Integrated, Inc. (ISI); ISI provides a wide variety of outsourced IT solutions to investment advisors nationwide. For details, please visit isitc.com or contact Kevin Shea via phone at 617-720-3400 x202 or e-mail at kshea@isitc.com.

View the original document.