Disaster Recovery for Government IT

By Courtney Macavinta

In the wake of several natural disasters, such as Hurricane Katrina -- not to mention the terrorist attacks on Sept. 11, 2001 -- government agencies at all levels are reconsidering how they can get back online quickly and perform essential government IT services. After all, the IT and communications backbones are essential tools in responding to such events and saving lives.

Disaster recovery and security are top of mind for public-sector CIOs. Sixty-one percent of those surveyed by Forrester Research said they had made significant upgrades of their disaster recovery capabilities a priority this year, according to the research firm's April report, US Government IT Spending Plans for 2007. 

This concern is echoed by the National Association of State CIOs (NASCIO). 

"Mostly the state CIOs are worried about any kind of disruption to the day-to-day operations -- from a power outage to something more severe like a fire, to something catastrophic like the area's data center being hit by a tornado, hurricane, or terrorist or cyber attack," says Drew Leatherby, the NASCIO Issues Coordinator for its Disaster Recovery and Business Continuity Working Group.

The task of improving IT disaster and recovery plans is not an easy feat for government agencies. Yet by adopting some strategies used by the private sector, public sector CIOs can take the following steps to improve their disaster recovery plans:

Step No. 1: Have a disaster plan especially for IT
Even before investing in backup technologies, government agency CIOs need to lead the development of a continuity of operations plan (e.g., disaster recovery plan), according to Leatherby.

"They need to resolve some of the details of how to get essential employees back online," he says.

They also need to make decisions about which essential services need to be brought back online first.

A disaster recovery plan needs to cover all critical IT services and data and potential disruptions. For instance, it's not just a hurricane that can cripple government IT services. Leatherby has heard stories of squirrels chewing through an agency's uninterruptible power line or a blocked rain drain causing a data center's roof to collapse. The NASCIO offers a free DVD "Government at Risk: Protecting Your IT Infrastructure" on its Web site. The DVD provides an overview of why a government agency needs an IT disaster recovery plan and will release a planning toolkit this summer.

Step No. 2: Use a multi-tiered approach to data backup
Government agencies are taking various approaches to data backup, because not all can afford multi-million dollar backup data centers. But no matter their size, budget, or critical function, experts say that important data -- from email to electronic records that must be archived to comply with regulations -- needs to be replicated, protected from unauthorized alteration, and stored off-site. Some agencies use daily backup tapes that are stored several miles away in a secure location. Leatherby says that some state agencies store their tapes in secure underground caves. Others use microwave radio systems or remotely served state Web sites or emergency blogs to keep the communication flowing to the public in the event that traditional phone service is disrupted.

Some agencies might need to rely on remote servers that are located much farther away. Jarad Carleton, senior consultant in the Information and Communications Technology Practice at the analyst firm Frost & Sullivan, says in the wake of geographic disasters like Katrina, essential data and systems should be mirrored in remote locations from the main hub.

"More of the dynamic, forward-thinking enterprises make sure their data is mirrored remotely in another state, such as in a far-away location where it's not going to be affected by a localized disaster," Carleton says.

Some CIOs would be best served by using an outside vendor to manage their day-to-day data backup, he adds.

Step No. 3: Know your vendor's backup plan, too
Though many government agencies might turn to vendors to manage their data backup and recovery, Leatherby says this doesn't mean CIOs are off the hook. They also need to know what type of backup plan their vendors have in place.

"You have to think sideways," he says. "Like for your backup generators, you need contracts for diesel fuel. So the gas station needs a backup plan, too. If their power is out, they can't pump it."

He notes that if a vendor has contracts with several agencies, CIOs also need to ask them about their capacity in case a disaster impacts a multi-state region.

Step No. 4: Sell and test your plan
A CIO can put in place the best disaster recovery technology, but unless it's supported, tested, and fine-tuned, it can be rendered useless.

"Communicate with the people who write the checks," Leatherby says, "about the necessity of keeping systems and data centers or telecommunications up and running."

Testing is equally important. "I have heard some horror stories of CIOs who had to use their backup plans but they couldn't get them to work -- so test and retest," he says.

Ultimately, when it comes to public sector IT disaster and recovery planning, Leatherby says: "The technology is out there and the money is out there. It's a matter of whether you can convince [agency leaders] it's a high priority. The ability to recover is the most important challenge CIOs face today."

Courtney Macavinta is a Silicon Valley-based business and technology writer. Her articles have appeared in CNET News, Business 2.0, Red Herring, Wired News, and The Washington Post. She also is managing editor of  the online program The Online Family.