Compliance is one of the most interesting elements of any data management plan - it's a microcosm of evolution in action. When many of the laws that impacted data retention were first enacted, business wasn't collecting a lot of information. Now, data collection happens everywhere. And, as citizens have come to realize that more and more of the information about their daily lives is recorded, they demand their governments provide privacy and protection from misuse of that data.
(Editor's note: In his column in the June E-Edition of Database Trends and Applications, Kevin Kline considered issues that need to be taken into account when evaluating approaches to managing data for the long term, including cost versus performance, as well as business continuity requirements. Here, he looks at the impact of federal and state laws on data management policies.)
You might already be aware of federal legal precedent such as HIPAA (http://en.wikipedia.org/wiki/HIPAA), which governs the use of health-related information about individuals, and SOX (http://en.wikipedia.org/wiki/Sarbanes-oxley), which requires publicly traded companies to maintain at least seven years of financial records and subjects them to audits of that data. Europe also has implemented many of its own legal protections for data, and is much more rigorous in defending personal privacy than what is currently seen at the federal level in the U.S. There also are international trade group standards that affect data management.
However, in my opinion, compliance is experiencing an accelerating rate of change at the state level here in the U.S. More than 40 states have passed laws governing data breach notification, such as California's SB 1386, with more likely to follow. Massachusetts recently implemented a new data security law, 201 CMR 17.00, requiring business to have data security and protection processes in place "appropriate to the size and resources of the business." A remarkable aspect of this law is its application not only to the common computer-based data IT people think is important, but also to smartphones, USB thumbdrives and MP3 players.
All of this compliance overhead means that businesses will have an increasingly more complex maze to navigate to ensure their data complies with the law. At the same time, they will have to determine how to store important data longer, and from a wider variety of sources, than ever before.
Looking for Examples
The good news is that many examples that can be used as templates already are available on the internet. For example, Educause.Edu has a nice selection of 25 resources you can review and possibly draw upon for your own data management policies. (Refer to http://www.educause.edu/Resources/Browse/Data%20Retention%20Policies/30410 for more details).
Keeping data on disk or tape is a pretty simple job. It gets a little more complex and costly when we need to scale up the amount of data we store, or need to offer better performance. Making sure important business data never disappears or goes offline when it's needed puts more issues into the mix. And, making sure our organizations comply with international standards, plus federal- and state-level laws, can simply make your head spin. The answer is a thoroughly planned data management and retention policy that meets the requirements of the business, while staying within budgetary constraints. If you're new to planning a data management and retention policy, take advantage of the many resources on the internet to help fill in the gaps in your current approach to data management.