For many organizations, application information lifecycle management, or ILM, now offers expedient - and badly needed - measures for properly defining, managing, and storing data. Many enterprises are being stymied by a massive proliferation of data in their databases and applications. Growing volumes of transaction data are being digitally captured and stored, along with unstructured forms of data files such as email, video, and graphics. Adding to this tsunami are multiple copies of all this data being stored throughout organizations. At the same time, increasingly tight mandates and regulations put the onus on organizations to maintain this data and keep it available for years to come. Much of this data still resides on legacy systems, which are costly to operate and maintain.
Are organizations coming to grips with all this data surging into their systems? A new survey of 227 managers and professionals affiliated with the Oracle Applications Users Group (OAUG) finds that most are only just beginning to take the necessary steps to address these challenges and keep massive amounts of data from overwhelming their enterprise applications. The survey, which covered a range of organizations, was fielded by Unisphere Research, a division of Information Today, Inc., in partnership with Informatica Corp.
Essential ILM
The ability to properly manage information through its lifecycle-up through the archiving stage-is becoming essential to enterprises running Oracle E-Business Suite, PeopleSoft, Siebel, SAP, and other ERP, CRM, or supplier relationship management (SRM) applications, which are increasingly beset by growing volumes of data within them. ILM employs policies, processes, practices, and tools used to align the business value of information with the most appropriate and cost-effective IT infrastructure from the time information is conceived through final disposition. "The move towards ILM is a definite trend for the better, especially if you consider that data is probably the only corporate asset that has yet to be optimized," Daniel Teachey, senior director of marketing for DataFlux, tells DBTA. "Companies have traditionally stored data without regard to its management."
There is a growing convergence of forces that are reshaping the way data is deployed and presented to decision makers. Increasingly, data functions are being offered as data services that are accessible via a common, enterprisewide service layer, as explained by Robert Eve, executive vice president of marketing at Composite Software. "Standards-compliant data services leverage XML industry standards, data services and data virtualization middleware components," he says. "As enterprises become ever more distributed and complex, data integration becomes increasingly difficult."
ILM offers a way for enterprise applications to be effectively managed throughout the stages of their existence-from development, test, production, and archive to retirement. This is important in terms of cost savings and business agility because as data volumes grow, production system performance and responsiveness slows. There is only so much hardware and application and database tuning that can be thrown at the problem. Maintaining application service levels while keeping costs down becomes virtually impossible. "ILM starts to put the same emphasis on data that a company places on production equipment, buildings, fleet and other assets," Teachey says.
The foundation for the ILM discipline comes from records management, Noel Rath, worldwide product marketing manager, TRIM, HP Software & Solutions, points out. Managers pursuing an ILM strategy need to "take an enterprise approach and consider all business records, regardless of source, as enterprise assets and capture and manage these for their life in a unified manner in an enterprise records management system," he tells DBTA. "Records management underpins information governance and is an essential platform for information lifecycle management."
Organizational Issues
At this point, however, ILM is still relatively new to many organizations. Just under three out of 10 enterprises have adopted variations of ILM, and another 16% are considering such plans. However, the survey reveals that a majority, 56%,
either do not have or plan to implement any ILM efforts, or simply are not aware if they even exist within their organizations. Many companies "are still very undisciplined and reactive about their data challenges," Teachey remarks. "They fix problems as they occur, but as data accumulates across the enterprise, this ‘firefighting' mode isn't sustainable."
In many organizations, expenditures for ILM are either unclear or considered to be a minimal part of IT budgets. Among respondents that have ILM efforts underway, more than 40% are not aware of exact expenditures for staffing and solutions. About a third of respondents, 32%, estimate that their ILM efforts equal 5% or less of their total technology and staff budgets.
The proliferation of social networking platforms further exacerbates the challenge of effectively managing information lifecycles. "Potential challenges are the proliferation of sources of content such as the widespread adoption of Microsoft SharePoint for collaboration," HP's Rath points out. "Organizations are building business applications on top of SharePoint which produce business records that must be managed. This increases the onus on the business and its users to capture these business records from these collaboration system and business applications as seamlessly as possible. Compounding this is the need to manage content from all other sources, such as business applications, desktop applications and email, all in context with the activities of the business in an enterprise records management system."
Organizational issues pose one of the most vexing challenges, since many parts of the enterprise will be involved in the process of identifying and managing information. In the words of one respondent, the challenge for his organization is "a lack of a cohesive strategy across the organization for using information effectively and determining the applications best suited for meeting business objectives."
In many cases, the challenge of driving ILM across the enterprise "has proven too difficult with existing technologies and amongst competing priorities," Bill Brydges, managing director at MorganFranklin, tells DBTA. "Organizations have historically taken a siloed view of ILM, considering data archival, backup, and retention policies within the context of a single functional area such as finance."
Ultimately, such efforts need to be rolled up into an enterprise view, adds Brydges. "ILM cannot be attacked from a single point of view, it must be considered on an enterprise basis. Data that may no longer be relevant to one functional area, such as HR, may be absolutely critical to other areas with different retention requirements or compliance needs."
People Issues
Identifying the right people to manage information as part of ILM is another challenge that needs to be addressed. "The first thing that organizations need to understand about ILM is that it requires the right people to manage each part of the lifecycle," Teachey says. "There has traditionally been a turf war between business and IT about who owns the data, but smart organizations know that it requires a blend of these two groups to effect the change that's needed. The business side knows what the data should look like and how long it needs to be retained. IT manages the systems themselves and must take those rules from the business and implement them within the IT ecosystem."
Two out of three OAUG survey respondents say data-no matter how old-should be readily available when needed with a reasonable amount of response time. "The enterprise has to adapt its information management practices by deciding how long the information will be kept and how quickly the enterprise will need to access it," Thomas Vernersson, CEO and founder of Northern, tells DBTA. "Both of these requirements are tricky because information loses value at various rates. In implementing an ILM solution, you must answer basic business questions: ‘How valuable is this data? Can I ever delete it? How do I balance needs for the availability of this data and my storage budget?' "
Legal Challenges
Then there are the legal and compliance ramifications that may force some organizations to adopt ILM strategies - especially if data needs to maintained and retrievable for long periods of time. The OAUG survey found that a third of companies need to maintain data for longer than a decade, particularly to meet government mandates.
Another aspect of the issues around ILM is burgeoning requirements to maintain both active and inactive data for extended periods of time-and in many cases, keep it highly available to end users. Organizations may need to hold on to data for more than 10 years-further increasing data volumes and management costs. If data is not properly archived - moved off production systems - managers and administrators may end up continuously tuning their production systems to handle the growing volume of data. Moving data to archived systems enables enterprises to store information on less expensive media, such as tape. In fact, a majority of respondents, six out of 10, report that they hold on to their data for extended periods of time, more than 7 years, either because of company policy or compliance mandates. In 16% of cases, data is held onto "forever."
There is often debate within enterprises as to when data ceases being "current" and needs to be moved to the next stage. As one OAUG survey respondent put it: "The weakest link in our information lifecycle management is getting all parties to come to an agreement on when data is no longer active and should be classified as historical data and moved off to an archive."
Of course, for many, the government may have a say in this matter. The main reason for holding on to data, cited by 61% of the group, are federal, state, or provincial government compliance mandates that need to be met. Health Insurance Portability and Accountability Act (HIPAA) rules, for example, have varying directives for the length of time that patient records and imaging files need to be maintained.
Another reason for maintaining data for extended periods of time is to be prepared in the event of litigation, cited as a driving factor by 37% of survey respondents. There are now well-documented instances of companies needing to provide email records, for instance, in various lawsuits. "It is very important to reinforce a company's requirements to preserve data in relation to current or future litigation - many companies have found themselves on the losing side of million-dollar settlements because they failed to preserve key information," says Dr. Gavin Manes, founder, president, and CEO of Avansic, a digital forensics company that preserves and investigates corporate data for litiga-tion. "The landmark digital forensics case Zubulake vs. UBS Warburg set the requirement to preserve as soon as you ‘know or should have known that evidence was potentially relevant to future litigation.' Since many lawsuits last for years, and the scope of preservation may be substantial, this is a major issue facing companies."
Still, close to half of survey respondents say there are purely business advantages to maintaining such records. Forty-six percent of survey respondents say they need to hold on to data to improve their organizations' ability to track and analyze customer history and other needs.