Biographical Information
Joe McKendrick
Joe McKendrick edits the 5 Minute Briefing: Data Center email newsletter, serving the SHARE community.
Articles By Joe McKendrick
For many organizations, application information lifecycle management, or ILM, now offers expedient - and badly needed - measures for properly defining, managing, and storing data. Many enterprises are being stymied by a massive proliferation of data in their databases and applications. Growing volumes of transaction data are being digitally captured and stored, along with unstructured forms of data files such as email, video, and graphics. Adding to this tsunami are multiple copies of all this data being stored throughout organizations. At the same time, increasingly tight mandates and regulations put the onus on organizations to maintain this data and keep it available for years to come. Much of this data still resides on legacy systems, which are costly to operate and maintain.
In only a few years' time, the world of data management has been altered dramatically, and this is a change that is still running its course. No longer are databases run in back rooms by administrators worrying about rows and columns. Now, actionable information is sought by decision makers at all levels of the enterprise, and the custodians of this data need to work closely with the business.That's because, in the wake of the recent financial crisis and economic downturn, there's a push from both high-level corporate management and regulators to achieve greater understanding and greater transparency across the enterprise, Jeanne Harris, executive research fellow and a senior executive at the Accenture Institute for High Performance, and co-author, along with Tom Davenport, of Competing on Analytics and Analytics at Work, tells DBTA. "In many ways, I think the ultimate result of the financial crisis is that executives realized they cannot delegate analytics to subordinates; they can't view it as technology or math that doesn't really affect them."
In the midst of turbulent times, many successful businesses learned an important lesson: The closer IT works with the business, the better an organization can weather the storms that blow in. Thus, many savvy companies understand that the managers and professionals who oversee information technology and applications need to be well incentivized to stay on. At the same time, these professionals understand the need to develop expertise in business management and communications. Many companies are looking to information technology to provide an additional competitive edge, and see their Oracle enterprise systems as the cornerstone of this strategy. As a result, a survey finds that Oracle enterprise application managers and professionals appear to have weathered the economic storm. The survey, conducted among 334 members of the Oracle Applications Users Group (OAUG) by Unisphere Research, and sponsored by Motion International, finds an increase in the number of Oracle technology professionals who are near or surpassing the $100,000 mark in their base salaries.
As we enter the next decade of the millennium, we will see information technology becoming more ubiquitous, driving an even greater share of business decisionmaking and operations. IT has proven its muster through the recent downturn as both a tactical and strategic weapon for streamlining, as well as maintaining competitive edge. Now, as we begin the next round of economic recovery, companies will be relying on IT even more to better understand and serve their markets and customers. Yet, there are many challenges with managing a growing array of IT hardware, software, and services. To address these requirements, businesses continue to look to approaches such as analytics, virtualization, and cloud computing. To capture the trends shaping the year ahead, Database Trends and Applications spoke to a range of industry leaders and experts.
Corporate management is complacent about data security. Efforts to address data security are still ad hoc, and not part of an overall database security strategy or plan. Companies are not keeping up with the need to monitor for potential risks. More monitoring tends to be ad hoc or on-the-fly, versus more organized or automated systematic approaches. These are the findings from new research from Unisphere Research and the Independent Oracle Users Group (IOUG), which shows that the recent economic downturn has taken a toll on data security efforts within enterprises.
An overwhelming challenge - expanding volumes of data - threatens to gum up any productivity improvements seen to date as a result of information technology deployments. All that data is coming in from systems, sensors, and storage area networks, pressuring organizations to expand database inventories, while grappling with associated licensing and hardware costs. Plus, many compliance mandates demand that this data be stored for long periods of time, but remain accessible to auditors and business end users.
Oracle is a fast-changing company, and in recent years, its pace has accelerated to blinding speed. The software giant has expanded well beyond its relational database roots to encompass applications, management tools, service-oriented architecture and middleware, and even hardware. There are now many components to Oracle - from three major databases, to enterprise resource applications, to web applications to development languages to open source desktop tools.
The SHARE conference convenes on February 27th in Anaheim, with an agenda packed with industry initiatives and knowledge-sharing on the latest best practices and technology trends. In this Q&A, SHARE president Janet Sun provides her vision for the IBM users group in the coming years. "We see the mainframe as the center of the enterprise IT universe. If you don't think so, try unplugging it," says Sun. "Our organization focuses on enterprise IT, and that includes the mainframe. Today's SHARE membership continues to strive to leverage advances in information technology, and SHARE is a great place to do that."
The recent public release of thousands of leaked U.S. State Department cables by WikiLeaks continues to shake up governments across the world. The information captured and sent out to the wild is not only an embarrassment to U.S. government officials whose candid assessments of foreign leaders were exposed but also to the fact that that the organization with the tightest and most comprehensive data security technologies, protocols, and policies in the world unknowingly fell victim to a massive data breach. Can private corporations or smaller government agencies with less-stringent security protocols and standards expect to do any better? Securing data is tough enough, and now, with the increase of initiatives such as virtualization and cloud computing, the odds of loss of control and proliferation of sensitive data become even greater.
A new survey of database administrators and managers reveals that a pervasive culture of complacency hampers information security efforts, and as a result of lax practices and oversight, sensitive data is being left vulnerable to tampering and theft. While tools and technologies provide multiple layers of data security both inside and outside the firewall, organizations appear to lack the awareness and will to make security stick. The study, "Data in the Dark: Organizational Disconnect Hampers Information Security," was conducted by Unisphere Research among 761 members of PASS, the Professional Association for SQL Server, in September 2010. The survey was fielded in partnership with Application Security, Inc.
Many organizations now have, in their possession, the sophisticated analysis tools and dashboards that connect to back-end systems and enable them to peer deeply into their businesses to assess progress on all fronts-from revenues to stock outs to employee performance. However, a recent survey of 279 Oracle applications managers reveals that when it comes to decision making, simple spreadsheets still remain the tool of choice. And business users still wait days, weeks, and months for their IT departments to deliver reports, despite significant investments in performance management systems.
These days, many companies recognize that there are severe repercussions to ignoring or undervaluing data security, and a sizable segment of organizations-at least one-third in many cases-have been taking additional measures to bolster their data security.
The year 2010 brought many new challenges and opportunities to data managers' jobs everywhere. Companies, still recovering from a savage recession, increasingly turned to the power of analytics to turn data stores into actionable insights, and hopefully gain an edge over less data-savvy competitors. At the same time, data managers and administrators alike found themselves tasked with managing and maintaining the integrity of rapidly multiplying volumes of data, often presented in a dizzying array of formats and structures. New tools and approaches were sought; and the market churning with promising new offerings embracing virtualization, consolidation and information lifecycle management. Where will this lead in the year ahead? Can we expect an acceleration of these initiatives and more? DBTA looked at new industry research, and spoke with leading experts in the data management space, to identify the top trends for 2011.
This year, despite a turbulent economy marked by painful layoffs in many sectors, database professionals appear to be weathering the storm. In fact, database professionals reported higher incomes and bonuses this year over last. Still, a sizeable segment of professionals saw changes in their jobs as a result of economic conditions, and many are concerned going forward about the impact of tighter budgets on their departments' performance.
The old maxim, "may you live in interesting times" certainly holds true for IT managers and professionals these days. The year 2008 was full of changes and challenges, and 2009 promises even more.
Despite efforts to "democratize" business intelligence, it has remained stubbornly confined to a chosen few within organizations. Although vendors have worked hard to convince enterprises that their BI solutions could be extended to line-of-business managers and employees, high-end analytic tools have remained confined to power users or analysts with statistical skills, while the remainder of the organization relies on spreadsheets to cobble together limited pieces of information.This disconnect was confirmed in a 2007 survey conducted by Unisphere Research for the Oracle Applications Users Group, which found that most companies are still a long way off from the ideal of BI for all. The OAUG survey found that for the most part, BI reporting remains tied up in IT departments, and is still limited to analysts or certain decision makers. The majority of survey respondents said that it takes more than three to five days to get a report out of IT. Overall, the survey found, fewer than 10 percent of employees have access to BI and corporate performance management tools.
An entire industry has sprung up in response to the never-ending battle against complexity, server sprawl, and rising power consumption. Virtualization is now the mantra for beleaguered data center managers looking for ways to consolidate, better utilize, or abstract away their farms of physical servers and hardware. However, in many cases, virtualization itself can lead to even more complexity andoffer uncertain value to the business. Many businesses are finding that virtualization is not ready for core mission-critical applications.
Now more than ever, data has evolved into an asset more strategic and valuable than any raw material or capital construction project. Companies are scrambling to "compete on analytics," recognizing that the one to most effectively leverage information coming out of their systems gets the greatest competitive advantage.
Even the best economists can't agree on what the business landscape will look like in the year ahead, but IT industry leaders agree that enterprises are likely to be more cautious than ever in how they spend their IT dollars (or euros, pounds, or rupees). Most also agree that any downturn that may be looming won't be a repeat of 2001, when IT departments were ravaged.
In today's competitive and crisis-ridden market, companies are under pressure to rapidly deliver results and make necessary changes—which requires that decision makers have accurate and timely information readily available. However, many executives have doubts about the timeliness of the information they now receive through their current BI and analytics systems.
This is a time of great change for data centers. Technology is advancing and getting smarter, and workloads and performance demands keep growing. For this issue of Database Trends and Applications, we sought a range of industry views on the most profound—and perhaps unexpected—changes reshaping data centers and enterprise it.
Why do business decision makers need to wait for IT to deliver performance reports on the business? Why can't they build their own reports, and gain rapid access to answer the questions they have?
More than a decade ago, some IBM researchers began pitching a bold vision of information technology, called utility computing at the time, in which processing power would be made available just as electricity is available, as easily accessible as plugging into an outlet in a wall.
Many IT and business managers are now familiar with the concept of virtualization, especially as it pertains to the ability to run a secondary operating system within the same hardware that already supports a separate OS brand. Seasoned data center professionals have been aware of virtualization as a capability available on mainframes for years. The ability of virtualization to provide advantages to data center operations in terms of systems consolidation and simplifying administration has been well-documented.
Business intelligence (BI) and analytics solutions have been available for years now, and companies have learned to employ these tools for a variety of purposes, from simple report generation and delivery to more sophisticated data integration, executive dashboards, and data mining. They also recognize the need to get beyond spreadsheets, and to be able to provide more sophisticated, pervasive, and automated BI solutions to more end-user decision makers. However, most see their efforts stymied by the historically high cost of BI software and the complexity of available solutions.
Are today's data systems — many of which were built and designed for legacy systems of the past decade — up to the task of moving information to end users at the moment they need it? And is this information timely enough? In many cases, there's a lot of work that still needs to be done before real-time information, drawn from multiple sources, becomes a reality. A new survey of 338 data managers and professionals who are subscribers to Database Trends and Applications reveals that real-time data access is still a distant pipe dream for at least half of the companies represented in the survey. The survey, conducted by Unisphere Research, a division of Information Today, Inc., in partnership with Attunity in March of 2012, finds that close to half of the survey respondents, 48%, report that relevant data within their organizations still take 24 hours or longer to reach decision makers. This suggests that much data is still batch-loaded overnight.
Are today's data systems — many of which were built and designed for legacy systems of the past decade — up to the task of moving information to end users at the moment they need it? And is this information timely enough? In many cases, there's a lot of work that still needs to be done before real-time information, drawn from multiple sources, becomes a reality. A new survey of 338 data managers and professionals who are subscribers to Database Trends and Applications reveals that real-time data access is still a distant pipe dream for at least half of the companies represented in the survey. The survey, conducted by Unisphere Research, a division of Information Today, Inc., in partnership with Attunity in March of 2012, finds that close to half of the survey respondents, 48%, report that relevant data within their organizations still take 24 hours or longer to reach decision makers. This suggests that much data is still batch-loaded overnight.
In the never-ending battle for enterprise data security, industry experts say there has been progress on several fronts, but there is still much work that needs to be done. There is an enormous amount of data that tends to leak out of the secure confines of data centers, creating a range of security issues. "There are many copies of data which have less security and scrutiny than production environments," Joseph Santangelo, principal consultant with Axis Technology, tells DBTA. "The increased reliance on outsourcers and internal contractors leave sensitive data within corporate walls open to misuse or mistakes." Or, as another industry expert describes it, the supply chain often proves to be the greatest vulnerability for data security. "A typical organization has a direct relationship with only 10% of the organizations in its supply chain — the other 90% are suppliers to suppliers," Steve Durbin, global vice president of the Information Security Forum, tells DBTA.
In recent years, the networks of developers, integrators, consultants, and manufacturers committed to supporting database systems have morphed from one-on-one partnerships into huge ecosystems in which they have become interdependent on one another, and are subject to cross-winds of trends and shifts that are shaping their networks. Nowhere is this more apparent than the huge ecosystem that has developed around Oracle. With Oracle's never-ending string of acquisitions, new functionality, and widespread adoption by enterprises, trends that shape this ecosystem are certain to have far-reaching effects on the rest of the IT world. Concerns that percolate through the ecosystem reflect — and influence — broad business concerns. New paradigms — from cloud computing to big data to competing on analytics — are taking root within the Oracle ecosystem long before anywhere else.
Are today's data systems — many of which were built and designed for legacy systems of the past decade — up to the task of moving information to end users at the moment they need it? And is this information timely enough? In many cases, there's a lot of work that still needs to be done before real-time information, drawn from multiple sources, becomes a reality. A new survey of 338 data managers and professionals who are subscribers to Database Trends and Applications reveals that real-time data access is still a distant pipe dream for at least half of the companies represented in the survey. The survey, conducted by Unisphere Research, a division of Information Today, Inc., in partnership with Attunity in March of 2012, finds that close to half of the survey respondents, 48%, report that relevant data within their organizations still take 24 hours or longer to reach decision makers. This suggests that much data is still batch-loaded overnight.
In recent years, the networks of developers, integrators, consultants, and manufacturers committed to supporting database systems have morphed from one-on-one partnerships into huge ecosystems in which they have become interdependent on one another, and are subject to cross-winds of trends and shifts that are shaping their networks. Nowhere is this more apparent than the huge ecosystem that has developed around Oracle. With Oracle's never-ending string of acquisitions, new functionality, and widespread adoption by enterprises, trends that shape this ecosystem are certain to have far-reaching effects on the rest of the IT world. Concerns that percolate through the ecosystem reflect — and influence — broad business concerns. New paradigms — from cloud computing to big data to competing on analytics — are taking root within the Oracle ecosystem long before anywhere else.
While no one can dispute the importance of enterprise resource planning (ERP) systems to organizational performance and competitiveness, executives in charge of these systems are under intense pressure to stay within or trim budgets. Close to half of the executives in a new survey say they have held off on new upgrades for at least a few years. In the meantime, at least one out of four enterprises either are scaling back or have had to scale back their recent ERP projects due to budget constraints.
Databases are restricted by reliance on disk-based storage, a technology that has been in place for several decades. Even with the addition of memory caches and solid state drives, the model of relying on repeated access to information storage devices remains a hindrance in capitalizing on today's "big data," according to a new survey of 323 data managers and professionals who are part of the Independent Oracle Users Group (IOUG). The survey was underwritten by SAP Corp. and conducted by Unisphere Research, a division of Information Today, Inc.
Data keeps growing, systems and servers keep sprawling, and users keep clamoring for more real-time access. The result of all this frenzy of activity is pressure for faster, more effective data integration that can deliver more expansive views of information, while still maintaining quality and integrity. Enterprise data and IT managers are responding in a variety of ways, looking to initiatives such as enterprise mashups, automation, virtualization, and cloud to pursue new paths to data integration. In the process, they are moving beyond the traditional means of integration they have relied on for years to pull data together.
Data keeps growing, systems and servers keep sprawling, and users keep clamoring for more real-time access. The result of all this frenzy of activity is pressure for faster, more effective data integration that can deliver more expansive views of information, while still maintaining quality and integrity. Enterprise data and IT managers are responding in a variety of ways, looking to initiatives such as enterprise mashups, automation, virtualization, and cloud to pursue new paths to data integration. In the process, they are moving beyond the traditional means of integration they have relied on for years to pull data together.
In the never-ending battle for enterprise data security, industry experts say there has been progress on several fronts, but there is still much work that needs to be done. There is an enormous amount of data that tends to leak out of the secure confines of data centers, creating a range of security issues. "There are many copies of data which have less security and scrutiny than production environments," Joseph Santangelo, principal consultant with Axis Technology, tells DBTA. "The increased reliance on outsourcers and internal contractors leave sensitive data within corporate walls open to misuse or mistakes." Or, as another industry expert describes it, the supply chain often proves to be the greatest vulnerability for data security. "A typical organization has a direct relationship with only 10% of the organizations in its supply chain — the other 90% are suppliers to suppliers," Steve Durbin, global vice president of the Information Security Forum, tells DBTA.
In the never-ending battle for enterprise data security, industry experts say there has been progress on several fronts, but there is still much work that needs to be done. There is an enormous amount of data that tends to leak out of the secure confines of data centers, creating a range of security issues. "There are many copies of data which have less security and scrutiny than production environments," Joseph Santangelo, principal consultant with Axis Technology, tells DBTA. "The increased reliance on outsourcers and internal contractors leave sensitive data within corporate walls open to misuse or mistakes." Or, as another industry expert describes it, the supply chain often proves to be the greatest vulnerability for data security. "A typical organization has a direct relationship with only 10% of the organizations in its supply chain — the other 90% are suppliers to suppliers," Steve Durbin, global vice president of the Information Security Forum, tells DBTA.
Data keeps growing, systems and servers keep sprawling, and users keep clamoring for more real-time access. The result of all this frenzy of activity is pressure for faster, more effective data integration that can deliver more expansive views of information, while still maintaining quality and integrity. Enterprise data and IT managers are responding in a variety of ways, looking to initiatives such as enterprise mashups, automation, virtualization, and cloud to pursue new paths to data integration. In the process, they are moving beyond the traditional means of integration they have relied on for years to pull data together.
Social media network-based business intelligence represents the next great frontier of data management, promising decision makers vast vistas of new knowledge gleaned from exabytes of data generated by customers, employees, and business partners. Mining data from Facebook, Twitter, blogs, wikis, and internal corporate networks potentially may surface new insights into impending market shifts, patterns in customer sentiment, and competitive intelligence. It's a rich opportunity not lost on today's organizations, a new survey of 711 business and IT managers from across the globe reveals. A majority of respondents are either planning to collect and analyze data from both proprietary and public social media networks, or are doing so already.
As companies learn to embrace "big data" - terabytes and gigabytes of bits and bytes, strung across constellations of databases - they face a new challenge: making the data valuable to the business. To accomplish this, data needs to be brought together to give decision makers a more accurate view of the business. "Data is dirty and it's hard work; it requires real skills to understand data semantics and the different types of approaches required for different data problems," Lawrence Fitzpatrick, president of Computech, Inc., tells DBTA. "It's too easy to see data as ‘one thing.' "
As data grows, the reflex reaction within many organizations is to buy and install more disk storage. Smart approaches are on the horizon but still only prevalent among a minority of companies. How is it data has grown so far so fast? Technology growth along the lines of Moore's Law (doubling every 18 months) has made petabyte-capable hardware and software a reality. And data growth itself appears to be keeping pace with the hardware and systems. In fact, a petabyte's worth of data is almost commonplace, as shown in a new survey conducted by Unisphere Research among members of the Independent Oracle Users Group (IOUG). In "The Petabyte Challenge: 2011 IOUG Database Growth Survey," close to 1 out of 10 respondents report that the total amount of online (disk-resident) data they manage today-taking into account all clones, snapshots, replicas and backups-now tops a petabyte.
Until recently, companies were only warming up to the possibilities of cloud computing. Lately, however, for many enterprise-IT decision makers, cloud is hot, hot, hot. The sea change now underway means many companies are quickly moving from "dipping their toes into cloud computing" to a full-fledged immersion, says Thom VanHorn, vice president of marketing for Application Security, Inc. In 2012, expect to see those same companies dive right in. "The move will only accelerate," he tells DBTA.
Is the day of reckoning for big data upon us? To many observers, the growth in data is nothing short of incomprehensible. Data is streaming into, out of, and through enterprises from a dizzying array of sources-transactions, remote devices, partner sites, websites, and nonstop user-generated content. Not only are the data stores resulting from this information driving databases to scale into the terabyte and petabyte range, but they occur in an unfathomable range of formats as well, from traditional structured, relational data to message documents, graphics, videos, and audio files.
Is the day of reckoning for big data upon us? To many observers, the growth in data is nothing short of incomprehensible. Data is streaming into, out of, and through enterprises from a dizzying array of sources-transactions, remote devices, partner sites, websites, and nonstop user-generated content. Not only are the data stores resulting from this information driving databases to scale into the terabyte and petabyte range, but they occur in an unfathomable range of formats as well, from traditional structured, relational data to message documents, graphics, videos, and audio files.
As the economy shifts to expansion mode, and businesses start hiring again, a familiar challenge is rearing its head. Companies are scrambling to find the talent needed to effectively run, maintain, and expand their technology platforms. This is not a new problem by any means, but this time around, it is taking on a greater urgency, as just about every organization relies on information technology to be competitive and responsive to growth opportunities. A new survey of 376 employers finds a majority depend on the educational sector - universities and colleges - to provide key IT skills, often in conjunction with their own internal training efforts. However, few of the executives and managers hiring out of colleges are entirely satisfied with the readiness of graduates.
A new survey of 421 data managers and professionals affiliated with the Independent Oracle Users Group (IOUG) members finds that while most companies have well-established data warehouse systems, adoption is still limited within their organizations. Many respondents report a significant surge of data within their data warehouses in recent times, fueled not only by growing volumes of transaction data but unstructured data as well. Now, the challenge is to find ways to extend data analysis capabilities to additional business areas.
There's no question that cloud computing is a hot commodity these days. Companies of all types and sizes are embracing cloud computing-both internally and from external service providers - as a way to cost-effectively build new capabilities. With the rapid growth of cloud comes new questions about responsibility within organizations, in terms of how services will be paid for, who has ultimate say over cloud decisions, and how cloud fits into the overall strategic direction of the business.
There's no question that cloud computing is a hot commodity these days. Companies of all types and sizes are embracing cloud computing-both internally and from external service providers - as a way to cost-effectively build new capabilities. With the rapid growth of cloud comes new questions about responsibility within organizations, in terms of how services will be paid for, who has ultimate say over cloud decisions, and how cloud fits into the overall strategic direction of the business.
Companies are scrambling to learn all the various ways they can slice, dice, and mine big data coming in from across the enterprise and across the web. But with the rise of big data — hundreds of terabytes or petabytes of data — comes the challenge of where and how all of this information will be stored. For many organizations, current storage systems — disks, tapes, virtual tapes, clouds, inmemory systems — are not ready for the onslaught, industry experts say. There are new methodologies and technologies coming on the scene that may help address this challenge. But one thing is certain: Whether organizations manage their data in their internal data centers, or in the cloud, a lot more storage is going to be needed. As Jared Rosoff, director of customer engagement with 10gen, puts it: "Big data means we need ‘big storage.'"
Until recently, companies were only warming up to the possibilities of cloud computing. Lately, however, for many enterprise-IT decision makers, cloud is hot, hot, hot. The sea change now underway means many companies are quickly moving from "dipping their toes into cloud computing" to a full-fledged immersion, says Thom VanHorn, vice president of marketing for Application Security, Inc. In 2012, expect to see those same companies dive right in. "The move will only accelerate," he tells DBTA.
For enterprises grappling with the onslaught of big data, a new platform has emerged from the open source world that promises to provide a cost-effective way to store and process petabytes and petabytes worth of information. Hadoop, an Apache project, is already being eagerly embraced by data managers and technologists as a way to manage and analyze mountains of data streaming in from websites and devices. Running data such as weblogs through traditional platforms such as data warehouses or standard analytical toolsets often cannot be cost-justified, as these solutions tend to have high overhead costs. However, organizations are beginning to recognize that such information ultimately can be of tremendous value to the business. Hadoop packages up such data and makes it digestible.
For enterprises grappling with the onslaught of big data, a new platform has emerged from the open source world that promises to provide a cost-effective way to store and process petabytes and petabytes worth of information. Hadoop, an Apache project, is already being eagerly embraced by data managers and technologists as a way to manage and analyze mountains of data streaming in from websites and devices. Running data such as weblogs through traditional platforms such as data warehouses or standard analytical toolsets often cannot be cost-justified, as these solutions tend to have high overhead costs. However, organizations are beginning to recognize that such information ultimately can be of tremendous value to the business. Hadoop packages up such data and makes it digestible.