What matters most in data management in 2015? There are a lot of moving parts that data managers and professionals need to attend to in today’s enterprises. Databases need to be wide open and accessible to all parts of the business, but at the same time, secure and free of tampering. Unstructured forms of data—such as log data, documents, graphics, video, and social data—need to be prepared and ready for analysis in the same way structured files have been ready for years.
Databases need to be able to leverage all the advantages of their underlying platforms, yet also be capable of being moved to different platforms as business demands require. Critical data needs to be highly available at all times, without so much as a hiccup, but businesses often can’t agree on what data is considered critical, versus data that doesn’t have business urgency. Open source solutions offer ways to manage data that has heretofore been treated as unmanageable, but many organizations don’t have the skilled people who can work with these tools and platforms. Decision makers want to get at their organization’s data with many types of devices, but IT managers and professionals are hard-pressed to adequately support the variety of devices now used by employees, partners, and customers.
There’s no question that lately, data has been on the move, and it’s moving fast. The pressure from organizations may be on to facilitate even quicker transfer of data, but at the same time, database managers and professionals have never had more choices available to them to promote analytics, boost productivity, ensure security, and enable collaboration. A recent survey of 300 data managers and professionals conducted among readers of Database Trends and Applications, published by Unisphere Media, a division of Information Today, Inc., finds database administrators are busier than ever, and working in highly diverse environments (“The Real World of the Database Administrator,” February 2015).
Within the past year, NoSQL and NewSQL databases—rapidly deployable data stores for unstructured data—have moved from the novelty or experimentation stage to become standard parts of database shops. Close to one-third of managers report the DBAs that are responsible for managing relational database management systems are also involved in managing nonrelational systems such as NoSQL and Hadoop. Eight in 10 report employing multiple database platforms to support the multitude of applications within their organizations.
New configurations are also being introduced into busy enterprise data sites that offer to rearrange the way data is ingested and processed. Data lakes, for example, mean raw data can be brought in and stored until needed for a variety of uses —whether it means being transported to a Hadoop File System, data warehouse, or data virtualization environment.
Real-time analytics has become a key part of today’s enterprise database scene, fueled by the push into the Internet of Things. The market has been responding with tools and platforms that help enterprises ingest, manage, and develop meaningful analysis of data streaming in from various devices, sensors, and chips. The relational world is rapidly evolving as well, with new technologies such as in-memory processing which rapidly accelerates data processing and analytics to blazing speeds. Relational databases have also increasingly been opening up to support unstructured data and easily integrate with open source frameworks such as Hadoop.
Among the many options now available to database managers and professionals is cloud computing in all its various forms—as on-premises private clouds, public cloud services, or a hybrid of the two. This past year, cloud has become a ubiquitous part of mainstream enterprise computing, with security and reliability concerns quickly melting away. Along with the many suppliers of cloud (be they external or internal), there are many ways in which data can be managed—either supporting online applications through software as a service, through online databases in platform as a service, or infrastructure as a service.
Database managers and professionals have never had more choices available to promote analytics, boost productivity, ensure security, and enable collaboration.
This year, the market has also been seeing a variety of solutions to address the growing levels of mobility within enterprises. Cloud-based resources help connect empowered employees and consumers to back-end resources, and increasingly, powerful analytics are making their way to users via apps. There’s a recognition that more end users may be accessing applications and data via devices than PCs or laptops. At the same time, enterprises have been seeking and implementing mechanisms to ensure data security within these devices, from remotely wiping them clean when necessary to partitioning.
There’s one thing that can be said for all this—things have gotten interesting for data managers and professionals in recent years. They have become the caretakers and enablers of their enterprises’ most important assets, and, as a result, all eyes are on them to deliver the vital ingredients for building data-driven enterprises. Here are the eight things that matter the most in today’s market.
1. Empowering End Users
Increasingly, both the focus and locus of enterprise success have shifted to end users. The key is paving the way to help users get at the information they need, through as much self-service and automation as possible. This empowerment can be boiled down to one trend that has rocked the technology space in recent years: consumerization. As a result, IT and data management departments need to accommodate the preferences of their users. They need to work with end users to understand what technology—both in terms of devices and applications—they prefer to use, and what types of data and analytics they seek.
At the same time, many users are seeking the same simplicity, ease of use, and speed that they may find outside the workplace, such as that found on Google or Bing. Increasingly, this means less involvement with the IT department and more self-service. Of course, this doesn’t mean the need for IT or data managers will go away—if anything, they need to step up with more user-friendly tools to help enhance that experience—such as diagnostic and monitoring tools. The focus needs to shift from monitoring storage capacity or logins to managing the health and performance of back-end applications and services as well.
2. Keeping Data Secure and Trust in Data Strong
When it comes to competing on analytics, trust is everything. That trust goes in two directions. First, consumers need to trust that the data they provide to enterprises—be it identity information or financial details—remains in a secure place. Second, the organization’s executives need to trust the data they are receiving accurately represents what is going on across their enterprises. That’s why it’s even more important to ensure that data is timely, consistent, and of the highest possible quality. Plus, as countless examples of data breaches in recent years show, enterprise success is closely tied to data security. There needs to be trust.
3. Connecting to the Internet of Things
The Internet of Things, or IoT, may seem like the latest buzzword, but it goes deeper than that. Increasingly, organizations are recognizing there is a wealth of data that can be gathered long after products have shipped or services have been rendered. Within the IoT realm, devices, sensors, and chips—essentially, anything with an IP address—spread across a global network will stream data back to their designated users—or even original manufacturers. To a large extent, all these sensors, devices, and chips can be interconnected—and thus, less likely to end up in silos, where their benefits would be severely limited. The rise of IoT means data about customers and usage patterns no longer stops once the product leaves the shipping dock. Data is now built on a continuous feedback loop with customers through the lifetime of the product or service and well beyond. Data managers need to be prepared for the constant streaming of data well beyond the production floor or point-of-sale terminal.
Accomplishing this requires not only adopting or leveraging technologies and platforms that can handle large data volumes but also being able to cost-effectively discover, collect, process, and analyze large amounts of data in various forms. There has been some movement to establish connectivity standards for IoT, such as those being developed by the Industrial Internet Consortium and the Open Interconnect Consortium, but many of these efforts are still getting off the ground, and are representing competing vendor approaches. Data management platforms are needed to support and enable enterprises to ingest and analyze the wide variety of data coming in from IoT.
4. Building, Enabling, and Supporting Cloud-Based Services
Moving to the cloud may open up answers—and new vistas—for organizations struggling with big data, IoT, or real-time data requirements. With cloud, there’s an acknowledgment that no enterprise is an island anymore. Rather, enterprises can now gain functionality through multiple sources, many of which are outside third parties. Data managers will increasingly take on roles as advisors or consultants to the business, who can point business leaders to the right resources for their requirements, be they from within the organization’s own data center, or from outside cloud sources or software as a service providers. Cloud platforms also will help facilitate IoT data, with specialized applications built on NoSQL column-family store databases and integration capabilities that may be too complex to be administered on-site.
5. Promoting Deeper Understanding Through Analytics
Data analytics brings the power of enhanced decision making well beyond a few quants or high-level executives in the organization. It means managers and employees all across the organization have access to the insights that analytics can provide. Data managers need to open up data sources and tools to everyone who needs it—even outside the organization, as there is value in enabling partners and customers to do analysis from their perspective. In addition, this analytics output often needs to be delivered at real-time speeds.
6. Defining With Software
The software-defined data center—the amalgamation of server virtualization, software-defined storage, and software-defined networking—is bringing about new efficiencies that finally free applications and enterprise systems from underlying infrastructure and the complexity that has built up over time with layers of multiple platforms and devices from a wide array of vendors. For data managers, this means greater flexibility to provision new databases quickly and dynamically for new workloads and to scale up or down as users access the systems. Manual processes are reduced to a minimum, as much of this configuration or alignment work is automated.
At the same time, a software-defined enterprise also supports the rapid movement of data to where it is needed, assuring business continuity that meets the requirements of the 24x7 business. For example, backing up data has required investment in an array of equipment, software, systems, and network gear to enable the movement of data between servers and disk arrays. In an optimized software-defined data center, this could be accomplished virtually, with functionality and capacity added on a moment’s notice at no additional cost. The administrative overhead associated with data processing and storage imposed by databases can be mitigated, or even eliminated, if these functions are abstracted within software. Performance is also advanced as well, as these actions are contained within a virtual space.
7. Creating New Career and Business Opportunities for Data and Nondata Professionals
As mentioned above, data analytics is a capability that is increasingly being shared across organizations, including business decision makers. As a result, new jobs and roles are being created, often outside the data center. Within the data center, there is rising demand for data scientists and analysts who can take the data, decide how and what needs to be analyzed, and tie it to business needs.
Business professionals will see these kinds of capabilities in their own jobs, as well. The big data revolution isn’t just for data scientists, programmers, or statisticians—though individuals with these skills are essential to the success of data analytic efforts. Rather, to make big data actionable and convert it to success for organizations, there are a range of skills required. All types of managers and professionals—from marketing to finance to content creation—will need to understand the power of big data, and how it can provide their organizations the edge in today’s hypercompetitive economy. Along with technical data skills, organizations need people who understand the implications of social media data, IoT data, corporate performance data, and customer data. Executives, doctors, pilots, architects, librarians, and teachers all need to be adept at understanding and being able to dig into big data analytics. They will need the front-end tools and apps that can pave the way for them.
8. Creating Data-Driven Organizations
Ultimately, the goal is to build up these capabilities to the point at which the enterprise becomes data-driven. In this type of environment, analytics is embedded, shared, and available to all decision makers across their enterprises. This is supported by business leaders who paint the vision in which any and all decisions and processes are underpinned by data analytics. Accordingly, all necessary resources—from technology purchases to employee training to skills acquisition—are made available to continue this effort.
The way these initiatives are advanced is through the establishment of cross-enterprise committees, or even centers of excellence, to ensure that data analytics and management are promoted across all silos and channels, free of encumbrances such as organizational politics. These efforts go right to boosting the bottom line. The value comes out of both the operational streamlining analytics enables, as well as the embedding of analytics capabilities into products or services—or the development of data products themselves.
Database managers and professionals have never had more choices available to promote analytics, boost productivity, ensure security, and enable collaboration.