Earlier this year, I had the privilege of serving on the organizing committee for the DataTune conference in my hometown of Nashville, Tenn. Unlike many database-specific or platform-specific conferences, which tend to focus on analytics and data science, DataTune covers a wide variety of topics such as database engineering, development, administration, and analytics. I was delighted to be joined on the committee by two friends, Dustin Dorsey and Cameron Cyr, authors of Unlocking dbt: Design and Deploy Transformations in Your Cloud Data Warehouse.
Posted April 11, 2024
2023 was a year marked by innovation and change in the enterprise technology landscape. Companies of all sizes continue to accelerate their digital transformation efforts and leverage artificial intelligence and machine learning-powered solutions, including observability tools, to keep up with today's increasingly complex network environments.
Posted December 21, 2023
Finding a needle in a haystack is next to impossible—so much so, it's become shorthand for the frustration of a fruitless search. Now, imagine you've been told there may be a needle in the haystack, or one could appear at any time, even in a place you already searched. You spend the most valuable hours of your day searching for the needles, but in our case the needle is performance or reliability anomalies. Meanwhile, the haystack grows larger by the second, as more and more is added to the overflowing pile.
Posted December 14, 2023
Mankind has long dreamed of the promise of automation. Our earliest ancestors imagined it as something the gods could wield. In Homer's The Odyssey, Hephaestus, the Greek god of smithing and craft, uses automation to complete simple repetitive labor. Buddhist legends speak of automated guards who watched over the Buddha's relics.
Posted October 12, 2023
Data is the cornerstone of the modern digital business. IT teams, database professionals, and business leaders use data to make informed decisions, increase efficiency, and stay ahead of business objectives. Applications also need data to run efficiently to ensure teams don't encounter any bottlenecks.
Posted August 10, 2023
Database professionals are the backbone of any modern company. They are responsible for managing, organizing, and analyzing vast (and rapidly increasing) amounts of information. Their work helps organizations make informed decisions, remain secure, increase efficiency, and stay ahead of the competition. The next generation of data management capabilities is essentially a prerequisite for companies seeking to implement cutting-edge technology and processes such as AI and AIOps.
Posted June 08, 2023
2022 was a big year for observability. Many businesses made concerted efforts to gain a better understanding of its value. Leading technology research firms applauded its potential impact to simplify increasingly complex IT environments, with Gartner featuring observability in its "Top 10 Strategic Technology Trends" report. Analysts predicted a bright future for the technology, with some suggesting observability adoption would increase at a compound annual growth of 8.2%.
Posted April 13, 2023
In 2023, companies will live and die by their data strategies. Massive increases in information generation, the cloud, and innovative technologies will be top of mind for database managers, all while an uncertain economy continually threatens budgets. For companies to build and maintain exciting applications that customers love while increasing the bottom line, they need to plan, adapt, and execute effective strategies for their data and databases.
Posted February 09, 2023
The British royal family safeguards its crown jewels—the treasures held by the English monarchy comprising more than 100 objects estimated to be worth billions—with world-class security systems, including two-ton steel doors and bomb-proof glass. When the crown jewels are transported or moved, such as for a royal wedding or the recent coronation ceremony, they are guarded closely by elite, ex-military members at all times—with access restricted to only a few security-cleared individuals.
Posted December 08, 2022
British mathematician Clive Humby famously said in 2006 that "Data is the new oil." In the 16 years since, companies of all sizes have drilled for and stored more and more data about their customers and business operations to drive performance and growth to reach their goals. In conjunction with the increased use of the consumer internet, data creation has exploded in recent years—considering the majority of the world's data collected over the course of human history has occurred in just the last two years alone. As a result, entire businesses and industries have been built solely on having access to unique and useful data.
Posted October 06, 2022
Working with databases causes friction. On one side, nontechnical managers want data pros to skip the jargon and clearly describe their needs. On the other side, data pros wish their managers had a deeper understanding of the tech involved. Nontechnical managers may think they need to understand the technology better and get into the weeds to close this inherent trust gap. But this isn't the case—managers are busy tracking resource allocations, Gantt charts, project plans, and punch lists. They must also fill open slots, upskill employees, and keep people from falling prey to the Great Resignation.
Posted August 11, 2022
Data plays a vital role in any successful business. Protecting this data is mission-critical, and it falls to database administrators (DBAs) to organize, maintain, and secure it. The most effective way for DBAs to protect data is through database backups, a process designed to copy the data and schema from an existing database and save it elsewhere for future retrieval. Like any process in IT, however, there are a few factors to consider when devising a database backup strategy to ensure it meets your business needs.
Posted June 02, 2022
Most database administrators know database servers didn't initially come in a cloud or cluster. Once upon a time, DBAs had to reconfigure disk files and handle data manually. Now, with virtualization and the shift toward the cloud, the evolution of database administration yields more opportunities to automate tasks and fewer reasons for DBAs to get their hands dirty.
Posted February 08, 2022
Modern businesses make decisions based on data that's been carefully collected, extracted, formatted, and analyzed. Data powers competitiveness, brings new products to the marketplace, and improves the customer experience. The use cases are nearly endless, but there's a catch: Working with all that data is becoming increasingly complex.
Posted October 05, 2021
The global pandemic has forced companies to pivot over the past year in significant ways. Whether it was transitioning to a remote workforce or reimagining how to interact with customers, digital transformation initiatives within businesses were happening tenfold. As part of that, the pandemic notably accelerated corporate planning for cloud computing.
Posted August 02, 2021
Traditional DBAs who have spent a lot of time with all the nerd knobs at their disposal need to think more about the movement of data. Instead of thinking in terms of data administration—and running the risk of becoming redundant—they should become more familiar with how data moves in and out of their enterprises.
Posted June 10, 2021
Remember, all data is dirty—you won't be able to make all of it perfect. Your focus should be on making it good enough to pass along to the next person. The first step is to examine the data and ask yourself, "Does this data make sense?" Data should tell a story or answer a question. Make sure your data does, too. Then, before you do anything else, make a copy—or backup—of your data before you make the smallest change.
Posted April 06, 2021
Slow is the new broke. But things don't have to be slow to be broke. A poor user experience with your online ordering system will hurt your bottom line.
Posted February 10, 2021
COVID-19 is accelerating the drive toward SaaS apps and cloud adoption, and many companies are now adopting a hybrid cloud approach. Most are going about their transformation wisely, with experienced personnel who are measured in their approach. But we've all heard of companies going dark when a cloud provider did the unthinkable and went down: Everything was great until it wasn't.
Posted December 10, 2020
Citrix and Microsoft have a complicated history, but they've recently taken their relationship to new heights by announcing an extension of their partnership to help businesses better manage a remote workforce. Since the pandemic began, CIOs have been looking for ways to bring the office experience to the home and in the process put their enterprises anywhere. At the same time, CFOs are wondering just how to do so with the evolution of OpEx subscription options.
Posted October 08, 2020
Virtualization of x86 servers is ubiquitous for obvious reasons. Cost savings, efficiencies in provisioning virtual machines (VMs), recoverability, and the ability to move workloads are a few of them. There are, however, key VM and host metrics and events you should keep an eye on if you suspect your database performance is being impacted by running in a virtual machine. Let's walk through those metrics using VMware ESXi as the basis of the discussion.
Posted August 11, 2020
While Postgresql.org has been around since 1996, its origins go back to the mid-1980s, when Michael Stonebraker, who led the Berkeley team that developed Ingres, started a new project called Postgres to build upon the ideas in Ingres. While most PostgreSQL users rely upon the free and open source implementation, there are several variants.
Posted June 10, 2020
Even before the IT elements of data optimization begin, aligning organizational culture around a data-driven mindset will be a major challenge. Making the case for data optimization is important. Even before the IT elements of data optimization begin, aligning organizational culture around a data-driven mindset will be a major challenge. Making the case for data optimization is important.
Posted April 08, 2020
As with everything, there are trade-offs when it comes to indexes. A sign of a good database performance tuning and optimization solution is that it will analyze every query per database instance, look at all the tables used with each query, and then make a recommendation for table tuning if there's a positive impact.
Posted February 10, 2020
Remember when standing up a new database instance involved modeling the application using the underlying database to figure out what hardware (server, storage, etc.) was needed to support the new instance? And, since most organizations did capacity planning on a spreadsheet using linear metrics, there was a lot of estimating involved, thus over-provisioning the hardware was standard procedure.
Posted December 01, 2019
Hard skills, while still valuable, are only part of the necessary skill set to be successful as a data professional. But hard skills have a ceiling. Hard skills can be replaced. Soft skills have no ceiling and offer more value than many hard skills.
Posted October 01, 2019
Here is a list of common wait types and techniques that every DBA (or wannabe DBA) should know. While there are many more wait types than listed here, understanding these will give you a leg up when it comes to optimizing and tuning your SQL Server database performance.
Posted August 07, 2019
How do we define database workload? Good question! I've researched many approaches to this question and still don't have a definitive answer or way to measure. Let's dive into some of the research and personal thoughts I've had around this topic.
Posted April 09, 2019
There are a lot of folks out there who make a living (many of them quite a good one) doing database performance tuning. Why? Tuning requires a high degree of knowledge and performance skill, is time-consuming, and means knowing the right diagnostics to collect when performance hits occur. Because of these and many other reasons, database performance experts thrive—and let's not forget about job security.
Posted February 08, 2019
Anomalies—Predicting the Past
Posted December 04, 2018
As we enter a world of machine learning and data science, are there any gotchas or negatives? It sounds as if it is all sunshine and rainbows, but, as the title to this post alludes, there are.
Posted October 10, 2018
Shark attacks and ice cream sales follow the very same trajectory when charted. When ice cream sales go up, so do shark attacks. When ice cream sales decrease, shark attacks decrease. With a correlation coefficient of very close to 1, we can deduce that sharks like to attack after we've recently had ice cream!
Posted August 08, 2018
Our data capture and retention requirements continue to grow at a very fast rate, which brings new entrants in the SQL and NoSQL market all the time. However, not all data is created equal. Companies recognize that disparate data can and should be treated differently. That means the way we persist that data can be extremely varied. Now, enter applications that need to access all that data across a very heterogeneous landscape, and we get to the point where we're reinventing the data access wheel every time someone needs to spin up another application or introduce another data source.
Posted June 01, 2018
Moving to Automation Means Many Decisions
Posted April 12, 2018
The CTO of a major database vendor recently stated at a conference that basically DBAs would soon be out of a job. The gist of the keynote was autopilot flies better than humans, autonomous cars will be safer, so why would we not do the same for our databases (remove the human error)? What is the validity of this line of thought, and how might it be implemented? What does an autonomous database look like?
Posted December 01, 2017
If I were to ask if your database load profile is good, bad, or ugly, how would you reply? If you are unsure about how to answer that, you are not alone. There are a lot of factors that come into play—one of the biggest being, "Compared to what?" Most people would agree that having a historic baseline or profile of database activity is important. So, let's move beyond that and get into why it's important and what we can do with those historic datapoints.
Posted October 18, 2017
Poorly written SQL statements can cause significant performance problems in your database environment. According to some experts, poorly written SQL can cause up to 70% of performance problems overall. Adding resources can mask many issues surrounding poorly written SQL, but comes with a cost. Is writing good, quality SQL (including block code—stored procedures, packages, functions, etc.) a dying art? And if it's so important, why is that?
Posted August 09, 2017
Resources used to be expensive. Resources used to be scarce. Resources used to take a long time to provision. As such, it made sense to put resource consumption at the top of the list when talking about database performance. Those days are gone. With more than 80% of databases running in virtual environments, where hardware is more commoditized every day, access to physical resources—CPU, memory, network, and disk—whenever needed is much easier. In fact, Moore's Law predicts that technology advancements will double every 2 years. Well, most physical resources are certainly on pace with that, or better.
Posted June 01, 2017
There are many points in life where you may ask yourself whether it is better to build or buy. Think of a new house, a business, or an application. Regardless of the object of discussion, answering certain upfront questions can act as a guide to help you along the path to the right solution. Given the increasing importance, complexity, and breadth of database systems, the question of whether to build or to buy database monitoring is an important one to consider.
Posted February 08, 2017
There are three big challenges facing today's DBAs—a shift to an application-centric focus, the need to support multiple database platforms, and expanding responsibilities for managing database performance in the cloud as well as on premises.
Posted December 01, 2016
As technology professionals, one of the most important aspects of our jobs is to advise our organizations on the use of new technologies. However, the challenge is identifying the right technologies. This revolves around three very specific risks.
Posted October 07, 2016
When you think about the role of a database professional, you probably don't include "cost savings" in the list of responsibilities. Maybe if there were a clearer correlation between the work of database professionals and money, people would pay more attention. Well, it turns out, there is.
Posted August 04, 2016
Almost all organizations have migrated at least some infrastructure to the cloud. In fact, just 9% of IT departments have not migrated anything. Furthermore, databases rank in the top three for both infrastructure already migrated to the cloud and infrastructure with the highest priority for future migration.
Posted June 09, 2016
It seems every week there is another data breach in the news, which translates to millions and millions of personal records, credit card numbers, and other pieces of confidential information stolen each month. The victims of these breaches include important companies with professional IT staff. Now, you may be thinking: "Shouldn't the network guys be responsible for security?"
Posted March 31, 2016
It is sometimes said that a key difference between an IT administrator and a developer is that admins are all about stability, while developers are all about change. However, life isn't easy for admins nowadays because there is so much change. There is the usual change, such as replacing a host or upgrading to a new OS or DBMS. Then, there are the really big changes impacting every IT department.
Posted February 10, 2016
Database performance tuning is a complex but extremely important task. However, it can be difficult to effectively optimize databases when there are other "fires" to put out, limited resources, and an increasing number of databases to look after. But that doesn't mean it's impossible, especially with the right approach.
Posted December 02, 2015
Ever since Linux became a viable server operating system, organizations have been looking to all kinds of open source software (OSS) to save on license and maintenance costs and to enjoy the benefits of an open platform that invites innovation. If you're considering MySQL or another open source DBMS as either your primary database or to, perhaps, operate alongside your existing commercial systems, such as Oracle or Microsoft SQL Server, for one reason or another, here are seven things to keep in mind.
Posted October 07, 2015
Whatever the reason, database administrators (DBAs) frequently sit in a corner alone, siloed from the rest of IT, with their potential for impact ignored. Organizations in which this happens—which is most companies—are typically characterized by an attitude that the role of the DBA is to simply keep things running. The order from the top is "Just make sure the database doesn't break anything, OK?" This is a missed opportunity and a waste of a very valuable resource that hampers IT organizations everywhere.
Posted August 10, 2015
By and large, it seems like today's IT professionals belong to one of two camps: either traditional on-premises IT or the cool, new age cloud. Sometimes, it can even seem like we are talking about two different species. After all, for many of us who come from the client-server era, cloud can be intimidating. Even worse, it can be a source of real anxiety; becoming obsolete is a constant risk in an industry that evolves as fast as ours.
Posted June 09, 2015
Four years ago, moving a database to the cloud required courage, optimism, and confidence (or ignorance). Cloud platforms were young, and fundamental security, performance, and management issues were far from solved. How much things have changed.
Posted April 06, 2015