“Temporality” is a term that database managers know well, but it may be a new one for business managers. That has to change, as the temporality your database supports—or, how it handles time—could be the difference between whether or not the business will increase revenue, pay a fine, or identify new opportunities. Especially important in this regard is “bitemporality,” which is the ability to examine data across different points in time.
Indeed, a bitemporal database is much more powerful than a temporal database. While a temporal database tracks valid time, capturing data “as it happened in the real world,” a bitemporal database involves valid time and system time. System time records when something was recorded to the database.
In other words, bitemporal databases capture information “as it actually was.” For example, a temporal database can tell us where John Smith lived on Dec. 6, but a bitemporal database can tell us where John Smith lived on Dec. 6 as we knew it on Dec. 15. With a bitemporal database, organizations can answer the critical questions: What did you know, and when did you know it? It helps ensure that there is always a full and accurate picture of data at every point in time.
Regulatory Considerations
Bitemporality is important to any company that wants to more effectively leverage historical data, but it is particularly important to organizations in regulated industries.
In the financial trading business, for example, trades can be amended after they are made. If there is no time-based record of these changes, future analysis of trading positions could be compromised—and organizations could face major fines by regulators. Bitemporal design provides an accurate picture of the entire lifecycle of a trade review, including when changes to counter-party names, transaction IDs, or price corrections occurred.
In the healthcare industry, bitemporal capabilities can provide a more accurate picture of a patient’s history and can help connect the dots between an illness and its antecedents.
In the insurance industry, bitemporal helps by providing a clear determination of coverage over the course of history, ensuring that even if there are retroactive changes, data is never overwritten.
But while bitemporal capabilities make especially good sense for regulated industries, any business can use databases with bitemporal support to make sense of seemingly disparate data, enabling them to better understand why something happened and even predict future events. This data can help determine customer patterns and behavior, for more strategic development of new products and services.
Generally speaking, bitemporality makes sense for the following scenarios:
- Regulatory requirements, enabling companies to avoid fines
- Audits, preserving the history of all data, including the changes made to it
- Investigations and intelligence, maintaining information so it is possible to see exactly how data was updated based on what was known at the time
- Business analytics, enabling companies to run more complex queries based on time-based data
The Cost of Doing Bitemporal Business
While data analytics capabilities can come at a premium, bitemporal technology can actually save companies money and provide opportunities for increased revenue.
The cost per gigabyte of data is decreasing, but organizations today are spending more on storing historical data because they are dealing with so much more of it—for regulatory reasons, but also because companies tend to hoard data. Bitemporal design helps keep storage in check because it avoids the need to set up additional databases for historical data.
In addition, by enabling developers to write queries that can easily access historical customer behavior data, bitemporal design helps developers purpose-build products based on customer demand.
Barriers to Bitemporality
With that said, while there is no question that bitemporality will benefit businesses across all industries, getting there is not always easy. That’s because traditional relational databases are not well-suited to handle bitemporal implementations. Here are some of the reasons:
- Integrity constraints: Relational databases have referential and entity integrity constraints, not to mention defined schemas. When bitemporal columns are added to a relational table, they can wreak havoc on the relational data model.
- Schema evolution: It is difficult to change schema in a relational database in general, let alone when adding bitemporal data.
- Multiple data models: Bitemporality involves integrating multiple data models and data silos into a single source of truth, a difficult task for schema-bound relational databases.
- Performance decline: Bitemporal queries consider multiple axes of time and often span multiple servers.
- Add-on costs: Bitemporal is often not built into the database, it is an add-on component that comes with additional costs.
Database developers and vendors have made a number of attempts to work around the issues that relational databases have with bitemporal capabilities, but with little success. In the end, the workarounds are just that—workarounds, not solutions.
Many companies have been able to fully leverage the bitemporal model through the use of NoSQL databases. With NoSQL—or “not only SQL”—databases, data is not modeled according to tabular relations, providing the level of flexibility required to fully exploit bitemporality. Enterprise-class NoSQL databases also offer features that ensure high levels of performance, data resiliency, and security.
Bitemporal design is about historical data, but it’s really much more than that. It’s about strong data governance, strategic planning, risk management, and competitive advantage.
Jim Clark is senior director of product management at MarkLogic.