Traditional approaches to analytics and AI design consider outcomes in the context of the complete systems. As AI-driven systems become more prevalent and interconnected, profiling each decision point in the process becomes crucial. Automated systems aside, this discipline also enhances strategic decision making and design of traditional business intelligence (BI) and analytics information, which also operate in complicated ecosystems.
Stipulating that the problem to be solved or question to be answered is properly identified, comprehensive decision profiles are the harbinger of effective system design. To make good choices around analytically driven decisions, the following must be considered:
- Complexion—Is it informative, influential, or determinative? Often overlooked, if not assumed, this simple rubric is the foundation upon which decision intelligence rests. Establishing what constitutes a good decision sets the stage for all other design decisions, including what type of analytic or AI model may be appropriate or not.
- Error tolerance—How accurate do the results have to be to be useful? Can you make a good decision with poor or incomplete information? When considering tolerance for errors, the organization must factor in its own risk tolerance, regulatory and legal requirements, and business policies. Trust is often freely given, but it’s quickly lost when the expectations of the organization do not align with those of external parties. Therefore, the expectations of your end user and customers must be factored in as well.
- Knock-on effects—What happens next? Tracing upstream and downstream dependencies is often discussed in the context of autonomous agentic AI systems in which small deviations or errors create an escalating failure cascade. It applies equally to more traditional business processes and strategic decision making. It is the rare decision that happens in a vacuum, so not considering what comes before or after can result in widgets that work while systems fail.
- Ability to adopt—What knowledge or understanding does the end user or recipient require to make this decision with the provided input? Considerations include incentives or psychological factors that may influence usage and adoption beyond data and AI literacy, as well as supplemental inputs or out-of-bound conditions.
- Data coverage—When choosing what to decide, it is worth noting that decisions are rarely all or nothing. Clarity regarding the scope of the available or expected data input (What cases or populations are represented? Are inputs well-bounded?) determines the weight to be given model outputs or what aspects of a process can be confidently automated.
- Input reliability—Are data inputs expected to be stable or volatile? If it’s the latter, what level of volitivity can the system absorb and still support a confident or correct decision?
Intelligent decisions are not just the outcome of good analytic or AI models. Decision intelligence lives or dies in the application of a model’s outputs. This requires a heightened level of discernment well before the model or system itself is designed. Simply put, effective analytic systems rest on decisions and data. Whether it’s BI or agentic AI, meaningful analytics adoption requires renewed attention to these seemingly basic factors.