‘Ethics isn’t about knowing how to ask questions. It’s about knowing “how to navigate” the answers to those questions.’ This illuminating statement opens Reid Blackman’s short but incisive LinkedIn treatise on the need for ethicists in AI governance.
Replace “ethics” with “governance” and you will find the reason so many well-intended governance efforts fail. Your governance program is only as good as the questions asked, the answers proffered and the actions that result. Each element—asking, answering, and acting—is critical. When it comes to sound decision making, two out of three does not a responsible organization make.
Questioning whether your governance efforts are merely inquisitive? Here are five signs.
Prescribed Questions
Do your governance evaluations rest solely on predefined questions or a set checklist? If so, take heed. Preset questionnaires or constrained (e.g., exclusive) validation questions can narrow your field of view: constraining rather than encouraging inquiry. When it comes to technology, and AI in particular, context counts. The issues confronting a manufacturing quality control system are different than those of a credit assessment or a self-driving vehicle. A checklist can make sure the discussion happens. It cannot adequately account for the nuances of every discrete application.
Note: This phenomenon is not new to AI governance. The seeds of many a data or business governance program’s demise lies in singular, purportedly exhaustive rules or “do’s and don’ts” to guide them all. The problem? Miss one and chaos ensues. When it comes to effective governance, critical thinking—not careful check-boxing—is key.
Predetermined Outcomes (aka ‘Proceed Until Tackled’)
To quickly identify whether your governance process is just for show, simply ask, “What if?” What if concerns are identified? What if the concern is ethical versus legal? What if the system doesn’t behave or isn’t used the way we expect? How will we know? What is the tolerance and threshold for error? Will (can) the project be paused, scrapped, rethought? Have those outcomes ever come to pass? If the answer is not clear, the default is “go,” and there is no precedence for “stop,” take heed.
A Singular Focus on ‘Why’ (aka ‘Yes’)
I had pizza for dinner. Again. Smart decision? Not particularly. But, by leaning into the “why” (i.e., hangry, no cooking required, fewer dishes, it’s only one night) at the expense of “why not” (i.e., unhealthy, food coma, expensive, this is the X night in Y days), pizza prevailed. Is pizza fundamentally bad? Of course not. The composition and frequency of intake as well as the eater’s objectives all play a role.
A silly example, but the point remains. Responsible innovation often requires compromise. Without deliberately evaluating both the upsides and the downsides of a given solution, the cost of unintended compromises may be catastrophic.
A Single Source of Truth
Does responsibility for asking questions and probing the implications of the solution at hand rest with a single entity? Be it a data science team, risk management group, or a product owner, if questions are solicited from a single source—whether a group, function, or individual—take heed. Critical perspectives are being overlooked. Likewise, is there is a single source responsible for answering said questions? If so, also take heed. Is the single source you? Proceed with caution.
Unclear Accountability
Is the person “who decides what” clearly defined? What rights and responsibilities are afforded to the development team? And to the product manager or process owner? Risk manager? Legal? Executive steering committee? How are conflicting opinions and competing concerns addressed? Without clear decision rights and procedures for appealing or escalating decisions, take heed. There is simply no reason to spend time posing questions, much less mindfully considering them, if resulting answers will fall on deaf ears—or no ears at all.
Which is not to say that your efforts to bring visibility to issues confronting your organization should be abandoned. Rather, it is that attention may first need to be paid to what and how your organization intends to govern.