Clarity of vision is absolutely the most important part of database design. The data architect must understand the shape and patterns of the data being modeled. This lucidity arises when the designer understands the subject area, the goals of the target database, the nature of the data sources involved, and the internal lifecycle of the database objects in scope.
This understanding does not necessarily mean data architects must be able to recite the definition and valid values for every single data element; but it does mean they should understand the expected granularity of every table within their designs, as well as the expected sources, plus the anticipated data flow to get from initial source to final target. Occasionally the logic applied along the way is so complex that a business rules engine is needed; and while no one can be expected to recall all those directions at an ask, still the data architect must know that those rules exist along with the effect they are intended to have on the results. The data architect must find the balance for what is a practical implementation and what is not, while designing a path through to a solution that is achievable.
As a new subject area or industry is investigated, the designer’s thoughts are initially uncertain. The fundamental concepts are where everyone starts, and those highest-level concepts are exposed right at the beginning. The rest of the area of discourse is likely to seem ambiguous. Time is needed to comprehend further detailed concepts, lurking beneath the major players, such as descriptive niceties, hierarchies that categorize objects, and secondary activities.
Subtleties in relationships can drag on for quite a while, as supertypes/sub-types are sometimes just perspectives. Knowledge can be gathered by reviewing previous solutions’ databases set to be replaced or used as a source, as well as interviewing experts on the subject area of the solution space. If experts are not available, and one is left to decompose a previous or existing implementation, then the data structures themselves may be not be enough. Code creating and moving data will need to be evaluated to learn how content is actually used. Needing to slog through code very often can feel like a punishment; but jewels arise as areas of logical pay dirt expose critical business rules and transformations. Whether one is designing a new operational system, a data warehouse, or a data mart, a point is reached where the targets inside that design become clear. This understanding may be a subtle process, like the sun slowly rising on a calm morning, or it may be an “Ah-ha” moment as a final number hits the lock tumbler allowing the safe door to swing open.
In that moment of clarity, the goal is understood, the pieces work and flow in the mind of the data architect and can be expressed to the development team. Though the complexities will still exist, the designer now knows where and how to find those details. At this point, the designer should be clear on where to go to unravel even the most complex business rules.
Those moments of recognition are the joyful noises comprising the music of data modeling. Give a listen. While arriving at this moment of visionary coherence is a wonderful thing, it does not necessarily mean the work is as good as done. Flaws may still exist in the vision as each detail is probed and worked through. Misunderstandings about how things should work will be exposed. Such bumps in the road are a natural part of the process. Thus, data architects will need to adapt and evolve their designs for these hiccups. The larger the snag the more the previous clarity was flawed. Once the clear vision has arisen, only occasionally should a good architect run into larger-than-minor issues.