Making Data Governance "Agile" Again
Introduction
In Can data governance be agile? Nancy Couture identifies the most important first step in establishing an "agile" data governance program:
An alternative, and more agile approach, is to identify smaller data governance initiatives based on strategic projects or business needs, and build from there. In this way, the organization can keep everyone apprised of progress and decisions, but the work effort is limited and focused. And the planned business value can be realized more quickly, thus increasing interest.
The First Step
The importance of this first step, especially the part about "... based on strategic projects or business needs," shouldn't be underestimated.
Since beginning my own research into big data project management over a year ago I realized that data governance shares a lot with many other organizational priorities that generate a call for a comprehensive and strategic approach. Data when looked at as a resource permeates the organization. There's a tendency when examining how best to manage data comprehensively to suggest a wide-ranging solution that might be needed but that unfortunately might be doomed to failure.
No More Ocean Boiling
The prospect of failure exists partly because the day of the "boil the ocean" approach to massive enterprise level projects is long gone, especially in organizations that are legacy-system dependent and heavily siloed. Managing comprehensive programs that involve significant changes to technology, culture, and business processes requires a great deal of support and management skill, time, attention -- and money.
Also needed is the realization that change is inevitable and plans that attempt to plan too much inevitably have to be changed -- this is one of the reasons for the popularity of the "agile" movement which focuses on delivering value more quickly in manageable chunks.
Needed: Both Tactics and Strategy
As Couture suggests, it makes more sense to start off with something that's (a) important and (b) definable.
This doesn't mean you shouldn't think strategically when faced with "upping" your data governance game. When thinking about data governance it pays to think both strategically and tactically. We want to deliver value in the short term while at the same time providing a foundation for the future. But how?
An Approach
The following is based on ongoing discussions with consulting colleagues William Moore and Mark Bruscke. Will has evolved a structured approach to data governance that emphasizes how language and terminology are used. Mark is an experienced data architect with much enterprise level experience. My own focus is on IT strategy and project management especially in data intensive projects.
What follows are some of the actions we recommend to address both the tactics and strategy associated with improved data governance:
- Establish Data Governance Management Structure
- Understand Language and Terminology
- Manage Metadata
- Perform Data Stewardship
1. Establish Data Governance Management Structure
Even for "tactical" projects focusing on data governance applied to selected high-importance problems or issues, key stakeholders must be represented including not only IT but potentially all groups impacted by how data are managed. This might include groups as diverse as Legal, Marketing, Customer Service, Research, Data Science, Sales, and Administration. Any group that might somehow be directly or indirectly associated with producing, managing, or using data should be considered for some sort of stakeholder role.
Sometimes management of the data governance process is formalized in a "data governance council" which oversee policies and practices regarding data and metadata. This may not be appropriate for short term or tactically focused projects managed following an agile process. Eventually, though, such "council" will have to set policy and even resolve disputes when different groups or systems use different terminology to refer to what appears to be the same concept. Starting such a group requires settling on a data governance council structure (e.g., centralized, federated, hub-and-spoke, etc.), establishing and sharing a group charter, recruiting members, and overseeing the organized data governance operation.
Initially a "data advisory council" should include, even in a tactically focused short term project, those who are best able to articulate requirements and user stories concerning data associated with the problem or issue being targeted.
Given the potential need for experimentation at this stage, the team must be ready to change as requirements (and user stories) evolve. Flexibility and collaboration are needed. The more focused the initial targets are, the better able the team will be to share information and make decisions.
2. Understand Language and Terminology
Whether dealing with a short term tactically-focused project or a longer term project with strategic implications for how the organization manages and uses data, participants need to understand how language and terminology are managed and used in relation to the specific problems or issues being addressed.
A structured and open process will be needed to identify and document the most important concepts that need to be understood and documented, regardless of the variety of ways people and machines refer to these concepts.
Moore calls these “key business attributes." The process of discovering, defining, and modeling them is "business attribute analysis."
The basic approach for identifying key business attributes involves not only reviewing how data and metadata are currently defined but also analyzing how important business concepts are referred to in the organization's documentation, emails, reports, and other communication media. This is one of the reasons why initial steps need to focus on well define projects or issues: scope control.
One output of this collaborative business attribute analysis, carried out with support of the data governance team, is a high level model of key business attributes and how they relate to the problem or issue that's initially being targeted. Two important questions to address during this process are:
- What do we know now about solving this problem or making this decision?
- What do we need to know to solve this problem or make this decision?
The business attribute model that emerges from this analysis will surely evolve but will establish the basis for the next step by identifying what data need to be managed.
3. Manage Metadata
Our goal here is to create a metadata repository that catalogs key data concepts (see above) and how these are expressed and used throughout the organization’s business processes, systems, and databases.
For a particular problem domain defined by the process or issue being addressed, we create an inventory and catalog of target data related to the key business attributes identified above. We document where and how these data are used and who is responsible for them.
Our basic approach is to review all relevant business attributes, data, definitions, decision rules, data models, and the manner in which data are linked, expressed, and used throughout the targeted problem domain.
Process and information are what are important here, not the tools used. We recommend in order to proceed quickly at this stage that existing collaboration, database, and document management tools be used. Accessibility and ease of use are major concerns. We want to move quickly and deliberately but not in secret.
The primary deliverable of this process is an evolving catalog that supports data governance, consistency, data transformation requirements, and (where necessary and justified) intelligent standardization.
Most of all we want to clearly and transparently document how data are used in machine to machine, machine to human, and human to human communication. And we want the systems and processes used here to be reused and where possible scaleable.
4. Perform Data Stewardship
We need to establish a dedicated and staffed data stewardship process that works across and supports the systems and processes described in (1), (2), and (3).
The Data Steward provides the "boots on the ground" in the data governance process. Our role as consultants is to help establish the above systems and processes and to function initially as the client's data stewards while simultaneously mentoring others so the client can manage its own data governance and stewardship processes and system
Once a specific problem or issue is addressed and improvements identified for how data and metadata should be governed and used to address that problem, the processes and procedures we have gone through are represented as documented processes that can then be adapted for the next problem.
Conclusions
In summary we recommend:
- Start with a well defined data-dependent problem or issue.
- Move quickly but stay disciplined.
- Be collaborative and transparent.
- Treat this as a learning process; knowing in advance what benefits better data and analytics will bring is impossible.
- Keep track of costs.
- Don't just focus on existing technology and structured data.
- Keep management informed and involved.
- Document what can be done better next time.
- Focus on detail but keep the big picture in mind.
- Use collaboration and transparency to overcome resistance, not hierarchy and authority.
Copyright (c) 2017 by Dennis D. McDonald. Interested in applying these ideas to your own organization's data governance? Contact me in Alexandria Virginia at 703-402-7382 or by email at ddmcd@ddmcd.com.
