My consulting, project management, and research focus on planning and managing data intensive projects. My contact information is here.

A complete index to this web site’s technology related articles is here or use the search engine located here. This site also includes occasional book and movie reviews.

Dennis McDonald's Weekly Top Ten

For a given 7-day period this list shows the most popular Managing Technology posts.

Click to read more ...


The Commerce Data Advisory Council's 2nd Meeting: Storytelling, Staff Recruiting, and Complex Processes

There’s an understanding represented by this group that the data resources being stewarded by Commerce programs both reflect and are critical inputs to U.S. technical and industrial competitiveness. Hopefully this group will be able to facilitate an exchange of useful “lessons learned” and resources across the varied Commerce programs.

Click to read more ...


Ten Questions for People Who Manage Large Data Intensive Projects

I’ve been researching how to manage projects where the goal is to make large amounts of data open, accessible, and useful.

Click to read more ...


10 Basic Suggestions for Planning and Managing Data Intensive Projects

It is also impossible for the project manager to ignore how data management and data governance are handled in the organization as a whole. An individual project cannot be expected to change how the organization manages data overnight. Part of the project planning process must therefore include an assessment of the organization’s ongoing data management and governance practices and how they overlap with the goals of the individual project.

Click to read more ...


Problems and Opportunities with Big Data: a Project Management Perspective

By Dennis D. McDonald

Tap or click above image to download a .pdf of this article.Being able to make wise investments in “big data” capabilities may require more collaborative approaches to project management and decisionmaking than are currently being practiced. Two recent articles illustrate this:

  1. Big data’s big problem: How to make it work in the real world
  2. From DevOps to DataOps

The first article states an often heard complaint that, despite much marketing hype, real-world examples of big data value are few:

“It does not help when so-called experts suggest that an information revolution is changing—and will continue to change—the nature of the workplace itself. Big data projects can have a transformative effect on business operations and processes. Yet evidence of a knowledge lead revolt in the offices of global blue chips is thin on the ground. Instead large organizations seem to be drowning under a sea of information.”

The second article focuses on the opportunity side of the equation. It addresses how technology management needs to evolve in the face of growing technical data handling capabilities:

“From the top of the stack, more users want access to more data and more combinations. And from the bottom of the stack, more data is available than ever before—some aggregated, much of it not. The only way for data professionals to deal with pressure of heterogeneity from both the top and bottom of the stack is to embrace a new approach to managing data that blends operations and collaboration to organize and deliver data from many sources to many users reliably with the provenance required to support reproducible data flows.”

My own interest — making organizational data more open and accessible to both internal and external user groups — dovetails with both these perspectives.

Both views show how people are grappling with the challenge of using modern tools to make sense of the increasing volumes of data in digital form. If there is no clear understanding of what benefits will flow from big data investments, investment justification will be a challenge. Anyone who has ever sold a new tool or technique understands this. This is why practical demonstrations and understandable testimonials are so important. 

It is clear that tools and technology are improving in ways that can make sophisticated data usage more accessible and immediate to those with the necessary knowledge. For example, the turnaround time for processing large amounts of data is dropping as new platforms and tools are introduced. Processing speed and the ability to combine analyses of different data types can be documented objectively and technically.

One question is, can we demonstrate the real world benefits of such techniques in meaningful ways without requiring management to possess data science degrees? Just talking about the benefits of speed and volume in technical terms is not enough. Showing how these benefits translate into useful insights and support for useful planning and decisionmaking is required.

Healthy skepticism

The first quotation above, in my view, reflects a healthy skepticism brought on by much of the hype surrounding big data. Such skepticism can be a good thing if it forces proponents to articulate the “whys” and “wherefores” instead of just the “hows” of big data. This need for proof is why supportive “case studies” are in such demand at meetings and conferences. Consultants and vendors understand this. This is why so many stories about big data “successes” are industry sponsored.

The second article’s emphasis on infrastructure is typical of tech-oriented solutions that focus more on the technology side of the equation. Yet there is more here than just a veneer of newness. Combining “big data” with “DevOps” makes great sense. I especially like the emphasis on the need for communication and collaboration which are critical to figuring out how best to grapple with taking advantage of powerful new tools.

Déjà vu

I admit to having a sense of déjà vu when reading articles like these. “Tech hype” has always been with us. Mature tech managers and their bosses long ago mastered the art of healthy skepticism when hearing vendors expound on new technologies. A “show me” attitude is a healthy one to promote here especially when price tags are high, a lot of changes are required, and business value may be difficult to pin down.

There’s always a lot of faith and trust required at this stage of technology adoption. That tech vendors are investing heavily in tools, advertising, marketing, and hype need not divert us from asking the tough questions. Inevitably open discussions about how best to take advantage of new tools will lead to serious discussions about strategy, governance, quality, and costs.


Pushback can come in many forms ranging from good old-fashioned resistance to change (how many times have we heard, “You’ve got to change the culture!”) to major implosions of high-profile projects or vendors. What may be different this time around with big data may be the speed with which new tools are introduced  and the seeming ease with which tools can be used to analyze and visualize vast amounts of data quickly.

Range of changes

New tools and technologies may also stimulate the need for changes to current management processes which, more often than not, generates pushback from those being changed. Adopting big data tools and process changes may be associated with a range of organizational changes including:

  • The move to cloud services as a replacement for current infrastructure.
  • The need to learn new tools and techniques.
  • Resistance from business process owners who don’t want to change.
  • Overemphasis on tools and technologies while shortchanging business and strategy.
  • The need to align data services with core business needs, not just with “easy to do” and “low hanging fruit” initiatives.

While it’s certainly important for IT folks to understand the new data tools, employees and managers in all functional areas are impacted and need to articulate what they want from the data and the new tools.

This is where the need for collaboration and communication emerge along with the need for basic project management techniques and support. Stakeholders need to be identified and brought on board. Resources need to allocated and managed. Progress needs to be tracked and reported in some fashion. New products and services need to be aligned with goals and objectives that are important to the organization. Inevitable changes need to be addressed as learning takes place. Most important, project goals and objectives need to be clear and understandable to all.

No best practices

With that in mind I would have to say that, regarding the first article cited above, skepticism is healthy but this skepticism needs to be couched in terms that are specific not general. Searching for “best practices” and examples of whether or not other organizations are able to take advantage of big data applications may have little relevance to one’s own organization. Understanding one’s own needs and requirements must be the starting place, otherwise we run the risk of doing the same old things but with new tools.

One criticism of the second article is related to this. That is, while I do believe that we may need to re-think how we manage infrastructure and IT services to take advantage of new tools, services, and platforms, we also need to make sure that the needs of the organization are reflected in how priorities are set and initiatives managed. Again, that’s basic project management but it points out the need for communication and collaboration in how priorities are set and how solutions are implemented.

Whether we call this “DevOps” or “DataOps” is not the issue. The issue is whether or not we can effectively manage projects and programs that take advantage of big data while involving all necessary stakeholders throughout every point in the data value chain, not just IT staff and data analysts.

Data literacy and strategic alignment

If it is inevitable in today’s fast paced world that many fingers are going to be stuck in the “data management pie,” we need to make sure that the heads operating those fingers have a basic common understanding of how data are generated, managed, and used; let’s call this “data literacy.” Without this participants will talk past each other with the end result being inefficiency, blind alleys, and disappointment.

The connection between better data services needs to be aligned clearly with the organization’s goals and objectives. Again, this is a basic requirement for effective project management. Management, IT, and data analytics experts need to communicate and collaborate right from the start so that everyone is on the same page.

Related reading:

Acknowledgement: The author is grateful to Julia Glidden of 21c Consultancy Ltd for her helpful comments on an earlier draft of this article.

Copyright © 2015 by Dennis D. McDonald, Ph.D. Dennis is a management consultant based in Alexandria, Virginia. His experience includes consulting company ownership and management, database publishing and data transformation, managing the integration of large systems, corporate technology strategy, social media adoption, statistical research, open data, and IT cost analysis. Clients have included the U.S. Department of Veterans Affairs, the U.S. Environmental Protection Agency, the National Academy of Engineering, and the National Library of Medicine. He has worked as a project manager, analyst, and researcher throughout the U.S. and in Europe, Egypt, and China. His web site is located at and his email address is On Twitter he is @ddmcd.


Are you still distributing documents in .pdf format?

Are you still using .pdf documents as the only format for distributing your documents online? Here are some possible implications of such a strategy:

Click to read more ...


How I'm Using the New Google+ "Collections" Feature

Google+ recently introduced an easy to use “Collections” feature. It gives users the ability to create specialized online public collections of linked articles that can in turn be subscribed to and read by others.

Click to read more ...


Managing Data-Intensive Programs and Projects: Selected Articles

Digital and cloud-based services are changing how government IT resources are procured and managed. Of personal and professional interest to me is how data intensive programs are governed given growing interest in big data and open data. I’ve created this special compendium of posts that are relevant to planning and managing data related programs and projects. There are four groups:

Click to read more ...


Progress Implementing the DATA Act (draft)

I heard David Lebryk of the U.S. Department of Treasury speak on Data Act implementation this morning at the Johns Hopkins/REI Systems Government Analytics Breakfast Forum in Washington DC.

Click to read more ...


Data Program Governance and the Success of Shared Digital Services

Leadership and governance are key. If the tip of the data management “spear” is analyzing data in new and interesting ways to help solve important problems, someone still has to manufacture the spear, carry it, and train throwers to hit the target. Managing and governing the logistics involved in getting that spear to the battlefield may not be as much fun as hacking away at the data but it sure is important!

Click to read more ...