All in Big Data

The Changing Culture of Big Data Management

In some ways managing “big data” tools and processes is no different than figuring out how to manage any other type of technological innovation. The technology is introduced, experts emerge and help control and shape evolving practical applications, and management eventually figures out what is worth keeping and what can be discarded.

Understanding How Open Data Reaches the Public

In the long run, traditional top-down efforts at management control may be insufficient to ensure that both agency goals and the potential benefits of open data access — including unanticipated cponsequences — are realized in an efficient and cost effective manner. While leadership by the Federal government’s IT infrastructure will be necessary, it will also be necessary to ensure that ongoing efforts to advance open data access are managed efficiently both across agencies and in accordance with individual agency and program priorities; IT staff cannot do this on their own and will have to work closely with agency management.
It may be that the greatest challenge facing private entrepreneurs in developing new and valuable information products and services based at least partially on public data will be public resistance to paying for information, no matter how new, innovative, or unique these producrts or services are.

The State of Government Data Transparency, 2013

What I found most interesting and useful about the Data Transparency 2013 conference held September 10 in Washington DC was the diversity. The ability of Hudson Hollister’s Data Transparency Coalition to bring together such a high-quality group of speakers and attendees representing so many different perspectives was quite an accomplishment.

How Our Increasing Digital Connectedness Improves Government Program Evaluation

There will always be a need to conduct formal evaluations of how well government programs perform. Such evaluations must take into account the complexity of programs and the need to distinguish among short term and long term impacts and the intervening conditions that also impact program effectiveness.

Learning from the World Bank’s “Big Data” Exploration Weekend

If you’re serious about data analysis there’s probably no substitute for getting “down and dirty” with real, live, messy data. Sometimes you just have to sift through the numbers with your “bare hands” if you really want to extract meaning from descriptive statistics, predictive models, and fancy visualizations.

Transparently Speaking, Are Bad Data Better than No Data At All?

I’ve been researching government program transparency and the hype surrounding “big data.” Given OMB’s recent statement of support for improving access to accurate Federal spending data I’ve also been giving some thought to what improved access might actually mean, based on my own experience with data conversion and consolidation projects.