Development Gateway and the Eggheads of Open Data

Nearly ten years ago when I began working with fine folks at Development Gateway, I was still an untenured assistant professor at The University of Texas at Austin (UT). Josh Powell, now CEO, was freshly promoted from his internship position with DG. As a young start-up, DG was doing something quite daring. It was reaching out to work with “eggheads” in academic institutions like the University of Texas, Brigham Young University, and the College of William & Mary (home to AidData). The goal was to find willing partners to dive into the relatively new field of open data, and specifically to test one risky hypothesis: If we could geospatially map global aid flows at a highly refined level, could we make aid data transparent in a way that would empower key stakeholders within donor organizations and aid-receiving countries to make better decisions about where they put aid dollars?

Determined to give the “eggheads” a dose of reality, then DG CEO, Jean Louis Sarbib, and Director of Innovation, Steven Davenport, invited Dr. Michael Findley and myself to participate in the Aid Management Program (AMP) Best Practices Workshop in December 2010 in Nairobi, Kenya. Here we pitched what we thought was a simple idea: we would work with one AMP country to use ArcGIS to map the subnational locations and detailed activities of all aid projects in their country. It was an ambitious attempt to show proof of concept on the promise of data visualization previously launched by the World Bank under its Mapping for Results program. As academics, we had the job security and grant money to take it on. Through its World Bank ties and AMP program, DG had the in-country connections, technical capacity and the street-smarts to carry it off. The wonderful Aid and Debt Division of the Ministry of Finance in Malawi, led by Stan Nkhata, had the courage to support us. And the first multi-donor aid mapping exercise was born.

We achieved that proof of concept after nearly two years of hard work. It took months of digging out and digitizing hard files from various storerooms and file cabinets in Malawi, mobilizing a small army of UT and William & Mary interns to manually hand code thousands of pages of documents, and mining these same documents for the locations of projects to feed into Geonames and ArcGIS. In late 2011, our work resulted in a map of all the aid data (we could find) in one country (Malawi), visualized through maps and dashboards that we hoped (!) would be useful to decision-makers in countries.

To be honest, we learned as much about what could not be done at the time as what could be done. Read our report here. While we produced some very pretty maps which garnered considerable excitement internationally, we were also keenly aware of the “gaps and traps of maps.” For example, there was a lot of missing data due to huge variability in the availability and quality of information in project documents, locations of aid activities (when provided) were far from precise, and activity coding was difficult because of the death of project description information. Moreover, using the maps, even when we could build incentives and capacity for such use, was problematic. These early maps were fundamentally built upon spotty data because we were still much in the early days of aid transparency. We were caught in a giant catch-22: you couldn’t get robust use until the data were comprehensive, timely, and accurate, but you couldn’t get donors to report unless they were convinced that the resources and time they put into reporting data would lead to ethical and meaningful use of that data. 

Very quickly the excitement of the initial multi-donor aid map wore off, and bigger and better initiatives took off. These included the AidData Center for Development Policy consortium (funded by USAID) to scale up multi-donor mapping and data use studies, the International Aid Transparency Initiative – already established previously – expanded the geospatial component of its common standard for improved donor reporting, the expansion of groundbreaking work on open data by numerous NGOs such as Development Initiatives and Open Data Watch, the continued advocacy efforts of Publish What You Fund, research endeavors such Open Data for Development (OD4D), and the launch of numerous annual global forums such as the International Open Data Conference. DG was centrally involved in all of this.

That was a decade ago. What does this story say about DG and where it is today? 

Without a doubt, DG was then – and is now – recognized and respected widely as a global innovator and leader in the open data space now working across numerous sectors (including results data, gender data, extractives, open contracts, and agriculture). DG continues to take big risks and establish important partnerships across academics, NGOs, IGO, government, and civil society. More importantly, DG is, and has always been, willing to ask tough questions – often about the effectiveness of its own prior work – and to learn from failure. Its work with the AMP systems also reveals DG’s successful business models: build and sustain partnerships, share knowledge, listen and learn, adapt.

Finally (and I think this is the most important point), DG is patient and persistent. The dedicated staff and leaders of DG know that the real work of open data requires digging in when the excitement fades, doing the hard and tedious work maintaining and adapting systems, constantly reflecting upon the underlying assumptions of our theories of change around open data, and testing those assumptions again and again with painstaking, rigorous evaluation.