RDI allowed us to test some of our assumptions and develop new solutions, methodologies, and approaches to more effectively implement our work. In particular, DG’s Custom Assessment and Landscaping Methodology (CALM), our approach to co-designing with users, and our work to promote strategic data use with development agencies and the development community writ-large are all the result of RDI. From the perspective of the rear-view mirror, it is clear just how important this lucky combination of factors was in shaping our work.

What is RDI?

Launched in 2015 in partnership with the Bill & Melinda Gates Foundation, this five year, two-phase program worked with four country governments and two development agencies – Ghana, Malawi, Sri Lanka, Tanzania, Global Affairs Canada and UK’s DFID – to address critical barriers to results data use. In phase 1, we focused on research and exploration, and in phase 2 we focused on applying our lessons learned. While the first goal of RDI was to elevate the results focus of a few governments and agencies, our broader program goal was to provide real examples for how the “Data Revolution” can improve development policy and practice. DG created a combination of tools, datasets, resources, and approaches to help dynamic officials in each partner institution enable data-driven decisions.

Outcomes of RDI

During phase 1of RDI we focused on how data are collected, managed, and used in the health and agriculture sectors in Tanzania, Ghana, and Sri Lanka. We published a policy brief “How Should The Development Community Invest In Results?”, which highlighted three main recommendations for the broader development and ICT communities:

1. Sponsor technologies that promote data use – not just data reporting. During our RDI interviews, we heard multiple times that nearly all the data related technologies being used at the subnational level were for data reporting. Acting on these lessons, we developed an Open Contracting system for the Government of Makueni County, Kenya. As part of the technical development process, we have worked with local Project Management Committees, which are citizen groups who monitor project implementation. These citizen groups were part of our co-design process, and their monitoring reports are also included in the system. Our goal was to develop a system that reflects the user needs of both government and community stakeholders.
   Example from our current work: Makueni Open Contracting
2. Generate local-level outcome data. Frontline workers reported that they are rarely asked to collect outcome level data, as highlighted by this quote from Agriculture Extension Officer “If someone modifies livestock farming, builds a good house, and takes children to school after selling crops, [increases] from one meal to three meals… These are among changes that you can notice on the farmer though we rarely reach that level. [However,] the nature of data we access most of the time does not track to outcome level…”
   Example from our current work: Des Chiffres et Des Jeunes support to health clinics 
3. Respond to local data demands (this paved the way for development and publication of DG’s Custom Landscaping and Assessment methodology, CALM to facilitate use to systematically identify local demand and design tools fit-for-purpose). We used CALM to conduct data landscaping studies for several UNICEF Country Offices, with the aim of identifying “smart demand” of data — collecting information with specific decisions and problems in mind”
   Example from our recent work: UNICEF Country Data Strategy Development

Through our policy and advocacy efforts, we have sought to elevate these findings to the international level, while simultaneously applying the lessons to our own programming through phase 2 of RDI and beyond.

In phase 2 of RDI we expanded and further developed our data landscaping methodology to inform strategies for increasing the use of results data within development agencies such as DFID and Global Affairs Canada. Before RDI, our assessments were largely focused on identifying the technical and data requirements. With CALM, we are able to probe deeper into where the tool will live in the broader ecosystem, who will be using it, which decisions they will be making, and what information they will need to make that decision. CALM was developed as a user-centered design methodology, which puts the need to make decisions at the center of the tool(s), policies, and data governance models being designed Keeping in mind the decision spaces of our target users, we aimed to codesign tools that increase results data use within the Governments of Malawi and Tanzania.

RDI and Its Impact on Future Work

RDI has been influential in three key focus areas for DG.

1. Expanded data strategy network

Through RDI we were able to develop a rich network of results-oriented development practitioners with development agencies, which has been instrumental in our data strategy thematic area. At the end of phase 2 of RDI, we received additional funding from Gates Foundation to expand the data strategy work already started and to support agencies to better use data to make decisions. We continued working with Global Affairs Canada and UK’s DFID to publish the Decision Making and Data Use Landscaping Study, which again emphasizes the need to consider a user’s role in decision making – and to leverage data to support development of sector and country-specific analysis to meet user needs. Our expanded network enabled us to facilitate several knowledge exchanges between these agencies and the Icelandic Development Agency, Swiss Development Cooperation, USAID, MCC, UNICEF, and SIDA. We continue to make these linkages between practitioners and facilitate cross-agency learning, and have recently supported multiple agencies and INGOs in developing internal data strategies.

2. Expanded policy work

The lessons we learned through RDI built on our prior fifteen years of project work, and enabled us to test and refine feedback from DG programming into policy papers, guidance, and strategy and to disseminate them with the development community. During the course of RDI, we published the white paper Understanding Data Use and Managing for Feminist Results, which outlines the challenges and opportunities that development agencies may face when adopting new policies, and provides an overview of how other development agencies can integrate a feminist lens into their M&E frameworks. Our policy engagement also included working with the OECD Development Cooperation Directorate on research into the use of the Sustainable Development Goals (SDGs) as a shared results framework between development cooperation providers and partner countries. In addition, we led a closed-door consultation around measuring feminist international assistance policies with representatives from Global Affairs Canada, the embassies of the United Kingdom and Australia, the Millennium Challenge Corporation, Oxfam, Publish What You Fund, the International Center for Research on Women, Data2x, Cooper/Smith, the Center for Global Development, and the Council on Foreign Relations.

3. Refined co-design strategies

RDI allowed us to experiment with methods for co-design and co-creating, ultimately to develop tools that meet the end-users’ actual needs. While refining our co-design process, we adapted our approach and learned several valuable lessons on how to link the tools we develop with the end-users’ needs for decision-making. In Malawi for example, we developed a data ecosystem map using CALM, which provides a comprehensive picture of the Malawi agriculture sector, highlighting which data are currently gathered and used by which actors. Once we had a full picture of the ecosystem, we used it co-design the Conceptual Framework For the National Agriculture Management Information System (NAMIS) with the government of Malawi

We’re Still Learning

Looking back on the last five years of evolution resulting from RDI, it is clear that the program was a perfect fit for DG’s extensive background in aid effectiveness, open data, and our organizational culture of learning. It allowed us to reflect on how we practice technical development and support our partners use data for decision making. Although the RDI grant has technically come to end, its impact on our work continues, and the culture of learning and adapting at DG is as strong as ever.

AMP Basics

The Aid Management Platform helps governments and development partners gather, access, and monitor information on development activities, with the goal of increasing aid effectiveness. Using the AMP software, stakeholders can track activities through the planning, implementation, and evaluation stages. With online workspaces, data entry and reporting modules, and interactive dashboards and maps, decision-makers can better understand how aid is directed throughout the country. To ensure sustainability, the program implementation includes hands-on training for government and development partners, data management plans, ongoing technical support, policy support for program management, and a semi-annual Good Practices Workshop. AMP was first conceived and created in collaboration with the OECD, World Bank, UNDP, and Governments of Ethiopia and India. Anchored in the principles of the global aid effectiveness agenda, as expressed in The Paris Declaration, and specifically focused on country ownership of the development process and alignment of aid with country priorities to improve effectiveness.

In May 2005, DG completed its first AMP implementation in partnership with the Government of Ethiopia. Between 2005 and 2009, AMP rapidly increased from a single implementation in one country to 15 implementations globally. Accelerated growth meant expanding and adapting the platform for each country. During the early years, change requests from almost two dozen country partners had to be integrated into weekly stable builds. Since 2005, AMP has been implemented in 25+ countries. Much of this growth is made possible by the fact that AMP is a highly customizable off-the-shelf solution. One country’s improvement and any new features are made freely available to all other countries that receive the upgrade. This increases economies of scale, but it also creates complexity in the system. Because of this, over the years we have taken important steps to update, re-engineer, and streamline our tech and processes around code development.

The AMP process always begins with an in-depth assessment to best understand the needs of government, development partners, and CSOs. Based on the assessment findings, as well as previous implementation experiences, DG is able to propose the AMP configuration and improvements that meet the country partners’ unique needs, as well as suggesting institutional policies and processes to sustain the AMP.

Because the AMP platform has a standard source code with built-in customization, DG can quickly deploy a pilot system and then collaborate with the stakeholders to fine-tune system configuration. For larger system improvements, DG works closely with end-users to define and validate system requirements, receive feedback through user acceptance testing – first on design mockups and again once the final improvements are live. DG provides a series of trainings for four categories of users: 1) general users on data entry and analytics tools, 2) trainings for trainers 3) administrators on system management and configuration, 4) system administrators on the maintenance and troubleshooting of system hosting. The vast majority of AMP countries host the system locally. DG typically provides 1-2 years of technical support to ensure stakeholders are comfortable using the system, and to answer any questions, resolve bugs, and make small additional configuration tweaks.

AMP Technology

Read the previous blog for more about DG’s tech evolution.

When AMP began, support for version control was rudimentary. We initially used CVS (the very first version control system available!), migrating to Subversion in 2008, and again to Git in 2016. Git allows us flexibility in developing new features, and also saves time by providing quick fixes to up-and-running code. Similarly, we started using Jenkins as our continuous integration tool. Using Jenkins and Git also provides our quality assurance team with increased flexibility. As a user-focused team, we are constantly considering user requirements and environmental limitations. In 2016, with support from USAID’s Higher Education Solutions Network through the AidData Center for Development Policy, DG enhanced AMP’s GIS module based on findings from country-level research on how development data are used in Senegal, Timor-Leste, and Honduras. The GIS module meets country needs for more customized analysis tools. Additionally, to meet environmental challenges of internet speed and reliability, we developed AMP Offline, which enables users with limited or unreliable internet connections to use AMP without disruption. To avoid reinventing the wheel, we use open source frameworks – providing user-specific functionality, while also optimizing software development. AMP is built using the Java platform, and we have seen the Java ecosystem evolve – from plain JSP files and Struts back in 2005, to Apache Wicket in 2010, and APIs in 2012. We started building APIs with the IATI Import tool and with redesigning the reporting framework, and as for databases, we have moved from MySQL to PostgreSQL. In addition to iterating on the technology itself, the AMP team also continuously improves the modules making up the platform – for example, in building a new report engine. In reengineering how information is reported throughout the system’s modules, we have drastically improved the speed and stability of one of AMP’s core modules. In 2018 AMP took a big step by becoming open-source. While DG always granted full ownership of the source code to governments that use AMP, making it fully open-source was the next step to encourage innovation and sustainability of the system. Finally, this year, we launched a successful integration with an existing country budget management system. Read more about the evolution of DG’s technology here.

Encouraging a Culture of Data-Use for Decision-Making

When it comes to making sustainable, positive change, there is much more than technology involved. That is why we work closely with our partners to develop Data Management Plans that create processes and policies to improve use of the system, and that outline roles and expectations of those who enter, validate, and ultimately use the data. Our partners use data from AMP in many ways, including publishing annual development assistance reports, which analyze the collected information and allow for the development community to engage meaningfully on objectives and data-based planning. Data is also used to support government budgeting processes, increased awareness of development funding, and strengthening a community’s ability to hold development actors and government accountable.

While no single country’s AMP has the same exact setup as another, many of the needs and experiences are shared. This is why we encourage study visits between AMP countries, and why we host our AMP Good Practices Workshops. These workshops bring together AMP users from across the world to share how they are using AMP and their challenges and victories. The workshops also provide space to collect feedback from our partners on how to improve the program and the platform. Many specific changes – such as working toward partners maintaining their own AMPs – are the result of this type of feedback.

AMP Timeline

2005 – First AMP launched in Ethiopia
2008 – AMP launched in Burkina Faso, Madagascar, Malawi, Tanzania & DRC
2008 – First AMP Good Practices Workshop held
2009 – AMP launched in Niger, Senegal, Haiti, Kosovo & Timor-Leste
2010 – AMP launched in South Sudan, Laos PDR & Nepal
2010 – First public-facing AMP portal launched in Kosovo
2010 – First AMP GIS Module released
2011 – AMP launched in Uganda
2012 – AMP launched in Liberia, Honduras & Kyrgyz Republic
2013 – AMP launched in Moldova
2014 – AMP launched in Chad & Côte d’Ivoire
2015 – AMP launched in The Gambia
2015 – AMP architecture fully re-engineered, updated & modernized
2016 – IATI to AMP import tool released
2017 – AMP launched in Jordan with module for reading right to left
2018 – AMP Offline  released
2018 – AMP source code becomes open-source
2019 – Ethiopia’s developer team trained on independent source code development
2020 – AMP Offline becomes open-source

The Future of AMP

It is inspiring to look back on how far AMP has come, but even more exciting to look ahead and see where it might go. We are currently working on integrating our autogeocoder into AMP. This feature will automatically update the project location based on information in the project description and text. We are also expanding the South-South Cooperation Module, particularly for Haiti. Traditional aid tends to focus on financing and figures, while South-South Cooperation is often in-kind support, including technical assistance, which can be difficult to accurately capture in numbers. DG is also working in DRC to map its latest national development plan to the New Deal indicators. 15 years ago, AMP development was led by and co-designed with multiple partner country governments and international organizations. Today, we continue to expand the platform in partnership with our clients and based on actual requests – with the goal of building a platform that provides the most useful information for decision-making.

DG was spun out of the World Bank on July 14th, 2000. For those of you who knew DG twenty years ago, you might be surprised to learn just how much we have grown and evolved. As an organization that values learning, we have approached each project and partnership as an opportunity to better understand what works and why. We have become research and policy experts in data uptake and use, as well as have grown our footprint as global technology service providers. For individuals and organizations that have worked with us for the past twenty years, we look forward to celebrating our mutual successes and sharing what we have learned from you. For those of you who primarily know DG as we are now, our 20th Anniversary celebration will share more about how our work, learnings, and partnerships have shaped our approach and our values. We look forward to sharing this year’s important milestone with our global community (of the past, present, and future), which has shaped who we are.

Unfortunately, we do not expect to commemorate our 20th Anniversary in person, but we hope that you will follow along on our website and through social media. We also welcome you to share your own stories about DG anytime during the months of June through August and use the hashtags #DGTurns20 or #2Decades4Data.

Stay tuned, as we celebrate the past twenty years and share our plans for the future!

Co-Developing Solutions

Our main goal was to integrate Uganda’s PBS into the standard AMP portal. We have mentioned that the integration required two types of data: disbursement information, which shows how much is disbursed to implementing agencies, and medium-term expenditure framework (MTEF) data. MTEF data describes the amount to be spent over a given period of time. MTEF data’s objective is to help plan each year’s new budget, and identify expenditures for the next three years.

Working with the Ministry’s technical team, we jointly discovered a few of the existing challenges in integrating the two systems. First, the Ministry has its own internal processes built into the PBS and specific data importing requirements. The standard AMP portal exports reports in JSON format; However, the Ministry’s system requires XML format. Because AMP uses JSON format across the 25+ other country partners, changing the output from JSON to XML was not feasible, as making this change to AMP would impact all other partners.

Thinking through the Challenges

The two teams (the DG AMP team and the PBS team) discussed several different solutions, prioritizing options that would be most effective and result in the least error. One option considered was exporting reports in JSON format, then using the PBS to convert the reports to XML format before importing files into the PBS. Through this solution would solve the problem, the conversion and transfer process would create space for error or the potential for corrupted data. Once it was decided that modifying the data in AMP, then exporting it and re-importing into PBS was not an option, we faced another challenge: the two systems use different programming languages and servers. PBS uses microsoft.net with the MSSQL server, and the AMP portal uses Java. To assist the PBS team on how to consume the API developed and to make the integration less error-prone, DG developed a sample client in .NET to show how the API could be used by the PBS. DG had to find a solution that was less hierarchical and that could be interpreted by the PBS without major transformations.

The Solution

The final proposed solution was an integration in the PBS that interfaces with AMP, rather than an integration in AMP that interfaces with the PBS. After researching the options, XSLT was chosen to transform the data into different formats. DG worked with the Ministry’s tech team to format the XML output, via XLST so that it would transform the standard XML amp report output into the exact format needed. DG was able to make format adjustments to the XSLT without disrupting or having to modify existing AMP or PBS structures. Now, the solution is owned by the Ministry’s technical team, who can make adjustments to the XSLT as needed moving forward.

While DG has been working on AMPs around the world for 15 years, each implementation comes with its own puzzles and challenges that encourage growth and innovation across the platform as a whole. Through this successful integration, DG confirmed that success comes from a high-quality, uniform platform that is useful for all country partners but is also flexible and robust enough to meet varying partner requirements without comprising the existing system user base. As our platform evolves, each innovation and change becomes available to existing users and makes future versions of AMP stronger and more customizable.

Looking Forward

We are proud to have found a solution for integration that fits the needs of the client while also ensuring accurate data and governmental transparency. We look forward to using the new ability to export data in XML format together with XSLT transformations as part of the solution to further AMP country system integrations in the future.

The TCDI team learned the following from the assessment:

1. Data must be easy to understand:

Often, stakeholders struggle to find necessary data. They requested an online tool that will provide easy to understand information in one location.

2. On E-cigarettes:

E-cigarettes are still a relatively new product in South Africa, and researchers and tobacco control advocates are eager to understand more about e-cigarette prevalence and the associated health impact.

3. Policymakers want local data:

Policymakers want national data to make decisions. In the absence of South African data, they requested research from other countries to demonstrate the impact of tax increases or non-tax measures on tobacco use.

4. On illicit trade:

South Africa is battling high levels of illicit trade of tobacco products. This illegal activity decreases government revenue and limits the effectiveness of tobacco control interventions.

In the coming weeks and months, DG and partners will be working with stakeholders in South Africa to co-design data products that will best support decision-making around tobacco control. These top-level takeaways are an important guide as we design an online tool to meet stakeholder needs.