Thinking of Investing in a Data Ecosystem for Sustainable Development?

Three Questions to Get You Started 

The number of tools, frameworks, and initiatives to help you understand, analyze, map, and integrate data ecosystems at various levels can seem daunting. Below, we have included a select list of tools you can use for your assessment. Each methodology has a unique set of outputs, focuses on different parts of a data ecosystem, and analyzes different types of ecosystems. Before selecting a tool, ask yourself the following questions:

Q1: What is the objective?

This is the most important question to ask at the beginning of the process. The success of an ecosystem assessment depends on having a clear understanding of the reason for conducting it. This is important for assessments led both internally and externally. We see two major motivations for leading these assessments:

• To find possible solutions to a known problem, need, or interest – In this case, the assessment should be problem-driven and will help identify specific recommendations and investments that help find a solution to the known problem. Some notable examples are assessments to help identify possible data and digital investments, operationalizing strategic plans, or designing Monitoring and Evaluation (M&E) systems. A specific example illustrating this would be IREX and local actor D-Lab’s use of the Data Compass tool (as part of Data Zetu program) to help the Ministry of Health in Tanzania evaluate and sustain investments in public health care that included building capacities of rural health care facilities for more effective and systematic use of health data.
•   To evaluate where an ecosystem stands in relation to other ecosystems – Some data ecosystem assessment tools have been designed around evaluative frameworks used to conduct multi-dimensional research on the state of a data ecosystem — typically — across several countries. A reason for using these tools would be to determine where an ecosystem fits relative to other ecosystems or an ideal state. Examples of tools that help this type of assessment are scorecards, benchmarks, or indexes, like the recently released Global Data Barometer.

Q2: At what level do you want to focus?

Data ecosystems are multilayered, and there are multiple levels at which an assessment can be led:

• Country Level – These data ecosystem assessments are tools that provide a general landscape of a country. A country-level tool could be useful during investment or development planning. For example, a donor could use a tool that assesses the statistical capacities of a country to inform the design of a program or project that is data heavy. 
• Sector Level – These data ecosystem tools help actors to better understand opportunities to harness data within a sector. It considers all actors involved in a sector, including the private sector, government, civil society organizations, nongovernmental organizations, and donors. For example, DG led a data and digital assessment in Guadalajara, Mexico in the open procurement sector. The objective of this sectoral assessment was to identify key areas of investments in open procurement from the perspective of multiple actors, including government contractors, civil society organizations working in the corruption sector, and government employees making purchases.
• Organizations or Institutions Level – These data ecosystem assessments allow actors to understand the data ecosystem of a single organization or institution. For example, a social impact organization may want to understand the extent to which they are utilizing their data (i.e., data maturity level) in order to identify areas for improvement to become a data-driven and data-informed organization.
• A Combination of Assessments at Multiple Levels – Sometimes multiple data ecosystem assessments are needed. For example, a country-level assessment can help a donor identify areas of investment, and a second assessment can allow us to drill down on a possible solution. This was the case of the Results Data Initiative Program led by DG in Malawi. First, an assessment was led to understand the data ecosystem of the agriculture sector in the country. During this assessment, we identified the need for a National Agriculture Data System as a key priority across all actors involved. During a second separate assessment, however, DG focused on the Ministry of Agriculture to design the National Agriculture Management Information System (NAMIS).

Q3: Which aspects of the ecosystem do you want to understand?

A data ecosystem consists of many different elements. Having clarity on which aspects of the ecosystem you want to assess will ensure that the final product responds to your needs. Below are some examples of aspects you could focus on during the assessment. This is an indicative list that can be used as a starting point, but you can add more or select multiple foci.

• Available data – A key part of a data ecosystem is understanding what data exists. Additional information can be associated with available data, including data gaps; who produced the data; how it is collected; its infrastructure; how it is shared; the format in which it is collected; frequency of collection; interoperability(i.e., the ability of data systems to exchange and make use of information); data quality analysis; or the legal framework.
• Data openness – Knowing the level of open data of a government can help governments and international organizations recognize areas for investment. The level of data openness can refer to multiple variables, including the format in which data is published; the quality and frequency of data published; and coverage or legal framework.
• Data capacities and skills – Governments, donors, and civil society organizations need to have the capacity to produce and use available data. Depending on the assessment tool you select, you can assess statistical capacities, skills, access to training, or digital and data literacy.
• Use and reuse of data – Leveraging data to make decisions to advance the implementation of the Sustainable Development Goals is one of the key premises of the data revolution. Each assessment tool will evaluate the use of data differently. Some examples include having a positive impact, using data for decision-making, or incentives or disincentives for data use. IREX recently helped the Millennium Challenge Corporation (MCC) to better understand data ecosystems within Lesotho’s healthcare sector with a focus on the use of data to understand key processes ranging from budgeting to maternal mortality rates; recommendations also helped inform investments in data ecosystems driven by MCC’s new compact plan for Lesotho.

Select List of Tools You Can Use for Your Assessment

IREX and DG has created a curated list of tools to be used for data ecosystem assessments.