At DG, we pride ourselves in being a learning organization – focusing on continuous improvement and knowledge-building both internally and across our programs. We support this goal through a number of mechanisms, including holding post-mortem meetings at the end of every project. A post-mortem is a common method for project teams to review different perspectives on what went well, challenges faced, and what lessons could improve future projects. Our commitment to holding post-mortems for every project builds from our Agile development methodology, which includes retrospective meetings after every technical sprint to talk about what went well, what needs improving, and what the next steps are. Using a similar thought process to Agile, post-mortems allow for a thoughtful deep dive into the project cycle rather than simply moving to the next project without reflecting on lessons learned.
A few days prior to the post-mortem, each team member is sent a survey to provide feedback on how well they think the project went, if processes were followed and whether they worked well, and if desired results were achieved. The anonymized survey responses are then shared with the team, to ground the post-mortem conversation in real feedback.
Taking this dedicated time to pause and reflect on project processes and outcomes has been integral to our learning – and has furthered our ability to promote and integrate adaptive management into our overall approach.
Figure 1: Members of the DG team meeting at our hub in Bucharest. Post-mortems are conducted both in-person and via webinar, as our team is based all over the world.
To shed even more transparency on what this important process consists of at DG, below is an anonymized glimpse of questions and sample discussions taken from a cross-section of post-mortems.
Are you proud of our work? If yes, what made you proud? If not, what prevented you from feeling good about it?
- “Yes, I do. The whole project was successful in terms of the delivered functionality, the deadlines achieved, and the quality of final product.”
- “I’m proud of my work since the client is happy with the final product and finds it useful and usable.”
- “Generally speaking, I am proud of what we did. The project wasn’t easy and it was very complicated. I think we managed to deliver what the client asked for, even though their requirements were often changing and not completely clear.”
- “I’m proud of the resulting product, but not as much with the process to get it done. The final product complies with client expectations and is visually appealing. Regarding the process, I think that the discussion of requirements with client was difficult, as they went back and forth on decisions, which impacts the design and development process.”
What results were expected, and which ones did we achieve? Did our work make an impact?
- “I believe our work made an impact, because the project offered useful new features and a visual refresh on the tool.”
- “I expect that the added admin section gave more flexibility – both in configuration of the system, and particularly in allowing the client to independently update the information. The new French translations should also enable Francophone countries to browse the data.”
- “I think that the results match the expectations. We had some problems meeting timelines – which I also know that it was due to requirement changes and the complexity of having a pixel perfect product.”
Which of our methods or processes worked particularly well?
- “Jira sprint dashboards; data analysis spreadsheets with very clear quality assurance questions; Sprint retrospectives that gave specific feedback throughout.”
- “Consistent, daily meetings and extra check-ins as we neared the deadline. Also, we had good team collaboration.”
- “The full development process using Scrum – especially the requirements workflow (definition, grooming, validation, making changes).”
- “I liked how the technical team worked together to overcome obstacles on technical issues or unclear requirements.”
- “The strong coordination between the development team and the testing team.”
Which of our methods or processes need improvement?
- “Since this project has changed hands over the years, later-stage project staff had to resolve unexpected issues from previous phases of the project.”
- “We needed up-to-date technical documentation for internal use at the beginning, to assist new team members to get started faster.”
What did you enjoy the most about this project?
- “The team had great synergy, which allowed us to do our job without questioning our responsibilities.”
- “We were able to think through using different levels of data standardization, and why each is useful.”
- “It was an interesting mix between working with country-level teams on data management, and gaining exposure to/sharing with high-level organizations. We worked to strike balance between the two.”
What did you enjoy the least about this project?
- “I was not always sure what my role should be or where to help out, it was vague.”
- “Going back and forth deciding requirements meant that much of our early work had to be redone.”
Post-mortem meetings are not always comfortable. Holding them regularly means an open, running dialogue on the successes – and challenges – of projects that our team has worked hard, for months or years, to complete.
Conversations can be awkward or tense. However, the benefits from that awkwardness are clear – as teammates feel heard and eventually coalesce around how to better handle, or avoid altogether, those challenges in the future.
Perhaps even more important than holding post-mortems is actually using the feedback to improve project processes – so that the organization can continuously improve, and team members know their opinions matter.
Some of the overarching lessons that have come out of post-mortems that we’ve applied to new projects include:
- Do more user testing during the technical requirements and wireframing stage of our projects. This allows us to make needed changes earlier rather than later in the project.
- Include Technical Leads on client calls and interactions. This ensures that less information is lost as it’s shared from one person to the next (avoiding having to play “the telephone game”).
- Apply the Agile development process – or key elements – even for small projects, to ensure processes are uniform and effective.
- Improve internal and external communications on project progress. We’ve made effective changes to our communication processes as a development team, and we better understand how to support each other during project implementation.
Across the board, holding regular post-mortems have anchored DG’s culture of feedback and learning by allowing a dedicated “safe space” to think about how to improve as individuals, as a team, and as an organization.
We look forward to continuing to proactively foster this space and support team members in different ways, creating better processes and stronger teams – and in turn, making DG the most effective organization it can be.
For as long as Development Gateway has specialized in data, we have also specialized in data visualizations. In that time, we have discovered the pitfalls and learned ways that data visualizations can increase data use. In this post, we look specifically at selecting the right type of visualization for the story you want to tell.
If an organization with an existing culture of learning and adaptation gets lucky, and an innovative funding opportunity appears, the result can be a perfect storm for changing everything. The Results Data Initiative was that perfect storm for DG. RDI confirmed that simply building technology and supplying data is not enough to ensure data is actually used. It also allowed us to test our assumptions and develop new solutions, methodologies & approaches to more effectively implement our work.
15 years ago, AMP development was led by and co-designed with multiple partner country governments and international organizations. From a single implementation, AMP grew into 25 implementations globally. Through this growth, DG has learned crucial lessons about building systems that support the use of data for decision-making.