At Splash, we are constantly measuring the impact and sustainability of our programs. This means we are also continually learning.

When we started Project WISE in 2019, it was (and still is) groundbreaking in the WASH sector. Serving 100% of government-funded schools in selected cities with comprehensive water, sanitation, hygiene, and menstrual health infrastructure and programs had not been done before. We were treading into new territory.

We knew there would be learnings and adjustments along the way. That said, it is critically important that we don’t treat our work as an experimental proving ground. Good intentions are not enough. Our work impacts kids, families, and communities in a deep way and it’s our responsibility to ensure that our interventions put them first — making sure our programs have a positive impact.

So, how do we do this?

Monitoring, evaluation and learning! It’s more exciting than it sounds, we promise.

Over time, we have come to recognize that learning happens in many ways, over different timescales, and with varying levels of confidence and rigor. With this in mind, we have created a robust global monitoring, evaluation, and learning (MLE) framework built upon our theory of change, combined with periodic thoughtful review and analysis of our data and evidence, a process we call “learning loops.”

Some learning loops happen over multiple years and require large-scale randomized control trials (RCTs) to test fundamental theories about what we should be doing. Other learning loops are very small and happen on a daily, weekly, or monthly basis and require the use of dynamic information systems providing actors with the information they need right when they need it for decision-making. Each learning loop requires a nuanced understanding of the questions that need to be answered, and the innovative use of evaluation designs and analytical tools, to provide the decision maker with the right information, at the right time, with the right level of confidence.

To help us inform how we do what we want to do and how to manage it, Splash developed scorecards with key performance indicators (KPIs) for each of our intervention areas: infrastructure (clean water and sanitation), hygiene behavior change, and menstrual health. Examples of indicators include presence of soap at handwashing stations, drinking station tap functionality, pad availability, and toilet functionality. Each scorecard includes data for pre-intervention, post-intervention, and routine monitoring (ongoing checks at randomly selected Splash sites on monthly basis in Addis Ababa) so that we can view how each indicator is trending over time and adjust as needed.

An example of our scorecard is below:

So, what have we learned from our monthly KPIs?

First, the positives:

· Well-functioning and clean toilets — Across both Addis Ababa and Kolkata, we see that Splash designed toilets remain functional and that schools keep them clean.

· Lasting Gender Clubs and Pad Availability — Splash’s menstrual health (MH) work in Addis Ababa has resulted in ¾ of gender clubs remaining active and a very high percentage of sites with pads available at the time of routine monitoring.

· Infrastructure functionality and water access is strong — Kolkata has done a wonderful job across many aspects of infrastructure functionality and water access. Schools have invested more in ongoing maintenance and school administrators have carried out independent fundraisers to pay for improvements and supplies.

· Toilets are free from MH waste — an important behavior change indicator in Kolkata is whether students are throwing menstrual pads into toilets. We see limited presence of this behavior and have qualitative evidence supporting the proper disposal practices.

Things we need to work on:

· Tap breakage and theft — We identified several issues with handwashing and drinking station functionality in Addis Ababa, stemming from a lack of water, poor routine maintenance, and vandalism.

· Low presence of soap — The presence of soap was low in both Addis Ababa and Kolkata, driven in part by local norms about when soap is placed at the handwashing station and who is responsible for placing the soap (i.e., teachers, hygiene club members, etc).

· Hygiene club sustainability — Maintaining a hygiene club/child cabinet is a challenge and requires follow up with school administration and with the students themselves.

· Limited water availability — Water availability is a problem in Addis Ababa. Lack of water has a profound impact on other behavior change and infrastructure KPIs, undermining much of our theory of change.

How we address and hopefully fix these issues:

· Improved taps — Taps at stations in Addis Ababa are being retrofitted and redesigned to reduce leakage, breakage and theft. Longer term, we have re-designed our stations to have fully integrated taps, which are less prone to breakage and theft as well.

· Advocating for water — Splash is prioritizing water availability in conversations with the Addis Ababa government and advocating for water access solutions in communities where we work, including the use of rainwater harvesting in certain areas.

· School tiering — A system of tiering schools into three categories in Kolkata will help us prioritize support based on the number and types of challenges schools are facing and how much additional support we need to provide.

· Conducting citywide surveys — It will be critical as we expand into new cities to have a thorough understanding of water availability and water quality from the municipal utilities, so that we can design and install the best school WASH infrastructure possible.

· Presence of soap — The WASH Promotion Team will visit schools where soap is not provided in sufficient quantities to find out why. The Behavior Change Design team will review handwashing with soap programming.

· Hygiene clubs/cabinet refresher trainings — The WASH Promotion Team will reengage teachers and kids, while the Behavior Change Design team will review materials and methodology.

Our commitment to ensuring the positive impact of our programs is unwavering. Our responsibility is to prioritize the well-being of the children, families, and communities we serve. To achieve this, we have implemented a rigorous system of monitoring, learning, and evaluation. Using impact evaluations, periodic operational research, and monthly KPI scorecards that track key indicators, we assess the success of our interventions in infrastructure, hygiene, and menstrual health.

While we have seen notable successes, we also acknowledge areas where improvements are needed, such as handwashing and drinking station issues, soap availability, and the sustainability of child clubs. We are committed to addressing these challenges head-on, implementing solutions, and continually refining our approach to make a lasting, positive impact on the communities we serve. Monitoring, evaluation and learning may not sound glamorous, but they are essential tools enabling us to fulfill our mission in creating a better future so kids can thrive.