Conference day 1 – 16 October

8:30 am – 9:30 am
FOYER 1st floor

Registrations

9:30 am – 10:45 am
NUEVO LEON II Room
3rd floor

OPENING PLENARY

Welcome remarks from Coneval and Campbell Collaboration

Key note speech:

Integrating policy evaluation into policy design by Santiago Levy

10:45 am – 11:15 am
FOYER 1st floor

Coffee break – Networking

AGUASCALIENTES Room (2nd floor)

11:15 – 12:15 pm

Panel: PSW1

Early Childhood Development Programmes for Peacebuilding – the LINKS Study

Affiliation:

Queen’s University Belfast

Chair:

Paul Connolly

Presenters: 

Laura Dunne
Sarah Miller
Nicole Craig

Abstract:

The Centre for Evidence and Social Innovation has partnered with UNICEF, Yale, Harvard and New York University to lead a global research network, LINKS. The LINKS study seeks to support the development and evaluation of early childhood development programmes in societies affected by conflict, including Egypt, Kyrgyzstan, Mali, Tajikistan, Timor-Leste and Vietnam. At the heart of our work is the development of a measurement framework which can be used as a resource for governments, policy-makers, researchers and service providers to guide investment in, delivery, and evaluations of ECD programmes for peacebuilding and social cohesion. This session will introduce the measurement framework which underpins the overall programme of research. An overview of the research being undertaken in each of the network countries will be provided, with a particular case study focus on our work in Egypt.

12:15 – 12:25 pm

 

Moving to new room

12:25 – 13:25 pm

Panel: PE1

Building Research Projects at Scale with Government Ownership

Affiliation:

Innovations for Poverty Action

Chair:

Heidi McAnnally-Linz

Presenters: 

Sergio De Marco
Mónica Seminario
Besnart Simuchebu Kangalu
Christopher Neilson

Abstract:

Policymakers, educators, and researchers all want to use the lessons of rigorous research to improve education quality around the world. How can they collaborate to do this in practice? In this panel, researchers, policymakers, and leaders from international research and policy organizations will discuss experiences helping to deliver on the promise of rigorous research for improved education policy. In particular, presenters will discuss embedding rigorous research—full-scale impact evaluations and lower-cost studies with administrative data—within the work of ministries of education in Peru, Ghana, and Zambia. Panelists will also share results of rigorous original research on education quality from those countries, draw in synthesized lessons from systematic analyses of the body of rigorous evidence in education, and explore insights from their experiences working within governments.

13:25 – 14:30 pm
CIUDAD DE MEXICO
1st floor

Lunch

14:30 – 15:30 pm 

Panel: PE3

Evidence, Equity and Policy for Educational Attainment in Middle Africa: Lessons from a North South Collaboration

Affiliation:

Effective Basic Services (eBASE) Africa

Chair:

Loveline Lum Niba

Presenters: 

Aweh Annih Akofu
Patrick Mbah Okwen
Euphrasia Atuh Ebaig
Rigobert Pambe Miong

Abstract:

Our panel will focus on educational attainment for children aged 3 to 18 years in Middle Africa. It will explore and discus challenges with evidence informed decision making and equity in policies in education and how these impact attainment. Presenters will stimulate discussions along 4 strands: challenges with access to evidence; equity in evidence informed decision making; pedagogy audits and feedback and contextualization of foreign education guidelines using the EEF toolkit. These discussions will be stimulated from our experiences with inclusive education attainment projects in Cameroon, Niger, Nigeria and Tchad. Our team consists of 3 females and 2 males. Our panel will be on innovative approaches in evidence implementation. The session will be interactive, we will use results from our experiences in audio-visual data display formats on paper and slides to stimulate discussions.

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Campbell Collaboration Groups

Meetings open to all participants of the WWGS

DISABILITY

Introducing the Disability Coordinating Group of The Campbell Collaboration

 

Affiliation:

University of Central Florida and Co-chair of the Campbell Education Coordinating group

Presenter: 

Oliver Wendt

Abstract:

This presentation will describe the mission of the Disability Coordinating Group (DCG), share some recent disability-focused systematic reviews, and provide involvement opportunities. Objectives of the DCG are to: 1. Increase the number of high-quality systematic reviews (SRs) aimed at improving the quality of life and outcomes of individuals with disabilities; 2. Facilitate a network of individuals to contribute to disability-related SRs; 3. Encourage involvement of people with disabilities, their family members, and other stakeholders in the SR process; and 4. Facilitate training opportunities for disability-focused SR authors. The DCG originated from the Disability Subgroup that was approved in 2007 as part of the Education Coordinating Group. In May 2017, a stand-alone Disability Coordinating Group was approved by the Campbell Board.

EDUCATION

Campbell Collaboration Education Coordinating Group – Highlights

 

Affiliation:

Queen’s University Belfast and Co-chair and Editor of the Campbell Education Coordinating group

Presenter: 

Sarah Miller

Abstract:

In this session the Campbell Collaboration Education Coordinating Group (ECG) will report on their activities over the last 12 months. In particular, a number of key education systematic reviews will be highlighted and the potential for this type of evidence to impact on practice and policy, will be discussed. The session will provide a good opportunity for participants to meet the team and find out more about the work of the ECG. We warmly welcome everyone, including those who have an interest in undertaking a systematic review in education and/or would like to work with us in an editorial capacity.

17:00 – 17:10 pm

Moving to new room

17:10 – 18:10 pm 

Panel Fishbowl: FishE1

Accelerating the use of evidence and data: insights from the UK What Works movement

Affiliation:

Alliance for Useful Evidence, Nesta Foundation

Chair:

Morgan Kuranda

Presenters: 

Guillermo Rodriguez Guzman
Ligia Tiexiera

Abstract:

The UK evidence movement has grown at pace over the past decade. From the launch of a network of six ‘What Works’ Centres in 2012, there are now 11 evidence centres working on areas of policy from crime to wellbeing. In this interactive panel session, panelists offer cross-cutting lessons from their journey from designing standards of evidence that reflect internal validity, to ensuring research is accessible for end-users to spark policy change. Panelists will explore how these mechanisms can be replicated in policy contexts around the globe.
Guillermo explores the systematic synthesis of internal validity problems.
Ligia discusses incubating organisations to drive research responsive to the needs of homeless.
Anna’s expertise is in methods for improving research translation.
Kuranda explores building research capability through learning programs.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

NUEVO LEON I Room (3rd floor)

11:15 – 12:15 pm

Synthesizing evidence: the experience of Mexico, Colombia and Uganda building Evidence Gap Maps from different kinds of Evaluations

Chair:

Graciela Teruel, Coneval

Presenters: 

Lorena Trujillo TBC
Timothy Lubanga TBC
Thania de la Garza

Abstract: 

The aim of this session is to exhibit the experience of Mexico, Colombia and Uganda regarding evidence gap maps elaboration from the different kinds of evaluations that have been developed in these countries. The former tools summarize, in a visual way, the set of existing evaluations to public policy interventions. Its main objective is to show decision makers, researchers and key actors what evidence exists, where does gaps are identified and where do evaluations are concentrated. In this sense, evidence gap maps contribute to identify where does more investment is required and where does research duplicities may be avoided in order to inform and enhance evidence-based decision making.

12:15 – 12:25 pm

Moving to new room

12:25 – 13:25 pm 

Parallel session: Evidence-informed education initiatives
Chair: Sarah Miller 

Reducing School Exclusions in Children between 4 and 18 years in Cameroon Through Use of Best Available Evidence

 

Affiliation:

Effective Basic Services (eBASE) Africa

Presenter: 

Aweh Annih Akofu

Abstract:

Disparity in education at the expense of girls is more likely in LMIC (UNESCO 2018). We sought to Reduce school exclusions resulting from inappropriate correctional methods in children aged 4-18 years in Cameroon. We developed an evidence implementation project through our evidence based pedagogical fellowship program. The project was guided by a Campbell Review on school exclusions (Valdebenito 2018). 6 schools with 2400 students were involved in the evidence implementation project. 80 students (for criteria 1 to 3) and 60 teachers (for criteria 4) were sampled. 3 schools had a trained counselor. Baseline audit suggest no student received an academic skills enhancement program. 15 out of 80 children received counselling. 6 out of 80 children received some kind of mentoring or monitoring program. 3 out of 60 teachers had received training on correct disciplinary methods.

Teaching at the Right Level: Improving learning outcomes by adapting evidence-based approaches across contexts

 

Affiliation:

Abdul Latif Jameel Poverty Action Lab (J-PAL)

Presenter: 

Robert Rogers

Abstract:

Teaching at the Right Level (TaRL) is a pedagogical approach that assesses children in grades 3 to 5 using a simple tool and regroups them by learning level rather than age/grade. Six randomized evaluations in 7 Indian states consistently show that the TaRL approach improves learning outcomes across contexts and through multiple implementation models. Given TaRL’s successes in India, governments and organizations in several countries in Sub-Saharan Africa are formulating TaRL programs with technical support from Pratham and J-PAL. This presentation will share J-PAL’s experience with adapting the TaRL approach from India to Sub-Saharan Africa using a framework for generalizing evidence as outlined in Bates and Glennerster (2017). The framework describes how to apply evidence from rigorous research across contexts using impact evaluations, theory, descriptive data, and process monitoring.

Assessing and Labelling the Strength of Evidence for Decision Makers: A Case Study with Two UK What Works Centres

 

Affiliation:

University of Oxford/RAND Europe

Presenter: 

Yulia Shenderovich

Abstract:

The UK College of Policing and Education Endowment Foundation are two “What Works Centres” responsible for generating and appraising evidence for decision makers. They commissioned a project to investigate approaches for assessing and labelling the strength of primary research evidence on intervention effects. In this presentation, we will discuss our experience recommending the Cochrane Risk of Bias 2.0 tool for randomized trials, the Risk Of Bias in Non-randomized Studies-of Interventions (ROBINS-I) tool, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. We argue that these tools are applicable to assessing and labelling evidence about policing and education, though applying these tools to social policy requires special considerations about (1) sources of complexity and (2) challenges of working in time-sensitive decision-making environments.

13:25 – 14:30 pm
CIUDAD DE MEXICO Room (1st floor) 

Lunch

14:30 – 15:30 pm 

Panel: PSW2

Has Mexico’s prospera contributed to improve long-term human capital, welfare and intergenerational mobility outcomes?

Affiliation:

World Bank

Chair:

Maria Concepcion Steta

Presenters: 

Juan Pablo Gutierrez
Susan Parker
Iliana Yaschine
Giacomo de Giorgi

Abstract:

Positive impacts of Mexico’s CCT, PROSPERA, on human development outcomes are well-documented. However, evidence of PROSPERA’s long-term results was limited until recently. Using data collected in 2017-2018 combined with historic surveys and administrative records, recent studies show that PROSPERA’s short- and medium-term effects have translated into positive intergenerational social mobility and higher long-term wellbeing. These studies point that overall the program’s outlook is positive and identify some shortcomings. The studies find positive effects of PROSPERA on learning, on intergenerational mobility of human capital measures, on the youth’s likelihood to own durable assets, and on food insecurity reduction. The studies also show that half of the youth experienced upward occupational mobility, while the rest have not climbed the ladder in this dimension yet.

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Parallel session: Policy engagement in evidence synthesis
Chair: Patrick Mbah Okwen 

An impact framework as a tool for investigating the impact of policy relevant systematic reviews in International Development

 

Affiliation:

EPPI-Centre, UCL

Presenter: 

Mukdarut Bangpan

Abstract:

There has been a rapid increase in the number of systematic reviews in international development during the past decade (Waddingtion et al 2018). EPPI-Centre is a well-known research centre in developing evidence synthesis methods to influence policy and practice. We have supported research teams conducting more than 90 policy-relevant systematic reviews in international development since 2007. We will first describe the nature of the systematic review using a visual map and discuss how different stakeholders made use of these diverse types of systematic reviews using a newly developed impact framework. This impact framework recognises different types (instrumental, conceptual, capacity building) and dimensions (reach, significance, sustainability) of evidence use. Finally, we present challenges of how international development reviews are being used as the global public good.

CONEVAL’s Strategy for the Synthesis of Evidence

 

Affiliation:

Coneval

Presenters: 

Thania Paola de la Garza Navarrete

Abstract:

The objective of the presentation is to explain CONEVAL’s strategy for the synthesis of evidence, which consists of the elaboration of practical guidelines for public policies based on systematic reviews and principles such as standardization and transparency in the selection of evidence, exhaustive and thorough search of relevant studies and the guarantee of impartiality and quality in the chosen studies. In this sense, instead of seeking to obtain systematic reviews, an intermediate strategy is proposed that allows synthesizing the available evidence in an easy and accessible format.
For the time being, the guidelines address six issues related to social rights: chronic child malnutrition, educational development of girls, boys and adolescents, youth labor development, financial inclusion, food security, and long-term care for older adults.

The art and science of using evidence in Africa: an evidence map and synthesis of what we know about supporting increased demand for and the use of evidence in African countries

 

Affiliation:

Africa Centre for Evidence, University of Johannesburg

Presenter: 

Promise Nduku

Abstract:

African governments and institutions are increasingly leading innovation on evidence-informed policy-making (EIPM). However, despite existing activities and established institutions, there is no overall knowledge base on the learning and evaluation of the EIPM efforts in Africa. In short, we don’t know the size and the nature of the existing evidence on what works for EIPM in Africa. This presentation will summarise the results of the first evidence map collecting and organising the continent’s knowledge base on EIPM. The evidence map includes both scientific and Grey literature assessing African EIPM efforts. As a hub of all the available evidence on EIPM in Africa, it presents an opportunity for cross-learning and dialogue between African EIPM advocates. It also serves as an anchor and overview of what EIPM strategies have been tested and to what effect on the continent.

17:00 – 17:10 pm

Moving to new room

17:10 – 18:10 pm 

Panel Fishbowl: FishSW1

Strengthening data & evidence use in nutrition policy: Insights from India

Affiliation:

International Food Policy Research Institute

Chair:

Purnima Menon

Presenters: 

Esha Sarswat, IFPRI
Pulkit Agarwal, IDinsight
Neeraj Trived, EPoD

Abstract:

In 2018, India launched a high-level political effort to recommit to nutrition by consolidating efforts under a single policy umbrella and coordinating work across multiple ministries and partners. Data and evidence were deemed critical to support action in states and districts in India. Multiple sources of data, produced by multiple entities, are a feature of the data landscape in India. In this context, many entities have attempted to shape more robust data- and evidence-informed discourse on nutrition in multiple forums in India. This panel will feature insights from these efforts in India on the challenges of and opportunities for supporting use of data and evidence in a dynamic policy environment. After short presentations, we will hold a fishbowl discussion on data and evidence convenience, data politics, and supporting a cadre of data- and evidence-savvy administrators.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

SONORA Room (2nd floor)

11:15 – 12:15 pm

Panel:

Nutrition and Sustainable Development: how can evidence support policy?

Chair: 

Howard White

Presenters: 

Zulfiqar Bhutta
Oliver Rothschild
Annette O’Connor

Abstract: 

The Millennium Development Goals (MDGs) concluded in September 2015, and were succeeded by the world aligning behind new Sustainable Development Goals (SDGs). As opposed to the MDGs, there are 17 SDGs with specific targets to achieve. A distinctive feature of the SDGs is their inter-connectedness and the fact that achieving most goals would require collaborative efforts of multiple sectors of governments, experts and civil society to meet the objectives. The synchronized progress will minimize the competition for scarce human and financial resources and duplication of efforts, consequently enhancing the speed and sustainability of the progress.

12:15 – 12:25 pm

Moving to new room

12:25 – 13:25 pm 

Parallel session: Evidence mapping
Chair: Ekwaro Obuku

Evidence Gap Maps: a tool for promoting evidence and gaps in low- and middle-income countries

 

Affiliation:

Sightsavers

Presenter: 

Bhavisha Virendrakumar

Abstract:

Evidence of what works for development in low-and-middle income countries is relatively scarce. To address this, Sightsavers developed evidence gap maps (EGMs), which summarise, appraise and present evidence from systematic reviews in a user-friendly visual format. Each systematic review on the EGM was critically appraised using the SURE tool which gives reviews an overall rating of high/medium/low confidence based on the methodological quality assessment. On the EGM this is indicated using traffic light system – green, orange and red bubbles represent high, medium and low levels, respectively.
Each review was represented by a coloured bubble and placed in the cell corresponding to the relevant intervention along the x-axis and the strength of evidence along the y -axis. Each review was categorized as strong, inconclusive or weak based on the findings reported by review authors.

Blue Communities: Mapping the evidence for human health and wellbeing impacts form nature conservation in SE Asia

 

Affiliation:

University of Exeter

Presenter: 

Ruth Garside

Abstract:

Millions of people across the globe rely on marine and coastal ecosystems for food, employment and their general well-being. In recent years, the marine environment has come under increasing pressure from the multiple, and often conflicting, needs of the people that use it. This is the case for many SE Asian communities, where marine activities are important for livelihoods and wellbeing. Identifying “win-win” scenarios, where both nature and people benefit, may be particularly important.
In this evidence mapping project, a range of published and grey literature sources were used to identify research that explored health and wellbeing from conservation. As well as outlining the map findings and illustrating how the results were presented in an interactive data portal, this presentation will describe the challenges encountered, particularly in relation to grey literature.

What evidence exists for building peaceful and inclusive societies? An evidence gap map

 

Affiliation:

International Initiative for Impact Evaluation (3ie)

Presenter: 

Ada Sonnefeld

Abstract:

The overall aim of this evidence gap map (EGM) is to identify, map, and catalogue the existing, and ongoing, evidence on the effects of interventions to build peaceful and inclusive societies. The map takes a broad conceptualisation of peacebuilding, but is limited to fragile states and communities. The map includes a framework of interventions that promote reconciliation and security, build capacity of civil society and the state, and develop economic foundations. This is mapped against outcomes along the causal chain, including beliefs, psychosocial well-being, displacement, and levels of intergroup conflict. It will include primary studies and systematic reviews. The analysis includes looking at trends in the evidence base, identifying key areas for future research, and mapping the interventions onto a resilience framework (whether they are absorptive, adaptive, or transformative).

13:25 – 14:30 pm
CIUDAD DE MEXICO Room (1st floor) 

Lunch

14:30 – 15:30 pm 

Panel: PECO4

Policy-relevant evidence maps: a method to develop responsive evidence-bases for knowledge management and translation in government

Affiliation:

Africa Centre for Evidence, University of Johannesburg

Chair:

Harsha Dayal

Presenters: 

Carin van Zyl
Lauren’s Langer
Promise Nduku
Zafeer Ravat

Abstract:

This panel will discuss the development of policy-relevant evidence maps in South Africa. Led by the Department of Planning, Monitoring and Evaluation (DPME), since 2016 a methodology for developing responsive evidence-bases for knowledge management and translation in government was co-constructed with researchers. Following the application of policy-relevant evidence maps in five strategic policy areas, this panel will reflect on the use of these evidence maps for knowledge management and knowledge translation. It will also reflect on the innovation in the research process for evidence maps and elaborate on the distinct co-production and match-making model pioneered by DPME. The panel will consist of government officials and researchers involved in the development of policy-relevant evidence maps and serves to present a deep-dive into the more technical aspect of this methodology.

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Parallel session: International development
Chair: Scott Rozelle 

Transparency and accountability in the extractives sector: A narrative synthesis of what works and what does not

 

Affiliation:

International Initiative for Impact Evaluation (3ie)

Presenter: 

Francis Rathinam

Abstract:

The extractives rich countries are among the poorest in terms of GDP per capita, poverty headcount and have high risk of conflicts. Despite considerable efforts to support improved transparency and accountability (TAI) in the natural resources sector, the overall evidence is scarce. 3ie funded 7 impact evaluations in the TAI in natural resources sector and developed a theory of change for how TAI interventions work. This narrative synthesis demonstrates that interventions combining some form of information with deliberation seem to positively affect knowledge and awareness, trust, and civic actions for more accountability. Findings from the synthesis provides useful lessons for designing the scope and theory of change for future TA interventions and evaluations, and highlight some of the key evidence gaps.

Leveraging the SDGs for Better Planning

 

Affiliation:

AidData

Presenter: 

Jennifer Turner

Abstract:

The SDGs have the potential to materially impact decision making at international, national, and local levels—from prioritizing where to channel resources, to setting targets and measuring downstream results. Yet, the extent to which the global goals translate into action depends on how well the SDGs are integrated within planning, budgeting, and reporting processes.
As policymakers increasingly seek to align planning and budgeting with the global goals, the question remains: how can this process become more efficient, rigorous, and replicable? In this presentation, we will discuss innovative initiatives to help decision-makers leverage the SDGs to influence decision making. Examples will include AidData’s methodology to track project-level financing to the SDGs from multiple sources, such as development finance, domestic budgets, and private sector investment.

Re-thinking the implementation strategy of social protection policy in Uganda: An integrated approach to promote inclusion of informal sector workers

 

Presenter: 

Mary Baremirwe Bekoreire

Abstract:

The current legislation on social protection in Uganda promotes formal social protection schemes for workers in the formal sector, excluding 93 percent of the working population in the informal sector. This paper analyses the extent to which the implementation strategy of social protection policy in Uganda accelerates exclusion of informal sector workers, thus increasing their vulnerability to poverty. This study employed sequential mixed methods research design. Data was collected from 413 participants that included informal sector workers from registered CBOs with savings and investment and government and NGO staff was collected using questionnaires, Focus Group Discussions and interviews. Multi-stage, systematic sampling and purposive sampling were used to select the study participants. Content analysis was used to analyze qualitative data whereas descriptive statistics were used to analyze quantitative data. The findings reveal that, to a large extent, organization of workers into registered institutions has not brought significant benefits to the members of the CBO including access to social protection schemes. In addition, the current social protection schemes promoted by the government do not necessarily address the specific risks and shocks faced by informal sector workers. Whereas the social protection policy permits informal sector to access the National Social Security Fund and other insurance services on a voluntary basis, the private sector dominates service delivery thus, the social protection packages are unaffordable to majority of informal sector workers. The government should engage non-state actors such as CSOs, donor community, NGOs and private sector to build the capacity of informal sector workers to access to social protection. Active engagement of the informal sector workers in the entire policy process should be used as an important tool of enhancing prioritization of risks and shocks.

Context Matters: Insights from Multiple Impact Evaluations of Nutrition Behavior Change Interventions Factors Affecting Reach, Uptake and Impact

 

Affiliation:

International Food Policy Research Institute

Presenter: 

Purmina Menon

Abstract:

Appropriate infant & young child feeding (IYCF) practices contribute to optimal child growth & development. Behavior change interventions (BCI) can improve IYCF, but lessons on effectiveness at large-scale are limited. Over a decade, in Bangladesh, Ethiopia and VietNam, we conducted evaluations of BCI that used common design principles, but had diverse program platforms for delivery. We showed impact on breastfeeding and complementary feeding practices, but not all mothers were reached and of those reached, not all practiced recommended behaviors. Using evaluation data on program implementation, program reach and uptake, and household characteristics, we show that the match between intervention and delivery context, fidelity of implementation, demand-side constraints to using delivery platforms, and finally, household constraints that are behavior-specific contribute to mixed impact.

17:10 – 18:10 pm

Panel: EXT2

More Than Numbers: Impact Evaluators in Africa

Affiliation:

Hewlett

Chair:

Norma Altshuler

Presenters: 

Dr. Amos Njuguna, chair of the Network of Impact Evaluation Researchers for Africa (NIERA) Dean of Graduate Studies at the U.S. University of Kenya
Joy Kiiru, Senior Lecturer at the School of Economics, University of Nairobi, and Vice Chair of the board of the Mawazo Institute

Abstract:

How can impact evaluations provide the kind of information local decision-makers need—and that they’ll use? National researchers are well-positioned to understand and influence their own governments and other decision-makers. We’ll share exciting data from a recent African Centre for Evidence study about the number of African scholars who publish impact evaluations and the potential to expand their work and impact. We hope you’ll join us for a lively and interactive conversation with colleagues from around the globe on how to work with and champion African impact evaluation to improve government programs and policies and benefit more people. We’ll also unpack when and how partnerships across continents can be mutually beneficial.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

NUEVO LEON II Room (3rd floor)

11:15 – 12:15 pm

Panel: PBIG2

The Opportunities and Challenges of Using Data and Evidence to Drive Innovation in Governments in Latin America

Affiliation:

J-PAL

Chair:

Maya Duru

Presenters:

Elianny Medina
José Merino
Enrique Seira Bejarano
Edoardo Trimarchi

Abstract:

Many governments around the world are incorporating data analysis and evidence into the design, implementation, and evaluation of their social programs. However, significant opportunities and challenges remain in administrative data that governments already collect to both improve program delivery in real time and reduce the cost of policy experimentation and evaluation to identify effective programs. This proposed panel will bring together perspectives from government, research, and evidence-to-policy organizations to describe existing government efforts in big data and experimentation in the Dominican Republic, Mexico, and Peru, discuss shared challenges, and present a vision for the future of administrative data and evidence use inside governments.

12:15 – 12:25 pm

Moving to new room

12:25 – 13:25 pm 

Parallel session: Health systems
Chair: Promise Nduku 

Policy concerns for Pay for Performance of Health Workers in Uganda: An evidence brief for policy in Uganda

 

Affiliation:

Makerere University – Africa Centre for Systematic Reviews & Knowledge Translation

Presenter: 

Ekwaro Obuku

Abstract:

Uganda’s ministry of health has considered pay for performance (PFP) to improve effectiveness of health workers. We synthesized evidence on strengths and limitations of PFP models in Uganda’s context. We conducted a protocol driven systematic review of published and unpublished studies, searching 5 data bases and included 9 studies. Only one low quality study reported about death and showed no significant effect of PFP on neonatal mortality. Low quality evidence suggests PFP increased utilization, quality of care and reduced out of pocket costs.
PFP may distort human resource dynamics at the workplace & accounted for a significant portion of annual district health expenditure. In conclusion, PFP has benefits, but the effects are disparate from none or minimal or significant. Policy makers should examine contextual factors that influence effectiveness of PFP in Uganda.

Using District Health Information System (DHIS) 2 to enhance performance monitoring and accountability for the promotion of reproductive, maternal and child health in Kenya

 

Affiliation:

African Institute for Development Policy (AFIDEP)

Presenter: 

Bernard Onyango

Abstract:

In 2017/2018, we assessed the use of the District Health Information System (DHIS)2 as a an accountability platform to improve reproductive, maternal and child health in Kenya. We found critical barriers to the use of DHIS2 capabilities such as performance scorecards and dashboards important to improving outcomes. These included: lack of the necessary infrastructure to operationalise the system at the counties; understaffing at national and counties, and lack of trained health workers in monitoring and evaluation. We recommend strengthening the use of DHIS 2 through: (1) continuous capacity building of staff on data management and analysis for decision making; supporting county governments adapt M&E policies to the local context; and, advocating for the use of the score cards as a data visualisation tool to support decision-makers at both national and county levels.

Behavioral evaluation of Más Familias en Acción, the conditioned cash transfer program in Colombia

 

Affiliation:

National Planning Department of Colombia

Presenter: 

Lorena Trujillo

Abstract:

Mas Familias en Accion is the conditioned cash transfer program for poverty alleviation and education enrollment in Colombia. Targeted at more than 3 million households, the program offers additional to the cash incentive, the intervention called “Bienestar Comunitario” (BC) aimed at strengthening the social capital of these families by creating spaces for interaction between them. Unlike the first component of the program, the BC component was lack of evidence due to the shortcoming of measurement its expected outcomes. We designed an evaluation of the BC component by using behavioral economic techniques. We ran 54 lab-in-the-field experiments among three comparison groups in which the decisions they took led to quite accurate measures of reciprocity, altruism, confidence, cooperation and coordination, which shed light on the contribution of the program further than school enrollment.

13:25 – 14:30 pm
CIUDAD DE MEXICO Room (1st floor) 

Lunch

14:30 – 15:30 pm

Panel: PSW5

What works to improve outcomes for people experiencing, or at risk of experiencing, homelessness

Affiliation:

Queen’s University Belfast

Chair:

Ligia Tiexiera

Presenters: 

Howard White
Sarah Miller
Jane Hamilton
Christopher Coughlan
Charlotte Kincaid

Abstract:

The Campbell UK & Ireland Centre (Queen’s University Belfast) – funded by the Centre for Homeless Impact (CHI) – have undertaken three Campbell Systematic Reviews to synthesise the evidence in relation to what works to improve outcomes for people experiencing, or at risk of experiencing, homelessness. The reviews focus on the effectiveness of: accommodation based interventions; access to health and social services interventions, and; discharge programmes. All three reviews build on two Campbell Evidence and Gap Maps (EGM) that CHI commissioned in order to better understand the extent to which the evidence exists (or not) to tell us what works and why. The session will begin with an overview of the context underpinning the commissioning of the EGMs and the systematic reviews. The methods and results of the two EGMs as well as the results of the three systematic reviews will be presented.

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Parallel session: Evidence to policy tools
Chair: Tamara Lotfi 

CANCELLED Bridging the capacity gap in evidence use in Cameroon using a tailored training program

 

Affiliation:

Effective Basic Services (eBASE) Africa

Presenter: 

Emmanuel Berinyuy Kamga

Abstract:

We aimed at developing a tailored contextualized training program covering the entire evidence ecosystem through an evidence-based pedagogy and clinical leadership fellowships.
We used the JBI approach and the EEF toolkit for evidence synthesis, transfer and implementation to contextualize a training program for clinical and pedagogy leaders. Clinical and pedagogy leaders developed review questions based on work place challenges encountered.
A total of 09 clinicians, 12 teachers, 02 environmentalists and 02 finance experts have been trained since 2018. A total of 18 review questions were developed. A total of 04 implementation projects have been done. The trainings highlighted participant interest in using evidence in their different workplace but a lack in capacity to synthesize and use best available evidence. A tailored training program works in bridging this gap.

Systematic review of methods of reducing risk of bias in the evaluation of knowledge translation strategies for evidence-informed health policy making

 

Affiliation:

Tehran University of Medical Sciences

Presenter: 

Ayat Ahmad

Abstract:

Review studies have reported on the low quality of study methodologies and poor reporting of Knowledge Translation(KT) in the context of Public policy. This review aims to determine how much the risk of bias(ROB) contributes to the heterogeneity in the effectiveness of KT strategies.
Intervention: methodological technique to reduce the risk of bias
Population: studies aiming to evaluate KT strategies health policy
Outcomes: Standardized mean differences for KT interventions for each condition via meta-analysis (meta-epidemiology) across all type of KT interventions will be performed, considering ROB conditions as the intervention.
The contribution of the estimated ROB to the heterogeneity will be assessed through meta-regression.
Type of studies: Randomized and non- randomized control trials
Databases: MEDLINE, EMBASE, SCOPUS, Web of Science and ICTRP to identify registered trials.

Business Models for Research Centres: A Scoping Review

 

Affiliation:

American University of Beirut

Presenter: 

Nadia Nameh

Abstract:

Be it in for-profit, not-for-profit, or hybrid institutions, research centres depend mainly on grants and external funding to support their work and ensure their sustainability. Very little is known about how research centres function and the business model they follow. The purpose of this scoping review was to identify published business models of research centres and explore their characteristics. We searched seven databases: ProQuest Central, ABI/Inform, Science Direct and others using the following terms “Business Models” and Research Centre. We identified 2778 citations and included 12 papers. While the majority of the included studies did not identify published business models of research centres nor did they explore characteristics of research, a few followed the business model canvas and stressed on engaging with users and to outsourcing some of the work.

17:00 – 17:10 pm

Moving to new room

17:10 – 18:10 pm

Panel: PCJ1

Using data for crime prevention policy: Government-academic partnerships

Affiliation:

Abdul Latif Jameel Poverty Action Lab (J-PAL) – Latin America & Caribbean

Chair:

Trimarchi Edoardo

Presenters:

Nathalie Alvarado
Santiago Tobón
Pablo Vazquez
Xavier Vitores

 

Abstract:

A growing number of government agencies are turning to evidence and data to design effective strategies that can prevent, manage, and reduce crime and social violence. Despite the urgent need for policy solutions on crime and violence, there remain many open questions as to what works, even more so when thinking of the inherent challenges of working on citizen security and the local and contextual nature of crime. This panel will present clear examples of how to contextualize global evidence and think about what promising interventions are likely to apply in LAC, and successful experiences of government-academic partnerships to translate evidence and data into action.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

JALISCO Room (3rd floor)

11:15 – 12:15 pm

Panel: PBIG1

Synthesis without meta-analysis: Approaches to synthesis when meta-analysis of effect sizes is not performed

Affiliations:

University of Glasgow/Cochrane Public Health

Chair:

Hilary Thomson

Presenters:

Elizabeth Kristjansson
Vivian Welch

Abstract:

Meta-analysis (MA) of effect sizes is often considered an essential component of systematic reviews. Yet many, around 16% of Cochrane reviews, systematic reviews of intervention effects do not perform MA. The default term to describe synthesis methods when MA is not performed is often “narrative synthesis” (NS). However, there is lack of clarity about methods for NS, and recent work supports criticisms that reporting of NS is typically inadequate and opaque.
Session aim: To initiate discussion about approaches to synthesis of quantitative data when MA is not considered appropriate.
The session will include short presentations followed by structured discussion with participants on the following:
• Current practice in NS;
• Rationale and approach to synthesis without MA: case study;
• Synthesis without MA in Campbell reviews;
• Synthesis Without Meta-analysis (SWiM) reporting guidelines.

12:15 – 12:25 pm

Moving to new room

12:25 – 13:25 pm

Satellite event

Introducing a new partnership of organisations to advance research synthesis internationally

Affiliation:

Evidence Synthesis International (ESI)

Presenters:

David Gough
Jeremy Grimshaw

Abstract:

In order to know how research can inform policy, practice, personal decision making and future research we need to know what is known from research. We need systematic and transparent synthesis of evidence to clarify what is known about the evidence base. There are an increasing number of initiatives and organizations that produce, develop, use, and promote evidence syntheses around the world. The topics they cover the whole range of social science and science disciplines from education to conservation to veterinary medicine.
Evidence Synthesis International (ESI) is a new partnership to bring together these organizations and networks as a global hub to build capacity, share resources, and advocate for the synthesis and use of research evidence in policy and practice decision making. Working together we can learn from each others’ endeavours, have a powerful collective voice and reduce duplication of effort.
Please come to our session to learn more of our work and our plans and to see if your organization is interested in joining ESI.

13:25 – 14:30 pm
CIUDAD DE MEXICO Room (1st floor) 

Lunch

14:30 – 15:30 pm

Panel: PHEA2

Adaptive Learning for Systems Change: how systems thinking and small-scale testing can drive large-scale, sustainable change across health systems

Affiliation:

Results for Development Institute

Moderator:

Katie Bowman

Panelists:

Jean Arkedis
Christina Synowiec
Luke Heinkel
Taylor Salisbury

Abstract:

Health systems are complex, adaptive ones, and driving systems change is therefore a complicated task. How can we use systems thinking and methods (e.g., mapping) to identify leverage points that can create the conditions ripe for change across the system? With those leverage points identified, how might we use evidence generation and adaptive learning to rapidly test novel intervention approaches? In this panel, we will ask these tough questions, and share how we have tried to answer them within a variety of projects: USAID-funded Rapid Feedback Monitoring Evaluation Research and Learning (RF MERL); GiveWell-funded childhood pneumonia project; Australia DFAT-funded adaptive learning engagement with Tupaia Health Mapping App; and USAID-funded Health Systems Strengthening Accelerator. Teams will discuss the practical ways they have used lean, small-scale testing and continuous learning to create the conditions to produce sustainable systems change in the health sector.

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Parallel session: Automation in evidence synthesis
Chair: Erin Eldermire

The importance of Evidence Synthesis Technology in supporting systematic reviews and maps

 

Affiliation:

Stockholm Environment Institute/Africa Centre for Evidence

Presenter: 

Neal Haddaway

Abstract:

Evidence syntheses are resource-intensive, and we see an exponential increase in published research, challenging timeliness. Technology may support and improve evidence syntheses by: simplifying review conduct; assisting transparency; supporting group working; increasing efficiency; increasing rigour. Yet, Evidence Synthesis Technologies are typically isolated in research groups/disciplines, with little long-term support, and are hidden behind paywalls. The Evidence Synthesis Hackathon (ESH) is a series of interactive workshops bringing together experts in systematic review and programming to brainstorm and code new technologies to support evidence synthesis. The ESH is guided by Open Synthesis (Open Science in evidence synthesis), producing tools to increase transparency, efficiency, repeatability, rigour and accessibility. Here, we discuss why we must futureproof evidence synthesis.

Pilot study of the utility of machine learning tools to accelerate systematic reviews in economic evaluation

 

Affiliation:

Campbell Collaboration

Presenter: 

Denny John

Abstract:

This abstract describes the method and process to screen the reviews reporting costs and/or effectiveness/benefits, and quality of conducting and reporting economic evaluations in the Campbell library, using the machine learning filter of EPPI Reviewer. 155 protocols and reviews of Campbell library published till Feb 2019 were uploaded in EPPI reviewer. EPPI-Reviewer has an economic evaluation classifier model under its list of prebuilt classifiers. The classifier functionality allows one to create its own model based on a number of studies that have been identified as economic evaluations. The presentation will present the process and challenges of using machine learning filters for systematic reviews of economic evaluations and the prospects of implementing these strategies in future systematic reviews.

A data science revolution for global environmental assessments

 

Affiliation:

MCC Berlin/ University of Leeds

Presenter: 

Jan C. Minx

Abstract:

Scholars can no longer read and digest the huge volumes of scientific text currently being produced in their fields of expertise and track progress comprehensively. Securing the credibility and integrity of the IPCC and other global environmental assessments will require computer-assistance through scientometrics, machine learning and other data science applications. Developing data science capacities will prepare global environmental assessments for this age of “big literature” and upgrade them to a higher level of synthesis and rigour.

17:00 – 17:10 pm

Moving to new room

17:10 – 18:10 pm

Panel: PHEA1

Virtual Center for Operations in Emergencies and Disasters (CVOED acronym in Spanish)

Affiliation:

Conferencia Interamericana de Seguridad Social (CISS)

Chair:

José Antonio Hernández (Interamerican Conference on Social Security)

Presenters:

Dr. Sandra Elizondo (IMSS)
Dr. Víctor Hugo Borja (IMSS)

Abstract:

The Virtual Center for Operations in Emergencies and Disasters is an information and communications system that, in emergencies or disasters, allows immediate communication with the affected areas, as well as appropriate coordination between local, state and national levels, also allowing for appropriate decision making to provide rapid, organized and effective response, and allows the continuity of operations.
CVOED is a low-cost system that prepares situation rooms quickly using smartphones, tablets or computers using internet and also concentrates useful information during and after a crisis, such as availability of beds and services, availability of blood components, and nominal census of patients, among others. The CVOED has been used successfully by the main health sector institutions in México since 2009.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

HIDALGO Room (1st floor)

11:15 – 12:15 pm

Panel: PSPA1 – Presented in Spanish

Use of evaluation synthesis for programme and project improvement: examples from the environmental and rural development sectors

Affiliation:

FAO

Chair:

Carlos Tarazona

Presenters:

Teresa Romero
Oscar Garcia, Director, IFAD Independent Office of Evaluation

Abstract:

La FAO utiliza en los últimos años sintesis de evaluaciones como una herramienta de aprendizaje interno. La sesión propuesta explorará los objetivos, metodos y uso de sintesis llevadas a cabo en temas como biodiversidad, protección de areas protegidas y medio ambiente, a fin de identificar buenas practicas y lecciones aprendidas para la mejora de programas y proyectos futuros de la organizacion. Igualmente, se propone realizar una presentación sobre metodos para identificar contribuciones inesperadas de proyectos de cooperación internacional a la mitigación del cambio climatico.

12:15 – 12:25 pm

Moving to new room

12:25 – 13:25 pm 

Campbell Collaboration Knowledge Translation and Implementation Coordinating Group Meeting

 

Affiliation:

AIR and Co-Chair of the Campbell Knowledge and Translation Coordinating group

Presenter: 

Cindy Cai

Abstract:

The Campbell Knowledge Translation and Implementation Coordinating Group is a global group for anyone around the world who is in the business of producing high quality systematic reviews in the areas of Knowledge Translation and Implementation science and/or applying KT strategies to improve the use of high-quality evidence in practice and policy across a broad range disciplines (e.g., education, social welfare, mental health). Our community of members aims to bring together practitioners, policy makers, decision makers and other professionals that produce and use KTI focused research evidence. This annual session will provide an open forum for our group to update on activities, provide a highlight our reviews and create opportunities for member involvement.

13:25 – 14:30 pm
CIUDAD DE MEXICO Room (1st floor) 

Lunch

14:30 – 15:30 pm

15:30 – 16:00 pm
FOYER 1st floor

Coffee break – Networking

16:00 – 17:00 pm 

Parallel session: Mixed methods evaluation
Chair: Elizabeth Kristjansson

Opening the Black Box: A Mixed-Methods Investigation of Social and Psychological Mechanisms Underlying Changes in Financial Behavior

 

Affiliation:

Universität Göttingen, University of Oxford

Presenter: 

Janina Isabel Steinert

Abstract:

We use a mixed-methods approach to open the “black box” of a combined financial literacy intervention targeted at low-income families in South Africa and elucidate the key mechanisms through which changes in financial behavior are realized. Drawing on qualitative data, we find evidence for three causal pathways of change. Higher confidence in financial management skills, a more optimistic future outlook and emotional support are described as key facilitators of improved financial behavior. Mechanisms are cross-validated by quantitative data from a randomized control trial with 552 households. A causal mediation analysis indicates that the program’s effect on financial behavior is significantly mediated by financial self-efficacy (24% of total effect) and optimism (22% of total effect). Our findings motivate the inclusion of psychosocial program components in financial training curricula.

The Utility of Psychometrics for Evaluating Social and Educational Interventions

 

Affiliation:

Queen’s University Belfast

Presenter: 

Liam O’Hare

Abstract:

This presentation will discuss the various applications of psychometrics for evaluating interventions.
Two main areas of application will be discussed. 1. The importance of psychometrics in assessing the reliability and validity of outcome measures. 2. Using psychometric techniques to explore theories of change in social and educational interventions.
In addition, an overview of psychometric techniques will be provided including: reliability analysis, exploratory factor analysis, confirmatory factor analysis and path analysis.
Discussion of all psychometric applications and techniques will be embedded within examples of completed randomised controlled trial evaluations of social and educational interventions.

Better Data for Children on the Move

 

Affiliation:

AidData

Presenter: 

Jacob Sims

Abstract:

Development policy makers and practitioners working to help vulnerable children survive and thrive need accurate data and analysis to make evidence-based decisions as they design policies, programs, and advocate for change. Unfortunately, policy responses to many child-related issues are exacerbated by major gaps in data and evidence.
AidData is currently working to understand and improve the data landscape for assistance programs seeking to help migrant, refugee, and trafficked children. This project aims to address three key research questions via data scoping, desk research, and key informant interviews: 1) What role does data play in identifying COTM and directing resources towards them? 2) What are key gaps that prevent using data more effectively? 3) What are promising avenues for improving data collection/sharing/use to reach COTM?

17:00 – 17:10 pm

Moving to new room

17:10 – 18:10 pm

Book launch: Information and Communication Technologies for Development Evaluation

 

Affiliation:

ICT4Eval

Presenters: 

Oscar Garcia
Edoardo Masset

Abstract:

The panel will consist of Oscar Garcia, Director, Independent Office of Evaluation of International Fund for Agricultural Development and Edoardo Masset, Deputy Director, Centre of Excellence for Development Impact. The panel members have contributed to a book titled “Information and Communication Technologies for Development Evaluation (ICT4Eval)”. Written by a group of experts and development practitioners, the book presents on the theory and practice of applying technologies such as big data analytics, machine learning and remote sensing to the field of development evaluation, with a focus on measuring indicators of the Sustainable Development Goals and the panel will delve into it. The panel will also delve into the softer aspects of ICT4Eval viz. ethical and privacy concerns, concerns pertaining to bias and inclusion in using technology for evaluation, an area also covered by the book. This is a book written by practitioners, for practitioners. Come and see what ICT4Eval looks like in practice.

From 18:10 pm
FOYER 1st floor

Poster presentations Chair: Julia Littell
Exhibition area

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

WELCOME EVENT

Copyright 2019 wwgs2019. All rights reserved