Conference day 2 – 17 October

7:30 – 9:30 am
Jalisco Room
3rd floor

Campbell Members’ conference

open to all participants of the WWGS

9:00 am
FOYER 1st floor

DAILY REGISTRATIONS

9:00 – 9:40 am
FOYER 1st floor

Coffee served – networking

9:40 am-10:45 am
NUEVO LEON II Room
3rd floor

PLENARY

Chair:

David Ameyaw, CEO/President, International Center for Evaluation and Development

Presentations:

Building capacity in developing countries to promote greater participation of local experts in policy debates by Jane Mariara, Executive Director, Partnership for Economic Policy
Abstract: PEP is a global network that brings together researchers and research institutions from around the world, to build capacity and undertake research that provides contextualized solutions to key economic and social policy challenges in developing countries. Its research programmes cover Community-based monitoring systems (CBMS), Macro-micro policy modeling (MPIA), Experimental research (PIERI), and Microeconomic analysis (PMMA). PEP’s “Grant Plus” process is designed to produce high quality research which informs policy by providing mentoring to both the research process and knowledge translation. This presentation will describe examples of this process, success stories in influencing policy, and lessons learned for the research community.

Funding research to be policy relevant by Manny Jimenez, Executive Director, International Initiative for Impact Evaluation (3ie)
Abstract: The International Initiative for Impact Evaluation (3ie) is a global grant-making body for impact evaluations and systematic reviews. 3ie specialises in increasing access to, demand for and use of evidence by governments, parliaments, programme managers, civil society, programme participants and the media. All 3ie grantees complete a Stakeholder Engagement and Evidence Uptake Plan which emphasizes the value of planning and engaging with stakeholders to ensure that evaluations and reviews are relevant and useful. Robust and effective monitoring is put in place to measure evidence use to capture evidence of impact on programmes and policies with greater confidence. This presentation will describe this process, with examples of 3ie-supported studies which have led to policy impact, and lessons learned for researchers wishing to enhance the policy impact of their work.

Supporting a culture of evidence use in a development organization by Sheree Bennett, , Research and Evidence Advisor, International Rescue Committee (IRC)
Abstract: The International Rescue Committee is an international development agency which responds to the world’s worst humanitarian crises, helping people whose lives and livelihoods are shattered by conflict and disaster to survive, recover and gain control of their future. IRC provides clean water, shelter, health care, education and empowerment support to refugees and displaced people in more than 40 countries and in 26 U.S. cities. IRC has made a commitment for 100 percent of its interventions to be evidence-based or evidence-generating by 2020. To achieve this, IRC undertook an extensive programme of evidence mapping to develop the Outcomes to Evidence Framework (OEF), a tool identifying the evidence for what works to achieve IRC’s target sectors. This presentation will present the OEF, IRC’s experience in using it, and lessons for other agencies wishing to adopt a more evidence-based approach.

10:45 – 11:00 am

 

Moving to new room

AGUASCALIENTES Room (2nd floor)

11:00 – 12:00 pm

Panel: PSW3

Beyond cash transfers: innovative solutions

Affiliation:

CONEVAL

Chair:

John Scott Andretta

Presenters:

Leonardo Lacovone, Banco Mundial
Rodrigo Salazar, FLACSO
Juan Alfonso Mejía López, Ministry of Education and Culture, Sinaloa, Mexico

Abstract:

Recently, alternative policies known as nudge policies, that aim to influence the decisions of people through incentives, have been explored successfully. By constituting a complement (not a substitute) for traditional public policies, special units for their design and evaluation in small groups have been developed in several countries around the world with the main objective of measuring the effects and costs associated to these policies for assessing their scalability. In this way, these kinds of policies represent an outstanding alternative in contexts of low resources and low credibility of the operators. Therefore, the purpose of the panel is to discuss the fundamentals, usefulness and experiences of policies based on behavioral science, as well as their advantages, limitations and challenges, to identify the ways in which these policies could be applied in Mexico.

12:00 – 13:00 pm
CIUDAD DE MEXICO Room
1st floor

Lunch

13:00 – 14:00 pm 

Campbell Collaboration Groups

Meetings open to all participants of the WWGS

SOCIAL WELFARE

Campbell Collaboration Social Welfare Coordinating Group

 

Affiliation:

Saint Louis University, co-chair and Editor of the Campbell Social Welfare Coordinating group

Presenter: 

Brandy Maynard

Abstract:

This session will introduce the Campbell Collaboration’s Social Welfare Coordinating Group (SWCG). We will provide an overview of the people, scope/remit and processes of the SWCG. We will also discuss some of the reviews recently published through the SWCG and highlight some of the reviews in progress.

AGING

Campbell Collaboration Aging Coordinating Group

 

Affiliation:

Campbell Collaboration

Presenter: 

Vivian Welch

Abstract:

The Campbell Ageing group is interested in social dimensions of ageing. We will describe preliminary findings of a Campbell evidence and gap map on home-based social care and mobility devices, our ongoing Campbell reviews, our collaboration with Cochrane and describe our plans for the future and ways to get involved. Please join us!

14:00 – 14:15 pm

Moving to new room

14:15 – 15:15 pm 

Panel: PSW4

Strengthening the Evidence Base on Ending Violence Against Children in Low and Middle Income Countries

Affiliation:

UNICEF Office of Research-Innocenti

Chair:

Kerry Albright

Presenter: 

Howard White

Abstract:

This session aims to showcase some of the emerging thinking around strengthening the evidence base on ending violence against children (EVAC) in low and middle income countries and will highlight ongoing work of UNICEF’s Office of Research-Innocent and the Campbell Collaboration. The session will share findings of a new evidence and gap map on EVAC, highlight broader strategic and political linkages to multi-stakeholder fora in this space, discuss forthcoming strategies to both ameliorate poor evidence quality and to broaden the concept of quality evidence around EVAC, and introduce current conceptual thinking around strengthening evidence ecosystems to enhance evidence uptake and use through greater ownership, as well as to strengthen evidence architecture to take better account of common scaling up, transferability and implementation variability challenges in the EVAC sector.

15:15 – 15:45 pm
FOYER 1st floor

Coffee served – networking

15:45 – 16:45 pm 

Campbell Collaboration Groups

Meetings open to all participants of the WWGS

INTERNATIONAL DEVELOPMENT

The Campbell international Development coordinating group

 

Affiliation:

Cochair and Editor of the Campbell IDCG and International Initiative for Impact Evaluation (3ie)

Presenters: 

Marie Gaarder
Birte Snilstveit

Abstract:

This is an introductory session describing IDCGs aim, scope, processes and people. IDCG supports systematic reviews and Evidence and Gap Maps of evidence on social and economic development interventions in low- and middle-income countries. The group aims to build capacity to conduct reviews among authors, including those based in LMICs. We also advocate for the use of evidence to inform decisions and engage with evidence users. In addition to mentoring of authors registered with the group, we aim to build capacity among new authors by organizing, or participating in, training events such as WWGS. The session will have a brief presentation about the group followed by questions round from the participants.

NUTRITION and FOOD SECURITY

Introduction to the new Food systems, food security and nutrition group

 

Affiliation:

Co-chairs and Editor of the Campbell Nutrition and Food Security Coordinating groups

Presenters: 

Elizabeth Kristjansson
Annette O’Connor

Abstract:

In this 30 minute session, we will introduce and discuss the new Food systems, Food security and nutrition group. This new group resulted from the unification of the Nutrition and the Food Security groups. Our goal is to bring together the broad spectrum of issues associated with food and nutrition. If you are interested in any aspect of food from sustainable production to human nutrition, we invite you to attend. Similarly, if your interests include food policy, food-associated risk assessments, food security (at all levels), and systems models and interventions, we welcome your participation. The 30 minutes will be largely devoted to questions about the group, after a short introduction to the scope of our group.

16:45 – 17:00 pm

Moving to new room

17:00 – 18:00 pm 

Panel: EXT6

Social Protection, Productive Inclusion and Gender Equity: Graduation programs in Latin America

Affiliation:

International Development Research Centre (IDRC)

Chair:

Sonia Laslo, McGill University, Canada

Presenters:

Tatiana Rincon, Fundacion Capital, Colombia
Graciela Teruel, Universidad Iberoamericana, Mexico
Jorge Maldonado; Universidad de los Andes, Colombia

Abstract:

The graduation approach is one of the most effective programs for social and productive inclusion of households living in extreme poverty. Globally, the evidence points to considerable success in improving living standards and economic resilience, however Graduation programs impacts on women’s empowerment to date are unclear. This panel will share lessons learned at the regional level about this policy innovation and delve into the factors that explain the effectiveness of the program based on recent findings from the IDRC supported Latin America Research, Evaluation and Learning Platform on Graduation and other studies in four countries (Mexico, Peru, Paraguay and Colombia). It will discuss results and an agenda moving forward on how to strengthen gender equality and women empowerment through graduation. Beyond Graduation programming, this panel will also offer valuable lessons to inform policy and practice on other extreme poverty elimination and livelihoods programs.

18:00 pm-19:00 pm
NUEVO LEON II Room
3rd floor

PLENARY PANEL

The future of the evidence agenda for What Works in Education

Chair: Patrick Mbah Okwen

Presenter:

Mark Schneider, Director of IES Research

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

Poster presentations Chair: Annette Brown

HAPPY HOUR EVENT

NUEVO LEON I Room (3rd floor)

11:00 – 12:00 pm

Panel: PE4

Adaptive learning to understand what works in education: three case studies on approaches to using monitoring, evaluation, and research to improve the likelihood of success

Affiliation:

Results for Development Institute

Chair:

Jean Arkedis

Presenters: 

Luke Heinkel
Katie Bowman
Taylor Salisbury
Morgan Benson

Abstract:

The ultimate measure of an effective education system is whether it improves learning; while there has been evidence generated about interventions that do so, the handful of so-called “proven” approaches are merely a starting point. How to best apply, adapt, and scale them is a challenge that requires new ways of generating evidence. The first presenter will highlight how formative research was used to inform theories of change for a mobile reading program in India and an adult literacy and workforce platform in the US. The second will discuss how small-scale experimentation and monitoring helped low-fee private schools in West Africa (also the subject of a separate impact evaluation) improve their literacy program. Finally, we will share a new series of action research engagements to provide real-time feedback on how to best deploy education technology within local education systems.

12:00 – 13:00 pm
CIUDAD DE MEXICO Room
1st floor

Lunch

13:00 – 14:00 pm 

Parallel session: Implementation of evidence in education
Chair: Jeffrey Valentine

CANCELLED 3 key steps for evidence implementation in education: the ICT junior fellowship for TVET

 

Affiliation:

Effective Basic Services Africa

Presenter: 

Pambe Miong Rigobert Hanny

Abstract:

While Educational outcomes in sub-Saharan Africa are saddening, it even get worse when it comes technical and vocational education and training (TVET). Our objective was to assess the impact of best practices (ICT support and mentorship) on TVET learner’s performance and employability. Our tripod approach consisted of: exploring what works; contextualizing and implementing evidence; monitoring and evaluation and alterations. We recorded a significant improvement in participants’ performance at competitive test in schools. 70% of students were able to manufacture, market and sale their products using computers and social media. We recorded improvement in lifelong learning, desire to share, leadership and entrepreneurship. In a resource challenged setting getting the best evidence into practice is paramount. Our 3 steps approach facilitates this process.

14:00 – 14:15 pm

Moving to new room

14:15 – 15:15 pm 

Panel: PE2

Scaling education innovations in complex emergencies: Evidence from the Humanitarian Education Accelerator

Affiliation:

American Institutes for Research

Chair:

Thomas de Hoop

Presenters: 

Muy Cheng
Kelsey Hunt
Andrea Coombes

Abstract:

To address evidence-gaps on how to effectively scale education innovations, the UK Department for International Development, UNICEF, and UNHCR created the Humanitarian Education Accelerator (HEA). The HEA included a meta-evaluation that synthesized evidence from process and impact evaluations of five implementers that are scaling education innovations in humanitarian contexts. This panel will include reflections of researchers, UNICEF, and UNHCR on the quantitative and the qualitative findings of the synthesis. The quantitative synthesis estimated effects of the education innovations on educational and psychosocial outcomes, and compared those effects with systematic reviews on education programs. The qualitative synthesis adapted a meta-ethnography approach by organizing results according to three main themes: context; business model; and program ownership and advocacy.

15:15 – 15:45 pm
FOYER 1st floor

Coffee served – networking

15:45 – 16:45 pm 

Parallel session: Impact evaluation in education: methods
Chair: Molly Irwin

Using machine learning to improve program targeting: Predicting non-enrollment of girls in northern India

 

Affiliation:

IDinsight

Presenter: 

Ben Brockman

Abstract:

Machine learning has been heralded as the latest tool to help policymakers target programs and reduce poverty. We demonstrate a high-value policy application of machine learning to improve the cost-effectiveness of a large education program. In India, despite recent strides to improve enrollment, 4.1 million girls remain out of school. Educate Girls, a large NGO that helps families overcome obstacles to enrollment, seeks to reach as many non-enrolled girls as possible without having the resources to expand to all of India’s villages. We build a series of machine learning algorithms using Educate Girls’ existing administrative data and publicly-available government census data to predict the number of out-of-school girls in each of 200,000 potential expansion villages. Using our predictions, Educate Girls can enroll 50% to 100% more girls at approximately current cost levels.

How do the impacts of educational programs vary by gender? Evidence from randomized evaluations around the world

 

Affiliation:

Abdul Latif Jameel Poverty Action Lab (J-PAL)

Presenter: 

Priyanka Varma

Abstract:

Despite substantial increases in school participation around the world, girls of primary school age remain less likely to attend school than boys. Given that girls may face gender-specific barriers to attending school, do programs that focus on all children have different impacts for girls and for boys? We reviewed randomized evaluations of 25 programs to increase school participation. Most programs that improved school participation were as or more effective for girls than for boys. Successful programs aimed at increasing participation in general tend to help the disadvantaged gender most, likely since the most marginalized students are particularly sensitive to the costs and perceived benefits of education. Our gender analyses examine how student participation programs affect boys and girls differently, giving examples from programs in Ghana, Afghanistan, Colombia, and Nicaragua.

Have Effect Sizes Changed in English Language Arts Achievement Between 1985 and 2017? A Meta-Analysis of the WWC Reviewed Studies

 

Affiliation:

Independent Consultant

Presenter: 

Tsze Chan

Abstract:

What Works Clearinghouse has recently made available to the public properties of studies it has reviewed. Using an excel data file downloaded from the WWC website in February 2019, this study examines whether there has been a change in the magnitude of effect sizes for outcome measures in English Language Arts between 1985 and 2017. The analysis dataset has 45 interventions, 63 studies, and 233 effect sizes. We fit a linear trend line in a meta-regression using the Robust Variance Estimation which assumes a random effect model for significant tests. Preliminary result shows that effect sizes have been decreasing on average by about 0.01 unit (p < 0.05) per year over this period. The study examines concurrent changes in study properties such as sample sizes, types of evaluation design and comparison group used. The additional analysis offers some plausible explanations for the decline.

Does the impact of technology differ by students’ age? A meta-analysis of What Works Clearinghouse literacy interventions

 

Affiliation:

Mathematica Policy Research

Presenter: 

Andrei Streke

Abstract:

This study investigates how the impact of digital technologies on literacy skills differs by students’ age. Previously, we found that traditional interventions outperformed computer-assisted interventions (CAIs) on literacy outcomes for early readers (grades K-3) after controlling for various study properties. For the present study, we added data from the WWC database on preschoolers (ages 3-5) and adolescents (grades 4-12). The combined dataset, now includes 172 studies and 933 effect sizes, allows us to compare the impact of CAIs vs. non-CAIs overall and across 3 age groups (WWC topic areas). We use a meta-regression which assumes a random effects model and includes moderators suggested by education research. The coefficients and the standard errors are estimated by the robust variance estimation and by the synthetic effect size approach. Results from the two approaches are compared.

16:45 – 17:00 pm

Moving to new room

17:00 – 18:00 pm 

Panel: EXT3

Using Evidence to Strengthen Policy. Lessons from Climate Change, Education and Public Health Sectors in Mexico

Chair:

Gabriela Pérez Yarahuán, CIDE-CLEAR LAC

Presenters: 

Salomón Chertorivski, CIDE
Teresa Bracho, FLACSOs
Marco Antonio Heredia Fragoso, INECC

Abstract:

Mexico witnessed the creation of several public institutions devoted to evidence generation for policy during the previous decade. The National Institute for Educational Assessment (INEE, for its Spanish acronym), the National Institute of Ecology and Climate Change (INECC, for its Spanish acronym), and the National Council for the Evaluation of Social Development Policy (CONEVAL, for its Spanish acronym) were created between 2002 and 2005. The Popular Health Insurance (Seguro Popular in Spanish) was a complementary policy to the public health system in Mexico that heavily relied on evidence for its design. There are many lessons regarding how evidence has been used to inform policy decisions in the climate change, education and public health arenas that can inform how to better generate and communicate evidence to increase its use in the near future.

18:00 pm-19:00 pm
NUEVO LEON II Room
3rd floor

PLENARY PANEL

The future of the evidence agenda for What Works in Education

Chair: Patrick Okwen

Presenters:

Mark Schneider, Director of IES Research

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

Poster presentations Chair: Annette Brown

HAPPY HOUR EVENT

SONORA Room (2nd floor)

11:00 – 12:00 pm

Panel: PECO1

Evidence syntheses on the impact, implementation models and cost-effectiveness of women’s collectives

Affiliation:

American Institutes for Research

Chair:

Shubha Jayaram

Presenters: 

Thomas de Hoop
Leigh Anderson
Sapna Desai

Abstract:

Governments and NGOs have invested in different women’s empowerment collectives (WECs), including self-help groups (SHGs) and health groups. Systematic reviews suggest that WECs can improve women’s empowerment and health outcomes but provide limited documentation on implementation and cost-effectiveness. Furthermore, data on empowerment are difficult to quantify. This panel focuses on the learning agenda of a consortium that aims to address evidence-gaps through evidence-syntheses and primary research on implementation, pathways to change, and cost-effectiveness of WECs in India and East-Africa. We will present systematic reviews on the impact of SHGs on empowerment, and women’s groups and health outcomes, as well as methodological insights gained through an evaluation of the Gates Foundation’s investments in WECs. The presentations will also discuss the consortium’s learning agenda.

12:00 – 13:00 pm
CIUDAD DE MEXICO Room
1st floor

Lunch

13:00 – 14:00 pm 

Parallel session: Assessing methodological quality
Chair: Meghan Finch

The methodological and reporting characteristics of economic methods and outcomes in Campbell reviews: A systematic review

 

Affiliation:

Campbell Collaboration

Presenter: 

Denny John

Abstract:

This abstract describes the method and process to identify the studies reporting costs and/or effectiveness/benefits, and quality of conducting and reporting economic evaluations in the Campbell library. 155 protocols and reviews of Campbell library published till Feb 2019 were uploaded in EPPI Reviewer. These studies are currently being screened using an algorithm to identify if a review has an economic component as an outcome and/or bibliography/reference list of economic studies. A data extraction form with parameters; type of economic evaluation, number of studies, publication year, quality of study, and quality instrument used, study design and statistical analysis, and equity considerations, will be conducted. Finally, reviews will be grouped according to whether reviews assessed methodological quality or by intervention, or by specific methods used in economic evaluations.

GRADEing the quality of a body of evidence for complex interventions: challenges and the way forward

 

Affiliation:

University of Birmingham UK

Presenter: 

Paul Montgomery

Abstract:

Background:
Several commentators suggest that GRADE has limited usefulness for Complex Interventions as its constructs frequently rate them as being of low quality and in turn, policy uptake is reduced.
Objectives:
To discuss with participants the findings from the project and to invite thoughts on them.
Description:
First, I will review different conceptualizations of complex interventions and how these may affect many systematic reviews. I will then present empirical findings from our recent investigation into challenges and user-proposed modifications for adapting the GRADE approach to complex interventions. Finally, we will provide an overview of papers reporting a new extension for GRADE with respect to complex interventions.

Taking stock of systematic reviews in international development: a study of methodological quality, policy relevance and usefulness

 

Affiliation:

3ie

Presenter: 

Ada Sonnefeld

Abstract:

This presentation will summarise the findings of two studies designed to assess the reliability of systematic reviews in international development and the extent to which they provide policy relevant and useful findings: a qualitative study to explore stakeholders’ experience and views of systematic reviews; and a systematic review of the findings and methods of systematic reviews of international development interventions This is the first attempt to take stock of systematic reviews in international development through empirical investigations of their methodological quality, usefulness and relevance. The presentation will conclude with summarising the key findings and lessons learned from the last decade of systematic reviews addressing questions of relevance to international development, highlighting implications for the commissioning and conduct of systematic reviews.

14:00 – 14:15 pm

Moving to new room

14:15 – 15:15 pm 

Panel: PSW6

Systematic Reviews on the Effects of Interventions Targeting Child Health and Education in LMICs

Affiliation:

Stanford University

Chair:

Scott Rozelle

Presenters: 

Huan Wang
Yue Ma
Matthew Boswell

Abstract:

Improving the schooling and well-being of children from poor and disadvantaged areas has been a long-standing challenge for education and health systems in low- and middle-income countries. Many studies evaluate different interventions, but differences in treatments, settings, instruments, and methods make findings difficult to compare and may even obscure underlying trends. Understanding the best, most cost-effective child focused programs for LMICs remains an urgent need. The objective of the panel is to synthesize available evidence on the effects of key children’s health and education interventions. This panel includes five systematic review papers, each focusing on one type of intervention (vision care, long-distance learning, infant nutrition, and early childhood development). The reviews aim to provide evidence-based guidance for policy makers and program implementers.

15:15 – 15:45 pm
FOYER 1st floor

Coffee served – Networking

15:45 – 16:45 pm 

Parallel session: Child Welfare
Chair: Janina Steinert

Interventions for Violence against Children in Low and Middle Income Countries: An Evidence and Gap Map

 

Affiliation:

Campbell Collaboration

Presenter: 

Ashrita Saran

Abstract:

More than 1 billion children—half the children in the world—are victims of violence every year. Although experiencing violence in childhood impacts lifelong health and well-being, it is often preventable. However, there is an increased need to invest in generating sound evidence on the scale and nature of violence against children and to know ‘what works and what does not’. Evidence and Gap Map identify research gaps and to provide a resource for informing policy and practice. The intervention outcome framework is based on the INSPIRE framework (WHO 2016) which identifies seven evidence-based strategies to prevent violence against children viz. Implementation and enforcement of laws; Norms and values; Safe environments; Parent and caregiver support; Income and economic strengthening; Response and support services; and Education and life skills.

Evidence standards to prevent ‘K light’ Knowledge Mobilization in child and social welfare

 

Affiliation:

EPPI-Centre

Presenter: 

David Gough

Abstract:

This presentation will consider the evidence standards for making evidence claims to inform recommendations about policy and practice in child and social welfare programmes.
To assess evidence claims we need to know about the quality and relevance of the primary research, the totality of evidence from that research, and the quality and relevance of the methods by which the research evidence is synthesized / reviewed. Transparency in reporting of all these methods is also required in order to make it possible to assess the claims.
Data will be presented on the extent that evidence standards are met in mobilizing evidence to inform policy and practice. It is argued that there are dangers of weak evidence being used to inform policy and practice recommendations in particular with branded child and social welfare programmes.

Intergenerational mobility and differential of education, income and occupation between youths in Brazil

 

Affiliation:

Federal University of Latin American Integration

Presenter: 

Ana Lucia Kassouf

Abstract:

This article uses the School-to-Work transition survey (SWTS) conducted by the International Labor Organization in 2013 to measure educational mobility, occupational mobility, as well as to evaluate youths´earnings. The database presents retrospective questions regarding the socioeconomic characteristics of the parents, allowing the measurement of intergenerational mobility for individuals from 15 to 29 years old. The results obtained show great mobility between generations, especially when parents had lower levels of education. To analyze the returns to education on earnings, the income equations were estimated using the Heckman selection model. The education of youths and parents had a great impact on income.

Best of UNICEF Research: A global competition to reward research excellence and uptake

 

Affiliation:

UNICEF Office of Research-Innocenti

Presenter: 

Kerry Albright

Abstract:

UNICEF spends approximately US$120 million per annum on child-related research and evidence synthesis, but like other organizations, sometimes struggles to maximize recognition and uptake within and outside the organization. The annual Best of UNICEF Research competition, first launched in 2013 with the aim of enhancing the visibility and recognition of UNICEF staff involved in commissioning or undertaking research across more than 190 UNICEF offices worldwide, has now become a central part of the organization’s evidence culture. This presentation will highlight the conceptual thinking behind the exercise, how it has served to accelerate evidence brokering and replication of successful research and how it has given recognition to UNICEF’s researchers and research partners. It will also highlight some of the original award-winning research and ongoing impacts, particularly from the Latin America and Caribbean region.

16:45 – 17:00 pm

Moving to new room

17:00 – 18:00 pm 

Panel:

The future of evidence curation and translation in international development: opportunities and challenges

Affiliation:

International Initiative for Impact Evaluation (3ie)

Chair:

Marie Gaarder

Presenters:TBC 

Harsha Dayal, Department for Planning, Monitoring and Evaluation
Rhona Mijumbi-Deve, African Centre for Systematic Reviews and Knowledge Translation
Birte Snilstveit, International Initiative for Impact Evaluation
TBC, Epistemonikos

Abstract:

Coming soon

18:00 pm-19:00 pm
NUEVO LEON II Room
3rd floor

PLENARY PANEL

The future of the evidence agenda for What Works in Education

Chair: Patrick Okwen

Presenters:

Mark Schneider, Director of IES Research

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

Poster presentations Chair: Annette Brown

HAPPY HOUR EVENT

NUEVO LEON II Room (3rd floor)

11:00 – 12:00 pm

Panel: PECO2

Cost collection and analysis to increase the utility of research for policymakers

Affiliation:

Abdul Latif Jameel Poverty Action Lab

Chair:

Kyle Murphy

Presenters: 

Elizabeth Brown
Alaka Holla
Bethany Park

Abstract:

Cost-effective analysis can be powerful tools to increase the value and take up of research results by policymakers, yet too few projects collect comprehensive cost data. Costing requires information from multiple sources: academic papers or program reports for a description of the program structure, project documents for detailed costs, and public sources for components such as local wages and transportation costs. Comparable costing across programs requires consistent and careful decision-making about which costs to include and the application of the same methodology across multiple contexts and projects. This panel will bring together several organizations working on new efforts on uniform cost collection, improving the comparability of CEAs, benchmarks for CEA, and frameworks to consider value for money across different approaches to impact similar outcomes.

12:00 – 13:00 pm
CIUDAD DE MEXICO Room
1st floor

Lunch

13:00 – 14:00 pm 

Parallel session: Early years learning
Chair: Jane Hamilton

Enhancing young children’s language acquisition through parent-child book sharing: a randomized trial in rural Kenya

 

Affiliation:

University of Michigan

Presenter: 

Heather Knauer

Abstract:

Worldwide, 43 percent of children under five do not meet their developmental potential due in part to inadequate cognitive stimulation. Several parent support programs benefit children’s development, but are resource intensive. This cluster-randomized controlled trial tested four variants of a potentially scalable, cost-effective intervention to increase cognitive stimulation by caregivers (n=357) and improve emergent literacy skills in 24- to 83-month-old children (n=510). The intervention was an interactive reading training program that used culturally and linguistically appropriate books. Books and training improved the frequency and quality of caregiver-child reading, and storybook comprehension among children. The children of illiterate caregivers benefited at least as much as the children of literate caregivers, and for some outcomes, there were differentially larger effects.

A Cluster Randomized Controlled Trial of Reciprocal Reading: A Reading Comprehension Intervention for Elementary School Aged Children

 

Affiliation:

Queen’s University Belfast

Presenter: 

Liam O’Hare

Abstract:

Reciprocal Reading aims to improve reading comprehension in elementary school aged pupils. The intervention is delivered by teachers and teaching assistants who are trained on how to deliver the programme. The intervention can be delivered in a universal way (whole class) and in a targeted way (for pupils with poor comprehension, but good decoding skills).
To evaluate the programme 98 schools in England were randomised (49 intervention, 49 control and n= 5222). The intervention schools did both versions of the programme in two separate year groups.
A range of pre-post reading comprehension and attitude measures were used to identify effectiveness of the programme. Both versions showed a statistically significant effect for comprehension reading attitudes. Only the targeted intervention showed a statistically significant effects on reading comprehension ability (effect size g = .18).

Impacts of a low-cost pre-school program in rural Mozambique

 

Affiliation:

American Institutes for Research

Presenter: 

Juan Bonilla

Abstract:

We conduct a mixed-methods experimental evaluation of an accelerated school readiness program in Zambezia (Mozambique) on children’s school readiness, on-time enrollment, and academic achievement in Grade 1 relative to comparable children in communities with no pre-primary education. Program provided 120 hours to children aged 5 to 6 by trained community volunteers; strengthened school councils; and provided 13 weeks of parent-to-parent education sessions. We found large and statistically significant impacts on all child domains assessed (including emergent literacy, emergent numeracy, and motor skills) both right after the end of the program and one year later at the end of their first grade of primary school. We also found positive impacts on the likelihood that children enroll and attend primary school. We find minor impacts on parents’ attitudes and education practices.

Evidence-Informed Parenting Support Programme in South Africa: Programme Impacts and Implementation

 

Affiliation:

University of Oxford/RAND Europe

Presenter: 

Yulia Shenderovich

Abstract:

We conducted a cluster-randomised controlled trial of a parenting programme, including 552 families in 40 rural and peri-urban locations in the Eastern Cape, South Africa. Parenting for Lifelong Health: Sinovuyo Teen is a non-commercialised 14-session parent and adolescent programme delivered by trained community members. Pre-specified outcomes were measured at 1 and 5-9 months post-intervention. The intervention was associated with several reported improvements, such as lower maltreatment and corporal punishment (caregiver report), more involved parenting (caregiver and adolescent report) and others, while some of the outcomes did not show a difference between groups. In this presentation, we will also discuss patterns of attendance, engagement, and fidelity of the intervention, and whether participant characteristics affected attendance and engagement, or intervention outcomes.

14:00 – 14:15 pm

Moving to new room

14:15 – 15:15 pm

Panel: PECO6

Importance of evidence creation in women’s economic empowerment (WEE) in India

Affiliation:

IWWAGE-IFMR

Chair:

Yamini Atmavilas

Presenters:

Madhuparna Joshi
Sona Mitra
Soledad Prillaman
N Vijaya Lakshmi

Abstract:

WEE is a major policy concern in India. The panelists represent data and evidence to action initiatives operating at different levels of governance in India. They work through an initiative the Bill and Melinda Gates Foundation is supporting called the Initiative for What Works to Advance Gender Equality. The network of IWWs aim to fill evidence gaps critical for progress policy making on gender. They will share their theories of change, and discuss discrete pieces of evidence they’re generating on women’s economic empowerment. They will speak about the ‘what’ as well as the ‘how’ of evidence-based policy making, and share successes and challenges in generating and using evidence to shape policy discourse in India.

15:15 – 15:45 pm
FOYER 1st floor

Coffee served – Networking

15:45 – 16:45 pm 

Parallel session: Climate and Environment
Chair: Kate Ghezzi-Kopel

Seas, Oceans and Health: using evidence mapping to inform research and policy

 

Affiliation:

University of Exeter

Presenter: 

Ruth Garside

Abstract:

The health of the ocean and human health is intrinsically linked, and these linkages are numerous and varied: from the impact of marine nutrients on healthy diets, to the impact of anthropogenic pollutants and pharmaceuticals on the environment. Understanding these links requires health concerns to permeate environmental policy and for the role of the marine environment to be considered in health agendas.

Mentoring to support rigorous evidence synthesis for global climate assessments

 

Affiliation:

Stockholm Environment Institute / Africa Centre for Evidence

Presenter: 

Neal Haddaway

Abstract:

Climate change is one of the biggest threats we face, and significant effort is given to attempting to summarise evidence to direct decision-making for adaptation and mitigation strategies. However, we currently face an ‘evidence explosion’; an exponential increase in published climate research, meaning that global evidence assessments like the IPCC face an increasingly impossible task of reviewing all of the evidence. To date, the climate researchers have been slow to embrace formalised, rigorous evidence synthesis methods. Here, we introduce a recent large-scale mentoring and systematic review publication project that aims to provide rigorous synthesis for the IPCC and build capacity for synthesis in the climate community. We see this as a necessarily incremental process, rather than immediate transformational change. We report on the progress and initial results here.

Learning on climate solutions in the IPCC: towards cumulative social science research on climate change mitigation and adaptation

 

Affiliation:

MCC Berlin/ University of Leeds

Presenter: 

Jan Minx

Abstract:

The Intergovernmental Panel on Climate Change(IPCC) has been very successful in learning about the physical science basis of climate change. Yet, little progress has been made to systematically learn about policies that deliver climate solutions. Formal evidence synthesis is urgently needed to put forward the most promising options. This requires a cultural shift in the social sciences, similar to the one which transformed the health sciences into a stronger, more efficient and more credible discipline. This presentation summarizes on-going efforts within and outside the climate community to understand climate solutions.

Mapping the vulnerabilities of children and youth to the health impacts of climate change

 

Affiliation:

University of Alberta

Presenter: 

Denise Thomson

Abstract:

Objective: To present results from a scoping review on the question, “What is currently known about the vulnerabilities of children and youth (0-18 years) across the globe to the health impacts of climate change?”
Context: Children and youth are particularly vulnerable to the health impacts of climate change, but the nature and extent of this vulnerability have not been mapped. The health impacts will be categorized as: 1) direct Impacts of climate and weather on health (e.g. excessive heat); 2) ecosystem-mediated impacts of climate change on health outcomes (e.g. spread of vector-borne diseases); 3) health impacts heavily mediated through human institutions (e.g. nutrition; migration); 4) adaptation or mitigation strategies addressing health impacts or co-benefits. Findings from the review will be aggregated into descriptive and thematic summaries, and presented in evidence tables.

16:45 – 17:00 pm

Moving to new room

17:00 – 18:00 pm 

Fishbowl session

CEDIL: The frontiers of effectiveness research

Chair: David Ameyaw, President and CEO of the International Center for Evaluation and Development (ICED) and Chair of the CEDIL Advisory Board

Presenters:

Howard White, Research Director CEDIL: CEDIL – origins and motivations
Edoardo Masset, Deputy Director CEDIL: CEDIL – emerging themes in the frontiers of impact evaluation
Craig Bardsley, Programme Manager, CEDIL – What is CEDIL funding?

Abstract:

The Centre of Excellence for Development Impact and Learning is a DFID-financed research centre to identify, develop and test new and innovative way of assessing the impact of development interventions and synthesizing that evidence. This session is introduction to CEDIL and its work. A fishbowl format will be used providing audience members an opportunity to engage the panel in discussion.

18:00 pm-19:00 pm
NUEVO LEON II Room
3rd floor

PLENARY PANEL

The future of the evidence agenda for What Works in Education

Chair: Patrick Okwen

Presenters:

Mark Schneider, Director of IES Research

From 19:00 pm
CIUDAD DE MEXICO Ballroom
1st floor

Poster presentations Chair: Annette Brown

HAPPY HOUR EVENT

JALISCO Room (3rd floor)

11:00 – 12:00 pm

Panel: PSPA3 – This panel will be presented in Spanish with translation to English available

Strengthen local government decision-making based on evidence and oriented towards compliance with the SDGs: Experiences and lessons learned from the implementation of three academic models in Mexico and Brazil

Affiliation:

UNDP Mexico

Chair:

Annabelle Sulmont

Presenters:

Eduardo Sojo
David K. Bohl
Carlos Gomes Pessoa
Florian Chávez
Rafael Hernández

Abstract:

The objective of the panel is to generate a debate on the capacities and limitations of local governments in designing policies that promote sustainable development at the subnational level. Three concrete experiences of the application of practical tools at the local level will be presented and discussed:

1. Sustainable Cities Index: a tool that allows measuring the progress of metropolitan areas in Mexico towards compliance with the SDGs.

2. International Futures: a prospective model that helps local governments to formulate and compare future scenarios in order to identify the impacts of different policy approaches in the medium and long term.

3. Agent-based policy model: based on behavioral economics, this model accounts for inefficiencies in the implementation of public policy interventions, as well as the network of interrelationships between development indicators.

12:00 – 13:00 pm
CIUDAD DE MEXICO Room
1st floor

Lunch

13:00 – 14:00 pm 

Parallel session: Environment mapping
Chair: Neal Haddaway

A Topography of Climate Change Research

 

Affiliation:

University of Leeds/ MCC Berlin

Presenter: 

Jan Minx

Abstract:

Assessing the climate change literature has become a challange of “big literature”. In this presentation we use data science applications – namely topic modelling and a variety of scientometric methods – to understand, categorize and map out a landscape of more than 500,000 publications on climate change in the Web of Science and Scopus. We further use this rich dataset to re-assess a variety of biases in assessment reports by the Intergovernmental Panel of Climate Change (IPCC) that have been observed by scholars. We find that numerous claims in the literature do no longer hold when adequately assessed in the context of the entire available science. We conclude on the arising changes in policy implications.

Establishing a responsive evidence synthesis service in the environmental sector in South Africa

 

Affiliation:

Africa Centre for Evidence, University of Johannesburg

Presenter: 

Promise Nduku

Abstract:

Policymakers in the environmental sector require timely access to the most relevant and high-quality evidence in order to support decision-making. Rapid response services (RRS) present one mechanism to provide these environmental policymakers with such access. Since 2018, the Johannesburg Centre for Environmental Evidence has piloted such an RRS with the National Department of Environmental Affairs in South Africa. Modelled on the pioneering RRS in Uganda, our service too is based on rapid evidence synthesis methodology. In addition, our service places an emphasis on co-construction of evidence introducing an element of ‘responsiveness’ over ‘rapidness’. That is, the environmental policymakers are involved in each step of the rapid response and shape its design. This presentation will summarise our learning on the environmental RSS including the initial design and current piloting phase.

Librarian Collaboration on Ceres2030: Sustainable Solutions to End Hunger

 

Affiliation:

Cornell University

Presenter: 

Kate Ghezzi-Kopel

Abstract:

Research shows that librarian involvement in evidence syntheses improves quality. How do we ‘tip the scales’ to encourage researchers to partner with librarians across disciplines? Ceres2030: Sustainable Solutions to End Hunger is a global donor-coordinated effort to achieve Sustainable Development Goal 2. Ceres2030 has selected eight evidence-review topics targeting the impact of small-scale food producer interventions on household food security, rural economic livelihoods, and environmental sustainability. Each of the eight review teams include librarian co-authors with experience conducting multidisciplinary reviews. We will present insights from coordinating this large, multi-national project, which includes stakeholders, researchers and librarians, and will discuss the various roles that librarians and information specialists play in facilitating high-quality evidence synthesis.

Developing a Search Strategy for a Scoping Review of Adaptation to Climate Change

 

Affiliation:

University of Alberta

Presenter: 

Denise Thomson

Abstract:

We present the challenges in creating a search strategy for a scoping review evaluating the effectiveness of climate change (CC) adaptation measures in creating health impacts. Numerous iterations of this strategy were required. Challenges included: developing a working definition and search terms to adequately cover the concept of “health impacts”; creating a filter for the range of relevant study designs; encompassing adaptation initiatives with health co-benefits; capturing relevant studies not explicitly described as related to CC; teasing out health benefits from other social, economic and/or environmental benefits.
Strategies for CC adaptation can have multiple health benefits. Evidence synthesis on the effectiveness of adaptation interventions is a health priority; effective search strategies are a cornerstone of high-quality syntheses so appropriate methods are crucial.

14:00 – 14:15 pm

Moving to new room

14:15 – 15:15 pm

Panel: PECO9

Different methodological approaches to know the impact of three specific interventions in Mexico: PROMAJOVEN, PAJA and PETC

Affiliation:

Consejo Nacional de Evaluación de la Política de Desarrollo Social (CONEVAL)

Chair:

Agustín Escobar Latapí

Presenters:

Gabriela Sánchez
Manuel Triano
Francisco Cabrera

Abstract:

The purpose is to present and discuss the results of three studies carried out by CONEVAL with the objective of knowing the effects that three social programs had in solving the problems for which they were created. The programs analyzed were: Support Program for Basic Education of Young Mothers and Young Pregnant Women (PROMAJOVEN); Care Program for Agricultural Laborers (PAJA) and Full Time Schools Program (PETC). The PETC was analyzed through a quantitative approach (difference-in-differences methodology); while PROMAJOVEN and PAJA were analyzed under a qualitative approach due to the complexity of the interventions, lack of data and the mobility or difficulty to access to the beneficiaries. The qualitative approach implied generating a variety of alternative methodologies, which included fieldwork, ethnographic research, and individual/collective interviews.

15:15 – 15:45 pm
FOYER 1st floor

Coffee served – Networking

15:45 – 16:45 pm 

Parallel session: Development planning
Chair: Stuti Tripathi

A Simple Tool to Identify Poor Households: the Poverty Probability Index

 

Affiliation:

Poverty Probability Index

Presenter: 

Hugo Salas Rodriguez

Abstract:

Billions of dollars in aid are directed towards poor households, but it is extremely difficult for organizations to know if they are successful in reaching them because measuring poverty is expensive, technically challenging and requires several hours of interviewer time per household. This could discourage organizations to effectively identify the poor. We describe a faster, but yet rigorous, alternative to poverty measurement. Using modern machine learning methods, the Poverty Probability Index (PPI) leverages data collected by national representative surveys to create a 10 question, easy-to-use tool that is inexpensive to administer and places a low burden on respondents. The PPI is offered to the international development community as a public good and makes it feasible for organizations to estimate the poverty status of their beneficiaries and use that data in their decision-making.

Workers’ subjective well-being: a systematic review

 

Affiliation:

Universidade Federal de Goiás

Presenter: 

Viviane Castro

Abstract:

Subjective well-being is fundamental in this review and refers to any kind of evaluation made by people about their own lives. This systematic review aims to measure workers’ subjective well-being in different organizations, territories and professional categories. We included studies that evaluated subjective well-being levels and used some instrument developed by Ed. Diener. The databases were: SciELO, BVS, Portal, CAPES, MEDLINE and Embase. The descriptors were “subjective well-being” and “worker”, written in English, Spanish and Portuguese, from the beginning of the databases until March 16, 2018. 8 articles were included. The studies were published between 2012 and 2017. High heterogeneity was observed. Two studies were considered at high risk of bias. Workers’ subjective well-being level has been around the average, with indices ranging from slightly below average to above average.

16:45 – 17:00 pm

Moving to new room

17:00 – 18:00 pm 

Panel: EXT1

Citizen evidence and engagement in influencing policy and practice

Affiliation:

CLEAR Anglophone Africa

Chair:

Ian Goldman

Presenters:

Laila Smith
Mine Pabari
Katie Gibbs

Abstract:

With growing social movements around service delivery, climate change, rights etc., there is an increasing demand for the engagement of citizens and the wider public in policy and decision making. Many governments around the world also recognise the value of these approaches and have incorporated them in their legislative processes.
This panel session will discuss the value and importance public participation in the policy making and implementation. Three cases studies with very different perspectives and experiences will be shared to prompt discussion: i) Advocacy for evidence-based decision-making by a not-for profit organisation, Evidence for Democracy (E4D); ii) Citizen engagement facilitated by a parliamentary body in the development of the Kenyan Wildlife Conservation and Management Act (WCMA); and iii) the role of civil society in enabling evidence use in order to strengthen the sanitation sector in Ghana.

12:00 – 13:00 pm
Puebla Room
1st floor

What Works Lunch
By invitation only

18:00 pm-19:00 pm
NUEVO LEON II Room
3rd floor

PLENARY PANEL

The future of the evidence agenda for What Works in Education

Chair: Patrick Okwen

Presenters:

Mark Schneider, Director of IES Research

From 19:00 pm
CIUDAD DE MEXICO II Ballroom
1st floor

Poster presentations Chair: Annette Brown

HAPPY HOUR EVENT

Copyright 2019 wwgs2019. All rights reserved