Blog

Stay up to date with Giving What We Can's blog. Featuring the latest news about our community, how to donate effectively, charity evaluation, community events and more!

Giving What We Can's blog, featuring the latest news from our community.

Donate

Pledge

footerNavigation

About us

Contact us

Our team

Our values

Our members

Our events

Transparency

Our mistakes

Our impact

Donate to Giving What We Can

Careers

About

Our best charities

Take a giving pledge

Get involved

Make a bequest

Videos, books, podcasts

FAQs

Sitemap

Learn more

Press kit

Media coverage

Case studies: People who pledge to give

Media

Footer Navigation

DONATE

topNavigation

Best charities

High-impact causes

Why give to funds

How much to donate

Why donate publicly

Recommendations

Our research & approach

Comparing charities

Evaluating evaluators

Our approach

Giving guides

Videos, books, podcasts and essays

Hesitations about charitable giving

How rich am I?

Can money buy happiness?

Resources

Research

Team

The 🔸10% Pledge

🔹Trial Pledge

Company Pledge

Further Pledge

Bequests

Why pledge

Pledges

Newsletter

Podcast

Videos

Connect with our community

Sharing our ideas

Events

Giving games

Charity elections

Pledge drive

Fundraisers

Community

Donate now

Manage my donations

Tax deductibility

Australian page

My pledge

My donations

My income

My impact

Top Navigation

Andreas Mogensen

“Without this kind of work, we could make fantastically wrong predictions”: An Interview with Eva Vivalt

If we find that a deworming programme in one region improves school performance, how much have we learned? Should we be confident that a programme carried out by a different organization in another country will have similar effects? Will the programme be equally effective if it’s handed over to the government and scaled up?

 is a Post-Doctoral Research Fellow at NYU’s Development Research Institute whose work addresses questions like these. She founded 

 to synthesize data from different impact evaluations and improve the assessment of development interventions. Her paper 

“How much can we generalize from impact evaluations?”

described as a must-read for development economists

. I recently got in touch with Eva to learn more about her research.

Could you tell us about your work as a development economist? How does your academic research relate to your work with AidGrade?

Right now I'm focusing on leveraging AidGrade's data for research because there is a lot more that can be said with it. But I am interested in other topics in development as well and have several other studies in progress. I'm lucky because my academic work only allows me to set up the structure for and manage AidGrade on the side, and the staff are so great that they can take it from there.

Your paper assesses to what extent we can generalize from evaluations of programs implemented by one organization in a particular location to the efficacy of similar programs implemented by other organizations elsewhere. Why is this issue important? Does it matter for individual donors?

How much one can extrapolate across settings is critical to understanding what we can take away from any one study. Without this kind of work, we could make fantastically wrong predictions. For example, a similar issue came up in medicine, where many studies were traditionally done on men but there has been increasing acknowledgement that effects of some drugs might be different in women or children. The issue is likely to be even more important in development economics, because there are so many more things that could interact with the intervention and determine its effects. So one reason this work is important is because it teases apart some of the factors associated with the effects of a program and helps to improve our ability to predict.

Apart from the prediction issue, knowing the distribution of effects is also important. If we were equally unsure of the effects of all development programs, perhaps we could get away with only looking at the average effect for a particular intervention, but if we know some have spanned a broader range than others that could make a difference as to which we want to support. A former professor and inspiration, Adrian Wood, liked to say that first we should just do no harm; one can imagine perhaps we should pay more attention to the lower end of the range of possible effect sizes because if you do end up doing an intervention with poor results you might sow distrust, and research in psychology shows people dislike losses more than they like gains. Alternatively, if there are poverty traps, perhaps we should weight the large, positive effect sizes more highly. Either way, it is important to consider the full range of possible outcomes.

Finally, and less relevant to individual donors but more relevant for researchers, this kind of work can help tell us where we should allocate more attention -- where are the big unknowns and unexplained variation.

How do you assess the generalizability of impact evaluations?

AidGrade gathered data from about 600 impact evaluations of 20 different kinds of development programs. These were academic studies, and you can think of our data as essentially comprising the results tables along with characteristics of each paper (e.g. whether it was a randomized controlled trial, whether it was randomized by cluster, characteristics of the sample being studied, etc.). We also coded what kind of outcome the results captured (e.g. attendance rates in percentage points, height in cm, etc.).

For the main results, I run leave-one-out meta-analyses of all the papers within each intervention-outcome combination. For example, there might be six papers estimating the effect of school meals (intervention) on attendance rates (outcome). I take one of those estimates out, generate the meta-analysis result from the other five, and see how well that meta-analysis result predicts the one left out, cycling through all possible permutations. It's a bit more complicated than that, but that's the quick summary.

You find that a positive, significant result in one study is a poor predictor of finding a similar result if the same intervention is studied again: there’s only a 32% chance that the next results with be positive and significant. How worried should we be in light of this finding?

It does suggest some caution. One should also consider the sample size of the studies. Studies are very frequently underpowered, meaning that there is just too small a sample to be likely to observe a significant effect even if the program really had a positive effect. In general, especially with regards to how we commonly talk about programs or see them discussed in the press, I think we don't know quite as much as we think we know. Academics are better at being careful about this.

You find that government-implemented programs generally fare worse than programs implemented by academics and NGOs. Why do you think we find this pattern of results? What lessons can we take away?

There are a few possible explanations. The first thing that comes to mind for most people is that government-implemented and NGO/academic-implemented programs are qualitatively different – the government-implemented programs, in particular, tend to be much larger. There could be "equilibrium effects" at a large scale that reduce the observed impact. For example, perhaps microfinance programs help some business owners but at the expense of others -- if one looked at a small-scale program, it would appear to help, but it might not when scaled up. Alternatively, it could simply be harder to implement a program well at scale. Another possibility is that NGO/academic-implemented programs are simply better-managed or devote more effort or other resources to each beneficiary.

In my paper, I show that while it is true that larger programs tend to fare worse, government-implemented programs still perform worse even controlling for sample size, so size is not the full story and it appears to be a combination of factors. More work is needed to distinguish between the remaining possibilities.

Based on your findings, are there any changes you think we should make in how we go about trying to identify the best charities to support with our donations? For example, should we significantly increase our confidence in an organization like GiveDirectly, where we have an evaluation that looks directly at their program?

I think that is a correct inference to draw. There is definitely still reason to sometimes support unstudied charities, but the full range of possible results and our uncertainty about them should enter into that calculation. We should also be cautious as to whether the evaluation of a charity was cherry-picked in any way.

What steps can be taken to improve the generalizability of impact evaluations? Are development economists following these steps or is generalizability a neglected concern?

Great question. First, there are things that people could do to help measure the local generalizability of their results -- for example, include subgroup analyses in their pre-analysis plans. This means that they commit to examining differences in results for different subsets of their own sample. While this would only give us within-sample variation, I found that variance in results within a paper is associated with variance in results across the papers that look at the same intervention-outcome, and it might be relevant locally (such as if the program were to be expanded), so it would seem a step in the right direction. There should also be better coordination across people working on different papers so that they use more of the same outcome measures. Paul Gertler and co-authors, among others, have been setting up impact evaluations that ask the same questions in many different settings. This is very valuable work.

More generally, better integration of theory in development work would help. People like to say that impact evaluations are testing some causal chain, but if you delve into the papers, how they think the intervention actually works is often left unstated. It's a knotty problem, but the holy grail would be to uncover some stable parameters explaining at least some of the variation in an intervention's effects. Failing that, we would need many more iterations of experiments.

Michelle Hutchinson

Our recommended charities for the year

Giving What We Can is maintaining its list of recommended charities this year: we continue to recommend Against Malaria Foundation, Deworm the World, Project Healthy Children, and Schistosomiasis Control Initiative. We believe these organizations remain the 

 to which to give and we hope you’ll support them with your donations this holiday giving season!

Last year, GiveWell raised concerns about AMF's room for additional funding. We're pleased by the progress AMF have made in distributing funds to support net distributions. Beginning the year with approximately $12.5M, AMF has committed $10.3M to fund distributions in Malawi and the DRC, whilst raising around $2.65 in donations. They are currently engaged in a number of negotiations, though current funding levels would allow AMF to take on a maximum of three out of the distributions currently in discussion. AMF are also planning to partner with the Liverpool School of Tropic medicine to assist in studies designed to assess the extent of insecticide resistance in areas such as the DRC. We continue to recommend AMF as a top charity and we’re happy to see that GiveWell has placed AMF back on its list of top charities in the most recent update of their recommendations.

GiveWell’s recommendations this year also includes a second tier of stand-out charities: Development Media International, Living Goods, GAIN, and ICCIDD. We’re looking forward to learning more about these promising organizations in future. DMI has previously been on our radar: they are an exciting organization currently trialling a 5-year mass-media campaign designed to reduce childhood mortality and morbidity in Burkino Faso, with final results expected in the second half of 2015. Living Goods provides health counselling and sells health-products door-to-door: they are also engaged in an RCT of their program, with published results expected soon. Like Project Healthy Children, GAIN and ICCIDD work on micronutrient fortification: these organizations are focused on reducing iodine deficiency via salt iodization.

In terms of membership growth, we’ve had a fantastic year. We had 394 members at the beginning of 2014. There are now 690 Giving What We Can members in all! That means 296 people have signed the Pledge this year, compared to just 119 in 2013. We’re very happy that so many more people have been inspired to take the Pledge and we’re thankful to all our members, new and old. We can’t wait for the new year!

Rob Wiblin

80,000 Hours is seeking a Head of Research

 is our sister organisation - we are cross posting this 

As Head of Research, you will become part of our founding team to lead our research efforts into how to select the most high-impact careers. Our research is what drives the advice we provide via our website and career coaching.

Specifically, you become responsible for:

Deciding what questions to investigate and personally coming up with answers to these questions, or overseeing the delegation of this research to others.

Helping to translate these findings into practical products that help people make better decisions - books, web applications, workshops and others.

Fitting the research into the overall strategy for the organisation.

We are a small team, so you'll also have the opportunity to be involved in recruitment, fundraising, publishing our findings in the media and whatever else most interests you.

This role combines great potential for impact with great potential for growth and learning. We believe 80,000 Hours has the potential to transform careers advice, and you would be playing a central role in our strategy. As one of the key team members of a rapidly growing small organisation, you'll have a huge amount of responsibility, autonomy and room to grow.

If you work for 80,000 Hours, you’ll be part of a focused team of intelligent, enthusiastic, driven colleagues who want to change the world, located in our offices within the beautiful University of Oxford, just an hour from London .

What does the research involve in more detail?

Check out our existing pages for examples. Some big questions we've focused on so far are:

what factors should you look for when comparing two careers? e.g. how to prioritise building skills against your immediate impact?

what steps should you take to make your next choice?

which career paths are most promising in general?

We set about answering these with a mixture of expert interviews, literature searches and quantitative estimates. All our research findings are applied in our one-on-one coaching to test their usefulness and get further feedback.

We're looking for someone who is highly enthusiastic about 80,000 Hours and effective altruism, who could see themselves becoming obsessed with the question 'in which career can you make the most difference?'.

You'll need a combination of research skills honed on messy real-world problems, and exceptionally clear written communication.

Some examples of experience that would be especially useful include: journalism, graduate study (especially in social science), policy research, and philosophy study - though we're interested in anyone who can demonstrate the skills.

The ideal candidate could also grow into a major leadership role within the organisation, so evidence you can run your own thing and lead people is a major plus.

Candidates will be selected on the following traits, and should aim to demonstrate them in their application:

Applications close Midnight GMT, Friday 12th December.

See other job openings at the Centre for Effective Altruism.

Francesca Day

Christianity and Giving

I joined Giving What We Can because I am a Christian.

 The Christian call to charity can sound almost clichéd at times, but the fundamental message is radical. Jesus tells us “love your neighbour as yourself” (Mark 12:31). This sounds obvious and easy – but Jesus makes it clear in the Parable of the Good Samaritan that our neighbour is not just people who live near us or people who we like, but every human being in the world. We are called to value each and every human being in the world as much as we value ourselves.

Applying this principle to charitable giving, we see that putting the odd pound in a collection bucket just isn’t going to cut it. If we really love those living with poverty, illness or oppression as we love ourselves, we will do everything we can to help them. We will make real sacrifices to help them.

Charitable giving is a focus of Jesus’ teachings. He tells us “Sell your possessions and give to the needy. Provide yourselves with money bags that do not grow old, with a treasure in the heavens that does not fail, where no thief approaches and no moth destroys”. It is clear that Jesus had some fairly serious charitable giving in mind.

By sacrificing worldly goods to help those whose need is so much greater than our own, we build treasure in heaven, and we make the Earth a better place. Jesus is talking about big sacrifices here – He’s saying we should go as far as to sell our things to raise money to give to the poor! When we consider that we’re called to value them as much as we value ourselves, this idea doesn’t seem so silly.

Do I live up to these ideals? Absolutely not. I strive to model my life on Jesus’ teachings, and taking the Giving What We Can pledge is a step towards that goal. I have pledged to give 10% of my income to whichever charity I think most effectively helps those living in extreme poverty. 10% feels like a lot, but it shouldn’t. It feels like a lot because I haven’t succeeded in truly loving my neighbour as myself. Taking a public pledge to give helps ensure that my actions will live up to my ideals.

Why the focus on effectiveness? Now I’ve pledged to give this money, I have to choose where to give it. How should I choose? I should choose in a way that 

values every single human being as my neighbour

. Not just people who live near me, not just people who speak my language, and not just people suffering in ways that I might one day suffer. I should value the child in Ethopia dying of a disease I haven’t heard of as much as the lady down the road battling cancer.

With this in mind, I’ve decided I want to 

give to a charity that can help as many people as possible with my donation

. How can I work out which charity that is? It’s hard – there are several organisations researching this questions, but no definitive answers. Jesus does not give us lists of rules to follow – He tells us to love our God and love our neighbour and we have to figure the rest out ourselves. We have to figure out how and where we should give to best love our neighbour.

Jesus tells us “be as shrewd as serpents and as innocent as doves” (Mark 10:16). We are called to give wisely and to give generously. We are called to love our neighbours by giving freely of all the gifts God has given us – our worldly possessions, our intelligence and our humility. For me, being a member of Giving What We Can is part of this commitment.

Charlie Satow

The data revolution: what's behind the barricades?

A bed net is distributed in the forest and no one is there to track its delivery and use. Does it contribute to reducing the incidence of malaria?

This question gets to the heart of a key issue in development policy and one that is extremely important to Giving What We Can: data. And with the much-heralded ‘data revolution’ demanded by the UN High-Level Panel on the post-2015 development agenda, it seems to be rising up the list of priorities for governments and multilateral institutions. In this post I want to take a look at this ‘data revolution’ and why data are important for tackling global poverty.

But first, the ‘so what?’ question. Why is data important? The first reason comes from an efficiency argument central to Giving What We Can’s mission: you need to gather data 

to judge whether an intervention is having the effects it was supposed to have

, and thus whether it is a success. What we really mean when we talk of investing in data collection is investing in effective school systems, in well-run hospitals or in successful nutrition programmes. That much is familiar. But this can also 

contribute to the democratic political process

, by equipping people with the information they require to hold their leaders accountable.

Claire Melamed, of the Overseas Development Institute, makes a further, ‘equity’ argument for gathering data[1]: 

we tend to have least information about the people who are most marginalised

 (e.g. slum dwellers[2], women, the disabled), which leads them to receive less attention in terms of policy responses**.If we had more information about these groups we would provide better services for them**. These are three compelling reasons for caring about data or evidence.

What are the current problems associated with data collection in developing countries?

The problem is that the current state of data collection in many developing countries is extremely poor - so much so that Morten Jerven has written an 

 about the poverty of GDP statistics in Africa. And here we are not talking about minor mistakes: for example, when Nigeria’s GDP was re-based, the new estimate showed a 90% increase[3]. And to give another example, the High-Level Panel’s briefing document on the data revolution states that every year around 50 million births go unregistered.[4]

Given this situation, some sort of ‘revolution’ seems appropriate. But the term ‘data revolution,’ officially comprising ‘two main objectives’ and ‘four components’[4], is a slippery one. As Claire Melamed discusses on a 

 on the subject, it holds different meanings for different people.

bringing ‘big data’ into the mainstream of measurement and policy

. This could mean using mobile phone data to track refugee movements [5], or satellite imagery of trees to measure rates of deforestation [6].

strengthening capacity in traditional government statistics agencies

: training more statisticians, getting computer systems to work, etc. Perhaps, then, the ‘data revolution’ is simply an aspirational term, 

pushing policymakers to prioritise “more data, better data, disaggregated data, frequent data,”

 as Amanda Glassman of the Center for Global Development expresses it in the same podcast. It has certainly succeeded in drawing political attention.

For Glassman, one of the key issues is addressing the fundamental problem of incentives underlying the poor data. Administrative, government-collected data remain low-quality in many countries partly because of donor actions: they have demanded accurate household surveys to be able to evaluate their programmes, and to track indicators of their interest (e.g. the Millennium Development Goals). This has produced a system of little coherence and with certain types of information prioritised over others, for example at the national level at the expense of more data disaggregated into population groups.

Faced with this situation, one concrete proposal for 

donors is to engage in funding compacts with recipient-country governments to achieve certain standards on data

, following the payment by results strategy discussed in an 

, and with civil society oversight [7]. This could perhaps be nested within what the High-Level Panel calls a 

‘Global Partnership for Development Data’

, which would also help set common global standards for data reporting, reducing fragmentation and double-collecting.

Visible, user-friendly and practical data

What else can be done here and now to improve the data we collect? Melamed identifies some such steps, for example 

, particularly by publicly-funded bodies such as the UN, and also 

 to create a baseline for 2015 by which we can judge progress on the Sustainable Development Goals, the successor to the Millennium Development Goals. These are both linked to the further point that the final packaging of data must be 

 i.e. visible, user-friendly and practical [8]. The UK Department for International Development leads the way on this matter with its 

Whichever particular proposal you favour - making better use of ‘big data’ for development, forming ‘data compacts’ to improve statistics agencies in developing countries, standardising the way data are reported, or any other initiative - the key point is to take advantage of the current political moment to create a list of specific actions which can be taken forward even after attention has moved on from the data revolution (which it will), balancing the need to strengthen existing systems with the potential advantage to be gained from tapping into new and exciting data projects.

Giving What We Can puts gathering good data on an intervention at the heart of its philosophy. The ‘data revolution,’ if successfully translated into a series of concrete measures and integrated into the post-2015 development settlement, can reap the benefits of this approach on a global scale.

The Guardian: The Data Revolution is Coming

Post2015: We need a 'data revolution' for the global poor

Reuters: Nigeria surpasses South Africa as continent's biggest economy

CGDev: Delivering on the Data Revolution on Sub-Saharan Africa

SciDev: How to Engineer a Data-for-Development Revolution

Sebastian Roing

A Brief Update on Ebola - Are There Any Effective Interventions in Sight?

The outbreak of Ebola in West Africa is the only outbreak of Ebola to this day that has taken place in a densely populated, urban environment. There is no vaccine against the horrible virus and tragically few medical options for those infected. The most effective interventions at the time of writing seem to be information campaigns and isolated treatment centres. It remains impossible to say whether these interventions are as effective as other 

charities recommended by Giving What We Can

. Ebola may also be difficult to compare with certain other issues due to its acute and plague-like characteristics.

Ebola is a virus which is transmitted to humans from wild animals and spreads human-to-human. Monkeys, fruit bats and consumption of raw meat are a few documented sources of transmission to humans. Human-to-human transmission occurs through bodily fluids. Contact between broken skin or mucous membranes (e.g. in the nose and mouth) with an infected fluid allows the virus to enter the body and be transmitted. The 

 and has varied from 25% to 90% in past outbreaks.

Past outbreaks have been limited to rural settings, but the most recent outbreak in West Africa (of a virus subtype called Zaire[1]) has affected both urban and rural areas. Since the disease can spread rapidly and has a high fatality rate, it puts a huge strain on the healthcare systems of affected countries.

Blog

“Without this kind of work, we could make fantastically wrong predictions”: An Interview with Eva Vivalt

Our recommended charities for the year

80,000 Hours is seeking a Head of Research

Christianity and Giving

The data revolution: what's behind the barricades?

A Brief Update on Ebola - Are There Any Effective Interventions in Sight?

The Cost of Violence II: What can we do?

Problems are in the eye of the beholder

The Cost of Violence I: What should our priorities be?

The Centre for Effective Altruism is seeking a Chief Operating Officer