Dear Journalists: What to look for in aid programs…

22 Nov

Dear Journalists: I know I’ve been hard on you in the past (and yes, you kinda did deserve it). But I’m turning over a new leaf. For the second time (not than anyone’s counting), I’m going to try to be helpful. Here is the first in a series that I’ll add more posts to over time about how you can get the most of your visit to an aid program or project:

Understand that you cannot evaluate a project, program or organization during one-day visit. I’m going to say it again: relief and development work are complicated. Just as it takes us concerted effort over time to understand a context, so it also takes an outsider (e.g. you) time and effort to understand what is going on in a program in the field. You can certainly gain impressions, and below I’ll share a few tips on what to look for that will help you gain a more informed opinion. But in general I’ll say that to spend one day or even one whole week with an organization’s relief team in the field and then print declarative statements about whether they’re doing well or doing poorly, or whether the overall relief effort is succeeding or flagging –  and whether your statements are positive or negative – is plain ludicrous. Things are very often not what they seem to outsiders. Things that look chaotic might simply be complex. And in the context of a short visit, both success and failure can, to an untrained eye, appear as the other.

It’s fair to ask for copies of evaluation documents, but be aware that many organizations will demure from providing this outright. Ask whether the evaluation was internal or external. Also be aware of where you are in the chronology of a program or disaster response: there may not be an evaluation available.

Ask about the learning. Good aid learns from experience. A good relief manager or program director or media spokesperson should be able to articulate how the current program or relief effort is based on learning gained from previous experience. Don’t be surprised if the learning expressed is incremental – development and relief evolve slowly over time through a series of tweaks and fine-tuning from one program or relief effort to the next. An NGO that can’t articulate specifically what’s been learned in the past that’s being applied now should raise a red flag in your mind, as should one that claims to have it all the way right, now.

Ask whether learnings have been published or if there are plans to publish them. Ask whether the organization has already or has plans to participate in a multi-agency learning event or interagency evaluation (fairly common following large disaster responses).

Ask about the process used to design the program, project, emergency response, etc.: Here you are specifically looking for evidence a couple of things: First, you want to hear that local people were involved in articulating the need as well as the design of the project or relief program. Second, you want to hear that there was, in fact, a process: there was an actual assessment (not just a sort of willy-nilly mix of observation and the odd interview), there was an actual program design exercise that involved the analysis of assessment data.

Larger organizations may have their own in-house organization-specific process(es) and an accompanying set of tools (available online in many cases). They’ll most probably be happy to provide this, but be aware that your media colleagues won’t have the documents at their fingertips – they’ll have to track down a programs person for this. Smaller organizations may not have their own model/tools, but should still be able to talk to you in specific terms about the processes that they follow for assessment, program design, monitoring and evaluations.

Ask about outcomes. Someone should be able to tell you what the expected outcomes of a relief or development program are. Or, in hindsight, what they were. In development programs this is frequently expressed as a percentage of change in something (infant mortality decreasing, literacy rates increasing, etc.), while in relief programs this is typically expressed in what we call outputs – numbers of something (number of transitional shelters put up, the number of families with access to clean water, etc.).

It’s fair to ask why proposed outcomes or outputs may have been underachieved (if your conversation is after-the-fact), but don’t assume that underachievement equals failure or incompetence. Hear the explanation. Prices change, overall context evolves, and aid worker’s understanding of situations deepens – all of which potentially affect the outcomes of an aid project or program.

It is becoming increasingly common for organizations to make their evaluation documents available externally. It’s fair to ask. If they won’t share a full evaluation document, ask for an executive summary.

Use logic. Understand Pythagorean logic. Understand that correlation does not equal causation. Know the difference between issues that humanitarian aid providers can fairly be expected to address or be held responsible for, and those that they cannot (e.g. cholera in Haiti = not the fault of NGOs). Don’t confuse anecdotes with data, or data with evidence, or evidence with proof. Understand the difference between simple and simplistic. Beware of magik bullets and one-size-fits all solutions: Anyone who’s selling their widget or approach as the thing that will solve the world’s problems deserves closer questioning around learning, and process. Watch out for solutions in search of problems.

Ask the “why this, not that?” question. It’s fair for an NGO to define it’s focus or niche in a particular sector or place. I tend to be skeptical, though, of an organization that only does one thing (only one tool in it’s tool box). There should be a logical explanation for why they’ve chosen to do what they’re doing in the place where they’re doing it.

Understand ambiguity. With few exceptions, neither successes nor failures are total. Even the most stellar, most widely acclaimed aid organization or approach or program has areas that didn’t work well, aspects for improvement. See also “ask about learnings”: Any organization or project that presents a 100% rosy picture probably need deeper scrutiny. On the other hand, despite bold headlines, few aid programs are unmitigated failures. Thing are never cut-and-dried.

Ask for descriptions of the context. Where programs seemed very successful, it’s fair to ask about challengs and learnings. Where programs seemed to have failed or been extremely marginal, it’s fair to ask why. Also ask about learnings. Also ask about areas where there may have been unexpected successes (although these might not have substantially affected the overall outcome).

Understand that things are almost never the way they seem at first blush. Recipients of aid, local authorities, and local partners can all have many reasons for telling you that they loved project X when in fact they hated it, or vice versa, none of which are related to the reality of what project X is or does or did. Understand that there is a ubiquitous dynamic called “screw the outsider”, and understand – further – that you may be variously treated an outsider or an insider due to factors over which you have precisely zero influence. In very simple language, just because a local person goes on about how much they love project Z, isn’t proof that project Z works or is “good” or even that that person really loves it.

Frustrating? Yes, welcome to our world. And also, see the very first point: it takes time to triangulate information; it takes experience and expertise and also time to be able to figure out what’s going on.

Any questions?

13 Responses to “Dear Journalists: What to look for in aid programs…”

  1. ansel 23 November, 2010 at 12:25 am #

    Dear aid groups,

    Do not invite us on one-day tours of your programs and expect them to be useful to us in any way. Do not bring out a single smiling hand-picked group of locals, who, of course, talk in gushing terms about everything you’ve done for them. You all do this constantly, and unfortunately, there are plenty of lazy journalists all too happy to be spoon-fed your bullshit.

    We need to be able to come out to where you’re working unannounced and talk with you – your people in the field. Do not act like a corporation. You are not one. You’re not accountable to shareholders, you’re supposed to be accountable to the communities where you work and donors, who rely on journalists for information. Do not as policy refer us to your PIO spinmaster and refuse to talk to speak, on- or off-record. It’s highly suspicious.

    Let us talk to the supposed beneficiaries of your program, as well as local hires, without you hovering nearby. Go far far away until you can’t identify who’s talking with us.

    Do not send out press releases over and over simply listing off the sheer numbers of stuff you’ve distributed or have stocked in warehouses as if it indicates how much you’ve accomplished. Quality of life is not measured by those (nearly impossible to verify independently) numbers. One often has nothing to do with the other. More fodder for lazy journalists and crap that the serious ones have to swat away in search of something closer to truth.

    The only way you will get us to stop pestering about how much money you’ve spent versus how much you raised to be completely open with your budget and how your projects/plans are funded. Do disclose your salaries and vacation packages.

    Do not try to get journalists kicked off mailing lists that contain meeting notes from clusters.

    Do not always blame your problems on the government, while never speaking ill of any other NGO or the UN. Again, highly suspicious.

    Do not claim, every time, that you are “rushing” to provide aid, as if in a never-ending cycle of emergency, when you know that the pace of operations has not really changed. You are not necessarily “urgently” or “swiftly” doing anything if every time you do something it is “urgent” or “swift.” It’s all the same. You are filing the same paperwork, using the same staff, with the same supplies, only there’s a new problem to deal with…

    It’s late, I’ll stop there… Any answers?

    • Verity Landon 24 November, 2010 at 12:44 am #

      Thanks for the great comment. I am sorry but I found this post incredibly pompous and self important.

      • J. 24 November, 2010 at 12:22 pm #

        Verity – I normally don’t feed the trolls…

        But let me get this straight: I’m the only aid blogger that I know of who even occasionally writes directly to journalists, who will address their questions (no matter how unelightened), who doesn’t summarily blow off the self-styled 22-year old “Independent Multi-media” freelance startups, and who actually tries to provide discussion of value instead of the usual “it’s complicated and you’re too dumb to get it”… and that makes me pompous?

      • ansel 24 November, 2010 at 2:22 pm #

        Um J? “Self-styled 22-year-old Independant Mult-media freelance startups”? You referring to me? Am I misreading or are you dripping with condescension?

        I’m a 22-year-old freelance journalist doing writing and radio, along with some video work. And why does *who* is making the comment even matter… judge it for what it says.

        You misspelled independent.

        That said, I don’t agree with Verity. I think it’s a post a lot of journalists should read.

      • J. 24 November, 2010 at 2:41 pm #

        Not condescension, although I can see why it might feel that way.

        The reality is that many large INGOs will not make time for less than CNN, New York Times, Miami Herald, or CSM.

        I did judge your comment for what it said. Which is why I wrote an entire post in response. You’re the only journalist thus far to get that on this blog. Take it for what it’s worth.

      • Verity Landon 24 November, 2010 at 6:56 pm #

        Wow, you really are pompous.

    • Jon 24 November, 2010 at 5:32 pm #

      Answers? here’s a hint, ansel: Put away the recorder you stick in everyone’s face; stop trying to disprove everything anyone says that doesn’t fit in your worldview; stop attacking and start listening; and go to a newsroom where you can learn what reporting is about. I’ve seen you disregard what sources have said in favor of reporting the exact opposite. I’ve heard you trash NGOs. You don’t have the ability to approach an issue “independently,” so stop pretending that you are an “independent journalist.” You have a point of view, an agenda, so stop pretending you’re a journalist and admit that you’re promoting your own agenda … for personal gain.

  2. Tim Ogden 23 November, 2010 at 9:21 am #

    I’m nominating this for comment of the year…

  3. Saundra 23 November, 2010 at 9:49 am #

    I’ll second that nomination. He brings up some excellent points about how nonprofits attempt to control the message making it very difficult to do real reporting. This is also a serious issue with any attempt to rate, evaluate, or hold nonprofits accountable for their work.

Trackbacks/Pingbacks

  1. Dear Journalist: What to look for (redux) « Tales From the Hood - 23 November, 2010

    […] An addendum to the previous post. […]

  2. Aid Workers vs. Journalists? « Brett Keller - 26 November, 2010

    […] going on at Tales From the Hood: First, J (the anonymous aid worker blogger behind Tales), wrote “Dear Journalists: What to look for in aid programs,” which includes suggestions like “Understand that you cannot evaluate a project, program or […]

  3. Interesting articles and posts – Late November 2010 | Good Intentions Are Not Enough Interesting articles and posts – Late November 2010 | An honest conversation about the impact of aid - 26 November, 2010

    […] Dear Journalist – Tales from the Hood – Advice to donors by J. as well as a counter advice to aid organizations by a journalist in the comments. […]

  4. Media Matters | Good Intentions Are Not Enough Media Matters | An honest conversation about the impact of aid - 2 December, 2010

    […] Dear Journalists – Tales from the Hood […]

Pearls of wisdom

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: