This is also interesting in how it relates to the study of airplane crashes and the strong, relatively immediate feedback that these provide. It seems that auto safety was much more slow to catch on, and required odd forces to bring the topic to the forefront.
This also has an interesting interaction with a small startup essentially searching for the important metric for its survival.
For profits have the benefit of having profit/positive cash flow as a guide that not for profits do not. These are probably much better metrics for small startups given that success in any stated altruistic endeavours are ultimately going to show up in these two metrics anyway (and must in order for a startup to be sustainable).
I take his essay to mean that, if you want results, measure results and hold them to it. Chess experts are experts because they are measured by their results. Political pundits are hit hit or miss because they are not held accountable for their results. Non-profits should be measured by their results, not their process, PR, or grants. Makes sense to me, but quantifiable results are often hard to get, witness the discussions here about how to to tell goo programmers from bad.
Having worked extensively with not for profits, the biggest problem is the lack of accountability - this has as much to do with governance as metrics. Managers often have a say in the appointment of board members whereas at least power ultimately rests in shareholders in for profits. Then there's the problem of changing missions - take the number of family foundations that would have their original benefactors appalled (e.g. Ford, Rockefeller, etc.)
And then another problem with metrics is that there has to be a cost benefit at some point. Obviously some are better than others, but if the cost to measure is as much to achieve, is it worth it? Though this may say as much about the mission as it does about the metric (microfinance is an area where this debate is raging).
Google "commercialization" and "microfinance" - a debate that's directly related to "social metrics"/"social impact" - you should find a number of forums (consolidated at nabble; but there are also a number of articles that you can find at USAID funded http://www.microfinancegateway.org/)
I've been involved with a couple of music/culture related 501-c3 non-profits that had very quantifiable results: class enrollments, ticket sales, sales of music CDs, and awards won by participants in international competitions.
I would say that both of them do extremely well in the non-quantifiable quality department as well.
How do you measure the "results" of a non-profit, when most of them work in non-quantifiable fields?
If McDonalds can work out how to quantify feeding hungry people then any charity working in Africa should be able to quantify feeding hungry people. If an insurance company can tell me the average lifespan of their clients to six decimal places and what exactly the risk factors are among that pool, then health-focused charities should be able to do it as well. etc, etc
(I have some familiarity with the literature and know the excuses for why charities operating in Africa with mobile phones and computers can't be expected to have as good a handle on their numbers as a British bank circa 1780. Yeah yeah, it must be difficult when you rely on globally distributed workers, many of whom are not literate. Oh wait, so does McDonalds.)
> This isn't to say that we should have companies replace nonprofits
Yes we should.
Charities I donate to are selling me a product just as much as companies I buy from. That the product is a better world--by some measure that seems reasonable to me, rather than a simple benefit for myself, matters not a whit to my decision making process.
I don't give without a very concrete understanding of what the money will be used for, any more than I buy stock without a concrete understanding of what I'm buying and for how much.
By all means, sell me peace, charity, and goodwill! Perhaps that will motivate you to learn how to produce it, and to convince me that you have done so.
It's worth noting that "non-profit" doesn't always translate to charity. Microfinance, for instance, exists as a whole spectrum of quasi-for-profit operations. And on the strictly non-profit side you have things like ROSCAs, one of the foundations of microfinance, which are micro-level, semi-formal, non-charitable "non-profit" groups that resolve a number of different financial issues for participants, similar to tiny credit unions, and are neither charitable nor reasonably mappable to "companies."
The whole impetus behind the growth of the microfinance movement is that traditional banking simply can't serve these groups. There are also numerous non-obvious major core issues involved, such as how gender roles have a huge impact on microfinance or why group lending is so prominent.
In short, the reason it's not simply handled by companies is because it's just not that simple.
Oh, indeed, I'm well aware that there are for-profit philanthrophic ventures and questionably profitable non-profit groups. I don't mean to suggest that it is simple. Indeed, groups that adopt a hybrid, hard-to-classify approach often strike me as the healthiest. I am very fond of certain microloan groups or folks like TechnoServe.
But I do advocate approaching philanthropy as a consumer, however the group you're working with is legally classified. Be honest with yourself about what you're buying -- even if what you're buying is a warm fuzzy feeling. Perhaps especially in that case.
For-profit companies selling you goods and services often spend enormous amounts on marketing, advertising, and branding in order to convince you to part with your hard earned money. This doesn't mean their product is any better than their competitors though. It just means they've spun you into believing it is.
While I think it's a good idea to research your charity or donation to a non-profit, I definitely do not think they should operate like a for-profit business.
I think this is a great article, and there are some really pointed areas of improvement for non-profits. However, let's not turn to the free market gods for a solution please.
I think it really depends on the specific area/charitable cause - but where it comes to economic development, markets are a far better solution than charities and aid which have often done far more harm than good (e.g. food aid). In general, the potential for inefficiency and even fraud is arguably even higher given the lack of transparency and often weak governance).
Anyone who donates money to a charity should at least spend as much time researching what they're "selling" as they would a comparable product/service. The tax returns are available but some of these firms spend as much as 80% of their funds raised in administration and overhead - which is often far worse than most for profits. Yes, not for profits also "often spend enormous amounts on marketing, advertising, and branding in order to convince you to part with your hard earned money".
Fortunately at least anyone can look up any charity's IRS Form 990 that lists such things as salaries (which are sometimes as much if not higher than private market salaries). I usually start here: http://www.guidestar.org/ (using bugmenot for login/password)
Marketing, advertising, and branding is a MUCH larger part of what non-profits sell than it is of what commercial operations with an actual product sell.
What does this even mean? Right now nonprofits do things in exchange for money from wealthy people. How would "running them like a business" be any different?
"This isn’t to say that we should have companies replace non-profits..."
Actually, that is not necessarily a good thing to just sidestep. A lot of things that non profits work on might work better if they were worked on by companies. A lot of things that companies work on might have to be filled by non profits if companies weren't doing it.
Think of second hand stores. They are often run not-for-profit in order to provide cheap stuff to poor people. These compete directly with commercial low cost stores and other second hand stores. If these beat the non profits, there is no societal loss. One of the worlds most celebrated not for profits, the Grameen Bank provides small loans to poor people. It seems very possible that this NGO (and its many imitators) will be put out of business by commercial banks.
The UN has started a free online University to help the world's poor. I would take an either-or bet on for profit variants making a lot of progress on this issue commercially.
A great way of making non profits more like chess masters is making non profits into businesses. You don't always have to. But if it is feasible, its worht considering.
The difference between a well-run non-profit and a well-run business: the non-profit puts 100% of profits back into its operations, while the business may re-invest the profit or distribute it to its owners.
Most states already require non-profits to operate like businesses; indeed, non-profits are expected to operate better than businesses -- most states require non-profits to maintain levels of capitalization that require devilish efficiency and reuse.
A bigger difference is that the donors aren't the customers. A charity has positive cash flow so long as it pleases its donors, who don't necessarily know what's really happening with the people who receive the charity's services.
I don't think you understand quite how the donor-charity relationship works. The donors are the customers. The charity earns money by pleasing the donors. The venue for pleasing the donors is by performing charitable acts.
If a donor is pissed, it will force the charity to conform to his/her demands. The recipients of the charity's services are usually an afterthought, b/c they are far more easily replaced than any donor.
"Grameen Bank provides small loans to poor people. It seems very possible that this NGO (and its many imitators) will be put out of business by commercial banks."
Traditional, for-profit banking simply can't, which is the entire point. Even when microfinance institutions lean further toward for-profit, they end up more and more like traditional banks making traditional big loans.
And that's the way it actually moves, with microfinance institutions edging toward becoming more commercialized, and when they drift too far away from poorer clients, new micro institutions will likely come in to take their place. In other words, the shift naturally moves in the opposite direction.
Indeed, it's former NGO that aggressively went for-profit. That's a good article and sums up some of the major debates around latin american microfinance.
The problem with for-profit microfinance is that, as a general rule, financial performance increases with loan size, and that's why commercialization is a cause for concern, along with a lot of the other baggage that comes with commercialization like pressure to increase profits and employee incentive programs that negatively impact service toward small clients.
Compartamos accounts for the problems of the small loan market (notably the inability to determine which borrows are good or not) the same way moneylenders needed to before them, by charging very high interest rates. But this ends up screening out lower end borrowers, hence the discussion of "mission drift."
Commercial banks have long dabbled in elements of microfinance, but there are fairly concrete obstacles, mostly operational and information-related issues, that make it tough to be profitable, and even tougher to approach it as strictly for-profit while continuing to be actual microfinance.
No, it absolutely does not suggest that at all. For example, ROSCAs are inherently non-profit, they don't even hold money at all, and yet provide needed services to their users in ways that for-profits simply can't.
Homeboys Industries, in Los Angeles, which has been very successful in transforming ex-gangbangers and low-income youth into productive members of society.
Legal Aid (anywhere) assists asylum-seekers, wrongfully evicted tenants, and indigents needing legal services.
Microfinancing (non-US operations) in India, China, and the Southeast have done so well that the pioneer of microfinancing received a Nobel.
Nearly any education reform organization that gets parents talking to other parents through channels not controlled by schools. Often parents can do a lot of self-help without public resources as soon as they can compare notes about what works to help children who need help. Setting up an email list and moderating it can be just about free, and often has great effect.
After edit: To relate what I say here to what was said in the submitted article, an organization that gets all of its members talking to one another freely through one-to-many, many-to-one, and one-to-one channels available to all members gets a LOT of feedback that other organizations don't get, and learns what works and what doesn't work.
They got caught out because they were blatantly astroturfing for themselves and slagging 'competing' charities all over the web. Then it turns out that even by their own generous accounting, more than half of their funds goes directly to their own overhead. On top of that, their 'research' is at best based on getting charities to rigorously assess themselves.
Their rubberstamp board publicly (in the NYT) pretended to fire one of the founders, but less than a year later he was quietly back at his old position.
You would expect that any organisation doing this kind of work would attract criticism. There is a taboo against harming a charities ability to raise funds.
Do you know of any organisation doing similar but better?
I'm not sure how any of this discredits their recommendations. E.g. it seems to me the Stop TB Foundation and Partners In Health probably still achieve measurable results.
I certainly wouldn't recommend donating to GiveWell, though. The MIT Poverty Action Lab seems like a much better use of the money.
I'm fond of Children International. They're very good about letting you know exactly and specifically what your donations achieve. I suppose any evidence of the effect on people is anecdotal, but in such a well-established relationship, it's also self-evident.
I'm not sure if this qualifies as measurable, but the ICRC does a lot of good in a lot of really bad places, and they produce testimonial and other documentation of what activities they can publicly document. (As you may or may not be aware, it's their policy to hold all reports of detainee treatment in strict confidence. Their site can explain why this is better than I can.)
This is extensible well beyond non-profits. The general form is still rough in my mind but it is something like "avoid situations where accountability and feedback mechanisms are not essential parts of the environment"
I would argue that there is a second, equally-important, albeit interrelated, problem with non-profits in general: they are highly risk averse. Many non-profits still operate under a create an endowment and dispense funds to proven -- or at least perceptive -- good projects model. Consequently, there is a "if it ain't broke, don't fix it" mentality, which is to say weak experimentation.
I should mention that most of my non-profit experience deals with medical research foundations. Political think tanks are limited by the general impossibility of proving all but trivial problems.
As a board member of two very active nonprofit organizations, and former board member of another, I really appreciate the suggestion to devote attention to demonstrable results. I'll bring that up at our next board meeting for each organization. It's easy to track membership, and reasonably routine to track member satisfaction in membership organizations, and some organizations set policy goals (within the scope of the IRS rules about political activity by some kinds of nonprofits) that are verifiable. But, yes, organizations doing work far from most supporters (e.g., foreign relief organizations or missionary organizations) often can gain funds for years without having to show results. It's regrettable when a willing donor isn't giving money to best effect.
"But, yes, organizations doing work far from most supporters (e.g., foreign relief organizations or missionary organizations) often can gain funds for years without having to show results. It's regrettable when a willing donor isn't giving money to best effect."
Not to be a shill for my own blog again, but the reason so few organizations run real evaluations is that it's really, really hard, expensive, and time-consuming to do. Consequently, funders by and large don't offer real money or incentives for it, and if they don't, there's not much of an impetus to improve, and I describe this basic idea in much greater detail here: http://blog.seliger.com/2008/04/24/studying-programs-is-hard...
Furthermore, some funders don't want real evaluations, even if they say otherwise; for example, we wrote a bunch of Community-Based Abstinence Education proposals for various clients, where the main purpose of the evaluation was to try and demonstrate that abstinence-only education works, and I described the problems with that in this post: http://blog.seliger.com/2008/10/12/what-to-do-when-research-...
Another important thing to remember when comparing non-profits and companies is the power of the customer. Companies go bankrupt because the customer has the power to discontinue their patronage if they deliver a product or service poorly (unless there are anticompetitive practices in play - ahem comcast). However, in the non-profit world the recipient of the service or products has very little power to motivate a market. Those who contribute money to non-profits could theoretically be this force, but does someone who wants to invest in reducing malaria really have to invest in understanding the health metrics and do this assessment with each investment? I don't think that is a permanent solution either. No doubt feedback and accountability are vital to making non-profts more successful, but let's challenge ourselves to think of a more systematic way to implement this type of change rather than summing with measurement is needed.
What you're really addressing is the issue of incentives, as economists call it, and so is the original author of the story. I wrote this comment on his blog:
To my mind, this isn't news, but that might be in part because I've worked in my family's grant writing firm for so long. The real issue you're discussing is one of incentives: nonprofits don't have much incentive to measure how well they're doing because funders don't demand that they do, or at least don't as a large group.
If the funder is federal, state, or local, the funder doesn't have the resources (or the brains) to really measure effectiveness; if the funder is a foundation, the funder probably cares more about the appearance of doing good than effectiveness. I've addressed this point in some detail here: http://blog.seliger.com/2008/01/04/more-on-charities and here: http://blog.seliger.com/2008/01/23/foundations-and-the-futur... . The blow, which I contribute to, deals with the issues you raise on a regular basis, and if you want to learn more about why things are the way they are at the moment, take a look at the archives.
Companies go bankrupt because the customer has the power to discontinue their patronage if they deliver a product or service poorly
This definitely happens to many kinds of non-profit!
in the non-profit world the recipient of the service or products has very little power to motivate a market.
For 501-c3 organizations that provide classes, lectures, and performances, often the opposite is the case. Many non-profits have to be run much like a business. There can be something like a marketplace where the "customer" pays not with currency, but with their time. (And sometimes, both money and time are involved.) Also, very often, the "recipient" of services can easily directly involve themselves with running the organization. (KPFT radio in Houston is an example of this.)
>> Companies go bankrupt because the customer has the power to discontinue their patronage if they deliver a product or service poorly
>This definitely happens to many kinds of non-profit!
Unless I missed the sarcasm, it doesn't happen with charitable non-profits. They die when funding dries up, and since their customers don't provide funding, their customers have little/no effect on their survival.
Yes, a charity can continue to exist long after folks stop showing up to receive services.
That depends on your point of view. I'd argue that the people supplying the funding are the customers and that the work they do and people they help is the service they provide. So if their customers aren't happy with the quality of the service they'll take their money somewhere else, exactly like in the for profit case.
> I'd argue that the people supplying the funding are the customers
That definition is inconsistent with 'There can be something like a marketplace where the "customer" pays not with currency, but with their time.'
Charities, almost by definition, provide products/services to folks who can't afford the full cost of said products/services so the charity gets money from elsewhere.
This is why I don't donate. Charities usually don't publish exact reports what's the money is used on and what have they achieved.
Building schools doesn't help if you don't hire teachers and get kids attend school and money used on clever guilt marketing doesn't solve ecological problems.
"Making a bad prediction isn’t like that... it’s months or years before your prediction is proven wrong. And then, you make yourself feel better by coming up with some explanation for why you were wrong: ...And so you keep on making predictions in the same way — which means you never get good at it."
I think this also applies to many macro-economists
This is one of the reasons why non-profit companies are flawed. There are no incentives in place for monetary gain. Non-profits are good at raising awareness, not being a sustainable entity.
If someone wants to make a difference and change world (especially developing countries), I believe there has to be financial incentives : social entrepreneurship.
If you can't make money doing something, is there any good evidence that it is worth doing at all? Leaving aside goods with "free" competition from the gov't (schools, roads, libraries, etc). EDIT: Also leaving aside things you do for your more direct benefit, like growing your own food or contributing to open source projects you use.
There are cases where charity in its more pure forms are important - I do think from time to time that individuals need help as a result of an event that may be no fault of their own. I personally try to be compassionate - but I don't think we should shame/force people to be charitable - and even in implementation, local not-for-profits tend to be far more efficient than government at assessing needs and addressing them.
When it comes to foreign relief/charities, it's a really tough call. On one hand, oppressive governments are incentivized to actively show their people at their worst so that they can get more money and in turn collect graft (e.g. Saddam Hussein and baby incubators), but on the other what are the rest of us supposed to do? Let these people die? We play right into the hands of their governments when we don't. Following natural disasters, war/strife, I also personally feel a strong need to help. In these scenarios, there can be no room for sustainable profit because they tend to be unique/unusual events.
I have nothing against helping in emergencies, but Ayn Rand had a good essay on that - emergencies are situations outside of normal operations and the response to an emergency should be working to end the emergency situation.
Did you really ask "is there any value in leading a moral life if it doesn't pay off financially?"
Non-profits do not actually aim at financial gain (duh!), so measuring their success in monetary term is an error of categories. Of course it is difficult to assess success of an endeavour like that, and Aaron is right in raising awarness for these difficulties, but forcing an inappropriate kind of measurement (did your non-profit make enough profit?) on them doesn't help solving this problem.
The correct question to ask are "what are our goals?" "How do we go about reaching our goals?" "Did our past activities in fact further our goals?" and so on. "how much more money did we raise and distribute" may factor in the analysis, but on its own it is pretty meaningless.
In the world of development, it is often quite easy to get cynical over the multitudes of people who make rather significant amounts essentially tax free working abroad. Even in some local charities, we have to be careful about this notion that because their mission seems pure that they are in fact pure.
Sometimes (and I'm afraid more often than even I suspect) instead of generating profits what would be "profits" instead accrue to what I'll call local empire building - managers building their own little fiefdoms of employees who report to them. Generally there's not as much pressure to stop it if the organization is reasonably well funded. Alternatively, they pay themselves more to soak up any potential profit so they can perversely call themselves not for profit.
Otherwise I agree - choosing metrics is not an easy task to be taken likely. Profitability/+cash flow are however far easier metrics to measure and manage.
There are many reasons to do things other than money, but when you are dealing with people that you don't personally or directly know the fact that they are willing to pay for your services is basically the only real signal you have as to whether you are benefiting them. And if you are not making more doing this (for a combination of monetary and non-monetary benefits) than you could something else, then you need to switch to a better use of your time.
Of course, the contributors benefit from donating money, etc by signalling effects and warm fuzzies, but that doesn't mean the recipients are benefiting nearly as much as the contributors are spending.
As Friedman put it in "Machinery of Freedom", there are basically only three ways to work with others - love or shared goals, free exchange, or coercion.
For the love of Cthulhu! Repeat after me: Profit is not determined by how much value you create, it is determined by how much of that value you can capture.
Some value isn't easily captured and that's a perfectly legitimate reason for nonprofits to exist.
I didn't say you should try to make as much profit as you can, but if you are not able to make anything, what evidence is there that you are benefiting anyone or even doing more harm than good? On the other hand, I've been following OB and LW since they started so I can easily see why you want to believe in the goodness of non-profits.
"But it does mean that if you’re involved in nonprofits (or predictions), you need to be much more careful about making sure you’re doing a good job. Unfortunately, few nonprofits do that."
Oh really? So he can cite research that says, for example, that only x% (where x < 50%) of nonprofits with an annual budget of $500k have ever done any sort of evaluation, either comprehensive or more basic?
His assertion may be true. But just throwing this out there with absolutely nothing to back it up, in a blog post titled "The Trouble With Nonprofits", is frankly insulting to me.
I spent > 4 years working in a technical role for a nonprofit, and I can assure people that measurement and evaluation were very much on the minds of staffers. Everyone (funders, management) was becoming more data-driven, so our organization got a grant to conduct a multi-year evaluation. It was a very positive and necessary process, and the results were very positive. Like Aaron, I definitely recommend some sort of measurable goals for most every nonprofit out there.
I don't take issue with his opinion. What I take issue with is the tone of the article.
For example: "...but an actual attempt to measure how much they’re improving people’s lives. For most nonprofits, I expect these numbers will be depressingly small. "
It can be easy, as young paragons of technology and entrepreneurship, to look over to the nonprofit world and disparage it, to think of it and it's practitioners as "lesser than". I should know; it's the attitude I had before I started working at a nonprofit.
Truth is, running a nonprofit can be a hell of a lot harder than running a for-profit. At any stage, you have all the issues that for-profits have - hiring, firing, managing employees, management issues, getting money in the door, technological problems - PLUS a whole other layer.
A lot of this other layer comes because of the weak link between financial health and results. Am I accomplishing my mission? How do I measure success? How do I maximize success based on dollar spent? How do I judge where to allocate capital or resources - when maximizing revenue isn't a goal?
Why am I wasting my time writing this? Because this article uses a lot of conjecture and one very poor example to make the entire nonprofit sector seem like a bunch of bumbling idiots who don't know how to use spreadsheets. They're not, and it's not fair for this community to see such a distorted view.
I've worked at two social ventures that had non-profits as clients (Web Networks, and most recently Urbantastic.com).
There emphatically is something wrong with non-profits. Just because there are some excellent ones (Sturgeon's law applies) doesn't mean that the field at large isn't ill.
An anecdote: we had a meeting with a guy who worked at a large company with a philanthropic streak. They provided, free of charge, a "dashboard" for planning, along with free training and tech support to any non-profit that wanted it (it cost considerable money otherwise). It was simple but very powerful, allowing you to do sensitivity analyses on projections to determine which courses of action would result in the best returns - both monetary and any other metric you wanted to put in (i.e. the social goals of the project).
It was like he was selling popsicles at an ice rink.
We ran into a very similar issue with Urbantastic. We had a web service that was very effective at getting small things done for non-profits: it's called micro-volunteering. There's no supervision required, you just write down what you want and someone always did it.
When we founded Urbantastic we thought the challenge was going to be in getting skilled people to help out for free. That was not a problem. The problem was that we'd go up to non-profits and say: here's a machine where all you have to do is post your todo list, and people will cross off items for you. For free. And we never got uptake.
People lined out the door offering to help, and almost no one took them up on it. Despite our offers to walk non-profits through it, and despite the recent funding cuts which meant that they needed all the help they could get.
I'm not saying that Urbantastic is perfect, but in our pilot cities this was overwhelmingly the result, and I still can't think of any incremental change that we could have done to get the non-profits engaged. A big part of it was that most of them were still uncertain about the whole email thing. It's not an exaggeration to say that some of them didn't know how to use a spreadsheet.
With our fancy web 2.0 ajaxy website, we felt like we were giving out iTunes gift cards to the lost tribes of Papua New Guinea. They didn't value it because they weren't even close to being at the point where they could use it.
I care about social good - I wouldn't have founded a social venture if not. And twice now I've been floored by the state of the non-profit world. Again, 10% of them are brilliant, but the other 90%... It's a big problem, and it's not going to get better by pretending that it's not there. That world is amazingly backwards. And it's saddening to see such important mandates being handled by such ineffective organizations.
My advice to anyone here planning on doing a social venture: drop backwards compatibility with the non-profit world. (one giant caveat: working exclusively with one or two of the few clued-in orgs might work).
A lot of what you are saying here resonates with me. I work part-time for a local charity in my town and have been involved in various others at times. There are a number of problems charities normally face which may explain your findings to some extent, including:
- well meaning but incompetent staff. People don't seem to understand that wanting to do good doesn't naturally translate into the ability to do it.
- no real management. I suppose this may be local phenomenon, but where I am (UK), there seems to be a tendency for small charities to be managed by wealthy housewives as a kind of hobby job. There is very definitely a social network of them that dominate the charities in my town. Often this means the management will be consistently absent from the organization. You can also see my previous point.
- no real check on the management. Charity trustees are often people who just want their name to be associated with a charitable organization. I can think of several trustees of the charity I work for who probably have only a vague idea of what we actually do.
- funding constraints. Charities in the UK who receive funding from government departments are usually monitored in terms of the volume of work they do rather than the outcomes they produce (which are much harder to quantify). Thus the incentive is often to not try to improve efficiency as reducing the work load may negatively impact funding.
Obviously my examples are to some extent just personal experiences that may not be universal. However I would guess that most non-profits face similar problems and that these problems may help explain why you got such a negative reception with your projects.
With regards to volunteers though, what did you to do assess their competency to perform the tasks they were willing to help out with? I ask because part of my job is to manage my organization's volunteers (which mostly come to us on work experience placements from the job centre). They frequently start working for us claiming to have various skills and then we discover that they often don't. For this reason I am usually quite unhappy about accepting new volunteers - when I have to get rid of them is is much harder (emotionally, for me) than sacking an incompetent employee. This might also help explain some of the resistance you encountered to what you were doing.
I'm the "tech guy" (ie, code monkey) at Givology, and boy could we use some of your givers. We don't have a fancy website (since it's basically composed of my spare time), but the concept is to allocate money to specific students or projects (similar to Kiva and Global Giving). This is the first I've heard of Urbantastic, and it sounds like you've got the opposite problem we have; we have a lot of students and projects that we can put up on the site, but not a lot of donors.
A big part of it was that most of them were still uncertain about the whole email thing. It's not an exaggeration to say that some of them didn't know how to use a spreadsheet.
They have an irrational fear of technology, and it may be because they believe nonprofits should be modeled after the 60s and 70s counterculture-based nonprofits where technology was distrusted as being a tool of the Man (a lot of money was poured into computers via AI funding, defense grants, etc).
It is awful and sad that they would not use Urbantastic because it sounds like a great idea.
There is of coure a very important problem charities trying to do this kind of measurments of efficiency face. They are in a market for public perceptiom to get donations, and (if we take the numbers the article cites for Oxfam), can you imagine the loss of donations and goodwill after a few newspapers and TV programs run the headline "Charity X spends half their donations on statistics instead of helping people!"? Even if the costs could be amortized over a few years, the charity would be dead by then.
Perverse incentive factors, again. (And the news might be right, collecting in depth statistics about the help program might just be a pretext to give a lucrative consulting contract to the director's best friend).
True, but responding to feedback is a lot harder than being aware of it. And especially so in a larger organization. I'm not so sure that GM's problems stemmed from not being aware of the problems as not being able to respond for various structural, legal, and social reasons.
84 comments
[ 1089 ms ] story [ 1614 ms ] threadThis also has an interesting interaction with a small startup essentially searching for the important metric for its survival.
And then another problem with metrics is that there has to be a cost benefit at some point. Obviously some are better than others, but if the cost to measure is as much to achieve, is it worth it? Though this may say as much about the mission as it does about the metric (microfinance is an area where this debate is raging).
Can you show me where?
You end up with the quantity versus quality argument, favoring large, superficial efforts over narrowly tailored, substantive efforts.
http://www.gatesfoundation.org/learning/Pages/december-2008-...
I would say that both of them do extremely well in the non-quantifiable quality department as well.
If McDonalds can work out how to quantify feeding hungry people then any charity working in Africa should be able to quantify feeding hungry people. If an insurance company can tell me the average lifespan of their clients to six decimal places and what exactly the risk factors are among that pool, then health-focused charities should be able to do it as well. etc, etc
(I have some familiarity with the literature and know the excuses for why charities operating in Africa with mobile phones and computers can't be expected to have as good a handle on their numbers as a British bank circa 1780. Yeah yeah, it must be difficult when you rely on globally distributed workers, many of whom are not literate. Oh wait, so does McDonalds.)
Yes we should.
Charities I donate to are selling me a product just as much as companies I buy from. That the product is a better world--by some measure that seems reasonable to me, rather than a simple benefit for myself, matters not a whit to my decision making process.
I don't give without a very concrete understanding of what the money will be used for, any more than I buy stock without a concrete understanding of what I'm buying and for how much.
By all means, sell me peace, charity, and goodwill! Perhaps that will motivate you to learn how to produce it, and to convince me that you have done so.
The whole impetus behind the growth of the microfinance movement is that traditional banking simply can't serve these groups. There are also numerous non-obvious major core issues involved, such as how gender roles have a huge impact on microfinance or why group lending is so prominent.
In short, the reason it's not simply handled by companies is because it's just not that simple.
But I do advocate approaching philanthropy as a consumer, however the group you're working with is legally classified. Be honest with yourself about what you're buying -- even if what you're buying is a warm fuzzy feeling. Perhaps especially in that case.
While I think it's a good idea to research your charity or donation to a non-profit, I definitely do not think they should operate like a for-profit business.
I think this is a great article, and there are some really pointed areas of improvement for non-profits. However, let's not turn to the free market gods for a solution please.
Anyone who donates money to a charity should at least spend as much time researching what they're "selling" as they would a comparable product/service. The tax returns are available but some of these firms spend as much as 80% of their funds raised in administration and overhead - which is often far worse than most for profits. Yes, not for profits also "often spend enormous amounts on marketing, advertising, and branding in order to convince you to part with your hard earned money".
Fortunately at least anyone can look up any charity's IRS Form 990 that lists such things as salaries (which are sometimes as much if not higher than private market salaries). I usually start here: http://www.guidestar.org/ (using bugmenot for login/password)
Actually, that is not necessarily a good thing to just sidestep. A lot of things that non profits work on might work better if they were worked on by companies. A lot of things that companies work on might have to be filled by non profits if companies weren't doing it.
Think of second hand stores. They are often run not-for-profit in order to provide cheap stuff to poor people. These compete directly with commercial low cost stores and other second hand stores. If these beat the non profits, there is no societal loss. One of the worlds most celebrated not for profits, the Grameen Bank provides small loans to poor people. It seems very possible that this NGO (and its many imitators) will be put out of business by commercial banks.
The UN has started a free online University to help the world's poor. I would take an either-or bet on for profit variants making a lot of progress on this issue commercially.
A great way of making non profits more like chess masters is making non profits into businesses. You don't always have to. But if it is feasible, its worht considering.
Most states already require non-profits to operate like businesses; indeed, non-profits are expected to operate better than businesses -- most states require non-profits to maintain levels of capitalization that require devilish efficiency and reuse.
If a donor is pissed, it will force the charity to conform to his/her demands. The recipients of the charity's services are usually an afterthought, b/c they are far more easily replaced than any donor.
Traditional, for-profit banking simply can't, which is the entire point. Even when microfinance institutions lean further toward for-profit, they end up more and more like traditional banks making traditional big loans.
And that's the way it actually moves, with microfinance institutions edging toward becoming more commercialized, and when they drift too far away from poorer clients, new micro institutions will likely come in to take their place. In other words, the shift naturally moves in the opposite direction.
The problem with for-profit microfinance is that, as a general rule, financial performance increases with loan size, and that's why commercialization is a cause for concern, along with a lot of the other baggage that comes with commercialization like pressure to increase profits and employee incentive programs that negatively impact service toward small clients.
Compartamos accounts for the problems of the small loan market (notably the inability to determine which borrows are good or not) the same way moneylenders needed to before them, by charging very high interest rates. But this ends up screening out lower end borrowers, hence the discussion of "mission drift."
Commercial banks have long dabbled in elements of microfinance, but there are fairly concrete obstacles, mostly operational and information-related issues, that make it tough to be profitable, and even tougher to approach it as strictly for-profit while continuing to be actual microfinance.
For example, the free market will not provide free meals to homeless people with HIV in Manhattan.
Nonprofits should not be made into businesses. If a business can fill a need and turn a profit at the same time, great (in most cases)!
Legal Aid (anywhere) assists asylum-seekers, wrongfully evicted tenants, and indigents needing legal services.
Microfinancing (non-US operations) in India, China, and the Southeast have done so well that the pioneer of microfinancing received a Nobel.
One example:
http://homeschoolers.org/
Another example:
http://www.mcgt.net/index.html
A third example, with more expensive programs besides the kind of program I just mentioned:
http://www.davidsongifted.org/
After edit: To relate what I say here to what was said in the submitted article, an organization that gets all of its members talking to one another freely through one-to-many, many-to-one, and one-to-one channels available to all members gets a LOT of feedback that other organizations don't get, and learns what works and what doesn't work.
There are also meta-charities like GiveWell and the MIT Poverty Action Lab that investigate other charitable attempts.
http://metatalk.metafilter.com/15547/GiveWell-or-Give-em-Hel...
Summaries: http://mssv.net/wiki/index.php/Givewell
They got caught out because they were blatantly astroturfing for themselves and slagging 'competing' charities all over the web. Then it turns out that even by their own generous accounting, more than half of their funds goes directly to their own overhead. On top of that, their 'research' is at best based on getting charities to rigorously assess themselves.
Their rubberstamp board publicly (in the NYT) pretended to fire one of the founders, but less than a year later he was quietly back at his old position.
Do you know of any organisation doing similar but better?
I certainly wouldn't recommend donating to GiveWell, though. The MIT Poverty Action Lab seems like a much better use of the money.
I should mention that most of my non-profit experience deals with medical research foundations. Political think tanks are limited by the general impossibility of proving all but trivial problems.
Not to be a shill for my own blog again, but the reason so few organizations run real evaluations is that it's really, really hard, expensive, and time-consuming to do. Consequently, funders by and large don't offer real money or incentives for it, and if they don't, there's not much of an impetus to improve, and I describe this basic idea in much greater detail here: http://blog.seliger.com/2008/04/24/studying-programs-is-hard...
Furthermore, some funders don't want real evaluations, even if they say otherwise; for example, we wrote a bunch of Community-Based Abstinence Education proposals for various clients, where the main purpose of the evaluation was to try and demonstrate that abstinence-only education works, and I described the problems with that in this post: http://blog.seliger.com/2008/10/12/what-to-do-when-research-...
To my mind, this isn't news, but that might be in part because I've worked in my family's grant writing firm for so long. The real issue you're discussing is one of incentives: nonprofits don't have much incentive to measure how well they're doing because funders don't demand that they do, or at least don't as a large group.
If the funder is federal, state, or local, the funder doesn't have the resources (or the brains) to really measure effectiveness; if the funder is a foundation, the funder probably cares more about the appearance of doing good than effectiveness. I've addressed this point in some detail here: http://blog.seliger.com/2008/01/04/more-on-charities and here: http://blog.seliger.com/2008/01/23/foundations-and-the-futur... . The blow, which I contribute to, deals with the issues you raise on a regular basis, and if you want to learn more about why things are the way they are at the moment, take a look at the archives.
This definitely happens to many kinds of non-profit!
in the non-profit world the recipient of the service or products has very little power to motivate a market.
For 501-c3 organizations that provide classes, lectures, and performances, often the opposite is the case. Many non-profits have to be run much like a business. There can be something like a marketplace where the "customer" pays not with currency, but with their time. (And sometimes, both money and time are involved.) Also, very often, the "recipient" of services can easily directly involve themselves with running the organization. (KPFT radio in Houston is an example of this.)
>This definitely happens to many kinds of non-profit!
Unless I missed the sarcasm, it doesn't happen with charitable non-profits. They die when funding dries up, and since their customers don't provide funding, their customers have little/no effect on their survival.
Yes, a charity can continue to exist long after folks stop showing up to receive services.
That depends on your point of view. I'd argue that the people supplying the funding are the customers and that the work they do and people they help is the service they provide. So if their customers aren't happy with the quality of the service they'll take their money somewhere else, exactly like in the for profit case.
If McDonald's serves me a burger that is inedible, I know about it immediately and can stop patronizing McD's.
If Burgers4Africa distributes inedible burgers in Somalia, how do I know about it?
That definition is inconsistent with 'There can be something like a marketplace where the "customer" pays not with currency, but with their time.'
Charities, almost by definition, provide products/services to folks who can't afford the full cost of said products/services so the charity gets money from elsewhere.
Note that not all non-profits are charities.
Building schools doesn't help if you don't hire teachers and get kids attend school and money used on clever guilt marketing doesn't solve ecological problems.
I think this also applies to many macro-economists
If someone wants to make a difference and change world (especially developing countries), I believe there has to be financial incentives : social entrepreneurship.
When it comes to foreign relief/charities, it's a really tough call. On one hand, oppressive governments are incentivized to actively show their people at their worst so that they can get more money and in turn collect graft (e.g. Saddam Hussein and baby incubators), but on the other what are the rest of us supposed to do? Let these people die? We play right into the hands of their governments when we don't. Following natural disasters, war/strife, I also personally feel a strong need to help. In these scenarios, there can be no room for sustainable profit because they tend to be unique/unusual events.
Non-profits do not actually aim at financial gain (duh!), so measuring their success in monetary term is an error of categories. Of course it is difficult to assess success of an endeavour like that, and Aaron is right in raising awarness for these difficulties, but forcing an inappropriate kind of measurement (did your non-profit make enough profit?) on them doesn't help solving this problem.
The correct question to ask are "what are our goals?" "How do we go about reaching our goals?" "Did our past activities in fact further our goals?" and so on. "how much more money did we raise and distribute" may factor in the analysis, but on its own it is pretty meaningless.
Sometimes (and I'm afraid more often than even I suspect) instead of generating profits what would be "profits" instead accrue to what I'll call local empire building - managers building their own little fiefdoms of employees who report to them. Generally there's not as much pressure to stop it if the organization is reasonably well funded. Alternatively, they pay themselves more to soak up any potential profit so they can perversely call themselves not for profit.
Otherwise I agree - choosing metrics is not an easy task to be taken likely. Profitability/+cash flow are however far easier metrics to measure and manage.
Of course, the contributors benefit from donating money, etc by signalling effects and warm fuzzies, but that doesn't mean the recipients are benefiting nearly as much as the contributors are spending.
Some value isn't easily captured and that's a perfectly legitimate reason for nonprofits to exist.
...and you didn't pay for it, and we don't run ads. Do you feel that we benefited you, and did more good than harm?
Oh really? So he can cite research that says, for example, that only x% (where x < 50%) of nonprofits with an annual budget of $500k have ever done any sort of evaluation, either comprehensive or more basic?
His assertion may be true. But just throwing this out there with absolutely nothing to back it up, in a blog post titled "The Trouble With Nonprofits", is frankly insulting to me.
I spent > 4 years working in a technical role for a nonprofit, and I can assure people that measurement and evaluation were very much on the minds of staffers. Everyone (funders, management) was becoming more data-driven, so our organization got a grant to conduct a multi-year evaluation. It was a very positive and necessary process, and the results were very positive. Like Aaron, I definitely recommend some sort of measurable goals for most every nonprofit out there.
I don't take issue with his opinion. What I take issue with is the tone of the article.
For example: "...but an actual attempt to measure how much they’re improving people’s lives. For most nonprofits, I expect these numbers will be depressingly small. "
It can be easy, as young paragons of technology and entrepreneurship, to look over to the nonprofit world and disparage it, to think of it and it's practitioners as "lesser than". I should know; it's the attitude I had before I started working at a nonprofit.
Truth is, running a nonprofit can be a hell of a lot harder than running a for-profit. At any stage, you have all the issues that for-profits have - hiring, firing, managing employees, management issues, getting money in the door, technological problems - PLUS a whole other layer.
A lot of this other layer comes because of the weak link between financial health and results. Am I accomplishing my mission? How do I measure success? How do I maximize success based on dollar spent? How do I judge where to allocate capital or resources - when maximizing revenue isn't a goal?
Why am I wasting my time writing this? Because this article uses a lot of conjecture and one very poor example to make the entire nonprofit sector seem like a bunch of bumbling idiots who don't know how to use spreadsheets. They're not, and it's not fair for this community to see such a distorted view.
There emphatically is something wrong with non-profits. Just because there are some excellent ones (Sturgeon's law applies) doesn't mean that the field at large isn't ill.
An anecdote: we had a meeting with a guy who worked at a large company with a philanthropic streak. They provided, free of charge, a "dashboard" for planning, along with free training and tech support to any non-profit that wanted it (it cost considerable money otherwise). It was simple but very powerful, allowing you to do sensitivity analyses on projections to determine which courses of action would result in the best returns - both monetary and any other metric you wanted to put in (i.e. the social goals of the project).
It was like he was selling popsicles at an ice rink.
We ran into a very similar issue with Urbantastic. We had a web service that was very effective at getting small things done for non-profits: it's called micro-volunteering. There's no supervision required, you just write down what you want and someone always did it.
When we founded Urbantastic we thought the challenge was going to be in getting skilled people to help out for free. That was not a problem. The problem was that we'd go up to non-profits and say: here's a machine where all you have to do is post your todo list, and people will cross off items for you. For free. And we never got uptake.
People lined out the door offering to help, and almost no one took them up on it. Despite our offers to walk non-profits through it, and despite the recent funding cuts which meant that they needed all the help they could get.
I'm not saying that Urbantastic is perfect, but in our pilot cities this was overwhelmingly the result, and I still can't think of any incremental change that we could have done to get the non-profits engaged. A big part of it was that most of them were still uncertain about the whole email thing. It's not an exaggeration to say that some of them didn't know how to use a spreadsheet.
With our fancy web 2.0 ajaxy website, we felt like we were giving out iTunes gift cards to the lost tribes of Papua New Guinea. They didn't value it because they weren't even close to being at the point where they could use it.
I care about social good - I wouldn't have founded a social venture if not. And twice now I've been floored by the state of the non-profit world. Again, 10% of them are brilliant, but the other 90%... It's a big problem, and it's not going to get better by pretending that it's not there. That world is amazingly backwards. And it's saddening to see such important mandates being handled by such ineffective organizations.
My advice to anyone here planning on doing a social venture: drop backwards compatibility with the non-profit world. (one giant caveat: working exclusively with one or two of the few clued-in orgs might work).
- well meaning but incompetent staff. People don't seem to understand that wanting to do good doesn't naturally translate into the ability to do it.
- no real management. I suppose this may be local phenomenon, but where I am (UK), there seems to be a tendency for small charities to be managed by wealthy housewives as a kind of hobby job. There is very definitely a social network of them that dominate the charities in my town. Often this means the management will be consistently absent from the organization. You can also see my previous point.
- no real check on the management. Charity trustees are often people who just want their name to be associated with a charitable organization. I can think of several trustees of the charity I work for who probably have only a vague idea of what we actually do.
- funding constraints. Charities in the UK who receive funding from government departments are usually monitored in terms of the volume of work they do rather than the outcomes they produce (which are much harder to quantify). Thus the incentive is often to not try to improve efficiency as reducing the work load may negatively impact funding.
Obviously my examples are to some extent just personal experiences that may not be universal. However I would guess that most non-profits face similar problems and that these problems may help explain why you got such a negative reception with your projects.
With regards to volunteers though, what did you to do assess their competency to perform the tasks they were willing to help out with? I ask because part of my job is to manage my organization's volunteers (which mostly come to us on work experience placements from the job centre). They frequently start working for us claiming to have various skills and then we discover that they often don't. For this reason I am usually quite unhappy about accepting new volunteers - when I have to get rid of them is is much harder (emotionally, for me) than sacking an incompetent employee. This might also help explain some of the resistance you encountered to what you were doing.
They have an irrational fear of technology, and it may be because they believe nonprofits should be modeled after the 60s and 70s counterculture-based nonprofits where technology was distrusted as being a tool of the Man (a lot of money was poured into computers via AI funding, defense grants, etc).
It is awful and sad that they would not use Urbantastic because it sounds like a great idea.
Perverse incentive factors, again. (And the news might be right, collecting in depth statistics about the help program might just be a pretext to give a lucrative consulting contract to the director's best friend).