71 comments

[ 3.3 ms ] story [ 104 ms ] thread
Your blog looks like it's owned by Twitter. The twitter logo is in the usual location of the owner's logo. I suggest you repair this. Among other things, it's probably against their policy.

Also, I had exactly the same experience, but the difference here is the enterprise nature of the startup. Enterprise demands power, and investment, and capital. You can't do it lean.

As someone who works in the Canadian Government I'm not surprised at all that a deputy minister would have no interest in helping you iterate your product, at that level it's all about politics and to have a product fail in front of them and their coworkers is a big problem it makes them look bad which could have a negative effect on their job.
> the minimum viable product preached by Lean Startup has limited practical use. Customers aren’t interested in funding your “learning.” They want reliable software that delivers value consistently. You must build the minimum desirable product, and if you don’t have a good understanding of what’s desirable before you start, then, don’t.

Oh, how I wish I could tell myself this five years ago. Maybe it's the industry I do most of my development for, but users I interact with very rarely want to hear "that will be in 2.0!" The ones that even let you finish the sentence before hanging up/walking out usually respond with "That's great. Let me know when it's done, then we can talk about signing up."

MVP and incremental development is great when you're a VC-funded startup in Mountain View, but when you're a real business you either have a working product that people want, or you don't.

What is it about the word "viable" that seems to make everyone read right past it?
Unfortunately MVP has been bastardized from "the smallest thing people will pay for" to "the smallest individual piece of the application."
Great share to hear the more realistic side of some of the MVP pieces. Certainly one of the biggest challenges I hit almost daily with my saas is what feature is critical to success, and which can be 2.0, especially in light of the comment than they might not want to wait for a 2.0. Customer research only gets you so far.
Sorry, this doesn't add up. Lean doesn't mean your product breaks or is pushed out functionally incomplete. If no one wants it it's not viable.

The article says give your customers what they want. Unfortunately they thought the customers wanted a mobile app... but they didn't! Wait, what startup methodology could have prevented this...

The article is honest about their mistakes and that must have been hard to do and props for trying but don't try and pass the failure buck to a methodology you didn't even do right.

Exactly, viable = vendible. If there are no customers, than you didn't make your homework right and built something that nobody wants. I'm affraid that this misassumtion about mobile app can be caused by the feedback from customer interviews. You shouldn't rely too much on what people say, ask them to pay for your product instead.
Agreed (edit: though I thing the title of the post indicates awareness of not doing it right). Jumping from:

  > Our inaugural customer couldn’t access the software
to claim:

  > the minimum viable product [...] has limited practical 
  > use. Customers aren’t interested in funding your “learning.”
and blaming the MVP concept instead of the broken software?
"The purpose of your first release (and every other release) is to give your customer immediate value."

Exactly. That's a MVP, not a crappy product which doesn't even work.

Lean Startup theory can be pain in the ass for many kind of startups. I think it's wrong that Lean Startup doesn't encourage sales from the very beggining. Sales are the best validation of your idea, not feedback from customers who doesn't pay for your MVP anyways. Build a product for people who pay for it - that's my advice. It's not universal eighter, but more than 50% of startups can start sales before coding anything.
I think you guys misunderstood the purpose of MVP, since your lessons learned part explains pretty much what we can find in Lean approach books. MVP is not a product, it's a learning process. Bugs have nothing to do with being lean (or not), it's sloppy developers.

Not trying to be a wise-ass, but I still think your mistakes are avoidable while still following lean principles. Good luck on your next venture!

I worked for a enterprise software company where we practiced incremental development. The quality of software was hardly stellar but they had a great sales team. By the time I left, they had 6000 customers, lot of them Fortune 500. It is also worth mentioning that they were completely bootstrapped until recently.

Our customers were always ok with us informing them that features will be in x.0, as long as we delivered on them.

Maybe, it was your market that prevented the lean startup methodology from being applied?

I like the term "minimum desirable product". Minimum viable product suggests something that barely works, while minimum desirable product alludes to something people actually want.
I upped you on that one - good call on the clarity of desirable over viable.
Minimum Delightful Product is the variant that I like best.
> I like the term "minimum desirable product". Minimum viable product suggests something that barely works, while minimum desirable product alludes to something people actually want.

I think "minimum viable product" works, and is better than "minimum desirable product" at describing the goal, since to be viable as a product something must not only be desired, but be desired by customers willing to pay for it sufficiently to meet the marginal costs of delivering it.

Of course, an MVP can fail to be viable as a product and still deliver validated learning (as I understand, the original definition of the purpose an MVP) that progresses you toward something viable as a product, but the goal at each iteration should be to have something valuable as a product (and this is consistent with the more general lean methods which lean startup draws on, where each iteration aims to do the minimum work that will produce independent business value.)

Not specifying what browser/versions you would be supporting - when dealing with anything remotely government is a mistake. Having an access denied error message right after signing up means your basic happy path wasn't working correctly and neither were your development and QA processes. Testing features in production is different than testing logic in production.
Ironically, the first two of the "four livelihood-threatening fundamentals" that the Lean Startup methods distracted them from are pretty much verbatim parts of the Lean Startup method.
Many people i talk make the same mistake with lean and custdev.

And i have to say i did the same mistake.

We try to things the lean way and hope therefore we do it the right way. Eg. building an MVP that should validate our business assumptions.

All of the methods around lean and custdev sound perfect in theory. But validating that you do the right thing (apart of a super obvious strong market pull) is extremely hard.

A common mistake i see is when people speak about 20 different hypotheses or experiments - and usually none of them is related to a real make-or-break risk of their company.

They try to validate that they do the right thing. Which is extremely hard.

As entrepreneurs we are in charge of a strong product/customer vision. Any method we use is only in charge to check if we are lying to ourselves.

But many of the techniques used in custdev/lean are very strong tools for minimizing the downside.

And this is (imho) where we should use them.

Our main goal is not upside optimisation (waiting for the go) but downside minimization (avoiding the no-go)

Or differently put: They are very good to double check if you aren't doing the wrong thing, if we aren't lying to ourselves.

The other main mistake i see is "trying to do it the lean way":

There is no "lean" way. Lean is a reverse engineered model largely based on survivor bias and wrong self-reflection. But that's completely ok. We must not forget how young fast-growth oriented product-centric entrepreneurship is. Even custdev which is more fullstack than lean is mostly academic and best used only as a frame of thought. Not as a blueprint model. Lean will be replaced by other models using the same or improved stack elements.

Just like me, too many people are obsessed with the theory behind lean and see it as fullstack solution.

Instead of looking at lean as a fullstack, we should pick parts of that stack that make sense for our business - as we do eg in software engineering or ux.

E.g. MVPs tend to be a problem with b2B relationships. Agency like concierge code-for-hire models tend to work better. Also customer interviews work gold for B2B - by far better than for b2c models.

Simply put: We need to pick stack elements of lean and custdev as we would pick elements of software stacks, depending on our business's risks and challenges.

If you are interested what i mean with stack elements, take a look at - customer interviews http://www.slideshare.net/andreasklinger/actionable-customer... - metrics in early stage http://www.slideshare.net/andreasklinger/metrics-for-early-s... - jobs to be done http://hbswk.hbs.edu/item/6496.html - and several other aspects: http://www.hackertalks.io/robfitz/why-bother-talking-to-cust...

Thanks for summarizing. It seems like you have a lot of useful info here and I appreciate your contribution. However, I'm really having a hard time parsing this; it seems like you're repeating the same thing several times. Any additional effort you can invest in further editing your post would be greatly appreciated!

Edit: Might look at a higher sentences-per-paragraph ratio, I think that's what's throwing me off.

You are completely right. Sorry i just wrote it in one go.

I tried to restructure it. Hope it is now a bit clearer.

Critical Point: Viability is judged by your customer.

Developers are comfortable with dealing with some quirks in software. They typically use some of the fastest computers and the most up to date browsers. They live in and build the world of software.

It is very difficult for a developer to understand the mindset of a non-technical customer and see their software from the customer's perspective and experience-base. It's not impossible to do this. But if you don't try hard enough your project will fail.

Fist of all, it might be not the right choice of methodology as government and non profit clients are not the best audience to apply "lean". These guys are very conservative, especially in Canada.

Second, it appears a lot of problems appeared because founders didn't know what they are building or what they want to build and relied on "lean" as a magic bullet to help them figure it out.

Third, it's hard to believe but I've met people who applied "lean" and other methodologies without actually diving deep and at least reading the underlying manifesto books thoroughly. Lean methodology is so easy, isn't it? Build an MVP and then iterate.

To sum it up, there's nothing wrong with lean methodology. These guys first failed at understanding it. Then no wonder that they double failed at execution.

Wow. Reading these comments makes me think that the whole Lean Startup thing is more of a religion than a methodology.
It's not enough to simply survey people on whether or not they have a problem and then go build. Building good software means building something that people want to use. If you fail to properly dive into customer development and learn about your customers base (what tools do they currently use, who are the early adopters, what are the existing barriers to entry) then you will likely not succeed.

Simply reading one book and some blogs on Lean (or any other methodology) also does not make you an expert. You should read about many different business strategies to understand the key parts of them and how they are similar and different. It's a bit naive (arrogant?) to think you can just read a book and then go conquer the world without having to constantly learn and adapt during the process.

Sorry to add to my rant but I get really tired of hearing the "X methodology killed my business" story. There have been hundreds of thousands of successful businesses created and operated by people without formal training in any strategy so why should having read about a business strategy destroy your business? Did it remove your ability to talk to customers and respond to their concerns?

People often fail to address internal issues like limited domain expertise or limited business skills. It's not enough just to see a problem and be excited about solving it, you need some skills (either present or learned) and you need resources.

Lean is focused around learnings. If you had focused on learning rather than whether or not you completed a task (MVP, demo, etc), you'd realize that you've gained a lot from the experience but it wasn't until you reflected that you realized you failed to incorporate those learnings back into the business.

Also, failure to understand that your first customer wasn't an early-adopter really hurt you. If someone isn't willing to jump through hoops to use your product, it either 1) doesn't solve a big enough problem or 2) they aren't an early-adopter.

It's difficult to work out if the author is saying that his startup ended up 'fat', or that it would have been better if the startup had been 'fatter'.

I'd argue the problem was the former. It sounds like they spent a large part of the runway on things that were not part of the MVP (patents, funnel, extra employees, etc). This is exactly what the concept of 'lean startup' is designed to avoid - 'lean startup' and 'mvp' means you throw out everything that isn't targeting on creating a product that people will buy. You don't throw out parts of the product itself to be lean, you throw out everything that isn't required for the core offering. By the sound of it, the author threw out the core offering and focused on the 'fat'.

EDIT: The 'viable' in MVP is a very specific word that is straight out of the dictionary definition. As from Google dictionary:

  vi·a·ble  
  /ˈvīəbəl/
  Adjective
  Capable of working successfully; feasible: 
  "the proposed investment was economically viable".
Very clear. The MVP for humans going to space is a billion dollar ecosystem including runways, safety checks, logistics, long range communications, etc. An MVP can still cost a billion dollars - MVP does not mean cheap.
I'd like to suggest that all the lessons learned here are precisely what Lean Startup teaches-- with techniques that are meant to prevent you from spending two years in a death spiral.

Segue to edit of my standard rant on this subject: (originally posted on: http://news.ycombinator.com/item?id=3429906)

MVP != half assed, cheap, shitty product. It is exactly what it says: a minimum viable product, key word being viable. An MVP for a mission critical application or a cutting edge piece of research isn't going to be cheap or easy, because it needs to be viable.

Because of the name, many people make the assumption that doing a Lean Startup means not taking the time or money necessary to execute properly, this is not the case. Lean means conserving capital by focusing on iteration until you each product market fit, and only then ramping up sales or marketing, but again, if you're cracking a new market, untested technology, or going up against mature competition, even that phase of conserving capital and focusing in product may require a lot of capital.

Your product, when released to customers, needs to be usable, and needs to scratch an itch for them, otherwise it is inherently not viable. That's why customer development is so essential, it prevents you from making the worst mistake possible-- building a robust product no one wants. Even if you have the best product idea in the world, the only way to know if the features you add are having a positive impact is to measure the behavior of real, live, human users.

That being said, it's still your responsibility to stay relentlessly focused in your core value proposition. You can't build every feature you're customers ask for, only the ones that make sense for your product.

What we're seeing now is the lean startup backlash where people use the terms but don't really take the time to understand what they mean,.

The Lean startup movement is a powerful set of lessons that can empower entrepreneurs and save us from wasting our time and effort building the wrong things and chasing the wrong metrics, but I guess some of the lessons just have to be learned painfully through experience (it was certainly true for me).

Yes, yes, yes.

There are so many things they did that are obviously wrong from the Lean Startup perspective. And then he goes and blames the Lean Startup material, rather than their understanding of it.

For example, his notion that product management is supposed to be casual and erratic is not what the Lean Startup approach suggests. The essence of the approach is in proving that customers really need something before you invest in it. It's a disciplined, data-driven approach. But he says they built half-assed features that nobody needed, and somehow blames that on the Lean Startup stuff.

Also, he preaches about controlling expense. The word "Lean" in "Lean Startup" comes from Lean Manufacturing, the heart of which is relentlessly discovering and eliminating waste. How in the world did he get from that to being free-spending and wasteful?

And then this makes me crazy: "By this stage, Lean Startup theory told us we should be scaling up." No. No, no, no, no, no. No! The Lean Startup theory is that you start scaling if and only if you have achieved product-market fit and have a sustainable way to deliver. He described nothing to suggest they were close to product-market fit. Indeed he says they were limping.

Or his sudden too-late realization that software quality is important. The Lean Startup stuff is basically Extreme Programming plus Customer Development plus some Lean philosophy. Extreme Programming is one of the most quality-focused methods. At my Lean Startup, we had high unit test coverage, strong acceptance tests, lots of code review, and very high quality. 18 months ago I gave a talk on how quality practices were vital when using the Lean Startup approach:

http://vimeo.com/24843552

And it's not like I was the first, or even the 10th person to talk about this. Rather than blaming the Lean Startup approach, he should blame his poor understanding, and his cargo-cult adoption of various rituals he apparently couldn't fathom.

The problem I have is that everyone can rap the lean startup principles but when you ask who has successfully implemented lean startup, nobody says yes. How many people do you guys personally know that have successfully implemented lean startup?
By what measure do you want to define if startups have implemented it successfully?
I'm not the parent poster, but I'd basically say: are there any examples at all? Even if we're really loose with what qualifies as "lean startup" are there any examples anyone would be willing to share?

I don't mean to imply that there aren't any, but I just haven't heard of any.

If you read The Lean Startup there are examples in there. I mentioned above WPEngine did it, but also look at Food on the Table. These stories show the lean startup done right. The story above is so riddled with holes. It's like the guys heard some buzzwords, read a wikipedia page, and ran with their limited understanding of it.
Talking about business, entrepreneurship is fun. Doing it is another. Separates the Men from the boys.
Jason Cohen with WPEngine. Great lean startup. They rock, and are profitable.
Well, it depends on your value of success. At my last startup, as I explain in the linked talk above, we used Lean Startup techniques to kill a number of ideas that we would otherwise have implemented. The startup still didn't make it, so you could call it a failure, but the customer development and iterative market-testing techniques gave us many more tries for the same money, so I am very happy we used them.

But if you want a more traditional success story, I'd suggest Boxbee:

https://boxbee.com/

I was a mentor at a Lean Startup Machine event ~18 months ago where I met the founder. He had a very hazy idea, and over the course of the weekend refined it into something more focused. He then used the Lean Startup techniques to build a business that had revenue from the first month, and was quickly in the black. Relentlessly refining both the user experience and the back end, he found product-market fit and came up with a sustainable plan. And he did this all in his spare time, mainly with his own money.

A month or so ago he pitched it at Jason Calcanis's LAUNCH conference and took first place. Now he's raising money to build an infrastructure well beyond the initial MVP-grade product.

But you can also look at historical examples. It's important to note that none of the Lean Startup techniques are new. It's meant to be a crystallization of things that some people were doing before, but that had never really been talked about. At the local Lean Startup Circle event, we had the founders of Dropbox and Cafe Press talk about how they worked, and it was very much in the start-small-and-iterate-relentlessly style that the Lean Startup folks are promoting.

I would say that I have.

It's a B2B product that has a couple substantial value adds over the competition. I created the MVP and got a core group of people in a paid beta. There were tons of bugs, but because of the unique value add I offered everyone was more than willing to put up for it and pay for it (and they knew they would get locked in at beta pricing for when it improved).

I was then able to quickly and easily iterate fixes and improvements, based on my vision as well as what I heard from my paying customers.

So on launch day I was already profitable, I knew what my users wanted, all the major bugs were completely sorted out, and I had ~100 paying beta users (all of whom were referred directly by me or by word of mouth through a beta user) who felt like they had a vested interest in the success of the product who helped me reach a critical mass.

With that being said I don't consider my company to be a lean startup (I hate labels like that), and I didn't set out to do a lean startup, but a lot of what I did is consistent with the lean startup approach.

No, it's not just a question of Lean Startup education. I am questioning the rituals. The world doesn't work according to a theory, however much you might wish it does. Clients couldn't care less about Lean Startup, MVP, XP, unit tests etc. This idea that you can experiment on clients (in my humble experience) is a dangerous delusion. The comment about "couldn't fathom" is unprofessional (and ludicrous given my resume). Please: listen more, lecture less. Cheers.
What rituals? Extreme Programming and Lean are inspect-and-adapt processes. You might start with something out of the book, but then you are supposed to tweak relentlessly. If you are doing it in a ritualistic fashion, you're already doing it wrong.

The client never cares about what methods you use. That doesn't mean you shouldn't use any methods.

I agree that experimenting is dangerous: any powerful tool is dangerous. But it has worked for too many people to call it a delusion.

Regarding "couldn't fathom", I'm glad to substitute "didn't fathom". But you certainly didn't understand them, and many other people did.

Totally. Take the complaint about the mobile app. They built a mobile app that nobody wanted, and then blamed Lean!

"Nevertheless as devout Lean Startup followers, we kept building, measuring, and learning. Hidden inside our minimum viable product was a separate mobile application to gather data on logic model performance. This mobile app mystified our customers. Why didn’t we let them input data online at their desks as a starting step?"

On what planet can you be doing Lean development and get as far as building an entire app that not one of your customer wants with features that none of them needs? How can anything be "hidden"?

Wow. Was this published April 1st?

Ship early, ship fast, ask questions, write down answers, run tests. That's what "lean" is all about.
Oh, so now the whole "we fucked up so let's blame the tools/methodology" has arrived at Lean Startup?

Plus ça change...

Here's a wonderful nugget: "too often Agile development methodologies are used to excuse lack of process and planning". The irony is strong in this one.

The next book on Lean should be a dictionary with just a single word explained in way even a 3 year old can understand: "viable"

Viable means, strictly, able to live at birth, especially, in relation to a foetus (but not a 3-year old?). Being born doesn't mean you will survive. So, minimum viable product is a poor term, implying disrespect for your customers. Minimum desirable product (from your customer's perspective) reflects the standard required.
What I tell a lot of people: Iterate on ideas, do not iterate on quality.
I agree. Once one decides that there's a problem worth working on the iterations in early stages shall be to test different ideas how to solve the problem.
There seem to be two schools of thought here, both interesting to me. The author seems to think that his failure has something to do with problems associated with the "lean startup" model and other commenters (lean startup adherents, apparently) seem to hink his failure has something to do with failing to properly follow the tenets of the philosophy.

I suspect actually that the author adhered about as closely to "lean startup" as many self-identified "lean startups" who have succeeded in the past, and made a pretty average number of mistakes in the process. I don't think of 'lean startup' as a way to guarantee success, I think of it as a way to fail gracefully if you don't succeed. The author came pretty close to this, actually - it doesn't sound like they burned through enormous piles of venture capital money, it seems like they learned a lot in the process, it seems like they didn't spectacularly fail to make payroll causing half a dozen family-bound engineers to suddenly lose their livelihoods. The only less-than-graceful element of their failure was that it took two years, which is probably a little (though not a lot) on the long side. Frankly late in the process they came pretty close to scoring what sounds like it would have been a pivotal investor, and that combined with lessons learned could have produced a true MVP, product market fit, and a viable startup...

So, I think the author successfully followed a good chunk of lean software advice, made some mistakes, got unlucky, and failed. This is what most startups do - even most lean startups - and there should be no shame in it. Paul Graham himself has remarked he wishes he were able to bring himself to invest in more likely-to-fail black swans because that's the way you get to own enough vol to have a real winner in your portfolio - we all should be more comfortable with the risk of failure.

It may well be that your product just wasn't needed or that your focus was wrong, for instance customers might have wanted feature y before feature x: It is not uncommon to find it difficult to work out what customers want in isolation.

(Shameless plug alert!)We encountered this several times in our own failed startups where we have built the best software the world has never seen and decided to create WillThisFly? to address these issues: It allows you to develop your product with feedback from day one, validate ideas and features, iterate and develop before committing time and resources to it or indeed, find out that it's a non-starter.

Our MVP is online now at http://willthisfly.net/projects/1/

But since anyone hoping to make a new product should be talking to his real potential customers, why would he not talk to them directly, instead of say, trying to get them to use WTF?

Anyone other than his real-life potential customers giving him feedback is.. well, not going to give him the feedback he needs, no matter how well-intentioned he is.

Then there's the issue of someone potentially stealing your idea, etc.

What WTF provides is the ability to come up with an idea and get a quick gauge as to whether it has merit (it works with projects at various stages from new ideas to coming up with new features for existing projects and anything in-between).

For example, you have likely heard of "Facebook killer" apps and "New way to organize your inbox"-type ideas in the past and the general reaction is "Oh ffs, not another <insert product here> clone" but most ideas still have some merit even if their initial briefs are rather ambitious and somewhat tenuous.

Your Facebook killer may be too ambitious but with a bit of focus you may be able to come up with a smaller, more niche product that benefits a particular demographic given feedback and some direction.

This is what WTF provides: Sure, it will not give you a definite "If you build it, they will come" answer but it might just be able to help you salvage something that doesn't look like it is going anywhere.

In addition to helping you shape a project, it may also prevent you from spending time and money on something that just will not work as a result of feedback so you will know fast if it has any potential before spending time and money on it.

AS to stealing your idea, this is rarely the case. The thing that sinks most projects is lack of focus and aiming in the wrong direction - something WTF is designed to help prevent.

Thanks very much for your insight... this is exactly what WTF was designed for.

Mick

Well, if I have a good idea, post it there and get a lot of positive feedback, wouldn't that downright encourage someone to implement the idea now that it's (supposedly) been validated?

Don't get me wrong, I'm not hoping WTF will not succeed. It's just that as someone who's just wasted several years of his life on products that went nowhere, I'm a bit skeptical.

"The Lean Startup" seems like the way to go, but that involves speaking directly to your potential customers, because they're the only ones who know what they really need. Apparently it's really freaking difficult to confirm that you've really got a product-worthy idea on your hands, even if you're talking to potential customers.

What I've learned from my failed startup during 2000/2001 [1]: With enterprise software the MVP is a power point deck with mockup screen shots. You first need to validate the idea, words, concepts before you start developing. This also helps with the deep decision pipelines that enterprises usually have to buy your enterprise software.

Next time I'd start immediately going around with a powerpoint deck of mockup screenshots instead of developing software first (which we did and failed).

[1] http://codemonkeyism.com/6-reasons-why-my-vc-funded-startup-...

I have seen a lot of people who pointed out how wrong the OP had done and the OP should have done X, followed by Y, yadi yada. What I can't find is maybe a bit of sympathy, or encouragement.

Fine. It would be very useful if those smart people can share what and how they have applied the Lean Startup theory, and how successful their products/companies are. I will give most of them benefit of doubt for _reading_ that book, but applying it? I would say around 10% of them? Applying it and succeed? Even less, I just speculate it will be around the 1% mark. In other words, those who talk so loud about OP didn't get the MVP right etc etc, I will say they won't fare any better if their rubber hit the road. I am more than happy to be proven wrong.