Dos and Don’ts of Lean Startup- Top Takeaways from The Lean Startup Conference 2014

Lean Startup Conference 2014

Lean Startup Conference 2014

Main Takeaway- Continuous Experimentation Well Beyond The Startup Stage

Contrary to the generally held belief that lean startup principles advice experiments in early stages of a startup; many speakers at the conference showed how they are experimenting continuously at all stages of their ventures.

Eric Ries in said that “Product market fit and experimentation is not a one time activity. It’s a continuous flow of activities. There are no discrete big jumps! Think of these steps in continuous flow that lend themselves to go back if an experiment fails”

Hiten Shah of Kissmetrics reiterated that a meaningful metric leads to a hypothesis and then to an experiment to validate it. Startups should always be A/B testing. Empirically 1 out of 5 tests succeed. Strive to win 1.67 out of 5.

A/B testing can help not only at different stages of a startup; but also for various activities including website traffic, app installs, welcome emails, Web/Mobile onboarding, E-mail digests, Triggered notifications, dormant/churned users.

Des Traynor also said that having continuous feedback is more valuable than one time event driven feedback.

Experiments helped even established brands like Rally, Google and Vox Media to validate hypotheses at later stages of their product lifecycle

  • Rally launched a dummy brand targeted towards developers to protect the parent brand from the impressions created by the experiments. Finally Rally decided to have both brands.
  • Google Adsenses team validated Partner Problems using Lean Startup Principles. Blair Beverly said that they faced problem with new projects scaling too early and failing as they lacked historical data to go by. He got coworkers at Google ad senses team to use the Lean Startup. They scheduled office time to read the book, being helpful and not pushy. They also gave them a reading guide with questions. In the end they identified three hypotheses; put together templates like the partner problem hypothesis. People felt good about invalidating their own hypotheses as it saved them work that would’ve been wasted.
  •  Vox Media launched in 9 weeks using analytics to guide customer validation. Melissa Bell got her co founders and others from Vox media in the same room to get everyone on the same page about her vision. Many editorial staff came from Washington Post whereas Vox was an agile technology company. They used card stacks for flexibility. They had problems with the way editors used card stacks, as it was difficult to navigate-hence they analytics were used to solve the problem. Now has 22m users. Delivering content to users where they are-on social channels such as Facebook or YouTube instead of own URL.

Lean Startup- Dos & Don’ts

Max Ventilla

  • Pivoting statistics- 80% of failures didn’t pivot, 65% of successes Pivoted but 85% of Huge Successes (>$1B exit) didn’t pivot. Those didn’t pivot felt that evolution is safer than betting on intelligent design.
  • You need to eat your own dog food. Use your product to solve your own problems. If not you are at an enormous disadvantage.
  • Invert the org chart :customers & customer facing team should be on top. They should be heard and not told what to do.
  • Force yourself to pretend at the earliest possible moment what you want to be- to learn whether its worth being what you want to be. Landing pages, Concierge or Wizard of Oz are ways to pretend.
  • Don’t speed up for the sake of it. For startups not going fast enough is not the main risk. False summit is the reality. Journey of a startup is slow like that of a mountaineer. A new goal appears once you have reached what seemed like the ultimate goal.

Grace Ng

  • According to Grace Ng success criterion for any experiment is the weakest outcome that will give you enough confidence to move forward.
  • Testing the riskiest assumption on buy side in a two-sided market place could be deceptive in a sellers’ market. Sellers may not automatically follow even if you find many buyers.
  • Validated hypothesis doesn’t necessarily lead to a viable business. Grace Ng tested a hypothesis whether birdwatchers will post photos to ask questions. The Hypothesis was valid but the problem turned out to be too small – not a big pain-point.
  • Don’t validate the solution before validating the problem. As in the case above; the problem was not big enough though the solution was right.

Eric Ries

  • When it takes too long to learn as end results take time, use proxy metric like number of likes or start a cohort.
  • Don’t depend on one experiment to determine the product market fit. Keep testing and validating along the way as you grow. Growing too fast by taking product-market fit for granted is dangerous.
  • Don’t get misled by corporate America’s habit to underinvest or overinvest. “All Hands On Deck” sounds great but surely is a sign of overenthusiasm.
  • Avoid handing off innovation between silos. Handoffs kill innovation. What is learnt in one silo can’t be handed off to another silo.
  • Don’t add features for the sake of it. Its better to err on the side of being too minimal to get early feedback and learning. Its easy to add a missing feature later.
  • Pay more attention to paid users’ feedback than free users’ feedback. Free users ask for more; paid users ask for better.
  • Don’t use vanity metrics- Eric’s law: At any time no matter how badly you are doing there is at least one Google analytic graph that’s up into the right

Joanne Molesky

  • As you go through build-measure-learn cycles for product the same way you should be going thru build-measure-learn cycles for process compliance.
  • Beware of developers’ tendency to focus on how to do things than on outcomes. Developers tend to ignore security as they are dazzled by technology, so they focus on doing things faster-not safer. Security testing, threat model and risk metrics should be included right from the beginning and not at the end.

Dan Milstein

  • Don’t take idle pleasantries as positive feedback. People tend to be polite and cordial even though they are least interested.
  • Don’t choose to see what fits in a narrative that sounds good and makes you look awesome. That is self-deception. Realize that a startup is a series of unpleasant encounters with reality.
  • Don’t own a plan. Own questions. Plans will change.

Hiten Shah

  • Test small changes- Google sign on and changes to verbiage improved acquisition by 314% for KissMetrics.

Brant Cooper

  • Don’t as two questions that kill breakthrough innovation – what is the roi? When do we get it? In order to answer these questions we have to look at existing markets which kills innovation -innovator’s dilemma. We need to build cultures or safety net for innovators.


Most of the takeaways and dos and don’ts are common sense for any practicing entrepreneur. According to Eric Ries The Lean Startup process is more widely practiced than talked about. Most entrepreneurs are agents of long term change. They don’t think The Lean Startup is a big deal. As with most profound thoughts- it seems obvious after its well thought through, well organized and well presented.


Hiring 2.0- More Efficient and Effective

General frustration with the process of hiring.

In spite of wide application of technology in the process of hiring –with latest social media channels like Linked In automating the search and selection process ; there is widespread frustration with the delays, arbitrariness and ambiguity inherent to it.  There have been many attempts to automate the hiring process. Finally we have come to conclude that its impossible to remove the human element. Thus its hard to quickly scale the process of hiring to meet sudden rise in demand.

To add to the woe of the HR manager, its no longer enough to go through the resumes- there is lot more information available on social media channels. Importance of resumes is waning– postings in social media reveal what the resume does’t. Ensuring proper skill and capability fit is not enough. Hiring mistakes are common in companies where only job related skills, experience and qualification are evaluated while hiring. The selected candidate has to gel well with the team – for which we need to look at his social graph, likes and dislikes, hobbies, values and beliefs.

We need to look at new ways of staffing our teams – the traditional resume-interview-offer process is too slow and error prone. But before that lets see how the organizations themselves are evolving.

Changing face of the organization


As we saw in this earlier posting here – we need to implement cultural and structural changes – make organizations more cross-functional and starfishlike for social business to become a true engine of growth.

There are other reasons why organizational structures are challenged. With more VUCA business environment; organizations have to empower individuals to engage is smaller and more frequent transactions using their own smart devices. The BYOD movement is catching up and organizational structures are changing in tandem. In an earlier article we saw how BYOD is impacting Organizational Structure. Consumerization of the enterprise and bottom up transformation is driving corporates to structure themselves like communities.

Software product companies building products in an uncertain environment  – no matter how big or small are being run like startups. The Lean Startup approach is increasingly used by established organizations like Intuit. Intuit calls itself a startup that is 30 years old. Their cross functional teams can include anyone including HR and Legal professionals. As we all know small, cross-functional teams are the primary building blocks of Lean and Agile organizations.

We are also seeing more and more teams are geographically distributed. Automattic – the organization behind WordPress a few months back had 230+ employees distributed across 190 locations. They work from home. Geographical location really doesn’t matter. Geography is History.


Hiring 2.0

Let us examine the changing face of hiring in the new age enterprises

  • In some reputed companies most hiring happens at the entry level. These companies have been developing good relations with educational institutes from where they hire summer interns- who are often hired by executives who are most likely alumni of the same institute. These interns under the guidance of their seniors work on interesting projects that test technical skills and ability to be a team player. This mentorship chain dates back to several years.
  • Many companies prefer outsourcing to hiring their own teams. Jobs needing skills that aren’t  the “Core Competencies” can be outsourced to derive economic benefits.


  • Using social media to generate dialogue and engagement – both internal and external are important for getting talented people interested in your company. Enterprises are 3 to 4 years behind individuals in their adoption of social media tools –but the trend is positive and they are catching up. My company actively promotes talent communities such as Technext  or Google Developers Group. Many companies use talent communities to attract talent. Organizations have to consciously make their projects interesting.
  • Open Source software projects exemplify how communities demand by publishing their wish-lists. Success of the product is easily measured by the number of downloads and usage analytics. Communities of users also act as customer support teams by posting commonly faced problems and solutions on platforms like StackOverflow. Developers from the community who want to add new features are allowed to commit if their code is good-  a lightweight hiring process. Anyone is free to fork out his own project and hope to attract users to her own flavor of the product.
  • Every applicant is hired on trial in Automattic. This makes the initial screening and interviewing less tenuous. She is assigned a new guide every week and she is required to provide customer support by closing as many tickets as she can. She is evaluated on quality and quantity of work before she is hired.
  • Recently I attended the startup weekend – in which teams got formed as aspirants attracted members by pitching their ideas on Friday night. It’s a self selection process where members join startups that are appealing and where they can add most value. If the startup manages to get the right team their chances of success are higher and vice versa. More and more companies like Intuit are giving freedom to employees to solve business problems by forming their own teams that experiment to validate hypotheses – exactly as in The Lean Startup Model. My company has promoted an incubator where young and energetic teams work on cutting edge technologies to build their own product ideas. Those who fail are likely to get hired as employees – this provides a safety net that budding entrepreneurs need to improve their ability to take risks. Teams that come together by a self-selection process seem to be the future of hiring that can be effective and scalable.

If Its Painful Do It More Often!


“Do more painful things more often!” – In the Agile 2013 conference keynote at Nashville Gene Kim quoted Adrian Cockroft (Architect at Netflix) which got me thinking how we see it being applied not only in software development and delivery but also in diverse fields like inventory management and personal fitness. Here are some examples.

Continuous Integration: Development teams would independently build software components which were integrated in the end to work together.  Integration used to be a painful process given mismatch of assumptions, versions and expectations. Now we have CI tools like Jenkins, Bamboo and Cruise Control which execute automated steps that integrate at short intervals of a few minutes – this flags out the mismatches listed above early. This reduces the surprises and pain faced at the end of the iteration.

Continuous Delivery: Releasing a software product in the production environment is a nightmare for development and operations teams often requiring all team members to stay available over weekends to take care of any last minute snafus. Managers tend to delay release dates to the last acceptable date . Continuous delivery preceded by continuous deployment is done by running automation scripts  to deploy and test the application in development and test environments. This is done several times before final deployment in production. This re-use increases the confidence level in these scripts are run to deploy in production.

The Lean Startup Way: Launching a business venture is not easy to say the least. Entrepreneurs tend to spend time perfecting their dream before taking it to the market. Eric Ries acknowledges that business models are likely to fail. He advises businesses to go to market at shorter intervals – thus allowing them to learn from their early failures than getting cornered in a situation where there is no time left to fail and learn. This model places high value on learning from early failure and frequent build-measure-learn cycles.

Chirunning : This technique applies principles of Tai Chi to running. Formalized in a book by Danny Dreyer – one lesson of this technique advises runners to take shorter steps more frequently to reduce the pain and injuries caused by over pronating – or taking long strides.

Cycle Counting: Counting items in inventory at the end of the financial year used to be a painful exercise riddled with inaccuracies that could not be traced back to errors accumulated over the year. Most warehouse management software provide a solution by putting cycle counting in place. Warehouse managers don’t wait for the year end ; but cordon off areas of the ware house every week to physically count all items in that area and reconcile differences with book inventory. This flags out errors early – closer to their occurrence  – thus improving traceability.

There must be many more examples where an arduous , painful and inefficient task which often got postponed because of its nature is  now being completed more efficiently and with less pain as its being done more frequently.

Here are some candidate tasks; which we should consider doing more frequently.

1)   Employee Appraisals: Usually an annual exercise – dreaded by appraisers and employees. Would quarterly frequency reduce the pain?

2)   Elections: It’s the most painful part of democracy. Can we make elected representatives more accountable by having them go back to their electorates more frequently?

3)   Census: So much data is collected that it becomes irrelevant by the time its processed . Can we have an ongoing survey to account for births, deaths , relocations and other events that impact demographics?

4)   Tax returns: Year-end is busy time for accountants. They stay late to file returns as the last date approaches. Often accountants are required to request date extensions to complete their work. Can we have a financial period shorter than one year to make our lives easier?