The testing mindset - part 1

Hard things first

All the failures can be traced back to fuzzy & undisciplined thinking.

Fear is the obliteration of progress - Dune

The hard things

• if you don’t face the hard problems at first ; then the problem which waits you will be more rigourous then the hard problem - chaos theory

• you should measure your progress my number of hard problems you solve - rest all of the measures of progress are vanity metrics

• use the hard facts to your profits, use them to your benefits

Dimensions of hard problems :

1. they are hard to take & accept

2. they are based on hard objective data

3. they are hard in sense of being solid, unyielding & permanent - nothing can change them

Failure is most likely the outcome

Most new products will fail in the market even if they are competently executed

The equation of success is equation of multiplication not addition. You get one thing wrong and the whole equation sums up to zero. Still we are attracted to success because we have deep human rooted desire to be associated with success

\(\text{{Success}} = \prod_{i=1}^{n} \text{{Factor}}_i \)

Why do we fail ?

Failure is due to Launch, Operations Or Premise

1. Launch : You cannot get the word out to your target audience - be very clear of your target audience to avoid this

2. Operations : If the implementation is not good, they word will get out and the product will fail - success is harder than failure.

3. Premise : People just don’t care about what you have built

Most of the product that is being fails because of premise - people just don’t care about your product.

Lalala Land - Killer of ideas

In business there are no good ideas or bad ideas ; there are only ideas which succeed in the market or fail in the market

We need to figure out the right it : idea for a new product that, if competently executed will succeed in the market.

most of the market research for the product which is done is done in the fictional environment - the lalala land

The lalala land gives us opinions which are subjective & biased judgement. We all know that people might say somethings and do somethings, so one cannot rely on opinions while doing market research.

Why we shouldn’t be in lala Land?

1. Lost in translation problem : Your idea is just an abstraction, until it is translated into an actual product, so the interpretation of the idea is going to be distracted by the personal beliefs, preferences & prejudices.

2. Prediction problem : people are notoriously bad at predicting whether they would actually want or like something.

3. Skin in the game : Most people we survey don’t have any vested interest in an outcome - they don’t have something to gain or lose

4. Confirmation Bias : Most people don’t like their beliefs to be challenged, let alone totally proven wrong, hence people find it very difficult to see things the other way

Instead of dependable, objective & actionable data. Lala land gives us subjective, biased, misguided & misleading opinions which kills our product in the market.

Killers of ideas :

1. False positives : You go to the market and ask people their opinions and they say that “if you build it i will use it” - you leave your job and build the product only to find out they didn’t use it.

2. False negatives : You go to the market and people tell you “this is not going to work” - you drop your idea, only to find out someone else built it and it became insanely popular.

These might be the exteremes but always remember even expert opinions are just opinions, you need to get actual data to guide your decision making.

Fear is the obliteration of progress - if you fear it, you won’t be able to do it.

Remember your goal should not be success but to quickly learn from your failures, so that eventually your learnings will stack up to consistent success

Escaping Lala Land

We want to stay in the LaLa Land because it is easy to stay there, you don’t have to face any hard problems & consistently solve them. The metric you use to measure anything should be put in question first because that determines whether you succeed or not. We need to always look at outcome not the output, because outcome is the only thing which matters. To escape Lala land you need to clearly define and measure your metric.

Outcome vs output : define your metric of measurement clearly.

“Data beats opinions” is one of the core operating principles of google. If one couldn’t support their opinions with data then you won’t be able to communicate well - thus creating a plethora of problems.

Characterists of good data :

1. Freshness : Data has to be fresh - in the constant changing tech world, if you data is not fresh it might not be relevant. Throw it if it is not fresh

2. Strong relevance : Data must be directly applicable to the statement you want to put in test

3. Known provenance : Make sure you know where you data comes from and how it was collected and filtered - trustworthy source

4. Statistical significance : Data must be based on sufficiently large samples to ensure that results cannot be attributed to chance

You need to collect your own data to prove the hypothesis under consideration, it is very tempting to use other peoples data to prove a point because the efforts you take is quite less. Biggest concern is that other people data may not be applicable to your circumstances.

It is extremely unlikely that you came up with this idea first.

Collect your own data - it will be hard but it will teach you a lot about your target audience, give you clarity further and it will be directly applicable to your circumstances. To collect your own data is often more faster, cheaper & easier just we need to know the right principles, tools & methods for the same.

---

Note : this is the first part to the series, in the next part we will be looking at the right principles, frameworks, tools & methologies to bring clarity to your idea and test it in the market.