Market Research and Customer Insights Questions

Direct answer

I would treat the raw survey as an upper bound for early adopters, not as a read on the whole market. I would measure how skewed the sample is, reweight toward the real population where I have enough mainstream respondents, and replace stated willingness to pay (WTP, what people say they would pay) with behaviour where I can. Then I would give stakeholders a range with a plain statement of what it still cannot tell us.

Correcting the skew

1. Quantify it. Use screener questions (how quickly do you try new tools, where were you recruited) to split respondents into early adopters (people who try new products soon after launch) and mainstream.
2. Reweight (post-stratification). Give each group a weight equal to its share of the real population, taken from an external source or a stated assumption, instead of its share of the sample.
3. Respect small groups. If few mainstream respondents exist, weighting amplifies their noise, so report the uncertainty and collect more of them.
4. Discount stated WTP. People overstate what they will pay. Anchor with behaviour: what they pay today for substitutes, price-page clicks, pre-orders, deposits.

Worked example (illustrative numbers)

200 respondents: 140 early adopters (70% of sample) and 60 mainstream (30%). Assume outside adoption data says early adopters are 15% of the real market and mainstream 85%. Share saying they would pay $20 or more: early adopters 45% (63 of 140), mainstream 15% (9 of 60).

• Raw sample: (63 + 9) / 200 = 36%.
• Reweighted: 0.15 x 45% + 0.85 x 15% = 6.75% + 12.75% = 19.5%.

The raw figure nearly doubles the reweighted one.

Where the plus or minus comes from. The mainstream estimate rests on only 60 people, so chance alone moves it. A 95% margin of error means: if you repeated the survey many times with fresh random samples, about 19 times in 20 the true share would land within that margin of the sample result. The usual approximation for a share is:

margin = 1.96 x square root( p x (1 - p) / n )
       = 1.96 x square root( 0.15 x 0.85 / 60 )
       = 1.96 x 0.0461 = 0.090, about 9 points

So the mainstream share is 15% plus or minus 9 points, an interval (plausible range) of 6% to 24%. It is optimistic, because the sample was not random.

Carrying the interval to the whole market. Hold the early-adopter term fixed at 0.15 x 45% = 6.75%, and swap in each end of the mainstream range:

• Low end: 6.75% + 0.85 x 6% = 6.75% + 5.10% = 11.85%, about 12%.
• High end: 6.75% + 0.85 x 24% = 6.75% + 20.40% = 27.15%, about 27%.

Communicating what remains uncertain

Present "about 20%, plausible range 12% to 27%, and since this is stated intent (what people say they would do, not what they do) it is likely still high" and say which assumption drives it (the 15/85 split). Say what the survey cannot show: whether anyone actually pays.

What to do after the recommendation

• Extra data: a fresh mainstream sample, for instance 300 respondents from a panel recruited without a novelty screen. If it again gives 15%, the margin shrinks to 1.96 x square root(0.15 x 0.85 / 300) = about 4 points, so the interval narrows to about 11% to 19%.
• Cheap experiments: a priced fake-door page (a sign-up page for a product that does not exist yet), pre-orders, a paid pilot with a few accounts.
• Decision rules, set before the data arrives. Break-even is the point where revenue covers running costs. Illustrative derivation: the product costs $240,000 a year to run and sells at $20 a month ($240 a year), so it needs 240,000 / 240 = 1,000 paying customers. If the reachable mainstream market is 10,000 people, that is 10% of them. Proceed if the interval's lower bound (its bottom end) is at least 10%. Modify (re-price, target early adopters first, change packaging) if the interval straddles 10%, meaning 10% sits inside it. Pause if the upper bound (its top end) is below 10%.

Which interval the rule is applied to. The 10% break-even is a share of the 10,000 reachable mainstream people, so the rule must be applied to the mainstream interval (6% to 24%, from the 60 mainstream respondents), not to the blended 12% to 27%, which includes the early-adopter term. On the mainstream interval, 10% sits inside the range, so the call is "modify", not "proceed". The blended range also understates the uncertainty, because it holds the early-adopter share fixed at 45%: that figure rests on 140 people and carries its own margin of about 8 points (1.96 x square root(0.45 x 0.55 / 140) = 0.082). Letting both groups vary gives about 11% to 28% for the blend. Finally, every one of these shares is stated intent that the answer itself calls likely too high, so before comparing with a break-even of paying customers, apply a stated discount (for example, assume only a fraction of "would pay" respondents actually buy, and label that fraction as an assumption). The survey also did not say whether "$20 or more" meant per month, so confirm the price period before using the $240 a year figure.

Pitfalls

Weights fix who answered, not whether stated intent turns into purchases. Fixing one never replaces the other.

Direct answer

In 30 days I would test one question: will US parents of high-school students pay for a math tutoring app when free alternatives exist? I would set stop and go thresholds before collecting data, start with the cheapest tests of the riskiest assumption (willingness to pay), and use interviews and a pilot to learn why. Thirty days can catch a clear no or a promising lead. It cannot give a precise demand figure.

Desk research means reading published material (education data, competitor listings, reviews) instead of collecting new data. Qualified visitors are visitors who match the target buyer, here parents of grades 9-12 reaching the page from a targeted ad. Unit economics means what one paying family earns after the costs of serving and winning them.

Who matters. The student uses the app, a parent usually pays, and a school or teacher might buy. I start with parents as payers because that is the riskiest link.

Plan (four phases)

The first test is phase 2 and phase 3 together, because they are cheap and aim at willingness to pay.

Illustrative stop/go thresholds (set before the work; adjust to the unit economics)

• Go: at least 6 of 12 parents describe paying or trying a paid math aid in the past year; fake-door sign-up rate at or above 4% of qualified visitors at the shown price; at least 3 of 6 pilot families commit to continue.
• Stop: fewer than 3 of 12 parents have spent anything, and sign-up under 1%.
• In between: change the buyer (for example teachers or schools) or the problem (test prep) and retest.

Where the thresholds come from (illustrative inputs). Start from unit economics, not from a favourite number:

• Price $12 a month, a family stays about 6 months, so a paying family is worth about $72 (the margin after costs would reduce this; use your own).
• We will spend up to about $30 (roughly 40% of that) to win one paying family.
• Ad budget $600 at $0.60 per qualified visitor buys 1,000 visitors.
• If half of sign-ups pay (the 3 of 6 pilot line), cost per paying family = $0.60 / (sign-up rate x 0.5). At 4%: 0.60 / 0.02 = $30, exactly affordable. At 1%: 0.60 / 0.005 = $120, four times too much.

So 4% sign-up is go (40 sign-ups from 1,000 visitors), and under 1% (fewer than 10) is stop. The interview lines are a judgement: a clear majority (6 of 12) already paying for help, versus fewer than a quarter (under 3 of 12, since 3 of 12 is exactly a quarter and sits in the in-between zone) is a gap too wide for 12 conversations to blur. To set your own, write down the price, the lifetime, and what you can spend to win a customer, then work backwards the same way.

With 12 interviews these are directional. They are built to catch a clear no, not to estimate a percentage.

Things a strong plan accounts for

• Seasonality: demand rises with the school year and exams, so say when the 30 days fall.
• Minors: get parental consent for anyone under 18 and interview parents first.
• Free alternatives: ask what parents use now and why it falls short, because that is your competitor.

Pitfalls

Counting friends and family, measuring praise instead of payment, and moving the thresholds after seeing results.

Direct answer

I would not pick a winner. A large analyst number and a weak survey can both be true (willingness to pay is the most a buyer says they would spend; purchase intent is how likely they say they are to buy): the analyst figure usually measures what the whole category spends, while the survey measures whether your target buyers would buy your product. I would first diagnose each source, then test the gap with evidence of real behaviour, and I would agree the postpone triggers before collecting any new data.

Step 1: Interrogate both signals

• Analyst report: what exactly does the $2B include (which buyers, regions, services plus software)? Is it measured spend or a forecast? When was it published, who commissioned it, and can the method be traced?
• Survey: who answered (recruited from a general panel, meaning a paid pool of anyone willing to take surveys, or people who own this budget)? How many? Did the question describe the product concretely? Stated purchase intent (what people say they would do) tends to differ from what they do, so treat it as a rough signal, not a measurement.

Step 2: Reconcile

Re-cut the survey to the people who actually match your buyer. If 120 people answered but only 30 own the budget, that is 25 percent of the sample, and the weak result may come mostly from the other 90. Then compare definitions by narrowing the headline to the part you could serve. Illustrative numbers: of the $2B, perhaps only 40 percent is software (the other 60 percent, $1.2B, is services you would not sell), leaving $800M; if only 30 percent of that is in the regions and company sizes you target, the serviceable market is about $240M (2,000 x 0.4 x 0.3). A $240M market with a weak re-cut survey is a very different bet from a $2B market, and the two sources now disagree far less.

Step 3: Triangulate with behaviour-based research

Triangulation means checking a conclusion against independent sources that fail in different ways.

1. Interview buyers about what they pay today and what workarounds (manual or makeshift ways they solve the problem now) they run (current spend is stronger evidence than a future promise).
2. Expert calls with people who sold or bought in this category to learn why deals stall.
3. Public filings of incumbents (the established companies already selling in the space) to see whether revenue in your segment matches the analyst story.
4. A small commitment test: ask about 10 qualified budget owners for a paid pilot (a small, time-limited paid trial), deposit or signed letter of intent (a written, non-binding statement that they intend to buy under agreed terms).

Postpone triggers (set in advance, illustrative thresholds)

Postpone building if (a) the re-cut survey of true budget owners is still weak, (b) fewer than 2 of those 10 commit to a paid pilot, and (c) buyers describe their current workaround as good enough. The 2-of-10 line (20 percent) is a judgment call, not a benchmark: it is a placeholder chosen so that a result clearly below it would worry me, because if few warm, qualified prospects commit money, the sales funnel probably cannot reach the customer count the business needs. I would postpone only when all three conditions hold together; if only one or two hold, I would run one more targeted test (for example a different segment or price) before deciding. Adapt it by working backward from how many customers you need and what share of conversations you can realistically convert. Ten people is directional, so the thresholds decide whether to keep investing, not whether the market exists.

Trade-offs and pitfalls

• Dismissing the survey because the analyst number is bigger, or the reverse, is the common error.
• A low result from the wrong respondents proves nothing about the right ones.
• Deciding the trigger after seeing the data invites rationalisation.

Direct answer

Before building a minimum viable product (MVP, the smallest version that lets you test the idea with real users) for freelancers tracking expenses, I would answer six market questions. Questions about the outside world mostly come from existing sources (also called secondary research: material someone else already published); questions about freelancers' own habits and urgency need new primary work (research you run yourself, such as interviews or a survey). The most important one is whether enough freelancers hurt enough to change what they do today.

The six questions

What to do first

Start with questions 5 and 4 because they take hours and can kill the idea, then do question 1 for scale. Spend the new effort on 2, 3 and 6: a handful of conversations with freelancers across different fields (designers, consultants, drivers) would show whether the pain is shared.

Trade-offs and pitfalls

• Interviewing freelancers before reading about competitors wastes their time.
• Asking people if they would use it invites polite yes answers; ask about the past instead. For example: "Tell me about the last time you had to find a receipt for something you paid for months ago. What did you do, and how long did it take?"
• Rank the questions: commit to the one whose answer would change whether you build.

Direct answer

Analyst reports usually describe the companies that are still standing and still talking to analysts. I would first rebuild the full list of companies that entered the market, compare it with the list the report covers, and then restate the conclusion as a statement about all entrants, not only the winners.

Two biases are in play and they are different. Survivorship bias means the failures have dropped out of the sample, so what remains looks better than the market really was. Reporting bias means the companies that are present report their good news more readily than their bad news, so even the visible data is tilted.

How to check for each

1. Rebuild the denominator (the full population the report should have drawn from). Pick a time window, then assemble every entrant from sources that do not depend on the report: funding databases, company registries, app-store history, archived versions of websites, trade-show exhibitor lists from past years.
2. Find out what happened to the missing names. Closed, acquired in distress, pivoted, or still alive but small. Post-mortems, shutdown notices and former employees are the evidence.
3. Read the inclusion rules. Who chose the companies, and who paid? Rankings often include only vendors that met a revenue floor or took part in a briefing, which filters out small and failed players by design.
4. Speak to people on the losing side (customers who churned, ex-staff of failed firms, via expert calls) so the story is not told only by winners.
5. Look at the metric, not only the sample. Growth quoted as an average of survivors, or "customers" counted as logos signed rather than still active, flatters the picture even with a clean list.

Worked example (illustrative numbers)

A report profiles 12 meal-kit companies, all growing. A funding database and a registry show 40 companies entered the market in the same window. So the report covers 12 of 40, which is 30%, and 28 are missing. Checking the 28: 21 have shut down and 7 still operate but are small (21 + 7 = 28). Known exits are therefore 21 of 40, or 52.5%.

The report's headline, "the typical player grows 25% a year", describes only the 30% of entrants it chose to profile, a group selected because it is still visible and doing well. A defensible restatement: "Among companies that survived and were profiled, growth was strong, but about half of all entrants exited, so a new entrant should plan around a survival rate below 50% (19 of 40 entrants, or 47.5%, are still operating, and 7 of those 19 are small), with the growth figure as an upside case."

Adjusting the conclusion

• Change the claim from "what winners do" to "what separates survivors from exits", comparing the two groups on the same traits (channel, pricing, funding).
• Widen ranges and lower confidence on any number that comes only from the report.
• Keep the report as one input and say plainly what it cannot show.

Trade-offs and pitfalls

• The reconstructed list is itself incomplete (databases miss bootstrapped companies), so say which direction its error runs.
• Do not swing to the opposite error and dismiss the report. It is still good evidence about survivors.
• A company that vanishes from a list may have been acquired successfully, so classify each exit before calling it a failure.