Lesson 5 of 5

Metrics, experiments and the PM interview

3 min read

Metrics frameworks

AARRR (pirate metrics):

Acquisition: users arriving (visits, sign-ups, CAC).
Activation: the first "aha" moment (profile completed, first application sent).
Retention: users coming back (D1/D7/D30 retention, cohort curves).
Revenue: ARPU, conversion to paid, LTV.
Referral: invites, viral coefficient.

HEART (Happiness, Engagement, Adoption, Retention, Task success) for user experience.

Metric types: north star, input metrics (drivers), guardrails (must not get worse: latency, complaints, unsubscribes). Watch out for vanity metrics (total sign-ups, page views) that look good but don't reflect value.

A/B testing for PMs

Define the hypothesis and primary metric up front, calculate sample size, randomise, run for full weekly cycles, avoid peeking, check guardrails and segment effects, and decide based on statistical and practical significance. Not everything needs an A/B test: big strategic bets, low-traffic products and obvious fixes may not.

Common interview question types

Product sense / design: "Design a product for X" or "How would you improve Y?" Clarify the goal → pick a user segment → list their pain points → prioritise one → brainstorm solutions → choose and detail one (MVP) → define success metrics → mention risks and trade-offs.
Metrics / analytical: "How would you measure the success of feature X?" or "Metric Y dropped 15%. Why?" For drops: check data issues, time frame, external factors, segment (platform, region, user type, version), funnel steps, recent changes; form and test hypotheses.
Estimation: "How many UPI transactions happen in India per day?" Structured assumptions and sanity checks.
Strategy: "Should we enter market X?" or "How would you compete with Y?"
Behavioural: leadership without authority, conflict, failure, prioritisation (STAR format).
Technical (for some roles): how the internet works, APIs, system basics.

Example: improve a job platform for freshers

"Goal: more freshers getting hired. Segment: final-year students from tier-2/3 colleges. Pain points: no work experience to show, applications ignored, unclear status. Prioritise 'no proof of skills' because it blocks everything else. Solution: short paid-for-by-employer skill tasks with scored feedback that become verified profile badges. MVP: 20 tasks in 3 hiring areas with partner companies. Metrics: task completion rate, interview invites per candidate (primary), time to first interview; guardrail: application quality ratings from companies. Risks: task fatigue, cheating; mitigations: time-boxed tasks, plagiarism checks."

Build your PM profile

Write product teardowns and improvement proposals for apps you use, run a small side project end to end (even a no-code product), learn SQL and analytics, and practise structured thinking out loud. Communication and clarity are visible in every interview answer.

Check yourself

Define activation for a resume-builder app.
DAU fell 10% yesterday. List your first five checks.
Answer: "How would you improve WhatsApp for small business owners?"

← Previous