How do you collect product feedback that isn't noise?

The core problem with product feedback isn't volume — it's that most of it reflects what users think they want, not what they actually need. Signal lives in behavior, pain, and patterns across many conversations; noise lives in feature requests, one-off complaints, and survey responses. The discipline is learning to separate the two before you write a line of code.

Listen to the pain, not the feature request

When someone tells you to 'add a daily briefing,' they are handing you a solution. What you actually need is the problem underneath it — the reason they want a daily briefing at all, what they're currently doing instead, and what breaks when they don't have it. The feature request is noise; the pain behind it is signal.

Garry Tan's gstack project illustrates this vividly. The design philosophy is explicit: the system is meant to hear the pain a user is describing, not the feature they named. A user says 'daily briefing app' and the system surfaces something bigger — because it was trained to work backward from frustration, not forward from requests.

You can apply this in every user interview with one simple habit: never write down what someone asked for; write down what made them ask for it. What task failed? What did they have to do manually? What was embarrassing, slow, or broken? Those answers are durable. The feature request they derived from those answers is almost always wrong.

Filter by behavior, not opinion

Opinions are cheap and plentiful. Behavior is expensive and rare — which is exactly why it's more valuable. A user who tells you the product is great while churning is giving you noise. A user who says nothing but logs in every day and invites three colleagues is giving you signal. The feedback that matters most is revealed, not stated.

This is why usage data should sit next to qualitative feedback, not replace it. When you see a drop-off at a specific step in your funnel, that's a question, not an answer — but it's a much sharper question than anything you'd get from a survey. Go find the users who dropped off and ask what happened at that exact moment. Now you have a specific, anchored conversation instead of a vague one about what they'd hypothetically prefer.

The bias toward opinion is understandable: it's faster to run a survey than to watch someone use your product. But watch ten users actually use what you built and you will learn more in an afternoon than you'd extract from a hundred responses to a satisfaction poll. What users do under real conditions — what they click, where they hesitate, what they ignore — is the most honest feedback channel you have.

Talk to the right people, not the most available ones

One of the most common ways founders collect noise is by only talking to people who are easy to reach — warm network contacts, early adopters who like you personally, or whoever responds to a mass email. These samples are systematically biased. The people most willing to give you feedback are often the people least representative of your actual market.

Paul Graham's observation about natural ability being unevenly distributed — explored across several essays — maps cleanly to this problem in a useful way: the insight you need is also unevenly distributed across your user base. It isn't spread evenly across everyone who signed up. A small number of users understand your product's potential deeply and can articulate what's missing with precision. Find those people and spend disproportionate time with them.

You can identify them by behavior: who uses the product most frequently, who has figured out use cases you didn't design for, who recommended it to others unprompted? These power users have thought harder about your product than anyone else. Their feedback isn't random; it's earned through repeated contact with your actual limitations. That makes it signal.

Ship narrow, learn fast, and treat real usage as the feedback loop

One of the most underrated feedback mechanisms is shipping something small enough that its failure is informative. A feature that doesn't get used tells you something precise. An MVP that gets used in unexpected ways tells you even more. But a full product built on feedback gathered before launch tells you almost nothing — because the users were imagining, not experiencing.

Garry Tan's gstack workflow makes this concrete: the recommendation after analyzing a product idea is always to ship the narrowest wedge possible tomorrow and learn from real usage, not to build the full vision based on your current assumptions. The full vision might be right, but you can't confirm it without contact with actual behavior.

This reframes feedback collection entirely. Instead of trying to gather enough feedback before you build to make sure you build the right thing, you treat each small release as a feedback instrument. You're not building features; you're running experiments. The question isn't 'what do users want?' — it's 'what does this usage pattern tell me about what they actually need?' That's a question you can answer. The former is often unanswerable until after you've shipped.

Create conditions where honest feedback is possible

Most product feedback is polite. Users don't want to hurt your feelings, especially if you're the founder and you're clearly excited. This means the conditions you create for feedback conversations dramatically shape what you hear. An interview that starts with 'let me show you what we built' primes the user to respond to your framing. An interview that starts with 'tell me about the last time you struggled with X' puts the user's experience first.

The same principle applies to written feedback mechanisms. Feedback forms that ask 'how would you rate your experience?' generate noise. Feedback triggered immediately after a user abandons a flow — asking specifically what didn't work — generates signal. The context of the request shapes the quality of the answer.

Finally, pay close attention to what people don't say. Silence in a user interview — a pause before answering, a hesitation, a change of subject — often carries more information than the words that follow. Users who are working around your product rather than using it as designed are giving you feedback with their workarounds. The workaround is the feature request they decided not to bother making because they'd given up on you building it. That's worth more than anything they'd tell you in a survey.

The one thing to do

After your next user conversation, write down only what was broken or painful — not what they asked you to build — and let that become your next prioritization input.

Frequently asked questions

How many user interviews do you need before the feedback becomes reliable?

Most founders stop too early or too late. Five to eight deep interviews with users who fit your target persona will surface the same core problems repeatedly — that repetition is your signal. Beyond that, you're usually getting diminishing returns until you ship something and collect behavioral data.

Should you trust what users say or what they do?

Prioritize behavior, but don't ignore stated feedback entirely. What users do reveals current reality; what they say can hint at what they'd do in conditions that don't yet exist. The most useful feedback combines both: a specific behavior you observed plus an explanation of why it happened.

What do you do when you get contradictory feedback from different users?

Contradiction is a segmentation signal, not a problem. Different users want different things because they're different customers. The question is which segment you're building for — and contradictory feedback forces you to answer it explicitly rather than trying to average your way to a product nobody loves.

How do you handle feedback from users who want features outside your core focus?

Log it, thank them, and don't build it yet. Feature requests outside your core are noise until you've nailed the core. The exception: if three or more unconnected users independently ask for the same thing in the same words, that pattern is worth a closer look at whether your core is actually narrower than it needs to be.

Sources

More playbook answers · Growth Prophet home