Incorporating user feedback in an early prototype

We use user feedback to figure out what to do next.

The type of feedback you get depends on who you ask and where you ask for the feedback. Each type of feedback is important and should be weighted equally for prioritizing features

Observation
Watching people use your application or a particular feature for the first time helps you find user interface bugs and instances where the mental model that people have of your application differs from the designer's mental model (e.g., where users expect buttons to be located and what they expect those buttons to do)

Email
Email comments are typically a stream of consciousness -- what people say is almost always true and honest but not always literally what they mean. For example, when someone says, "I want video" there are many different ways to apply this feedback. Once you hear similar feedback more than once, you need to increase the priority of the feature and experiment until you stop hearing the feedback. It helps to re-read the comments every couple of weeks because they yield different insights when you are further along in the product and you can cross things off the list :-)

Phone
Phone conversations tend to start directed and then transition to hypotheticals/options. People wait for you to ask them a question and then respond with their gut feeling. "What did you think of the new rankings? Would you prefer fun/creative badges or more straightforward?" You can also get insight from people's intonation (e.g., are people excited about a feature or reluctant to discuss it because they don't want to hurt your feelings?)

Off hand conversations (coffee shops and Gchat)
These can be baselining -- someone mentioning a powerful concept or technical strategy that you may have overlooked entirely (e.g., "Did you know that Google App Engine supports the fan out problem out of the box? You should use the Adobe AIR for your desktop application"). These conversations can also be visionary statements that just make sense as a future goal but would never to occur to you as a designer because you are too deep in your current version (e.g., "You should make this real time")

Stats
Recording feature level stats (e.g., searches, plays) help to validate whether people are using a feature and how often. You have to be careful to record and analyze features that you can act on rather than vanity metrics which don't tell you much (e.g., page views)

Surveys,A/B testing and other mass data collection techniques only start to make sense when you have a large batch of users and you are very clear on what you are testing. In an early prototype, you have a vision but you end up throwing a bunch of features at the wall and hoping that something sticks