Recently Shopify launched its new forums. I played a pretty big role in designing the site and was anxious to see the response from the community.
Old:
New:
The initial response was great, everyone in the office thought the design was good and the community liked the refresh. After a week the “real” feedback started to come in, most of it being small bugs I missed. However some of the forum regulars began to discuss their distaste for the new wysiwyg editor (before we just had a manual text editor).
This made me realize that I don’t have a methodology in place to interpret feedback. After researching how other designers deal with feedback I came across a good presentation by ex-Digg UI designer Daniel Burka on how Digg interprets feedback.
To sum it up his talk: Daniel gives his 5 types of feedback (positive, bugs, negative, expert, implicit). He then discusses how he reacts to feedback:
- Don’t do anything – don’t fix it just yet you don’t have the full picture (wait 48 hours to a week)
- Identify themes and strong ideas – go look at what people are actually saying
- Engage your community – talk and interact with your community
- Iterate – ask yourself: is this a good way to respond to the feedback? What negative effects could our response to the feedback have?
After watching his talk and reading more about it. I came to these conclusions regarding the new forums
- I wasn’t ready to interpret any kind of implicit feedback and I couldn’t answer: How do I measure how users that aren’t registered or don’t post often react to the new forums?
- When I redesigned the forum I didn’t have any goals/metrics in mind other than “Give it a fresh new look and people will use it more”. I should have been ready to track how many new users were registering per week on the new forums vs. the old forums. On top of that I don’t think I explored the question “What do I want to achieve with these new forums” deeply enough.
- It might have been a good idea to involve the community during the redesign of the forums, but this can be a recipe for disaster. People will be offended if I don’t take their suggestions and I feel obligated to respond to all feedback since I asked for it in the first place. I did contact the forum’s most active members and ask for their opinion after launch and their feedback was very useful.
Overall I think the forums were a success, but there is a lot of headroom for improving my work flow especially in regards to anticipating and interpreting user feedback. If you want to read some more on interpreting feedback look up Mark Boulton, he gave an excellent account on how his design by committee approach is working with Drupal.