User testing on a tiny shoestring...

[Edwin]

I wanted to pass on a tip about using http://www.feedbackarmy.com/ to solicit user feedback about a web page.

What it basically does is put a friendly front-end on the feedback request then farms it out to Mechanical Turk and collates the responses.

It's the kind of thing you COULD do manually (I suppose) which is a point they even make themselves on one of their FAQ pages. But really, why would you when the service keeps everything so simple and is priced so cheaply (10 reviews for $20 or 25 reviews for $40)?

Actually, despite the mechanics, people write surprisingly REAL stuff. I've had to manually disapprove a couple of commenters (one click away on the response page) but in general they're thoughtful, helpful and the commentary - because it's coming from disinterested people far away from the original material - is worth its weight in gold. You have to be fair - it's not OK to disapprove a comment just because you don't like what they said - only if the person clearly took no time at all to actually look at the page!

It's important to keep your expectations manageable. Ask a small number of questions, keep it simple, and make sure they're not the kind of thing that can just be answered by a yes/no.

Normally the feedback is private, but I'll share the URL to one survey I did here so that you can see exactly what I mean. In this case, I did 2 sets of analysis on the same URL (the newer responses are in the top block and are on the site after I revised it based on the comments in the lower block)
http://www.feedbackarmy.com/get_feed...1eaa51f2933468

IMPORTANT: It's probably better not to be greedy like I was and get 2 lots of 25 responses on the same URL. It looks like the pool of "interested people" at Mechanical Turk isn't that large (there are eligibility criteria, it's not open to everyone) and I did get the sense that a couple of the people may have answered both my old and my new survey request!

IDEA: Not done this yet, but this could be a cheap way to analyse a competitor or to come up with a better design for a new website designed to compete head on with websiteX, websiteY and websiteZ. Spend $20 a time on each of the competitors and get a range of feedback on what they're doing well, what they're doing badly and what they're missing out on entirely...

[Sound]

interested to know why you chose ContentManagementSystem as the domain for the experiment?

[atlas]

That's a great website Edwin - thanks!

Some fancier usability services:
http://whatusersdo.com/ (fairly UK oriented)
http://www.usertesting.com/
http://www.gazehawk.com/ (eye tracking technology)
http://www.conceptfeedback.com/

Some related tools:
http://silverbackapp.com/ - an in house usability testing tool
http://www.clicktale.com/ - heatmaps of users
http://www.crazyegg.com/ - confetti of what users click on

As Edwin mentions, these services are great not only for your website, but to see what people like about your competitors' websites.

[aZooZa]

Looking at your site(s) Edwin, I don't think I'd change anything at all. I could understand the need maybe for this type of evaluation if you were selling anything else but the domain name. I know I'm 'in the business' and fresh eyes are useful, but at the end of the day, I don't think an outside evaluation like this is of use to you. Your only true evaluator in this case can be your prospective purchaser.

In that respect, as I say, the sites work just fine in my opinion. Simple and concise:

1. What?
2. Why?
3. Price.

[Edwin]

interested to know why you chose ContentManagementSystem as the domain for the experiment?

Random. Had it open on my screen at the time...

[Edwin]

Looking at your site(s) Edwin, I don't think I'd change anything at all. I could understand the need maybe for this type of evaluation if you were selling anything else but the domain name. I know I'm 'in the business' and fresh eyes are useful, but at the end of the day, I don't think an outside evaluation like this is of use to you. Your only true evaluator in this case can be your prospective purchaser.

Thanks, but the site that's currently there is significantly different from the site 10 days ago before I started tinkering. I have iterated a number of times and made lots of changes (many but not all as a result of the feedback from this tool). Anyway, it's good to hear I'm on the money now