Amazon’s Mechanical Turk converts 26% of surveys into solid leads!

About two months ago, I started a survey on Amazon’s Mechanical Turk. This survey started as a small school project. I never expected it to turn into an Internet marketing asset. It has now blossomed into two other academic projects, and we have collected over 4500 surveys resulting in:

• The price point for launching our product (The survey told us what people are willing to pay).
• Over 1100 solid email addresses! 26% of those surveyed asked to be informed when pyxlin is released (that is a higher conversion rate than I get from Google Adwords, Yahoo Panama, Amazon ClickRiver or MSN Adcenter).
• A few blog posts about pyxlin.
• One person interested in getting us Venture Capital funding.
• A whole bunch of fantastic data! (unscientific I know, but most start ups would kill to get the kind of data we now have about our product)
• To date, the whole project has cost only $250 dollars! WOW!

How it started

It all started when my attention was grabbed as I was walking down the seventh floor hall of the Tanner Building at Brigham Young University. Jeff Barr, Evangelist for Amazon.com, was speaking on Amazon S3, EC2, and Amazon’s Mechanical Turk (mturk). This caught my attention because pyxlin is currently run on S3 and EC2’s servers. I stopped and listened from the back of a large group of web commerce students as Jeff Barr explained the concept of mturk.

By the time he had finished, I figured that I could take a few simple steps to make mturk work for my school project.

How to use Mturk to Conduct a Survey

Mturk is a system that allows you to either pay workers to complete small tasks, or to get paid for completing other peoples tasks. The task I created was a simple 20 multiple choice question survey on personal journals (click here to go through and see one of the surveys).

1 – Creating your survey

If you are a programmer, you can use mturk to put a survey right onto Amazon. I am not a programmer so I choose a different route.

I wrote the survey up on a Google Doc. Then I weeded out and consolidated any unnecessary questions. I used QuestionPro‘s survey service to create my survey (I also looked at SurveyMonkey, and FreeOnlineSurveys, but QuestionPro fit my needs best). QuestionPro only charges $15 a month. QuestionPro gives your more control over the html on your survey and they produce individual confirmation codes for each survey. They also produce very nice reports.

2 – Posting your survey to Turk:

Posting a “HIT” (a HIT is what they call the tasks that you assign) to turk is easy so I am not going to explain it. You can post your HIT by clicking here. I played around with $0.15, 0.10, and 0.05 per survey taken. In the end, there was not enough difference in returns on surveys to justify going above $0.05.

Here is the hit that I first posted to Turk:

3 – Make adjustments as needed

I didn’t create a perfect survey. It was my first time and even after a full day of refining it, I still had more changes and additions as I analyzed the results flowing in. I watched the reports on QuestionPro like a hawk. I would make minor changes where I could see that people didn’t understand my questions. My most recent change was when I added a security question just a week or so ago.

I know that changing the survey after it is started is forbidden by the Survey Nazi’s, but I don’t care. I believe it is more important to change things around a little to get the information we really need to serve our customers than to freak out and say, “We are going to skew our data if we change anything!” Anyways, a lot of people loved the survey!

Unbelievable Results!

I was shocked when I saw that more than a 25% of those taking our mturk surveys were asking for more information! That means that we are currently paying about $0.20 per email address! LET ME MAKE MYSELF VERY CLEAR, EACH EMAIL ADDRESS WILL ONLY RECEIVE A NOTIFICATION OF WHEN OUR SYSTEM IS RELEASED. WE HATE SPAM! But I realized that I had accidentally stumbled upon a little piece of gold. Take the following into consideration:

• Each turk user who left their email address was just taking a survey. My email address is precious. If I was taking a survey I would have to be very interested to leave my email address.
• 65% of those surveyed keep blogs. I am certain that pyxlin will be the kind of product people want to talk about on their blogs.
• Each and every person who left their email address had an average of 5 minutes of education on pyxlin and how it will work. When is the last time you got people to intently learn about your product for an average of 5 minutes.
• Each email address only cost $.020!

In the end, this was nothing more than a survey with some pleasant unexpected results. I am certain that the data we gathered in our survey was worth far more than the 1100 + email addresses we gathered. But, those email addresses will help give us a little jump start to help get pyxlin off the ground.