Amazon’s Mechanical Turk converts 26% of surveys into solid leads!

About two months ago, I started a survey on Amazon’s Mechanical Turk. This survey started as a small school project. I never expected it to turn into an Internet marketing asset. It has now blossomed into two other academic projects, and we have collected over 4500 surveys resulting in:

  • The price point for launching our product (The survey told us what people are willing to pay).
  • Over 1100 solid email addresses! 26% of those surveyed asked to be informed when pyxlin is released (that is a higher conversion rate than I get from Google Adwords, Yahoo Panama, Amazon ClickRiver or MSN Adcenter).
  • A few blog posts about pyxlin.
  • One person interested in getting us Venture Capital funding.
  • A whole bunch of fantastic data! (unscientific I know, but most start ups would kill to get the kind of data we now have about our product)
  • To date, the whole project has cost only $250 dollars! WOW!

How it started

It all started when my attention was grabbed as I was walking down the seventh floor hall of the Tanner Building at Brigham Young University. Jeff Barr, Evangelist for Amazon.com, was speaking on Amazon S3, EC2, and Amazon’s Mechanical Turk (mturk). This caught my attention because pyxlin is currently run on S3 and EC2’s servers. I stopped and listened from the back of a large group of web commerce students as Jeff Barr explained the concept of mturk.

By the time he had finished, I figured that I could take a few simple steps to make mturk work for my school project.

How to use Mturk to Conduct a Survey

Mturk is a system that allows you to either pay workers to complete small tasks, or to get paid for completing other peoples tasks. The task I created was a simple 20 multiple choice question survey on personal journals (click here to go through and see one of the surveys).

1 – Creating your survey

If you are a programmer, you can use mturk to put a survey right onto Amazon. I am not a programmer so I choose a different route.

I wrote the survey up on a Google Doc. Then I weeded out and consolidated any unnecessary questions. I used QuestionPro‘s survey service to create my survey (I also looked at SurveyMonkey, and FreeOnlineSurveys, but QuestionPro fit my needs best). QuestionPro only charges $15 a month. QuestionPro gives your more control over the html on your survey and they produce individual confirmation codes for each survey. They also produce very nice reports.

2 – Posting your survey to Turk:

Posting a “HIT” (a HIT is what they call the tasks that you assign) to turk is easy so I am not going to explain it. You can post your HIT by clicking here. I played around with $0.15, 0.10, and 0.05 per survey taken. In the end, there was not enough difference in returns on surveys to justify going above $0.05.

Here is the hit that I first posted to Turk:

HIT

3 – Make adjustments as needed

I didn’t create a perfect survey. It was my first time and even after a full day of refining it, I still had more changes and additions as I analyzed the results flowing in. I watched the reports on QuestionPro like a hawk. I would make minor changes where I could see that people didn’t understand my questions. My most recent change was when I added a security question just a week or so ago.

I know that changing the survey after it is started is forbidden by the Survey Nazi’s, but I don’t care. I believe it is more important to change things around a little to get the information we really need to serve our customers than to freak out and say, “We are going to skew our data if we change anything!” Anyways, a lot of people loved the survey!

Unbelievable Results!

I was shocked when I saw that more than a 25% of those taking our mturk surveys were asking for more information! That means that we are currently paying about $0.20 per email address! LET ME MAKE MYSELF VERY CLEAR, EACH EMAIL ADDRESS WILL ONLY RECEIVE A NOTIFICATION OF WHEN OUR SYSTEM IS RELEASED. WE HATE SPAM! But I realized that I had accidentally stumbled upon a little piece of gold. Take the following into consideration:

  • Each turk user who left their email address was just taking a survey. My email address is precious. If I was taking a survey I would have to be very interested to leave my email address.
  • 65% of those surveyed keep blogs. I am certain that pyxlin will be the kind of product people want to talk about on their blogs.
  • Each and every person who left their email address had an average of 5 minutes of education on pyxlin and how it will work. When is the last time you got people to intently learn about your product for an average of 5 minutes.
  • Each email address only cost $.020!

In the end, this was nothing more than a survey with some pleasant unexpected results. I am certain that the data we gathered in our survey was worth far more than the 1100 + email addresses we gathered. But, those email addresses will help give us a little jump start to help get pyxlin off the ground.

About these ads

June 6, 2007. discussions, journal research, Uncategorized.

10 Comments

  1. Steve Walters replied:

    I had heard about Amazon’s Mechanical Turk before, but I wasn’t aware that it could be used to distribute surveys so inexpensively. I run a list of paid survey sites and browsing through Mechanical Turk has given me some additional fodder as well some good ideas. Congratulations on the great results and thanks for bringing this to my attention! BTW, my site at http://www.paid-survey-success.com is completely free – no subscription or registration required.

  2. andre nosalsky replied:

    Very interesting indeed. This can be used as a cheap focus group.

  3. Tony Wright replied:

    Do you have any data on the type of people who were filling out the survey? I always assumed that Mechanical Turk users were generally off-shore folks…

  4. Top Posts « WordPress.com replied:

    […] Amazon’s Mechanical Turk converts 26% of surveys into solid leads! About two months ago, I started a survey on Amazon’s Mechanical Turk. This survey started as a small school […] […]

  5. pyxlin replied:

    Tony,

    We do have data. A solid 64+% of those surveyed are from the US. About 16% are from India. In our data analysis we separated these into groups, US and Non-US. QuestionPro also lets you know what state and IP address each person is from. You will also see that a variety of incomes was also well represented.

    Here is a link to see the results of the survey:

    http://www.questionpro.com/akira/ShowResults?id=681868&mode=data

    Because this is not raw data it would be impossible for you to analyze it correctly, but it gives you an idea of the results.

  6. Brock Judkins replied:

    Hey, you got the attention of the BYU Alumni list at Amazon.com. Good work!

  7. Bill replied:

    Mechanical Turk is a very interesting system and it sounds like it is working well for you – but the success of this approach would depend very much on what kind of people you want to reach. There are easier ways to earn money than doing MT tasks for 5 cents a time and I would be interested to see some stats on MT worker demographics.

  8. FavoriteSurveys replied:

    Bill, it looks to me like a lot of MTurk users are from the US since as far as I know, you can only get paid by MTurk if you have a US bank account or can use the Amazon gift certs. Judging from message boards I’ve read about it, a lot of the users are college students killing time or they are actually doing MTurk work while at their regular jobs — I’m sure lots of employers would love to know about that LOL

  9. Donna replied:

    Greetings from an MTurk worker! I’m a 46 year-old, former office manager with a college education who used to spend way too much time reading gossip blogs on the Internet. When I discovered that I could earn money while propped up in bed, watching TV and eating bon-bons, I was thrilled. Sure, the pay is low, but I love seeing my Amazon account grow a little each day, until I have enough to treat myself to a new book or CD.

    I am a member of an MTurkers forum, and you might be surprised by how seriously we take the responsibility of these funny little tasks. Some of us pride ourselves on being super-sleuths, tracking down requested information all over the web for a few nickels and the satisfaction of solving a puzzle.

  10. Paul Allen: Internet Entrepreneur » Now Hiring: Mechanical Turk Project Manager replied:

    […] of my former BYU students wrote a great blog post about how he used the Turk to conduct a survey on journal keeping and the unanticipated side effect was that 26% of the survey respondents turned […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback URI

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: