TurkPrime Blog

TurkPrime Blog

Strengths and Limitations of Mechanical Turk

Posted by Leib Litman on Nov 20, 2017 11:12:00 PM

Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.

Read More

Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers

New Safety Feature: Assignment Rejections are not Automated and Require Manual User Action

Posted by Jonathan Robinson on Feb 19, 2016 10:39:00 AM

Many researchers set up their studies to use the TurkPrime AutoApprove feature so that they do not need to manually approve worker assignments based on the secret code that workers enter. On occasion, a researcher may set up his study incorrectly which results in many worker assignments getting automatically rejected. This was a significant cause for distress among the Mechanical Turk workers who received rejections for their work which then went unpaid and also lowered their MTurk approval rating. This was also a sore point for TurkPrimeresearcherswho had to deal with upset workers and correspond with them, and often reverse their rejections.

Read More

Topics: amazon mechanical turk, autoapproval, reverse rejection, study, survey, turkprime