CloudResearch Blog

TurkPrime Blog

New Feature: Exclude Highly Active Workers

Posted by Leib Litman on Dec 15, 2017 12:30:00 PM

Some workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year, and around 25,000 who take surveys each month, you are much more likely to recruit highly active workers who take a majority of HITs. About 1,000 workers (1% of workers) take 21% of the HITs. About 10,000 workers (10% of workers) take 74% of all HITs.

Read More

Topics: active workers, amazon mechanical turk, exclude, exclude workers, HIT, mechanical turk, mturk, online research, turkprime, workers, active

Best recruitment practices: working with issues of non-naivete on MTurk

Posted by Leib Litman on Dec 8, 2017 3:59:00 PM

It is important to consider how many highly experienced workers there are on Mechanical Turk. As discussed in previous posts, there is a population pool of active workers in the thousands, but this is far from exhaustible. A small group of workers take a very large number of HITs posted to MTurk, and these workers are very experienced and have seen measures commonly used in the social and behavioral sciences. Research has shown that when participants are repeatedly exposed to the same measures, this can have negative effects on data collection, changing the way workers perform, creating treatment effects, giving participants insight into the purpose of some studies, and in some cases impact effect sizes of experimental manipulations. This issue is referred to as non-naivete (Chandler, 2014; Chandler, 2016).

Read More

Topics: amazon mechanical turk, approval rating, experience, exposure, HIT, mechanical turk, mturk, naivete, non-naive, primepanels, qualification, recruitment, requester, workers

Are MTurk workers who they say they are?

Posted by Leib Litman on Dec 1, 2017 2:19:00 PM

The internet has the reputation of being a place where people can hide in anonymity, and present as being very different people than who they actually are. Is this a problem on Mechanical Turk? Is the self-reported information provided by Mechanical Turk workers reliable? These are important questions which have been addressed with several different methods. Researchers have examined a) consistency of responding to the same questions over time and across studies b) the validity of responses, or the degree to which the items capture responses that represent the truth from participants. It turns out that there are certain situations in which MTurk workers are likely to lie, but they are who they say they are in almost all cases.

Read More

Topics: amazon mechanical turk, anonymous, demographics, HIT, mturk, panels, qualification, turkprime panels, unique worker, worker groups, workers

Strengths and Limitations of Mechanical Turk

Posted by Leib Litman on Nov 20, 2017 11:12:00 PM

Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.

Read More

Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers

Anonymize Mechanical Turk Worker IDs

Posted by Jonathan Robinson on Jan 18, 2016 6:32:00 PM

We recently launched a ground-breaking feature that helps protect Mechanical Turk worker identities. It has been reported in the literature that Mechanical Turk Worker IDs can be used to identify the worker. This is because Amazon uses the same value for both the Worker ID on Mechanical Turk and elsewhere on Amazon properties like Amazon.com product reviews.

Read More

Topics: amazon mechanical turk, anonymize, anonymous, exclude workers, HIT, include workers, mechanical turk, mturk, turkprime

HyperBatch - Run Batched HITs at Hyper Speeds

Posted by Jonathan Robinson on Jan 5, 2016 2:24:00 PM

HyperBatch Feature

You can now run your Amazon Mechanical Turk studies at Hyper speeds. We worked hard to make the experience identical to our classic MicroBatch option so that you simply launch your study and TurkPrime does all the heavy lifting for you.

Read More

Topics: amazon mechanical turk, HIT, hyperbatch, mechanical turk, microbatch, mturk, mturk api, mturkgate, turkprime

TurkPrime Video Tutorials to Get You Started with Mechanical Turk

Posted by TurkPrime on Jun 26, 2015 2:34:00 PM

TurkPrime now has tutorials to get you started

If you're looking for a video tutorial that explains how to setup your account on TurkPrime, how to run a longitudinal study, or how to limit who takes your HIT on Mechanical Turk, these resources are for you.

Read More

Topics: amazon mechanical turk, exclude workers, HIT, include workers, mturk, turkprime

Maximizing HIT Participation

Posted by Jonathan Robinson on May 13, 2015 1:00:00 PM

Problem:

How can you increase Amazon Mechanical Turk HIT Worker participation rates and speed completion of a HIT? This is particularly an issue with HITs that have a large number of required participants or have Qualifications that limit the number of qualified Workers

Read More

Topics: amazon mechanical turk, completion rate, demographics, HIT, increase participation, qualification, speed, turkprime

Qualtrics Studies run on Mechanical Turk can be Integrated With TurkPrime

Posted by Jonathan Robinson on May 8, 2015 2:15:00 PM

Run Qualtrics Surveys on MTurk using TurkPrime

Running Qualtrics surveys on TurkPrime streamlines several major pain-points researchers using MTurk have dealt with:

Read More

Topics: amazon mechanical turk, HIT, mechanical turk, qualtrics, reverse rejection, secret code, secret key

Finding More Mechanical Turk Workers Faster with TurkPrime "Restart HIT"

Posted by Jonathan Robinson on Mar 24, 2015 2:47:00 PM

Problem:

Suppose you need to run a HIT with 1000 Workers. Or a HIT that is only open to Workers who have an approval Rating of 95% or more and have completed 500 HITs or more. Although when you launched your HIT the MTurk Workers arrived at a nice pace, over time, the pace has slowed to a trickle such that your HIT will never complete.

Read More

Topics: amazon mechanical turk, bump, HIT, increase participation, mechanical turk, restart, speed, workers

Subscribe Here!

Recent Posts

Posts by Tag

See all