TurkPrime Blog

TurkPrime Blog

Are MTurk workers who they say they are?

Posted by Leib Litman on Dec 1, 2017 2:19:00 PM

The internet has the reputation of being a place where people can hide in anonymity, and present as being very different people than who they actually are. Is this a problem on Mechanical Turk? Is the self-reported information provided by Mechanical Turk workers reliable? These are important questions which have been addressed with several different methods. Researchers have examined a) consistency of responding to the same questions over time and across studies b) the validity of responses, or the degree to which the items capture responses that represent the truth from participants. It turns out that there are certain situations in which MTurk workers are likely to lie, but they are who they say they are in almost all cases.

Read More

Topics: amazon mechanical turk, anonymous, demographics, HIT, mturk, panels, qualification, turkprime panels, unique worker, worker groups, workers

Strengths and Limitations of Mechanical Turk

Posted by Leib Litman on Nov 20, 2017 11:12:00 PM

Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.

Read More

Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers

Upcoming New Content on the Blog

Posted by TurkPrime on Nov 17, 2017 11:43:00 AM

Greetings Reader,

Read More

Topics: amazon mechanical turk, online research, primepanels, turkprime

Verified US Region Targeting

Posted by TurkPrime on Dec 2, 2016 1:22:00 PM

Problem

Many researchers wish to target participants from specific states or regions of the United States like the Northeast or the West. The problem they often encounter is that using MTurk's Geographic Qualification to specify a particular state is often not adequate to ensure participants actually reside in the specified state.

Read More

Topics: amazon mechanical turk, exclude workers, IP address, IP block, location, region, survey, turkprime

Dynamic Secret Completion Codes for SurveyMonkey and SurveyGizmo

Posted by TurkPrime on Dec 2, 2016 1:04:00 PM

What is the completion rate and dropout rate?

Dropout rate is defined as the percentage of participants who start taking a study but do not complete it. Dropout rate is sometimes referred to as attrition rate, and is the opposite of completion rate (dropout rate = 100 – completion rate). On MTurk, completion rate is defined as the number of Workers who submit a HIT divided by the number of Workers who accept the HIT. Note that, for the definition of completion rate used here, Rejected Workers are counted as completes.

Read More

Topics: amazon mechanical turk, autoapproval, completion code, dynamic, mturk, secret code, surveymonkey, turkprime

How to Create a Universal Exclude Worker List

Posted by Jonathan Robinson on Sep 9, 2016 10:41:00 AM

Problem: 

Requesters may observe that some workers, even those with high Approval ratings, may not perform to their expectations on a study. Sometimes this may result in rejecting their work which affects the Worker approval rating. But, often the work is not acceptable for research but is not worthy of rejection, or, it may simply be the policy of the research lab to approve all assignments for IRB or some ethical standard they may follow.

Read More

Topics: amazon mechanical turk, block worker, exclude, exclude workers, mturk, mturk api, qualification, turkprime

Reusable Include and Exclude Groups

Posted by Jonathan Robinson on Sep 9, 2016 10:36:00 AM

Problem:

You are running a longitudinal study and have identified 1000 workers who you want to allow to take your second phase studies. How do you easily group those workers for easy access.

Or you want to exclude certain workers from taking a number of your studies and wish to group them for easy exclusion in future studies. How can you do that?

Read More

Topics: amazon mechanical turk, exclude, exclude workers, include workers, Longitudinal, qualification, turkprime, worker groups

MTurk Panels on Your Own Requester Account

Posted by Jonathan Robinson on Sep 8, 2016 6:44:00 PM

Studies with Panels  for just $0.15 - 0.75 / complete

Now you can run Mechanical Turk studies using your own Requester account and specify over two dozen demographic traits!. The traits include gender, ethnicity, age, marital status and sexual orientation. But it does not stop there! The available options also include occupation, medical and health history, cell phone use and much more.

Read More

Topics: amazon mechanical turk, demographics, mechanical turk, mturk, qualification, traits, turkprime, turkprime panels

Survey Groups Ensure Unique Workers Across Multiple Concurrent Studies

Posted by Jonathan Robinson on Aug 23, 2016 2:41:00 PM

Problem

Have you ever wanted to run multiple studies simultaneously  and make sure that each worker only takes a single study? On MTurk, there is no simple way to block workers who completed one study from accepting and completing another study being run at the same time....until now.

(Note: TurkPrime's exclude feature excludes workers who have completed one study from taking another subsequent study, but not if both studies are being run at the same time.)

Read More

Topics: amazon mechanical turk, exclude, exclude workers, follow up, qualification

Demographic Consistency Over Time on Mechanical Turk

Posted by TurkPrime on Apr 12, 2016 12:15:00 PM

Problem

Ever wonder if workers are being honest with you when they answer a survey? Or, if you specify that your study should be taken only by Women, whether some workers take the study even though they are not women?

Read More

Topics: amazon mechanical turk, demographics, gender gap, mechanical turk, mturk, panels

Subscribe Here!

Recent Posts

Posts by Tag

See all