TurkPrime Blog

TurkPrime Blog

Moving Beyond Bots: MTurk as a Source of High Quality Data

Posted by TurkPrime on Oct 29, 2018 4:17:46 PM

Highlights

  • We collected high quality data on MTurk when using TurkPrime’s IP address and Geocode-restricting tools.
  • Using a novel format for our anchoring manipulation, we found that Turkers are highly attentive, even under taxing conditions.
  • After querying the TurkPrime database, we found that farmer activity has significantly decreased over the last month.
  • When used the right way, researchers can be confident they are collecting quality data on MTurk.
  • We are continuously monitoring and maintaining data quality on MTurk.
  • Starting this month, we will be conducting monthly surveys of data quality on Mechanical Turk.
Read More

Topics: data quality, bots, mturk, amazon mechanical turk, server farms

After the Bot Scare: Understanding What’s Been Happening with Data Collection on MTurk and How to Stop it

Posted by TurkPrime on Sep 18, 2018 2:07:52 PM

Highlights

  • Since early August, researchers have worried that “bots” are contaminating data collected on MTurk.
  • We found workers who submit HITs from suspicious geolocations are using server farms to hide their true location.
  • When using TurkPrime tools to block workers from server farms, we collected high quality data from MTurk workers.
  • We also collected data from workers who use server farms to learn more about them.
  • Our evidence suggests recent data quality problems are tied to foreign workers, not bots.

In this blog, we review recent data quality issues on Mechanical Turk and report the results of a study we conducted to investigate the problem.

Read More

Topics: turkprime, bots, server farms, data quality, high quality data

TurkPrime Tools to Help Combat Responses from Suspicious Geolocations

Posted by Jonathan Robinson on Aug 15, 2018 2:51:00 PM

Last week, the research community was struck with concern that “bots” were contaminating data collection on Amazon’s Mechanical Turk (MTurk). We wrote about the issue and conducted our own preliminary investigation into the problem using the TurkPrime database. In this blog, we introduce two new tools TurkPrime is launching to help researchers combat suspicious activity on MTurk and reiterate some of the important takeaways from this conversation so far.

Read More

Topics: amazon mechanical turk, bots, mechanical turk, mturk, workers

Concerns about Bots on Mechanical Turk: Problems and Solutions

Posted by Leib Litman on Aug 10, 2018 6:46:00 PM

Data quality on online platforms

When researchers collect data online, it’s natural to be concerned about data quality. Participants aren’t in the lab, so researchers can’t see who is taking their survey, what those participants are doing while answering questions, or whether participants are who they say they are. Not knowing is unsettling.

Read More

Topics: bots, mechanical turk, mturk, quality, turkprime, workers