TurkPrime Blog

TurkPrime Blog

After the Bot Scare: Understanding What’s Been Happening with Data Collection on MTurk and How to Stop it

Posted by TurkPrime on Sep 18, 2018 2:07:52 PM

Highlights

  • Since early August, researchers have worried that “bots” are contaminating data collected on MTurk.
  • We found workers who submit HITs from suspicious geolocations are using server farms to hide their true location.
  • When using TurkPrime tools to block workers from server farms, we collected high quality data from MTurk workers.
  • We also collected data from workers who use server farms to learn more about them.
  • Our evidence suggests recent data quality problems are tied to foreign workers, not bots.

In this blog, we review recent data quality issues on Mechanical Turk and report the results of a study we conducted to investigate the problem.

Read More

Topics: turkprime, bots, server farms, data quality, high quality data