MTurk Worker Dynamics: How Many New Workers Join Each Month?

Posted by TurkPrime on Jul 30, 2019 8:15:00 AM
TurkPrime
  

In a recent research article (https://psyarxiv.com/jq589/), we reported on the size of the MTurk worker pool. Using observed participation rates from the TurkPrime database, we found that over each of the last three years there have been approximately eighty to eighty-five thousand US workers active on MTurk each year (Figure 1). What these numbers do not illustrate is how often new workers join the platform—a number that is important for researchers who want to sample naive participants.

Using data from TurkPrime, we were able to ascertain that in each month from January 2016 through April, 2019 there were between 15,000 and 30,000 US workers taking at least one HIT and approximately 4,600 new workers joining the pool each month. As shown in Figure 1, the number of new workers per month has fluctuated over time. Nevertheless, within each of the last three years more than half of MTurk workers were new to the platform (Figure 2).

MTurk Worker Dynamics_Figure 1

Figure 1. The number of new US workers per month from January, 2016 - April, 2019.                                           On average 4,683 new workers joined the pool each month.

MTurk Worker Dynamics_Figure 2

Figure 2. The number of new and unique US workers taking a HIT posted through TurkPrime each year.

Research context

MTurk’s popularity has led to several questions about whether the participant pool is overused. One key to answering this question is ascertaining how many workers are active on MTurk per year. In addition, another key is finding out how often new workers join the platform. 

The numbers from our database reveal that MTurk has more workers than previously estimated and that there is a significant number of new workers joining the pool on a regular basis. Even though more research needs to be conducted, we believe these numbers offer a reason for researchers to be optimistic about the possibility of sampling participants from Mechanical Turk.

In addition to discussing how often new workers join MTurk, our article addresses several issues relevant for sampling on Mechanical Turk. The full preprint of the article is available here.

References

Difallah, D., Filatova, E., & Ipeirotis, P. (2018, February). Demographics and dynamics of Mechanical Turk workers. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (pp. 135-143).

Stewart, N., Ungemach, C., Harris, A. J., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10, 479-491.

Topics: mturk, turkprime, mturk workers, turkprime research, participant pool

Subscribe Here!

Recent Posts