Home > Crossway Blog > Let’s Mechanical Turk

Let’s Mechanical Turk

Update 3:20 PM U.S. Eastern Time: All the HITs have been uploaded and claimed, so none remain. Thanks to the 78 of you who participated. We’ll let you know how it works out.

We want to pay you two cents (US$0.02) to help us improve the ESV text database. You don’t need any special biblical knowledge.

What to Do

  1. Go to the Mechanical Turk website.
  2. Search for HITs containing “quotation.”
  3. Find our HITs, titled “Identify the speaker of a quotation” from Good News Publishers. If you can’t find any HITs, there may not be any available at the moment. We’ll upload HITs throughout the day, so you may want to try again later.
  4. Click “View a HIT in this group.”
  5. Read the instructions.
  6. Click “Accept HIT” if you want to work on this question. (You may have to sign in.)
  7. Answer the question, and you’re done (or ready to work on another HIT).
  8. We’ll review your answer and approve the transfer of two cents into your Amazon account.

Background

We’re always looking to improve the completeness of our database containing the ESV text. We thought we’d try out Amazon’s Mechanical Turk, which pays people a small amount to perform tasks that computers have trouble with.

In this instance, we’re looking to identify the speaker (when named) of the 5,500 direct quotations in the Bible.

We’ll let you know how this experiment in database normalization turns out.

June 13, 2006 | Posted in: Digital,ESV | Author: Crossway Staff @ 9:12 am | (18) Comments »

18 Comments

  1. I talk an awful lot about sharing on this blog. I have a really good non-photographic example of what I am talking about, because I know sharing really scares certain photographers: The English Standard Version of the Bible.

    Trackback by Free, as in beer — January 18, 2007 @ 4:23 pm

  2. Great advice from Fred Wilson on bringing someone in from the outside to run your startup. Interesting concept video showing off the ability of an animation company. Thanks Andy. The rm *nix command. Overheard on the phone.

    Trackback by fantasy is what people want, but reality is what people need ... — January 18, 2007 @ 4:23 pm

  3. When I first looked in on the Mechanical Turk it had about 500 hits available (all Amazon sponsored. I spent some time looking over it, but I couldn’t see anything worth bothering with even as it grew.

    Trackback by ESV Used Mechanical Turk — January 18, 2007 @ 4:23 pm

  4. Bible quote database built with Amazon Mechanical Turk for $75 2 cents a quote, with a 98.3% success rate [via]. read more at Waxy.org Links. rss2lj.

    Trackback by Bible quote database built with Amazon Mechanical Turk for $75 — January 18, 2007 @ 4:23 pm

  5. The Standard Bible Society used the “crowdsourcing” of Amazon.com’s Mechanical Turk to produce an English Standard version of the bible at $.02 a verse and 98.3% accuracy. I had forgotten all about the mechanical turk until I read this

    Trackback by Bible quote database built with Amazon Mechanical Turk for $75 — January 18, 2007 @ 4:23 pm

  6. There are some things that computers are good at and people aren’t. There are other things that people are good at and computers aren’t. Then there are the things that are trivially simple for people to do, but computers literally fail

    Trackback by Artificial Artifical Intelligence — January 18, 2007 @ 4:23 pm

  7. Bible quote database built with Amazon Mechanical Turk for $75. Okay the sheep were cute, but this is interesting.

    Trackback by Bible quote database built with Amazon Mechanical Turk for $75 — January 18, 2007 @ 4:23 pm

  8. I’m still working away (far too slowly for my impatient tastes) on the first complete first of New Testament Names, a semantic knowledgebase of named things in the New Testament and their relationships: you can get a sense of it from

    Trackback by Disambiguating Names in the New Testament — January 18, 2007 @ 4:23 pm

  9. Esv_blog There are some fascinating posts over at the ESV blog. ESV refers to the English Standard Version of the Bible. In order to increase the accuracy of their database of Biblical quotations, they used the Amazon Mechanical Turk.

    Trackback by Mechanical Turk for Metadata Collection — January 18, 2007 @ 4:23 pm

  10. From the blogroll… New Fink release. CameraHobby – Learning Modules and e-Books. Mechanical Turk Recap.

    Trackback by links for 2006-06-16 — January 18, 2007 @ 4:23 pm

  11. Our experiment with Mechanical Turk went better than we expected. We were able to approve 85% of submissions automatically, and we ultimately approved 98.3% of submissions. These figures came in higher than we planned: we thought we

    Trackback by Mechanical Turk Recap — January 18, 2007 @ 4:23 pm

  12. “We’re always looking to improve the quality of the ESV text database, especially as we prepare an OSIS version. (OSIS is an XML format for the Bible.) OSIS allows us to indicate who speaks each direct quotation in the Bible.”

    Trackback by Musings of a Chicagoan — January 18, 2007 @ 4:23 pm

  13. Another cool use of the Mechanical Turk. This has to be the most innovative thing I’ve seen in that it is an api for manual labour.

    Trackback by fantasy is what people want, but reality is what people need... — February 1, 2007 @ 7:34 am

  14. Producers of the English Standard Version Bible wanted an electronic database that would identify all 5500 direct quotes in the Bible by speaker. So they uploaded all of the quotes to Amazon’s Mechanical Turk…

    Trackback by ESV's use of Mechanical Turk to provide Bible metadata — February 1, 2007 @ 7:34 am

  15. Bible annotation using Amazon’s “Mechanical Turk” HIT service; a success. However they did invite their blog readers to participate, which would have skewed results by providing willing participants…

    Trackback by links for 2006-06-21 — February 1, 2007 @ 7:34 am

  16. Sounds like spam, but it’s not. Earn $0.02ca time to help the ESV people identify the speaker of a quotation.

    Trackback by Earn money from the Bible — February 1, 2007 @ 7:35 am

  17. The ESV Bible team crowdsourced their interlinked search database to Amazon’s Mechanical Turk…

    Trackback by Latest links from delicious — February 1, 2007 @ 10:38 am

  18. In addition to being a great translation, the ESV guys are very high-tech…my favorite combination Christians and geeks…

    Trackback by ESV uses Amazon’s Mechanical Turk — February 1, 2007 @ 10:38 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.