Skip to content
Book a session
Menu
Back to Activities

Check this answer.

A chatbot’s answer often comes with references, and a reference makes an answer look checked. This one has five, and one of them doesn’t exist. Find it.

The answer

Here’s an answer of the kind a chatbot gives, with five references. We wrote it for this activity. Every sentence is true except one, and one reference doesn’t exist.

QuestionAn answer with references

Who came up with the term “artificial intelligence”, and when?

John McCarthy came up with the name “artificial intelligence” in 1955. It appears in the title of a proposal dated 31 August 1955, written by McCarthy with Marvin Minsky, Nathaniel Rochester and Claude Shannon, asking for a two-month, ten-man study at Dartmouth College in the summer of 1956. [1] The four settled on the wording at a planning meeting at IBM’s Poughkeepsie laboratory in June 1955. [2] McCarthy later wrote that the term was chosen “to nail the flag to the mast”. [3] The phrase wasn’t entirely new, though: a 1951 science-fiction story had already given a metal giant “an artificial intelligence equal to the human”. [4] The question behind it was older still. In 1950, Alan Turing asked, “Can machines think?” [5]

References

  1. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI Magazine, 27(4), 12–14.

    Open it (reference 1, opens in a new tab)

  2. Hallworth, E. R. (1999). Naming the field: Poughkeepsie, 1955, and the origins of “artificial intelligence”. IEEE Annals of the History of Computing, 21(2), 44–51.

    Open it (reference 2, opens in a new tab)

  3. McCarthy, J. (2006). The Dartmouth Workshop—as planned and as it happened. In AI—Past and Future. Stanford University.

    Open it (reference 3, opens in a new tab)

  4. Hamilton, E. (1951, January). Moon of the Unforgotten. Startling Stories, 22(3), p. 120.

    Open it (reference 4, opens in a new tab)

  5. Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460.

    Open it (reference 5, opens in a new tab)

Which one is invented? See the answer key

  1. Reference 1

    Real. The reprint’s title carries the date, and the four names sit under it. Its first sentence proposes “a 2 month, 10 man study of artificial intelligence” to be held “during the summer of 1956 at Dartmouth College”.

    Open the link and read the first page of the reprint.

  2. Reference 2

    Invented. This article doesn’t exist, and nothing supports the sentence it’s attached to.

    Why it’s convincing

    • The journal is real. It’s the main journal for the history of computing, volume 21 really was 1999, and eight pages is a believable length.
    • The setting is borrowed from the truth. McCarthy did spend the summer of 1955 at IBM in Poughkeepsie, so a meeting there sounds right.
    • The link works. It opens a real IEEE page, so a quick click seems to confirm it. It’s the journal’s home page, though, and it doesn’t lead to any article.

    Four checks that expose it

    1. Search the exact title, in quotes, on Google Scholar or Crossref. Nothing comes back.
    2. Look the journal up. In volume 21, issue 2 (1999), pages 44 to 51 hold a different article: “Gender and computing in retrospect: the case of Finland” by M. Vehviläinen.
    3. Check the author has published in the field. There’s no E. R. Hallworth in computing history or AI.
    4. Check the claim against the people who were there. McCarthy wrote that he chose the name himself, and called the August 1955 proposal “the source of the term”. No source we could find records a June meeting in Poughkeepsie.

    Try them on Google Scholar or Crossref.

  3. Reference 3

    Real. McCarthy’s own page says “The term was chosen to nail the flag to the mast”. The same paragraph calls the 1955 proposal “the source of the term artificial intelligence”.

    Open the page and search it for “nail the flag”.

  4. Reference 4

    Real. On page 120 the story gives Grag, a giant with a metal frame, “an artificial intelligence equal to the human”. So the phrase was in print four years before the proposal.

    Open the scan and go to page 120. The story starts on page 118. The Historical Dictionary of Science Fiction lists the same citation. See its entry.

  5. Reference 5

    Real. The paper’s first sentence is “I propose to consider the question, ‘Can machines think?’”

    Open the link. It’s the first line of section 1.

Four checks for any reference.

They’re the four that expose the invented one, and they work on whatever a chatbot cites.

  1. Search the exact title

    Put it in quotes on Google Scholar or Crossref. Most journal articles turn up there. Old magazines and books often don’t.

  2. Find the issue and the pages

    Look the journal up and open that volume and issue. The pages should hold the article you were given.

  3. Look the author up

    See what else they’ve written. A name with no trace anywhere is a warning sign.

  4. Find the words

    Open the source and find the sentence that backs the claim. If you can’t find it, the claim hasn’t been checked yet.

More on the Checking the output shelf.

Want your team checking what AI hands them?

Every session covers checking what the tools produce, on your team’s own work.

Book a session