A wide array tools exists to detect use of generative AI, such as ChatGPT, but currently, the University Center for Teaching and Learning does not endorse or support the use of any AI detection tools, the center said in its newsletter.

These tools are becoming ubiquitous, the center said, but their accuracy varies considerably.

Based on their professional judgment, the Teaching Center has concluded that “current AI detection software is not yet reliable enough to be deployed without a substantial risk of false positives and the consequential issues such accusations imply for both students and faculty. Use of the detection tool at this time is simply not supported by the data and does not represent a teaching practice that we can endorse or support.”

Because of this, the Teaching Center will disable the AI detection tool in Turnitin effective immediately.

Below is the rest of the Teaching Center’s statement on these tools:

The creators of many of these tools claim extremely high accuracy rates in spotting AI-generated content. It is not unusual to see claims of 98 to 99 percent accuracy. However, when evaluated by third parties, most of these tools also have a high rate of false positives. Even AI detectors with a great record in spotting AI-generated content often flag human-written text as AI-generated. False positives carry the risk of loss of student trust, confidence and motivation, bad publicity, and potential legal sanctions.

Many of the software companies involved refuse to offer a specific false positive rate for their products.

On April 4, 2023, Turnitin released an AI detector that was available to Pitt faculty through its existing suite of tools. Although Turnitin claimed that its tool was more accurate than other AI detectors, some faculty voiced concerns about accuracy and usefulness. Based on our own testing at the Teaching Center, and discussions with other institutions, the Teaching Center decided not to endorse or support the use of this tool.

As of June 2023, there have been two new developments. First, Turnitin has now publicly acknowledged that its AI detection tool has a higher false positive rate than the company originally asserted. However, they have not disclosed a new false positive rate estimate. In addition, Turnitin has now updated their application so that the Teaching Center has the capability to disable this specific application within the suite of Turnitin tools.

We will continue to advise Pitt’s faculty about the value of AI tools as they continue to evolve, and we will work closely with all instructors on appropriate use of generative AI tools in the classroom and the appropriate best practices for managing potential abuses of this technology.

To request a consultation to discuss how to talk to students about the ethical use of AI tools or design assessments to mitigate academic integrity issues, email the Teaching Center at teaching@pitt.edu.

The Teaching Center also has a list of ChatGPT resources for faculty, including ways it can be used effectively in the classroom.