1/16/23
In late November 2022, OpenAI, the company responsible for the popular AI image generator DALL-E 2, released an AI text generator called ChatGPT, which outputs text content as well as lines of code in several different programming languages. While ChatGPT is certainly not the first AI text generation model, it quickly gained popularity due to the conversational style of the interface, and the model’s ability to ask for clarification, recall past responses, and point out errors in user prompts. ChatGPT can also detect inappropriate user prompts regarding drug use, hate speech, and other harmful subjects.
While ChatGPT is one of the most advanced AI language generators, it has many drawbacks common to other AI technologies. The data used to train the model is only relevant up to the year 2021, so queries concerning recent events may generate incorrect information or not generate text at all. A larger problem with its data set is the risk of introducing bias, which could result in racist, sexist, or otherwise harmful language being generated and perpetuated. This is an ethical concern for all types of AI systems, and there are several origins of bias including bias in the original data set and the biases of those choosing the data set. In earlier versions of the ChatGPT, some users were able to bypass initial prompt rejection models and “tricked” ChatGPT into revealing racist and sexist biases within its data set, where lines of code generated by ChatGPT implied that only white males would make good scientists. OpenAI has since updated the prompt filter to reject a wider range of inputs that could lead to inappropriate results. ChatGPT also has an integrated feedback system so that users can flag inappropriate or erroneous responses. However, the basis on which prompts are rejected is still determined by people at OpenAI; ChatGPT has no internal sense of morality and cannot independently determine which prompts are inappropriate. Even as more prompt restrictions are put in place, users will likely continue to find ways to bypass these safeguards.
ChatGPT and other AI language generators have also become the focus of academic integrity debates. Many AI essay generators already exist, but ChatGPT has quickly become popular among students because it generates shockingly human-like responses, and can output in several different styles. While ChatGPT can pass as human writing, it does not produce A+ quality work on its own. When presented with an essay written completely by ChatGPT, one AP English Literature teacher gave it a C. Many of the demerits stemmed from incorrect facts about the text from the essay prompt, i.e. attributing a quote to the wrong character. Nevertheless, the essay still received a passing grade without any human proofreading. The use of ChatGPT for school assignments has become so prolific that the New York City public school system has banned the website from its devices, while others argue that ChatGPT should be embraced as a teaching tool. The following “conversation” was conducted with ChatGPT about this issue:
KT: What are the ethical implications of using ChatGPT for school assignments?
ChatGPT: Using ChatGPT for school assignments can be considered cheating if it is used to generate original content. However, using it as a tool for inspiration or editing may be acceptable, depending on the academic institution’s policies and the student’s own ethical standards.
In its response above, ChatGPT highlights a problem in enforcing restrictions on AI language tools: Certainly, passing AI writing off as one’s own for an assignment is an ethical violation, but using ChatGPT to brainstorm or edit text blurs the line. Additionally, even if the use of AI text is not allowed by an institution, there is currently no foolproof way to detect AI-generated text. Plagiarism checkers like Google Classroom and Turnitin are able to compare previous works of a student to flag changes in writing style, and Turnitin reports that it will integrate an updated AI writing detector this year. One student at Princeton has developed his own program to detect AI writing, but the open demo does not flag all AI text. OpenAI has also stated they are developing a way to invisibly watermark their text to help educators accurately identify ChatGPT content.
Ultimately, AI text generators are tools, and the ethical ramifications of these tools are dependent on how they are used. The back and forth between AI text generators and detectors has resulted in rapid technological development, and it is a constant race to ensure that these tools can be used in an ethical and effective manner.