By Amelia Trello, Staff Writer
Photo courtesy of unsplash.com
On March 14, OpenAI released its newest advancement in AI technology, GPT-4 with claims that it can beat 90 percent of human bar test takers, as well as other professional exams.[1] According to the company, “GPT-4 exhibits human-level performance on the majority of these professional and academic exams.”[2] In fact, GPT-4 is said to have higher advanced reasoning, be more accurate, and is 40 percent more likely to produce factual responses compared to the previous model.[3]
According to OpenAI, “GPT-4 is a large multimodal… that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks.”[4] The program has scored a 163 on the Law School Admission Test and 298 out of 400 on the Uniform Bar Exam.[5]
Although GPT-4 is currently not available to the public, the older model, ChatGPT, is currently available for free.[6] The old model is equally as capable as the newest one.[7] OpenAI stated that ChatGPT is an artificial intelligence model “which interacts in a conversational way.”[8] The model can answer the user with follow up questions, admit when it has made mistakes, challenge incorrect statements, and refuse to answer inappropriate demands.[9]
Such a major advancement in AI technology creates a new set of issues in legal education. The free program can be accessed by any student with a computer.[10] Colleges are very aware of ChatGPT and are already finding students utilizing the program to complete schoolwork.[11] Unfortunately, high school and undergraduate students have been caught using it to write essays and other assignments.[12] Law schools are also becoming very aware of the dangers of AI and ethical education.[13] In an effort to understand how the program works, law professors have been conducting their own informal research.[14]
University of Michigan Associate Professor of Law Jon Choi submitted law school exam questions through ChatGPT and mixed in the reformatted answers with real student exams.[15] Choi gave the program Constitutional Law, Employee Benefits, Taxation, and Torts exams.[16] The tests were then graded blind by three human graders.[17] The AI program earned an average C+ on the exams and functioned better on essays than on multiple choice questions.[18]
According to Choi, “in essays, it could recite legal rules and correctly describe cases…but failed at ‘spotting issues’ and providing deep reasoning or analysis.”[19] Even though the AI program can complete exams, it cannot not trick graders just yet. During the grading process, two of the three graders stated that they “had a hunch” about which exams were written by ChatGPT and which were not and were able to correctly decipher between the two.[20]
Schools have begun taking measures to combat AI programs like ChatGPT.[21] For example, Niagra College has begun implementing trainings with the goal of preventing students from using the tool.[22] Additionally, public schools have started blocking the website on firewalls to prevent individuals from using the program on school campuses.[23] Finally, due to overwhelming criticism, OpenAI created the program Text Classifier to help people identify AI written work.[24]
Although many teachers and professors have very valid fears regarding ChatGPT, other educators are embracing it and using it as an education tool.[25] Initially, Seattle Public Schools blocked the AI program on all school devices.[26]However, the district “opened access to educators who want[ed] to use it as a teaching tool.”[27] Further, the Seattle district is in the process of potentially allowing teachers to use it as a “personal tutor” for their students.[28] Teachers within the district believe that it could help students generate new ideas when working on school assignments.[29]
Thomas R. Kline School of Law of Duquesne University Professor John Rago has some real and legitimate concerns regarding the program. “I know I sound like the old guy who says, ‘I remember when candy was a nickel,’ but this technology strikes me as having all of the markings that can overrun people and practices that are not fully prepared to use the tool thoughtfully, much less properly.” Rago stated. However, Rago also acknowledged the potential good the program could create within the legal profession. However, he believes the potential for abuse could outweigh it. “For a number of classes, it could be the ultimate shortcut and we are in a profession where shortcuts don’t usually turn out too well,” Rago said.
On the other hand, Noah French, a 2L student at Duquesne Kline, believes that AI resources such as ChatGPT could be beneficial for legal research and education. “I really believe that ChatGPT can be a helpful tool when conducting legal research in school. I think that law students can ask the AI program pointed and specific questions and then use the AI’s answers as a jumping point for in depth legal research.” French said. French also stressed that the AI tool should be used to support lawyers, not replace them. “I don’t think that ChatGPT will be replacing lawyers any time soon. However, I do think that if lawyers use ChatGPT effectively, they could conduct their legal research more thoroughly and in less time. I bet it could even cut down billable hours one day,” French said.
Finally, when asking ChatGPT how it thinks it will affect cheating in law schools, the program itself stated, “As an AI language model, ChatGPT may potentially increase the risk of cheating in law schools if it is used improperly or without proper safeguards, but it is ultimately up to individual law schools to establish and enforce their own academic integrity policies.” Just as the AI program stated, law schools will certainly need to develop regulations to combat the program to keep up with the times.
[1] John Koetsier, GPT-4 Beats 90% Of Lawyers Trying To Pass The Bar, FORBES (Mar. 14, 2023), https://www.forbes.com/sites/johnkoetsier/2023/03/14/gpt-4-beats-90-of-lawyers-trying-to-pass-the-bar/?sh=383588b33027
[2] Id.
[3] Id.
[4] GPT-4, OPENAI, https://openai.com/research/gpt-4
[5] Id.
[6] David Ingram, Kevin Collier, OpenAI releases GPT-4, artificial intelligence that can ‘see’ and do taxes, NBC NEWS (Mar. 14, 2023), https://www.nbcnews.com/tech/tech-news/openai-gpt-4-rcna74916
[7] Introducing ChatGPT, OPENAI, https://openai.com/blog/chatgpt
[8] Id.
[9] Id.
[10] GPT-4, supra.
[11] Bobby Histova, Some students are using ChatGPT to cheat — here’s how schools are trying to stop it, CBC NEWS (Feb. 2, 2023), https://www.cbc.ca/news/canada/hamilton/chatgpt-school-cheating-1.6734580
[12] Id.
[13] Mara Leighton, The University of Minnesota law school tested ChatGPT on exams — it was a C+ student, INSIDER (Jan. 25, 2023), https://www.insider.com/law-school-used-chatgpt-on-exams-c-student-2023-1
[14] Id.
[15] Id.
[16] Id.
[17] Id.
[18] Id.
[19] Mara Leighton, Supra.
[20] Id.
[21] Bobby Histova, supra note 10.
[22] Id.
[23] Id.
[24] Faced with criticism it’s a haven for cheaters, ChatGPT adds tool to catch them, CBC NEWS (Jan. 31, 2023), https://www.cbc.ca/news/business/chatgpt-academic-cheating-1.6732115
[25] Id.
[26] Id.
[27] Id.
[28] Id.
[29] Id.
