One of the most challenging difficulties in the next years will be distinguishing between AI-generated and human-generated material. Deepfake films that are practically indistinguishable from reality have already been seen. People are now utilising OpenAI’s ChatGPT AI to generate conversational text that might be misinterpreted as human writing. Fortunately, new research reveals that there is at least one technique to distinguish between ChatGPT and human writing.
The University of Applied Sciences and Arts of Southern Switzerland (SUPSI) released a study last week regarding a machine learning model that could recognise ChatGPT-generated text (via ZDNet). The researchers gave ChatGPT the duty of automatically creating restaurant evaluations and rephrasing human-written restaurant reviews.
The researchers discovered that the main difference between a ChatGPT review and a human evaluation was whether or not sentiments were communicated. ChatGPT typically reports the experiences but does not explain how it responded to those encounters. Meanwhile, as you may know from reading Yelp reviews, individuals are more than ready to tell others, sometimes in great detail, how a dining experience made them feel.
Other indicators indicating the material was created by AI included a lack of usage of personal pronouns, the presence of strange vocabulary, and the absence of any hostile or harsh language. Is it truly a user-generated restaurant review if it isn’t rude?
“Looking into SHAP (SHapley Additive exPlanations) explanations of the predictions gives some insights about ChatGPT writing style,” the researchers conclude. “It is extremely polite, aiming to please different types of requests from various domains fairly well mimicking humans, but that still does not have the profoundness of human language (e.g. irony, metaphors,…).”
Although the machine learning model proved effective in differentiating between AI and human writing, it’s important to note that OpenAI’s language models will only continue to advance in sophistication. ChatGPT is presently running on GPT-3.5, although GPT-4 is already under development. The capacity of these language models to resemble human writing will definitely increase as they develop and evolve.
