June 18, 2025
The GIST Psycholinguist talks nonsense to ChatGPT to understand how it processes language 
Stephanie Baum
scientific editor
Robert Egan
associate editor
Editors' notes
This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:
fact-checked
peer-reviewed publication
trusted source
proofread
A new study appearing in PLOS One by a psycholinguist at the University of Kansas explores how ChatGPT, the popular artificial-intelligence chatbot, responded to nonwords.
Nonwords are meaningless letters and sounds used in cognitive psychology to explore how humans learn, remember and process language.
Author Michael Vitevitch, professor in the Speech-Language-Hearing Department at KU, wanted to know how the "black box" of ChatGPT might react to similar experimental prompts.
"As a psycholinguist, one of the things I've done in the past is to give people nonsense to see how they respond to it—nonsense that's specially designed to get an understanding of what they know," he said. "I've tried to use methods we use with people to appreciate how they're doing what they're doing—and to do the same thing with AI to see how it's doing what it's doing."
By feeding nonsense to ChatGPT, Vitevitch found it was quite good at discovering relationships between things.
"It finds patterns, but not necessarily the same patterns that a human would use to do the same task," he said. "We do things very differently from how AI does things. That's an important point. It's okay that we do things differently. And the things that we need help with, that's where we should engineer AI to give us a safety net—to help us on those things."
In one part of the study, ChatGPT showed it could provide correct definitions for many actual English words that are no longer in use. One example is "upknocking," an occupation from the 1800s that involved waking people up, now made obsolete by alarm clocks.
As these words are no longer part of English, researchers consider them "nonwords."
The results are illustrative of the chatbot's internal workings. Of 52 carefully chosen extinct words, ChatGPT responded with 36 correct definitions. For 11 words, ChatGPT responded that it couldn't recognize the word. For three words, the chatbot gave definitions gleaned from words in foreign languages.
For two words, ChatGPT appears to have made up answers, giving so-called "hallucinations" in response to the word-definition requests.
"It did hallucinate on a couple of things," Vitevitch said. "We asked it to define these extinct words. It got a good number of them right. On another bunch, it said, 'Yeah, I don't know what this is. This is an odd word or very rare word that's not used anymore.' But then, on a couple, it made stuff up. I guess it was trying to be helpful."
In another part of the experiment, ChatGPT was given a set of Spanish words and asked to provide similar-sounding words from English. The task is much like the "phonological associate task" used with English speakers to gauge aspects of phonological similarity.
Vitevitch said the experiment highlighted the difference in how humans and AI process language.
"If I give you a Spanish word and tell you to give me a word that sounds like it, you, as an English speaker, would give me an English word that sounds like that thing," he explained. "You wouldn't switch languages on me and just kind of give me something from a completely different language, which is what ChatGPT did.
"The only time people switch languages like that is when multilingual parents who didn't teach their kids any language besides English want to talk about something without the kids understanding."
In a third aspect of the study, nonwords were designed to resemble real words in English. ChatGPT was then asked to rate the words on a scale from one for a "Bad English word" up to seven for a "Good English word"—its assessment of how much the nonword sounded like a word in English. ChatGPT's ratings were then compared against those given by human English speakers.
Lastly, the KU researcher prompted ChatGPT to create new English words that refer to a novel concept—much like the "sniglets" popularized in the 1980s by comedian Rich Hall on HBO's "Not Necessarily the News."
"He used to do 'sniglets,' which were words that don't exist," said Vitevitch. "Like with a vacuum cleaner, when there's a thread on the floor and you go over it and it doesn't get sucked up. So, you go over it—again and again. What is that thread called? 'Carperpetuation.' He came up with a name for that thread that doesn't get sucked up. So, we have many things that should have a word, but we don't have one in English."
According to Vitevitch, the AI chat bot did "kind of an interesting job there." After prompting ChatGPT for new words that matched certain concepts, he found it often relied on a predictable method of combining two words.
"My favorite was 'rousrage,' for anger expressed upon being woken up," said Vitevitch.
Other ChatGPT word coinages from the research include:
- "Prideify:" Taking pride in someone else's achievement.
- "Lexinize:" The process by which a nonword begins to take on meaning.
- "Stumblop:" To trip over one's own feet.
By talking nonsense to ChatGPT, Vitevitch hopes to better understand when AI might be able to help humans in certain language tasks rather than simply duplicate what humans can already do quite well.
More information: Michael S. Vitevitch et al, Examining Chat GPT with nonwords and machine psycholinguistic techniques, PLOS One (2025). DOI: 10.1371/journal.pone.0325612
Journal information: PLoS ONE Provided by University of Kansas Citation: Psycholinguist talks nonsense to ChatGPT to understand how it processes language (2025, June 18) retrieved 18 June 2025 from https://techxplore.com/news/2025-06-psycholinguist-nonsense-chatgpt-language.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Explore further
Like humans, ChatGPT favors examples and 'memories,' not rules, to generate language 0 shares
Feedback to editors
