You Asked For It: AI Transcribes Imaginary Images Into Word Soup

A lot of the time, machine learning & artificial intelligence researchers don’t understand exactly how their programs work. Specialists said in a research released on the pre-print site Arxiv that DALLE-2 has a well-known problem with text. Many text questions, such as this one that asks for a “picture of the word aircraft,” result in creating photos that show nonsense language.

There is a secret language that the system appears to have formed on its own in this generated text, which they find. Using this nonsense language as an example, the model is more likely to create aircraft.

When requested to caption a discussion between 2 farmers, a graphic released on Twitter by Computer Science Doctoral student Giannis Daras displays them chatting, but the conversation balloons are packed with what seems to be gibberish.

To test the AI’s interpretations of these meaningless phrases, Daras started feeding them back into the network. This led him to discover that the AI seemed to understand what the farmers were saying when they spoke about veggies and birds.

His research addressed the very first vulnerability concern, which was the use of nonsensical cues as backdoor exploits or techniques to evade the filter. Text prompts that break policy guidelines can currently be filtered by Natural Language Processing technologies however nonsensical prompts can be exploited to get around these monitors. More crucially, nonsensical prompts that create visuals on a regular basis put our faith in these large generative models under scrutiny.

There is a possibility that the speech is more akin to noise in certain instances. Peer assessment of the article will tell us more, but there may still be something happening around that we do not really understand yet.