AI is good at many things, like finding patterns in data, creating realistic images, and condensing thousands of words into a few paragraphs, but does it write great comedy scripts and jokes?
New research shows that it can, but only to a very limited extent. It’s an interesting finding that suggests AI can help with creative work, but only up to a point.
Piotr Mirowski is a researcher at Google DeepMind. In his spare time, he is also an improv comedian. He and other deepmind researchers studied the experiences of professional comedians who use artificial intelligence in their work.
They combine surveys and focus groups and aim to measure the usefulness of artificial intelligence in different tasks.
They found that while popular AI models from OpenAI and Google were effective at simple tasks, like constructing monologues or crafting rough first drafts, they struggled to produce content that was original, stimulating, or interesting.
They presented their findings earlier this month at the ACM FACCT conference in Rio, Brazil, but did not reveal the names of the participants to avoid damaging the actors' reputations (not all comedians want the audience to know They use artificial intelligence).
researchers asked 20 professional comedians who already use artificial intelligence in their artistic process to use large language models, such as chatgpt or Google gemini (or bard), to generate content they felt would be appropriate in a comedy performance situation.
They can use it to help create new jokes, or to repurpose existing comedy material. The results of the
experiments were mixed. While comedians say they largely enjoy using AI models to write jokes, they aren't particularly proud of the resulting material.
Some of them said that artificial intelligence can be used to draft on a blank piece of paper and help them generate something quickly.
One participant likened it to “a first draft that has just been brainstormed and I know I have to keep iterating and improving it.”
Many comedians also pointed to the ability of large language models to generate structures for comedy sketches, leaving them responsible for fleshing out the details.
However, the quality of comedy material generated by large language models leaves a lot to be desired. Comedians think the model's jokes are bland, generic and boring. One participant likened it to “1950s cruise ship comedy material, but less racist.”
Others believe that the effort is not proportional to the reward. One comedian said: "No matter how I prompt it, it can only give a very rigid, linear comedy strategy."
It is not surprising that artificial intelligence cannot generate high-quality comedy material. The safety filters that openai and Google use to prevent models from generating violent or racist content also prevent them from generating material common in comedy, such as jokes and dark humor that are offensive or sexually suggestive.
Therefore, large language models can only rely on content and information that is considered "safer", including large amounts of documents, books, blog posts, and other types of Internet training data. "If you make something that has broad appeal to everyone, it's going to end up being something that no one likes," Mirovsky said. The
experiment also exposed the biases of large language models. Several participants found that models would not create comedic monologues from an Asian woman's perspective but could do so from a white male's perspective. They argue that this phenomenon reinforces the status quo while erasing minority groups and their perspectives.
In fact, it’s not just safety guardrails and limited training data that prevent large language models from producing interesting content.
Tuhin Chakrabarty, a computer science researcher at Columbia University who specializes in artificial intelligence and creativity, said humor relies heavily on surprising and incongruous baggage, which is inconsistent with these models. Inconsistent ways of working."Creative writing requires a lot of imagination, and large language models can only imitate it," added
, who was not involved in the study.
He said: "Comedy, or any kind of good work, will play a long game (ambush), and the ultimate goal is to return to the theme or surprise the audience. It is difficult for large language models to do this because they only predict once One word."
"I've tried a lot in my own research to make AI fun, surprising, interesting or creative, but it just doesn't work," he said.
colleen lavin, a developer and comedian, participated in the study.
In order to perform a stand-up at the Edinburgh Fringe Festival in 2023, she trained a machine learning model to recognize laughter and "question" her when it found she wasn't getting enough laughs.
While she uses generative AI to create promotional material for her show or check her grammar, she draws the line at using it to generate jokes.
She said: "My daily work is full of technology, and writing has nothing to do with it. It seems to be sacred and inviolable. Why should I outsource something I really like to a machine?"
Chakraborty said, although Comedians assisted by AI might work faster, but their ideas won’t be original because they’ll be limited by the data the model was trained on.
He said: "I think people will use these tools to write scripts and commercials anyway. But real creative and comedy writing is based on experience and atmosphere, not algorithms."
Support: ren
Operation/Typesetting: He Chenlong
