The article I chose is "Can a horse ride an astronaut?" by James McCammon. The author talks about “antagonistic prompts”, that push the image model beyond it’s training and semantic understanding.
He divides it into two main categories: “Inversion Prompts”, creating the opposite of the given prompt, and “Discordant Prompts”, resulting in unexpected objects and misunderstood intentions.
Under the first category, there are 3 types of prompts:
• Prototypical Inversion Prompts (PIPs): creating images opposite to the prompt.
• Constrained Inversion Prompts (CIPs): creating images that partially align with the opposite of the intended prompt.
• Essentialist Inversion Prompts (EIPs): when using the form "X without Y" will result in an image of "X with Y”.
Under the second category, there are 4 types:
• Homonymously Discordant Prompts (HDPs): the model incorporates multiple meanings of a word into a single image.
• Positionally Discordant Prompts (PDPs): a discordance between the desired position or angle specified in the prompt and the resulting image.
• Missing Element Prompts (MEPs): These prompts lead to the omission of a specified object or element in the resulting image.
• Semantically Discordant Prompts (SDPs): a semantic incongruity between the intended meaning of the prompt and the resulting visual representation.
At first, I checked out some of the examples in the article:
Then I thought of another semantic phenomena the model might not be able to understand - idioms.
Some idioms the model understood very well, like a person under the weather. But other idioms were more complicated:
(all the prompts started with "create a photo-realistic, highly detailed, 4k photo of:").
These are the idioms:
raining cats and dogs • cloud nine • burying the hatchet • getting the axe • over the moon • in a pickle • butterflies in the stomach