The DALL-E are Coming
Can AI render illustration obsolete?
A new artificial intelligence tool was launched in April which gave users the ability to automatically generate images from text, instantly drawing a new path for the practice of illustration and raising new questions about the role of AI in the creation of art. Titled “DALL-E 2”, the project was built by OpenAI, a San Francisco-based research lab founded in 2015 by Elon Musk, Sam Altman, and Peter Thiel, among others, whose mission is to develop and promote AI technology that “benefits all of humanity”. The purpose of the DALL–E 2 program — a portmanteau of Salvador Dalí and Pixar’s WALL-E — is to use text-based prompts to create images and art from a neural network, essentially visualizing ideas automatically. While DALL-E 2 is far from the first and only AI Art tool — other prominent tools include Midjourney and Craiyon (fka DALL·E mini) — it is by far the most sophisticated, powerful, and increasingly, controversial.
The current policies given to DALL-E 2 users are firm. There is no hate, harassment, violence, sex, or nudity allowed, along with anything related to politics or politicians. Public registration is also closed — for now — partially in order to allow for the rules to be properly enforced. In line with OpenAI’s utopic vision and hope that it becomes mainstream in the image-making community, the policy also proactively blocks certain phrases and obscures faces in order to prevent malicious deepfakes.
In spite of OpenAI’s numerous steps taken thus far to mitigate and untrain bias and safeguard against the potential for abuse of the tool, DALL-E 2 in its current iteration does not exist without the potential for harm. As is commonly pointed out, AI technology often raises an array of moral and ethical challenges with outputs that can be incredibly discriminatory and biased. DALL-E 2 is no different, with clear potential to produce images that are discriminatory and based on racist, sexist stock artwork that it may draw its input from (their algorithms are opaque), a problem that could have been arguably been avoided had they trained the models on less prejudiced data.
Despite the fact that the program is far from perfect, it is also remarkable, a terrifying mirror to society that creates beautiful and confounding results.
Many have raised the question of whether or not AI art generators could replace the practice of illustration, a debate that has now been revived as DALL-E 2 increasingly becomes ubiquitous. And as a working illustrator, I say no. The principal challenge to using the tool to intentionally create images for dissemination is that the outcomes are predictably unpredictable. You might be looking for something specific and it will output something completely different, or even misunderstand the subject and style completely. The images are often cropped poorly, there is always some sort of AI artifact, and the resolution is inadequate. While I do think these technical problems will inevitably be fixed, I don’t think the issue of unpredictability is sufficiently surmountable to allow for the tool to replace working artists on a large scale.
Putting commercial concerns aside, I would argue that unpredictability is the key to the tool’s power. The robots are able to make connections between forms that our minds are unable to make, creating new formal relations between unrelated subjects and aesthetics. The practice of combining two art forms into one isn’t at all new — it’s one of the best ways to create new styles — but doing so in this manner is fully contemporary. Beyond converting text to images, DALL-E 2 also allows you to create “variations” of existing images, which creates new and unexpected forms from pre-drawn pictures and allows the artist to look at their work from a new perspective.
I was recently struck by this tweet by tech journalist Casey Newton where he used DALL-E 2 to generate a piece of art for his newsletter The Platformer:
The prompt he used to create the image was “facebook logo inside a tornado, digital art.” It created a visual that is both aesthetically unsurprising and conceptually expected. The form of the Facebook logo is in no way related to the form of a tornado, the metaphor of a tornado does make sense but is difficult to render in a 3D digital art style, and the output ultimately looks like a stock illustration from a site like Getty. I think this gets at the heart of the problem of artificial intelligence artworks: they are only as interesting as their prompts and the models they are trained on. The 3D Facebook tornado looks like stock art because the model was likely trained on stock art. The visual is formally confusing because the prompt is confusing. And as the great designer and illustrator Bob Gill once said "Just because you have an idea it doesn’t mean that it’s good; the solution has to be inevitable."
The DALL-E 2 outputs that I’ve most often seen praised are similar to artworks from an artist such as Beeple, grotesque digital forms not unlike what you’d see on a site like DeviantArt (this is not a dig, I love the DeviantArt aesthetic, but it’s hideous!) And the Facebook tornado example does point to one possible job for the robots to undertake: that of cheap, quick, disposable illustrations. The pace at which editorial illustration is executed is often breakneck and the rates paid to artists can be dismal. Illustrators rarely, if ever, get union representation, and their rights are always at risk of being exploited. There is also an opportunity here for those without illustration budgets to instead use DALL-E 2; in a piece for The Verge, Newton stated: “It’s also worth considering what creative potential these tools might open up for people who would never think (or could afford) to hire an illustrator”. Sure, the results will look cheap, but you get what you pay for.
I’m reminded of a former internship where I cut people out from photographs so that a motion designer could animate them. My months spent mastering the pen tool were eventually rendered obsolete with the development of sites like Slazzer and Adobe’s new background image remover, and I could not be happier. Cutting people out in Photoshop was boring and tedious, and I hated every moment of it because the work felt robotic. AI is best at mechanical tasks, and the ability to cut those corners is freeing, not dangerous. These tools could theoretically give illustrators the time and space to make work more meaningful to them thanks to the budgets freed up by automating out all the easy stuff.
Just as an experiment, I tried to see if DALL-E 2 could recreate a recent illustration I made of Elon Musk eating a Twitter bird for The New York Times. I put in the prompt “Elon Musk eating a blue bird” and it was rejected for violating its community guidelines. I tried something less violent — “Elon Musk puts a blue bird in his mouth” — but was also nixed until I realized the words that were banned were in fact, “Elon Musk”. I eventually gave up. Your work is only as limited as your tools, and hitting the walls this quickly was surprising, but maybe not so surprising, considering Musk is one of OpenAI’s founders.
In another example, I used DALL-E 2 to create the artwork at the top of this piece, and I got much better results. The illustrations I have made for this publication typically pull from clip art from the 90s, and DALL-E 2’s models certainly drink from that same pool. Typing in the brief for an illustration as the DALL-E 2 prompt or finding new variations from already made sketches or even drawing ideas using AI are some use cases that I’m excited to try.
I for one, welcome our artistic robot overlords and all their new forms and automatic illustration skills, albeit with a dash of skepticism. I do not fear them, and I remain perhaps naively hopeful of the possibility of freeing illustrators from making yet another low-budget piece and instead allowing them to make the kind of work they want to make — maybe even inspired by tools like DALL-E 2. Using the machine and sharing DALL-E’s delights with friends has been an absolute thrill, despite the many flaws that it presents. I don’t think that the AI robot artist army is ever going to replace the work of illustrators, but then again, maybe the machines are actually coming for all our jobs. For now, I will continue to joyfully experiment on DALL-E 2 and pray that maybe, just maybe, it may just bring about a new renaissance of form in ways we humans may have never been able to predict.
Like this newsletter? Feel free to support by donating to a local abortion fund.
Great to read a less alarmist take on Dall·E 2 from an illustrator. I've found reading people "worrying" about AI ending illustration often says more about their limited understanding of illustration than it does about the power of AI. Thanks for the thoughtful piece.