“Renaissance painting of a person sitting in an office cubicle, typing on keyboard, stressed”

After the news broke on AI-generated art winning a prize earlier in September, AI creative tools have been on my mind. On Sept. 28th 2022, the DALL·E image generator became open for everyone — so I decided to give it a try.

What is DALL·E?

DALL·E is a new AI system that can create realistic images and art from a description in natural language. According to the website, it combines text, images and art into new compositions, with over 1.5M users creating more than 2M images per day.

See Discord or Instagram for image examples.

As a new user, you’re greeted with a popup that describes different ways to use DALL·E:

• Generate images from a description in natural language
• Easily make realistic, targeted edits to images
• Create different variations of an image, inspired by the original

Expectation

I had hoped that Dall•E could serve as a powerful alternative to a stock photo library, creating unique imagery for use in a future project.

The homepage examples showcased various styles which you could add by appending the term to your prompt (ex. 3D, hand-drawn sketches, abstract/digital art, oil painting, impressionist, cyberpunk, vaporware, etc.). The more details you added to the prompt (ex. closeup, with dark background, in [location]), the better the result.

Adobe Photoshop: Neural Filters

The options were reminiscent of Adobe Photoshop’s Filters, which modify images with different styles or Getty images, which allow you to search for imagery filtering by color/mood/style/number of people/type of photography (editorial/creative).

Reality

I was pleasantly surprised with the quality of some prompts (pixel art, oil painting) and disappointed with others (ex. 3D render, doodle). However, without the search bar prompting me with suggestions, I also was often at loss as to how to modify my existing prompt or edit future prompts.

“Closeup of a whale and a scuba diver”

“Stuck on the Streets of San Francisco in a Driverless Car”

I could picture the photorealistic images above paired with a news story.

“Renaissance painting of a cat climbing the Tower of Babel”

“Oil painting of a bear, rabbit and chicken doing karaoke”

Different styles? Check.

But how well would it support implausible product ideas?

“Lamp in the shape of a lemon”

“Water bottle in the shape of a octopus”

“A bicycle with a lounge chair attachment”

Decent. The renderings were good quality.

How about an image representing a pixel character?

“Pixel art turtle with a backpack”

The results came out well.

Since all images were generated, prompts don’t need to be grounded in reality. Below were the coolest renders from all my experiments:

“A bowl of ramen that is a warp portal to space, digital art”

I also explored variations on top of a previously generated image, which had some fantasy novel vibes, a brooding character with a mysterious past:

Variations on “Blind man with long silver hair, digital art”

What is Outpainting?

Outpainting is a new feature, the latest addition to Dall•E which allows you to extend your image and “set your visuals free! No more simple squares… make landscapes!”

Outpainting

I used a previously generated image (“Oil painting of a bear, rabbit and chicken doing karaoke”) and extended it for a fun result (though you can still see the seams where I moved the generation frame).

Outpainting: “Oil painting of a bear, rabbit and chicken doing karaoke”

What didn’t work as well?

Credit: Giphy

In addition, I wanted to try out some terms that might be part of an impromptu story. Unfortunately there were fewer generated results with poorer quality.

Next, I wanted to see the quality of the 3D renders. The results were a bit disappointing. Also, one of these images does not belong.

Finally, I tried to generate something that might be part of an illustration asset library: “polygon sea creatures”. The style came out with decent quality. Though again, one of these images does not belong.

“Polygon sea creatures”

How might artists or designers benefit?

I can see how this tool has potential for art direction, mood board creation for branding, brainstorming or asset creation (ex. illustration, game assets, product rendering, avatar creation, etc.).

Having a shareable mood board with generated imagery (especially in the age of remote/hybrid work) certainly has potential to help people have more effective discussions around the long-term creative direction or getting alignment to clarify a vision. It’d be interesting to funnel imagery from an existing mood board (ex. Pinterest collection) to create new ideas with a similar mood from within DALL·E.

Today, while DALL·E might not have the nuanced editing or filtering capabilities of more mature design products, a quick export to another tool could partially meet that gap.

While there’s no way to download texture maps or 3D models based off imagery today, refining imagery with the help of a tool still could save time and effort. After I’ve already created the original artifact and defined the intended stylistic direction, I can see how Dall•E could help generate even more ideas, akin to musical variations on a theme.

What about the ethical elephant in the room?

There’s a common saying in machine learning which also applies to AI generated art: “garbage in, garbage out”. The quality of the dataset affects the subsequent results.

There are open questions around the content that I create that’s been used for training data and the degree of control I have over things I’ve shared online in the past. Mainly, I find the concept of some entity (person or company) profiting over my work without my knowledge and permission deeply undesirable.

The cynical part of me grudgingly accepts that this tool may eventually be misused for plagiarism in a way that harms a large community of existing artists, just as photo galleries have powered facial recognition systems without the clear consent of people in the photographs. And that the image generation platforms won’t sufficient oversight to be held accountable after the fact.

Several questions came to mind:

• How might we protect creative rights and attribution?
• How might we verify content authenticity?
• How might this tool support future creative work?

promptbase.com

If one wishes to earn money off what they’ve created, there’s a marketplace for prompts that allows people to earn by sharing the underlying prompts that created the artwork.

But my concerns remain around protecting artists and designers who have always had to fight distressing battles against plagiarism in a whack-a-mole fashion. Artists, illustrators, photographers or designers whose work likely will feed the systems underlying the generated images — I imagine that this tech will expose new legal questions around ownership and copyright.

Plagiarism is a deep and long-standing problem in the artistic community and the distinction between ‘inspired’ and original work is fuzzy. AI image generation tools threaten to push the definition of artistic ownership into even murkier waters. The recent attention around NFTs have likely led to a rise in plagiarism cases as well.

Common tactics against art theft (watermarks, low-rest images, preventing downloads) wouldn’t prevent your ‘style’ from being stolen.

Outpainting: “A chef cooking ramen, digital art”

On the other hand, my experience had memorable highlights— from rendering magical bowls of ramen to seeing how Outpainting gradually made a simple oil painting piece more vivid and colorful. The process was new to me and I played more of the role of a curator (picking among options) and art director (telling the system what I envisioned) than doing hands-on work to execute an idea.

I could also envision combining writing and image generation into a playful storytelling tool (ex. generated illustrations alongside a short story for an easy storyboard).

Conclusion

midjourney.com

When browsing Midjourney, an AI image generation tool similar to DALL·E, the output looked quite similar to trending illustrations from the gallery from any major digital art platform. The fantasy pieces were particularly stunning with intricate details.

curaturae.com

While my concerns remain, I don’t want to dismiss new tools pre-maturely.

Weeks prior, I explored poetry alongside Smithsonian’s Open Access: a large dataset bringing a museum’s artifacts to the public online. The experience reminded me of the ways art (or code) involve ‘remixing’ things from collaborators in the past and present, as well as how technology might enable fair access for people to create in new ways.

As new questions (and offenses) emerge around content authenticity and attribution through the rollout of DALL·E, I’m cautious to see how the image generation tool landscape unfolds, affecting the broader health of the internet and amplifying the creative process.