Category: Art and Audio

DALL-E, my new game jam partner

DALL-E has been getting a lot of my attention lately. I signed up for the wait list to become a user and got an invitation a few days ago. DALL-E is much more capable than DALL-E mini, later renamed Craiyon, that I had played with previously. It generates images higher quality images, faster.

I’ve been using DALL-E like a bad graphic artist’s client, pestering it every few minutes for free concept art with the promise of “exposure” as its own reward. DALL-E doesn’t seem to mind, though. And we love each other. I ask it for whatever comes to mind, and it seldom disappoints me.

I decided to ask DALL-E to create box cover artwork for Mega Man, since why not. It occurred to me that this would be a good John Henry competition, pitting the new machine up against the mightiest artist to ever lift a colored pencil, to see who was better.

The original Mega Man box cover.

It turns out that DALL-E was more than up for the challenge.

I mean, none of these is exactly good, but all of them are awesome.

Most of these don’t feel like Mega Man, but they all have a pretty good Japanese Giant Robot vibe, which is just fine. Mega Man is a small robot, not a giant robot, of course, but that hardly matters — instantly, my imagination is fired by the idea of a 1960’s or 1970’s Mega Man anime imported to the US in the 80s alongside shows like Star Blazers, Mobile Suit Gundam, Mazinger Z/Tranzor Z, and Macross/Robotech, and Diaclone/Transformers… What a “What If?” to think about! So the nostalgia factor for this alternative Mega Man that never existed is powerful.

It makes me think that if I got back into game development, I’d use DALL-E to give me the initial inspiration, concept art, cover art, what have to get me going.

I have so many questions about DALL-E.

Why does it get the title wrong? Megan? Meggian? Megman? It can’t be hard to get the exact text out of the description I entered and replicate it in the image. Is this a deliberate nerf on the part of the developers to prevent users from creating meme captions that could be offensive?

Due to the way DALL-E can’t seem to get the words right for the titles, a lot of what it generates reminds me of this hilarious meme from years ago, which parodied corporate fast food logos by turning them into dada-ish nonsense. Which makes it unintentionally (or not) hiliarious and entertaining, but it also limits its usefulness for creating images I want to actually use.

Eef Freef!

It’s exciting to be alive at this time, getting to see these developments in AI. I don’t get the sense that DALL-E is truly intelligent, but what it does is impressive nonetheless. The images it creates are often quite good, and about as often are complete nonsense. But frequently this “nonsense” is either hilarious or entertaining, or has a superficial feeling of artisticness to it. If I knew a human had created some of these “artistic” images, I would be more inclined to ascribe creativity or meaning to them. But what when DALL-E does it, it feels more like it is holding up a mirror to our culture’s human-created art, as though it put the conceptual ideas of art and design into a blender, and reassembled them for us, not quite at random, but according to rules that don’t quite work yet, but still give the impression of… I don’t know, “something going on in there.” Often times the results it spits out give me the impression that I’m looking through a window into an alternate reality; as though, the many worlds theories of physics are true, and DALL-E is somehow extracting images of alternate realities from one of an infinitude of alternate universes in our multiverse. And from an information theory standpoint, perhaps that might as well be what it is doing.

When the images are purely visual representations, the impression of intelligence at work is more effective, but when the images contain words, it’s clear that DALL-E is just throwing shit at the wall to see what sticks, following rules of some kind, but not intelligently, not with deliberation or intent. We get nonsense words that are a jumble of letters that the DALL-E parser decided it thinks we wanted it to put into the image. And sometimes we get strange alternative glyphs, as though from some alien alphabet.

Much of the time the images DALL-E creates could be used to stimulate human creativity, to give a human creator a spark that they could use to work from, an initial inspiration. That’s a valuable thing in itself. And I’m sure the potential is there for it to continue to develop its abilities, and I expect that in time AI development will give us more capable AI. That seems almost a given.

Seeing what DALL-E would create was so fun that I asked it to create alternative cover art for many of my favorite games. There were many that I thought would be fun to share.

Asteroids

Dig Dug

Double Dragon

Elevator Action

Commando

Bionic Commando

Metal Gear

Mega Man

Castlevania

Legend of Zelda

Tetris

Contra

Metroid

Super Mario Bros.

Super Mario World

Ninja Gaiden

Gradius

Pac Man

Ms. Pac Man

Space Invaders

Frogger

Punch-Out!!

Disney’s DuckTales

Teenage Mutant Ninja Turtles

Bad Dudes vs. Dragon Ninja

These are amazing, especially considering the amount of human effort it takes to create them — literally just typing in a few words and clicking a button. They are so fun to look at, to anticipate what DALL-E will come up with while the progress bar fills. Each one is as interesting for how it fails as it is interesting for how it succeeds. Each seems like an alternative universe’s version of the game title. The art style varies — some of it looks like the illustration on a children’s cereal box, a lot of it looks like the cover of a paperback book or movie, while others look like the box a board game is stored in. Tetris appears to have strong links with Rubik’s Cube, and here and there I can pick up hints of other influences. It’s especially fun to see when DALL-E “knows” the game title, and picks up on stuff that is recognizable from the game, and uses it in the images it composes.

Open World: Video Games and Contemporary Art

Open World opened last Saturday, October 19th at the Akron Art Museum. I attended the opening, and was very impressed with the exhibit. It is a large installation, covering three of the museum’s galleries. The works included cover a wide range of media, from ball point pen drawings to video to prints to sculpture to textiles to interactive media and virtual reality.

Experiencing Cory Arcangel’s I shot Andy Warhol, which is a romhack of Hogan’s Alley for the NES, for art’s sake. And yes, I got the high score.

It’s exciting to see the art world acknowledge the importance and influence of videogames on fine art.

It’s been about 15 years since famed film critic Roger Ebert famously proclaimed that videogames were not art, and could never be. He was wrong about that in so many ways, although to be fair to his argument, we should seek to understand what he meant by that. The word “art” has multiple definitions, and this is a confusing and contentious point, which can trip up many conversations before they even begin as people talk past one another without realizing it. Untangling that mess requires more words than I have time to type here.

But if I can bottom line it, Ebert was wrong, but he had a few good points.

Art is a very broad word, and to think it couldn’t include videogames is simply short-sighted and more than a bit bigoted. To make a pronouncement that games can never be art is arrogant. And of course games are art. Game design is an art, games are comprised of program code, graphics, and audio, and all of these require an artist’s touch in order to come alive.

But no, not every game is a work of high art. Just as not every book or film is art. Not every statue or painting is art. And sure, most video games are thought of primarily as commercial kitch intended for mass entertainment. But sure, a video game can be an object d’art. Why not? There’s an entire genre of videogames called “art games“, which are intended to be experienced as art.

But… wtf is art? Which definition are we using each time we say the word?

Well, that’s an important question, but never mind that. My goal isn’t to write a book about the definition of art, and argue that videogames are, or can be, art. We could spend time exploring that, and it’s not like that wouldn’t be worthwhile. But that’s not my point in writing this post; my point is to talk about the Open World exhibit at the Akron Art Museum, and how you should go see it.

Krista Hoefle

Why not simply go into the world and look at some art, and see if any of it is a videogame? And why not explore the world and find examples of art that show a clear influence from videogames, a clear sign that videogames are culture, that video games are a force that shapes and influences humanity, and has been, for decades, from very nearly the very beginning of the history of computing machines.

It turns out this is a rewarding endeavor. As much as it’s important to think about what art is and isn’t, its much better to experience art, and engage with it.

The exhibit does this very well, I think, by taking a broad survey of different approaches different artists have taken, and the different ways that video games have influenced them in the creation of art.

One of the artworks in the show is a video game: Cory Arcangel’s I shot Andy Warhol, a romhack of Hogan’s Alley for the NES, which simply substitutes images of Andy Warhol, the Pope, Colonel Harlan Sanders, the founder of KFC, and Public Enemy hype man Flava Flav for the usual graphics, to make a statement of some sort, about the historical fact that Andy Warhol was actually shot in real life. What that statement is exactly, I’m not entirely sure. But there it is, a playable video game, presented as art, in an honest-to-god Art Museum. Suck on that, Ebert.

Feng Mengbo‘s Long March: Restart, is another playable videogame, and incorporates numerous sprites from 8- and 16-bit run-and-gun games such as Contra, is another game, but was not playable on the opening day due to technical difficulties.

A lot of artwork that people might think of when they hear “art influenced by video games” would fall under the category of “fan art” — simply, works created by fans, done in homage to a favorite game, or character from a game, or to create feelings of nostalgia. This isn’t really what Open World is going after. The artwork doesn’t serve to celebrate commercial products. But there are a few pieces that might come close, such as the Colossal Cave Adventure quilt made by Krista Hoefle, one of my favorite pieces in the exhibit, or Butt Johnson‘s brilliantly executed ballpoint pen drawings, which simultaneously reference both 80’s video game culture and the Italian renaissance.

Butt Johnson
Butt Johnson

But most of the works in the show are not games. Some are digital works, such as Tabor Robak’s 20XX, or Angela Washko’s “gaming intervention”, The Council on Gender Sensitivity and Behavioral Awareness in World of Warcraft. Washko’s investigation of Warcraft players’ attitudes on feminism tends to be buried in the visual chaos of WOW’s cluttered UI and fantastical character avatars, but it is nevertheless interesting for its chat content and the social dynamics she frames and puts on display in the context of a popular fantasy MMORPG.

Many of the works in the exhibit reference games in some way, or excerpt from them. Others borrow cues from the new aesthetic of video game graphics in creating artistic compositions, such as Invader’s pixel art created out of Rubik’s Cubes, or Mathew Zefeldt’s life-size barrel and door from Duke Nukem 3D.

Barrel and Door by Mathew Zefeldt

Still others use games as raw material, taking elements out of them, repurposing or recontextualizing them, turning them into art. Still others use a game to stage a sort of theatrical performance, sometimes called machinima. These often are done for social commentary, as with Joseph DeLappe’s Elegy: GTA USA Gun Homicides, which is especially powerful in its depiction of gun violence through a modded version of Grand Theft Auto 5.

The above only covers about half of the total show, so to see the rest of it, you’ll have to go in person. Open World is up from October 19, 2019 – February 2, 2020, and after that will be travelling to Currier Museum of Art March 21 – June 28, 2020 and San José Museum of Art September 10, 2020 – January 10, 2021.

Pixel Art Chess Set: Communicating function through design

My five year old nephew started learning to play Chess recently, as I discovered on a visit a few weeks ago.  We played two games, and I didn’t have too much trouble beating him, but for a five year old he’s not bad. He knows all the pieces and their basic moves and their relative value.

I thought it would be fun to build a video Chess game that he could use to help learn strategy and how to see the board. So this is my latest project. I’ll be posting more about that as I work on it.

My first step was to design graphic resources. I didn’t want to spend too much time on it, just a basic “programmer art” chess set, that I could use to build the program with. Of course, it didn’t end up that way, and I’ve gone down the rabbit hole designing variations on sets of minimalist pixel art chess men. It’s too fun and fascinating not to.

My first attempt was actually rather good, I thought. I went for 16x16px representations of the classic chess pieces. I drew them very quickly, but was very pleased with my results, particularly the Knight.

I could have stopped right there, but it was so fun to make this set that I wanted to continue exploring and see if I could refine my pixeling technique further.

I decided to search the net for images of chess pieces to see what variety there was, and found a lot of examples of beautiful sets. I started to take notes and to infer design rules for chess men:

  1. Chess pieces are called “chess men” which seems antiquated and sexist, especially given that the most powerful piece in the game is the Queen.
  2. The modern standard chessmen are a design named for English chess master Howard Staunton, and have been in use only since the mid-19th century. A strength of its design is that each piece is easily distinguished from the others, making errors from mistakes in identifying a piece — a problem with other sets — unlikely. Previously, chess men of different types had a less distinct appearance from one another, and were not as standardized.
  3. In a Staunton set, the Knights are the most detailed, ornate, varied, and realistically represented pieces. 
  4. In Staunton sets, there is a standard height order: pawn, rook, knight, bishop, queen, king. (This surprised me, since Rooks are more valued in Chess I would have expected them to be taller than Bishops.)
  5. The pieces are differentiated by their tops. Each type of piece has a distinct, unambiguous shape.
  6. The body/base of the pieces have a common design, to create unity among the pieces in the set.

I tried to apply design choices to my chess set following these insights.

A follower on Twitter offered feedback that the pieces should be different heights, so I tried that. With a 16×16 pixel tile size, I could only shorten the back row pieces by 1-3 pixels.  I also tweaked the King piece by adding a few more pixels to its top, to make it a bit more distinct from the Queen, and moved the Pawn so that it would be more centered in its square.

I do like the result:

Staunton pixel chessmen

I think my initial 16×16 Staunton set look like they’re in ALL CAPS, while this set is more “readable” by using “mixed case” heights for the pieces.

I wanted my chess game to be focused on usability and instruction. I needed each piece to be immediately recognizable, and not to convey a bunch of extraneous information to the player that has nothing to do with play mechanics. 

My next attempt was a different take altogether. I wanted the look of each piece to suggest its rules for movement. I also thought that it would be clever if the pieces communicated the rules for using them through their visual design.

I ended up being very pleased with this set as well, although I went through many more variations, particularly with the Pawn. This one also came together easily and rapidly.  When your tile size is 16×16 and you’re working in just a few colors, it’s easy to work fast.

Things I like about this set:

  1. The shape of the piece is a built-in tutorial for how the piece moves.
  2. The Pawns still have a pawn-like shape (at least the black pawns; white pawns are “upside down”).
  3. The Knight’s shape may be read as an abstraction of the horse’s head shape of the Staunton piece.

I think out of these variations, my favorites are: P9, Kn2, B3, R1, K?  I’m least certain which King I like.  I think K4 and K5 are my top two, but I also liked the idea of incorporating a crown motif into the design, to signify the King’s special property of being the King.  K1, K2 and K6 were attempts at this, but I think K1 looks too much like a Staunton Rook, unfortunately.

I wasn’t sure which of my designs to use for my final set, so  I posted my sets on Twitter and a pixel art community on Facebook. @Managore responded to my request for feedback by coming up with a set of his own design, which I quite like.

His design was retweeted and liked over 500 times, and received many positive compliments from his followers, many of whom are game developers. One of my favorite indie developers, @TerryCavanaugh, who made VVVVVV and Don’t Look Back, pointed out an physical chess set that had been designed a few years ago which incorporated the same ideas.

It’s exciting to see my idea get picked up and reworked by fellow game developers who are inspired by the concepts I am exploring. So fun! Getting that validation that what I’m working on is interesting to others is very motivating. But it’s particularly good to get some attention from developers whose work I’ve admired for years, however modest.

I’m excited about this project and look forward to working on the program. I have more design ideas that I’m looking forward to getting into soon.