USING IMAGES, NOT WORDS.

This is the third and final post in a series about the search for a pictorial language, by Nigel Holmes.

When considering a visual language, it’s worth looking at artists’ efforts to communicate without any words, and there’s an illustrious history of wordless books. The graphic novels that Frans Masereel and Lynd Ward made in the early 20^th century are great examples. They are stories that are told through a series of black and white woodcuts, one to a page, without a single explanatory word or caption. As you turn the pages you follow the story, as you might if you flipped through a series of still pictures from a movie. You add your own interpretations and feelings, and your imagination fills in the story from picture to picture.

In 2013, Xu Bing published “Book from the Ground: from point to point”. Unlike Masereel’s and Wards’s books, “Book from the Ground” is comprised of thousands of existing symbols, icons, tiny pictures, emojis, trademarks, roadsigns, numbers and punctuation (including question and exclamation marks) arranged in rows like text, to be read from left to right. And you can “read” it. It’s a little difficult…but nevertheless it is a pleasure to slow down and take the time to work out the meaning. I can read about two pages at one sitting. It makes you think about what pictograms really mean; and how the meaning can change depending on the context.

Has Bing invented a new language? Not quite. The story is 24 hours in his life. You know that because there’s a little text on the back cover, the only regular words in the book. The more you read the book, the easier it is to understand how Bing is telling the story; what his “writing” style is. The tiny pictures don’t exactly make sentences, they represent a sequence of actions and thoughts.

One thing is clear in the book: all the pictures look like the things being described. And that must be the rule for a truly visual language.  If the inventors of such languages have to resort to abstract symbols and other marks to modify their pictures, then readers have to know what those marks mean. They have to learn another language, before they can see meaning in the string of pictures. Bing does use brackets enclosing a group of symbols to indicate thoughts about, or explanations of, the preceding “text.” Other forms of punctuation are used, but the images—the “words”—are all recognizable. He brushes his teeth, you see brush and toothpaste. Fedex delivers a package, you see the Fedex logo, and a box.

The forerunners of emojis were emoticons (emotional icons)—punctuation marks and other typographic characters arranged to make keystrokable pictures of faces. They first appeared in Puck Magazine, in 1881.

Modern emoticons were invented by Scott Fahlman in 1982. According to Pagan Kennedy (writing in the New York Times Magazine) Fahlman’s smiley face 🙂 was intended “to take the sting out of mocking statements” and other differences of opinion on online forums. He called his invention a joke marker. There’s a collection of 650 emoticons in David Sanderson’s Smileys.

To see the icons above, you tilt your head sideways. In 1993, Tota Enomota published “Niko! The Smiley Collection”, with 200 non-head-tilting emoticons. Compared to western keyboards there are many more characters on Japanese computers. Some of Enomota’s examples are quite subtle.

But somehow, five or six, or even just three :-), keystrokes proved to be too much for people who wanted to add a tiny picture to their email messages. Enter emojis (a literal translation from the Japanese is “picture letter”). In 1999, in Japan, Shigetaka Kurita created 176 12-pixel by 12-pixel characters. They were recently added to the Museum of Modern Art collection.

In 2010, emojis were added to the International Unicode Standard, a non-profit group of programmers who wanted to standardize the coding of fonts so that computers and phones all talked to each other in the same way. There are now roughly 1,850 emoji characters on the Unicode list. It’s a big number because many of the human faces are available in a range of different skin tones.  You can look them up on emojipedia.org.

Both Scott Fahlman and Shigetaka Kurita think “current emoji standards are ugly compared to their ancestors.” (From “Emojis, the secret behind the smile”, by Marty Allen, 2015.) I agree. Wouldn’t it be great if they were simple, flat, iconic pictograms? Why does everything have to be so fully rendered?

Funnily enough, one of the things that you can do with emojis is surprisingly similar to a principle that Otto Neurath used in Isotype, and Charles Bliss, too: joining two icons to make a third. Emoji ZWJ Sequence (Zero Width Joiner) is way of combining more than one emoji (digitally, on the keyboard) so that it displays as a single emoji.

This approach offers possibilities for a more nuanced language. A string of single emojis doesn’t constitute language at all; it’s just a way of adding fun to a message, and shortening it.

Fred Benenson has “translated” Moby Dick into emojis. “Emoji Dick” reprints all of Herman Melville’s words with the “translations” above each sentence. Benenson is a data engineer at the fund raising site Kickstarter, and he used Kickstarter to raise money to pay the 800+ Amazon Mechanical Turk workers to translate the novel’s approximately 10,000 sentences. The book is $200 for the color version ($40 for black and white), but you can download a free pdf of the whole thing (750 pages!) here: www.czyborra.com/unicode/emojidick.pdf

Unlike the Bing book, I have not cracked the code of how the emojis actually do translate the Melville’s text. Here’s the famous first line (“Call me Ishmael”).

I see a phone, and a whale…but it’s so cryptic, that I wonder if the whole thing is a joke. (An expensive joke.)

So unless it is a kind of joke, (and even if it isn’t!) Emoji Dick is not a wonderful advert for translating literature into emojis.

Here’s my point: strings of emojis, or emoticons of any type are not new pictorial languages. They are, at best, messages. Messages that the mind completes. “You get the message” or “Know what I’m saying” mean that the message is not all spelled out. Perhaps that’s enough for most people. Emojis are fun, but they are faddish—currently used by the pretty young (6-9 year-olds?), and the pretty old (60-plus, who may think they are being computer literate). Or perhaps we oldsters are just messaging with our emoji-addled young grandchildren. They certainly have more patience than I do going through all the tiny choices!

But if there is any way a universal visual language can be created, it’s probably via computing. Today, two thirds of people in the world have a smartphone. By 2019, that’ll be five billion people; this is how we communicate (sadly). If we can type, or say, “thank you” in Google Translate when writing to a friend in Germany, and get “danke” back in an instant, perhaps the newly updated and improved AI version of Translate will be able to “translate” regular written or spoken language into a new, as yet undrawn, set of (hopefully) flat and iconic pictograms.

Of course, since the point is to communicate, why not ask Google Translate to put your message into the receiver’s native language? Who needs pictures now? (And, if you have emojis on your phone now, they will pop up as stand-ins for some words).

I think designers will keep trying. We will look back to “Safo” (“mind writing”), a language developed from Chinese Hanzi by Andreas Eckhardt (1884–1974), who in turn looked at Gottfried Liebnitz’s work on universal scientific notation in the late 1600s, and “Solresol”, a language developed by Francois Sudre (1787–1862), which was based on the notes of an octave. It could be sung, spoken, or played on a musical instrument.

Language changes all the time. One day we’ll make a pictorial one that works. Everyone says we are in the age of visuals. With Otto Neurath alongside us, let’s get going, (and let’s have fun doing it).

Worth a look:
“Book from the Ground: from point to point”, by Xu Bing. 2013.
“Niko! The Smiley Collection”, by Tota Enomoto. 1993
“Emojis: The Secret Behind the Smile”, by Marty Allen. 2015

Thank you for reading! (Art by the great Gerd Arntz.)