USING IMAGES, NOT WORDS.

This is the third and final post in a series about the search for a pictorial language, by Nigel Holmes.

When considering a visual language, it’s worth looking at artists’ efforts to communicate without any words, and there’s an illustrious history of wordless books. The graphic novels that Frans Masereel and Lynd Ward made in the early 20^th century are great examples. They are stories that are told through a series of black and white woodcuts, one to a page, without a single explanatory word or caption. As you turn the pages you follow the story, as you might if you flipped through a series of still pictures from a movie. You add your own interpretations and feelings, and your imagination fills in the story from picture to picture.

In 2013, Xu Bing published “Book from the Ground: from point to point”. Unlike Masereel’s and Wards’s books, “Book from the Ground” is comprised of thousands of existing symbols, icons, tiny pictures, emojis, trademarks, roadsigns, numbers and punctuation (including question and exclamation marks) arranged in rows like text, to be read from left to right. And you can “read” it. It’s a little difficult…but nevertheless it is a pleasure to slow down and take the time to work out the meaning. I can read about two pages at one sitting. It makes you think about what pictograms really mean; and how the meaning can change depending on the context.

Has Bing invented a new language? Not quite. The story is 24 hours in his life. You know that because there’s a little text on the back cover, the only regular words in the book. The more you read the book, the easier it is to understand how Bing is telling the story; what his “writing” style is. The tiny pictures don’t exactly make sentences, they represent a sequence of actions and thoughts.

One thing is clear in the book: all the pictures look like the things being described. And that must be the rule for a truly visual language.  If the inventors of such languages have to resort to abstract symbols and other marks to modify their pictures, then readers have to know what those marks mean. They have to learn another language, before they can see meaning in the string of pictures. Bing does use brackets enclosing a group of symbols to indicate thoughts about, or explanations of, the preceding “text.” Other forms of punctuation are used, but the images—the “words”—are all recognizable. He brushes his teeth, you see brush and toothpaste. Fedex delivers a package, you see the Fedex logo, and a box.

The forerunners of emojis were emoticons (emotional icons)—punctuation marks and other typographic characters arranged to make keystrokable pictures of faces. They first appeared in Puck Magazine, in 1881.

Modern emoticons were invented by Scott Fahlman in 1982. According to Pagan Kennedy (writing in the New York Times Magazine) Fahlman’s smiley face 🙂 was intended “to take the sting out of mocking statements” and other differences of opinion on online forums. He called his invention a joke marker. There’s a collection of 650 emoticons in David Sanderson’s Smileys.

To see the icons above, you tilt your head sideways. In 1993, Tota Enomota published “Niko! The Smiley Collection”, with 200 non-head-tilting emoticons. Compared to western keyboards there are many more characters on Japanese computers. Some of Enomota’s examples are quite subtle.

But somehow, five or six, or even just three :-), keystrokes proved to be too much for people who wanted to add a tiny picture to their email messages. Enter emojis (a literal translation from the Japanese is “picture letter”). In 1999, in Japan, Shigetaka Kurita created 176 12-pixel by 12-pixel characters. They were recently added to the Museum of Modern Art collection.

In 2010, emojis were added to the International Unicode Standard, a non-profit group of programmers who wanted to standardize the coding of fonts so that computers and phones all talked to each other in the same way. There are now roughly 1,850 emoji characters on the Unicode list. It’s a big number because many of the human faces are available in a range of different skin tones.  You can look them up on emojipedia.org.

Both Scott Fahlman and Shigetaka Kurita think “current emoji standards are ugly compared to their ancestors.” (From “Emojis, the secret behind the smile”, by Marty Allen, 2015.) I agree. Wouldn’t it be great if they were simple, flat, iconic pictograms? Why does everything have to be so fully rendered?

Funnily enough, one of the things that you can do with emojis is surprisingly similar to a principle that Otto Neurath used in Isotype, and Charles Bliss, too: joining two icons to make a third. Emoji ZWJ Sequence (Zero Width Joiner) is way of combining more than one emoji (digitally, on the keyboard) so that it displays as a single emoji.

This approach offers possibilities for a more nuanced language. A string of single emojis doesn’t constitute language at all; it’s just a way of adding fun to a message, and shortening it.

Fred Benenson has “translated” Moby Dick into emojis. “Emoji Dick” reprints all of Herman Melville’s words with the “translations” above each sentence. Benenson is a data engineer at the fund raising site Kickstarter, and he used Kickstarter to raise money to pay the 800+ Amazon Mechanical Turk workers to translate the novel’s approximately 10,000 sentences. The book is $200 for the color version ($40 for black and white), but you can download a free pdf of the whole thing (750 pages!) here: www.czyborra.com/unicode/emojidick.pdf

Unlike the Bing book, I have not cracked the code of how the emojis actually do translate the Melville’s text. Here’s the famous first line (“Call me Ishmael”).

I see a phone, and a whale…but it’s so cryptic, that I wonder if the whole thing is a joke. (An expensive joke.)

So unless it is a kind of joke, (and even if it isn’t!) Emoji Dick is not a wonderful advert for translating literature into emojis.

Here’s my point: strings of emojis, or emoticons of any type are not new pictorial languages. They are, at best, messages. Messages that the mind completes. “You get the message” or “Know what I’m saying” mean that the message is not all spelled out. Perhaps that’s enough for most people. Emojis are fun, but they are faddish—currently used by the pretty young (6-9 year-olds?), and the pretty old (60-plus, who may think they are being computer literate). Or perhaps we oldsters are just messaging with our emoji-addled young grandchildren. They certainly have more patience than I do going through all the tiny choices!

But if there is any way a universal visual language can be created, it’s probably via computing. Today, two thirds of people in the world have a smartphone. By 2019, that’ll be five billion people; this is how we communicate (sadly). If we can type, or say, “thank you” in Google Translate when writing to a friend in Germany, and get “danke” back in an instant, perhaps the newly updated and improved AI version of Translate will be able to “translate” regular written or spoken language into a new, as yet undrawn, set of (hopefully) flat and iconic pictograms.

Of course, since the point is to communicate, why not ask Google Translate to put your message into the receiver’s native language? Who needs pictures now? (And, if you have emojis on your phone now, they will pop up as stand-ins for some words).

I think designers will keep trying. We will look back to “Safo” (“mind writing”), a language developed from Chinese Hanzi by Andreas Eckhardt (1884–1974), who in turn looked at Gottfried Liebnitz’s work on universal scientific notation in the late 1600s, and “Solresol”, a language developed by Francois Sudre (1787–1862), which was based on the notes of an octave. It could be sung, spoken, or played on a musical instrument.

Language changes all the time. One day we’ll make a pictorial one that works. Everyone says we are in the age of visuals. With Otto Neurath alongside us, let’s get going, (and let’s have fun doing it).

Worth a look:
“Book from the Ground: from point to point”, by Xu Bing. 2013.
“Niko! The Smiley Collection”, by Tota Enomoto. 1993
“Emojis: The Secret Behind the Smile”, by Marty Allen. 2015

Thank you for reading! (Art by the great Gerd Arntz.)

COMMUNICATING WITH SYMBOLS.

This is a guest post (in three parts) by a master of pictograms, the great Nigel Holmes. Parts 2 and 3 will appear on the next two Mondays.

Linguists, designers, social scientists, teachers, and a 12^th-century nun, among hundreds (yes, hundreds!) of others, have invented what they hoped would be internationally-understood languages. None have lasted. Esperanto (by Ludwik Zamenhof, 1887) came closest; Klingon (by Marc Okrand, 1984) survives as a pop curiosity for devoted Star Trek followers.

Very few of the invented languages were pictorial, and those that were have not fared any better than their purely alphabetic cousins.

These posts are not about a history of writing that starts with 30,000 year-old cave paintings then runs through Sumerian sign-writing, Egyptian hieroglyphs, Mayan calendar icons and so on, but are about how we might return to using pictographs, pictograms, pictorial symbols, icons —to mention a few of the many names for these tiny pictures—to communicate with people from countries who don’t know what you are saying when you speak or write in your native Dutch or Swahili or English (or Esperanto).

For a picture language to be universal, thousands of pictures are needed. An argument against inventing such a language (a two-way communication) is that while it might be fine for someone equipped with those thousands of pictures to “write” in that language, the average person cannot “write” it. It’s a one-way, read-only communication.

There are tons of pictures to be drawn (or somehow produced) by the “writer” of a visual language, but can we be sure that those pictures mean the same thing all over the world? A house is easy to draw. Or the sun. Or a woman with her child.

How do you picture hope, or sin, or longing, or fertility treatments, or financial backing, so that anyone, anywhere can see the precise meaning?

I’ve drawn lots of pictorial symbols, and I relish the quirks of regular spoken and written language. Against what look like difficult odds, I have often thought of trying to create my own version of a way to communicate visually. My inspirations are three picture languages that have come closest to any success in the past: Charles Bliss’s Semantography (1949; Bliss later changed the name to Blissymbolics); Yukio Ota’s LoCoS (Lovers’ Communication System, 1964); and Otto Neurath’s Isotype (developed in the 1920s). Actually, Neurath never claimed that he was inventing a total language. When he spoke about Isotype, the graphic system that he created with Marie Reidemeister (later she was his wife), and Gerd Arntz, his leading graphic designer, Neurath called it a “helping language” rather than a complete substitute for the written word. Neurath and Isotype will be the subject of the second post.

Some of the Blissymbolics and LoCoS images are understandable pictures (house, fish, car, man, woman) but many others are a vocabulary of abstract marks that the inventors have assigned meaning to, such as a wavy line (horizontal = water; vertical = smoke), that can be combined with pictorial symbols to make “words” (house + horizontal wavy lines = flooded house; although it could also be houseboat).

But what if a pictogram of a wavy line is positioned horizontally next to a factory chimney, indicating smoke? Now a horizontal wavy line means smoke, not water. Another inventor might use a teardrop shape to mean “water.” Besides, in certain contexts, Bliss used a vertical wavy line to mean fire. Weather maps today use three wavy horizontal lines to mean fog.

When it comes to completely abstract marks—those which don’t look like anything we know—there’s a bigger river of meaning to cross. We have to learn what the inventor intends these marks to mean, in the context they are used. Many of them act like accents common to written languages. They also address the question of grammar in language. (For instance, a mark might let readers know whether a pictorial icon is a noun or a verb.) But the use of such modifying marks means that pictorial languages that include them aren’t truly visual. If you can’t read a text until you have learned how to read the marks, it’s just another language to learn, not one you can “read” by looking at the pictures. And that’s the point of an international picture language. It’s easy to read. By anyone.

In fact, are there any marks that are truly universal? You’d have thought that some, such as up- and down-arrows are. They mean up and down, right? But do we all agree on what that means? For Bliss, a picture of a heart with an up-arrow meant happy. A heart with a down-arrow meant sad. So he was using “up” and “down” as figures of speech. But in American Indian picture-writing the arrow looked like an archer’s arrow with feathers at one end and a sharp point at the other, and it meant “protection.” We can’t even assume that a simple thing like an arrow means the same thing to everyone.

So before finishing the design of any pictogram, designers should ask themselves and others—non-designers, if possible—this question: what else could this pictogram mean that I haven’t thought of? And if the image is part of a proposed universal language, designers must ask that question all over the world before claiming it has universal meaning.

In 1961, IBM introduced The Selectric typewriter with its revolutionary, interchangeable “golf ball” printing element, that replaced a normal typewriter’s basket of spidery arms with characters on the end of them that sprang up to hit the page. If you typed too quickly, the arms got tangled up. Initially there were 64 characters on the golf ball; later, the ball had 96 characters. Before the age of the computer, Bliss saw the possibility of a typewriter golf ball with his symbols on it.

Now his language could be typed —“written”— by anyone. Typewriter companies were already making machines with letters and numbers and also with a range of symbols for scientific communication. Mathematical, chemical, biological, astronomical and other symbols were available. Bliss’s golf balls were never made, but his idea that anyone could simply type his pictograms on paper was way ahead of its time. Of course, to come up with a set of pictograms that would fit on the ball was difficult. Bliss had to simplify everything pictorially, and the result looked somewhat like written words—a string of marks—except that they weren’t actual letters of the alphabet. He wrote long tracts about how to combine his pictogrammic marks to extend the range of possible “words.”

The icons in Yukio Ota’s elegant visual language, LoCoS, are designed in a way that could be written, with practice. But like Blissymbolics, LoCos combines simplified pictures with abstract marks that have to be learned.

Any picture language excludes the blind—you can’t speak it or hear it—but perhaps we could consider a kind of braille version. Deaf people do have their own visual sign system, and there’s plenty of signage in airports, hospitals, zoos, at the Olympics, on the road—but that’s wayfinding, not a language. I hope we don’t give up on trying to design a whole universal pictorial language.

Ikea, the Swedish furniture store, hasn’t given up. They’ve attempted to make the instructions for their build-it-yourself items internationally understandable, eliminating multiple translations. However, it is good to know that for a modest sum, they will deliver the bits and pieces to your home and assemble them for you, thus preserving your sanity and fingers, and limiting the screaming of obscenities at the visual instructions. One day I may have the guts to show you my language. Then you can scream at me.

Worth reading: “In the Land of Invented Languages,” by Arika Okrent, 2009.

Next Monday: The Isotope revolution.