The Tyranny of the Visual

The Tyranny of the Visual by Chris Crawford is about "the way in which visual thinking has come to dominate our thinking" -- not that visual communication is a bad thing, just that there are alternatives that aren't often explored, and that the visual representation of things is often too closely intertwined with their conceptualisation.

Crawford uses the example of navigating through a house with his eyes closed, using his other senses to visualise his surroundings (like Jim Blinn's definition of visualisation as understanding something and then creating a picture of it in your mind, only here "picture" is an embodiment, not necessarily visual). He then goes on to explain that our visual perception is merely perception, and not necessarily reality, or at least not a complete conceptual understanding of reality:

"When you gaze upon a scene, do you imagine that you are perceiving reality? I certainly don't. I imagine that I am perceiving a tiny fragment of reality, perceiving reality through the narrow window of the visual. I look at a tree and perceive so much more than a simple visual image. I imagine the fluids slowly creeping through its cambium, the photosynthesis taking place in its leaves, the absorption of nutrients from the soil — all these invisible processes that are central to the life of a tree. My eyes don't tell me much about the tree; there's so much more going on out of my view. Note that this perception of the tree is informed, indeed driven, by my education. Because I have read about biology and trees and physics, I bring to bear an understanding that allows me to see deeper inside the tree. My perception of the universe is an integration of my knowledge and my senses".

Crawford also makes reference to the scene in The Matrix where Neo reaches a state of enlightenment:

"It comes at the climax of the first Matrix film. Neo has returned from the dead and can now see the Matrix for what it is. He looks down the corridor at the three agents and sees not the corridor, but the code behind it. The image communicates the idea of seeing the processes behind reality rather than just the visual skin of reality. Isn't it odd that we need a visual representation of an idea that attempts to get around visual thinking?"

This example is relevant to my idea with regards to visualisation of game elements and mechanics -- the visual representation of a game is merely a "visual skin" for the underlying attributes and processes that define the gameplay. This is also in keeping with Will Wright's and Raph Koster's ideas of wrapping simulations and game patterns in metaphor.

As Crawford says, when talking about interactive storytelling, "what is important is the function of the system" -- the (visual) presentation scheme is arbitrary. He then gives the use of stages in drama as an example where "space is composed of individual stages with no spatial relationships whatever between stages", citing literary examples such as the journey of Huckleberry Finn. This reminded me of graphs (vertices connected by edges) and state transition diagrams.

He also suggests an exaggerated screenplay involving Cary Grant to illustrate his idea that spatial considerations and details are sometimes unnecessary: "Overpowered with passion, he walked over to her, seized her in his arms, moved his head directly in front of hers, rotated his head slightly to avoid a collision of noses, then closed the gap between her lips and his and kissed her frantically". A simpler version, "Overpowered with passion, he seized her in his arms and kissed her frantically", would have sufficed and let Cary Grant's acting ability fill in the (spatial/visual) blanks.

At the end of the article, Chris expresses his dismay that some people cannot get past the visual representation of things, and that "to them, reality is WYSIWRE: What You See Is What Really Exists", and "those of us who confine their thinking to the purely visual are narrowing their vision". This is seemingly true for the current state of game design, and was alluded to by Doug Church in State of Church. I think that videogames can be better defined and designed in terms of a separation of their visual (including spatial) representation from the actual underlying game elements and processes.

Design Basics

Design Basics by Jim Frew is a Webmonkey article I often read aloud when teaching web design. While web design and game design are not the same thing, they are similar. The medium for both web pages and videogames is computers, both are interactive, and the visual aspects of both (should) have purpose and theme. The following section of Jim's article adresses these last two items:

"The first thing you need to do is ask yourself a few questions. What is the point of the site? What are your goals? Do you want to show the world pictures of your cat? Are you trying to sell worms through the mail? Are you promoting your new major motion picture? The answer will help you begin to focus your page. As you edit your material, you will quickly see that the picture of your cat has no business on the homepage of your new blockbuster. Next question: Who are you, and who's your audience? Are you a 12-year-old girl trying to communicate with other 12-year-old girls? The president of a start-up company trying to get some cash from an investment bank? Hint: Purple and unicorns will work really well for one of these situations".

The Role of Architecture in Videogames

The Role of Architecture in Videogames, says Ernest Adams, is the "short version" of a lecture he gave at the Ars Electronica festival of electronic and computerized art, and is one of his Designer's Notebook columns at Gamasutra. "They requested the topic, and although it sounded a bit odd at first, the more research I did, the more interesting it got".

Adams mentions that "The most popular PC game of all time, The Sims, was influenced and partly inspired by the work of an architect, Christopher Alexander's book A Pattern Language". He then lists some reasons why we construct buildings: "To protect people, goods, and animals from the weather. To organize human activity efficiently (factories, theaters, offices, sports arenas). To conceal and protect goods and animals from theft (warehouses, barns, shops, storage facilities). To offer personal privacy (toilets and private houses). To protect people from other people (fortifications, military installations, prisons). To impress, commemorate or simply decorate (civic monuments and religious buildings)".

He then explains that most of these reasons aren't applicable to architecture in games and says "a building provides a convenient metaphor for concealment and protection" and gives the Town Hall in Age of Empires and the Treasury in Dungeon Keeper as examples. The main idea of Adams' article is that there are 2 functions of architecture in games: "support the gameplay" and "inform and entertain".

"The primary function of architecture in games is to support the gameplay". Adams says the "architecture supports the gameplay by helping to define the challenges. There are four major ways in which this happens: constraint, concealment, obstacles or tests of skill, and exploration". He briefly explains each of these with examples. As for the secondary function, "If architecture were only about supporting the gameplay through constraint, concealment and so on, it could all be bare grey concrete. But architecture has a secondary, and still highly valuable role to play: to inform and entertain in its own right". Adams again lists several ways this is achieved: familiarity, allusion, new worlds require new architecture, surrealism, atmosphere, comedic effect, and architectural clichés. He again briefly explains each of these with examples. He had previously complained about surrealism in another article -- "I complained about pointless surrealism in my first "Bad Game Designer, No Twinkie!" column, but architectural surrealism does have a point if it's connected to the gameplay".

I'm glad I read this article, after initially thinking it might not have much to do with my ideas. It turns out it's exactly the kind of thing I want to point out about the visualisation in games: it consists of two parts, the functional and the aesthetic. Visualisation has to firstly communicate something, and because games are a form of entertainment, visualisation in a game must also have to appeal to audiences -- purpose and theme.

Visualization

Visualization by Jim Blinn is the basis (along with the works of Edward Tufte) for my idea of what visualisation is. I've often used the following quote verbatim in presentations (to the dismay of my supervisors, and audiences no doubt):

"Visualization is good. Visualization is valuable. Let's have more visualization.
THE MEANING OF VISUALIZATION
That's great, but what does visualization really mean? In the past I have always thought of visualization as primarily a mental process: you receive some knowledge (from any of various sources) and, when you understand it thoroughly, you can "create a picture of it" in your mind. Nowadays computer graphicists are trying to place this picture more directly in the mind by creating the pictures with a computer. (This, of course, has been done for some time using more conventional illustration media). The term "visualization" has come to be a proper noun referring to the actual picture or computer image itself, as in the phrase "I created a visualization of the process on the screen". Even though the visualization is on a piece of paper or a computer screen, the ultimate destination is the mind."

He then goes on to talk about "impossible visualizations", "dangerous visualizations", "designing good visualizations" and "the usefulness of visualization". The article is short and sweet, but is full of the kind of wisdom you'd expect from the author of a regular column in the IEEE Computer Graphics and Applications journal -- Jim Blinn's Corner. Another article of Blinn's, The Ancient Chinese Art of Chi-Ting ("cheating"), about graphical tricks and techniques, has an absolute gem of wisdom that has stayed with me: a "technique" is a "trick" that you use more than once.

Reality play: threads on digital games

Troy Innocent's Reality play: threads on digital games is mostly twaddle. Apart from the vague mention of Joseph Goguen's idea of Semiotic Morphisms, possibly to give the article some "technical" status, it's about a 3D videogame in which the visual representation of the player's avatar and immediate surroundings change, e.g. tree icons change into the word "tree", or a different graphical representation of a tree, as the player collects "energy" items.

The reason I include this article here is that, while I think Innocent could have written a much shorter article to communicate the same amount of "information", it does illustrate a videogame in which the underlying game has been separated from its visualisation. The article also mentions "ontology", but rather than present a conceptual schema for the videogame itself, Innocent waffles on about "being in the game space".

It's unclear whether he is sure about what exactly goes into a videogame at the beginning of the article: "Digital games are hybrid media built from a mix of simulation, experience design, rule systems, multimedia, and a fair proportion of the unknown" -- the unknown?! -- but he goes on to say, "I play games. I make game art. I teach students how to make games", so I suppose I'll have to take Troy Innocent's word for it, what with him being a senior lecturer. His profile at Monash University says he's "been exploring and charting the digital realm since 1989" -- I think he's gotten lost in this article.

Cartographic Cartwheels

In one of his Designers Notebook articles on Gamasutra, Cartographic Cartwheels, Ernest Adams talks about how cartography/mapmaking has some useful things game designers/developers could learn. He mentions important ideas about maps, such as they should have a specific purpose/audience, and they needn't be accurate in scale or space as long as they communicate symbolic relationships, and cites the London Underground Map as one of the best examples "where a map is easier to read if it expresses symblic relationships rather than physical ones".

Descent is given as an example of a game in which (he thinks) the map is difficult to understand, whereas Adams believes games like Interstate '76 and text adventure games, while not having overly detailed maps, give the player a better idea of where they are and what direction they're facing, and thus where they can/have to go: "The maps in Interstate '76, for example, looked literally like pencil scribblings on the back of a paper sack, but they were enough to get you oriented and explain the mission", and "the maps made by text adventure gamers [...] were usually boxes with names of rooms inside them, plus lines from room to room, indicating which direction you go from one room to reach another. (This is called a directed graph by mathematicians.) The size of the boxes and the lengths of the lines were completely irrelevant, as long as the relationships were accurately conveyed".

Adams recommends the series of books about graphic design by Edward Tufte: The Visual Display of Quantitative Information, Envisioning Information and Visual Explanations, about designing graphics to communicate, respectively, numbers, nouns and verbs. Adams illustrates Tufte's "data-ink" idea with an example table he'd presented at a Computer Game Developers' Conference, and then transforms it from what he says Tufte would call a "data prison" into a table that is much clearer and more concise. He does this by erasing grid lines, and replacing "Y" with "X" and using whitespace instead of "N", resulting in a better "data-ink" to "non-data-ink" ratio. Adams then suggests that the warning label on cigarette packets could be improved, and shows an example.

He goes on to explain two problems he sees with the map in Descent: "Looking back on Descent, I think it made two major cartographic mistakes. First, when you popped up the map, it always appeared with you oriented the right way up, and the world oriented with respect to you. Since the orientation of the world changed every time you opened the map, the rooms were very hard to identify - they all look different when slanted at odd angles. One of the first things we learn about maps as children is that north is at the top, regardless of what direction you personally are facing. Descent's maps would have been much more usable if they had kept a standard orientation and showed which way you were facing. Secondly, all the rooms were displayed as white wireframes, making very little use of the PC's capacity to display color. As a result, they often looked alike, especially long, narrow corridors. On the computer monitor, unlike on paper, color costs you nothing and contributes greatly to readability. Nobody would buy a black-and-white globe; why should you have to put up with a black-and-white map? Obviously you don't want a tacky, eye-popping mishmash, but a sensibly-chosen palette".

His final statement, that a map is "vital part of the user interface, and deserves just as much design attention as any other part" made me think that perhaps these other parts aren't given as much design attention, in terms of "data-ink" and other visualisation concepts, as they could be.

A Conversation with Will Wright

In Sims, BattleBots, Cellular Automata God and Go: A Conversation with Will Wright by Celia Pearce, Celia begins by asking Will what his "philosophy of interactive design" is, and things go from there. Wright talks about his childhood interest with models and making things, and how that led to programming and creating dynamic models on the computer, and how he later got into robotics and used computers to control his robots.

He expresses his ideas about "enabling the creativity of the player" by "giving them a pretty large solution space to solve the problem within the game" where "the game represents this problem landscape". This model/simulation problem/solution landscape/space fits well with comments elsewhere that Wright's games are actually toys. Wright says he was influenced by games such as Pinball Construction Set by Bill Budge -- "he kind of emulated what would later become the Mac interface" -- and Flight Simulator by Bruce Artwick -- "this little micro-world in the computer with its own rules" -- when he was first working on SimCity.

He also talks about playing hex-grid board games with 40-page rule books, and how arguments over interpretation of the rules with friends was "kind of half the fun of it -- both of you trying to find the legal loopholes for why your guy didn't get killed". Of note is his comment, "the [game] model was far more elaborate than you could really run in your head", which I think could be likened to the concept of Imperfect Information in Game Theory, although not exactly the same thing.

Wright says, "the types of games we do are simulation based and so there is this really elaborate simulation of some aspect of reality" (which relates to Chris Crawford's idea that "a game is a closed formal system that subjectively represents a subset of reality"). Wright creates games that are "this elaborate system with thousands of variables", and communicates this to players via an "overt metaphor" so that players can "bootstrap themselves into understanding that model", and establish their own "mental model". Although the metaphors for SimCity and The Sims are cities and people, he has other metaphors/analogies: he likens SimCity to gardening -- "you're kind of tilling the soil, and fertilizing it, and then things pop up and they surprise you, and occasionally you have to go in and weed the garden, and then you maybe think about expanding it, and so on" -- and The Sims to "juggling or balancing plates" -- "you're rushing from this to that to this, and then you're able to make these time decisions. So it feels very much like juggling and if you drop a ball, then all of a sudden, the whole pile comes crashing down". However, he adds "but other people play it differently [...] and it's not as clear to me what The Sims is".

Wright says, "with current technology, there are a lot of limitiations in terms of what we can do with character simulation [...] there are certain things we just cannot simulate on a computer, but on the other hand that people are very good at simulating in their heads. So we just take that part of the simulation and offload it from the computer into the player's head". He then goes on to talk about having "a consistent level of abstraction", and how this allows players to "fill in the blanks really well. And is something that kids do quite well" -- this is followed by a discussion about the game of Go and emergent behaviour from simple rules, and about how conflict, whether between players of a game, or between people in general, comes about from disagreement over their different mental models: "the game is in fact this process of us bringing our different mental models into agreement". Wright then cites a project called Sim Health that was a simulation of the national healthcare system: "the idea was that if people could come to a shared understanding or at least agree toward the model of the world, then they would be much more in agreement about the policy we should take" -- this idea of an agreed model, independent of vocabulary, is what I hope to achieve with my idea -- here's a brief excerpt following on from the above quote that I thought was important:

"CP: So in a way, a system like that could be used to externalize mental models and create a collective model.
WW: Yes, exactly. Which I think could have value, but at the same time I like this idea that there’s this diversity of models out there.
CP: Well, I think if you have a shared model, it’s not so much like you all have the same mental model, but that you have an externalized model that everyone agrees to abide by.
WW: Yeah, which is exactly the way science works."

Related to the idea of shared mental models is the idea of agency in The Sims, where "the pronouns change all the time". The characters are semi-autonomous, "and so at some point it's me inhabiting this little person [...] but when he starts rebelling, it's he". He makes reference to a toy called Rockenbok where players control one of several radio-controlled construction vehicles via control pads and can switch between vehicles at the press of a button. He likens the vehicles to "little avatars" and says, "it's really interesting to watch kids play with this because their identity is so fluid from truck to truck. And it's really interesting the situations they get in. They always end up pushing and fighting with these things. So, you're about to push me off the table, so I go to [truck] number 3 really quick and come up behind you, and all of a sudden, you see that I'm attacking so you turn and face me. You're very cognizant of which avatar I inhabit, even when I change. It's like The Sims in a way". I think this cognisance -- swapping of avatars, but keeping the interaction fluid -- is the players having a shared mental model.

The issue of narritive in games is raised with Pearce asking Wright what kind of god he would like to be, to which he replies, "I would try to be a God that surprised himself. I think being the all-knowing God would be, you know, hell". Pearce then mentions he's "creating a mental model of the whole [game] universe", which Wright corrects by repeating his idea about possibility space and says "I don't want to create a specific possibility that everybody's going to experience the same way. I'd much rather have a huge possibility space where every player has as unique an experience as possible". Pearce then adds, "you enjoy the unpredictable outcome. When people do things [in the game] that you didn't plan on, that seems to be something that you embrace". Wright replies, "to me, that feels like success".

In terms of a relationship between the size of the possibility space and the quality of the experience for players, and creating a metaphor for players to establish mental models, Wright says, "we can make the possibility space huge, just by giving the player a thousand numbers. [...] That's a big space. It's just not a very high quality experience. So we start wrapping graphics, sounds, scenarios an events around those numbers, and we're increasing the quality of the experience you have. It has more meaning to you. In some sense it becomes more evocative. You can start wrapping a mental model around that, as opposed to this pile of numbers". Wright is "trying to build the maximum possibility space in your head, not on the computer". He also talks about the possibility space in terms of terrain, where challenges take the form of peaks, and so a player's experience is how they navigate through that terrain. This leads to discussion about "the plausibility and believability of the [possibility] space. If the space starts becoming totally disconnected and random [...] your mental model will start breaking down", which Wright adds is "the main reason for keeping the [space/terrain] topography somewhat consistent" (see "a consistent level of abstraction" above).

Wright then discusses time. "I think that time is an interesting component because with games you can relate to time in a totally different way than in linear media. I can always back up, load my old saved game. I can pause whenever I want to, etc. You're starting to see little bits of that popping into linear media". He mentions the film Memento and Pearce mentions Run Lola Run and Wright says "I've always wanted a game that had a smooth slider where you could go forward and backward and rebranch" -- the game Prince of Persia: The Sands of Time has since implemented this with a reload/rewind facility, when the Prince (the player's avatar) dies. However, I think Wright meant the ability to do the same thing at any moment during a game. Wright's argument is that we currently have "spatial mobility": "I can click and move the screen wherever I want really fast. I should be able to do the same thing with time", and he goes on to say, "actually, we're experimenting with scales of time and space. The plan is that the time is going to be totally based on the zoom level. So your zoom level and your time slider are the same. If you want to speed the game up, you have to zoom out, if you want to slow it down you have to zoom in" -- this is very much to do with visualisation I think.

"Time and space are related scale-wise, they correlate" (is this related to the Theory of Relativity?). He closes by pondering this scaling in combination with reversing: "I'm kind of curious behaviourally what that's going to feel like. To change time I have to change scale. You know, it might be a total screw up. It might just be a total pain in the ass. But then if there was a way to go backwards, that would be cool. [...] Reversible simulations are hard in some sense, though. I mean, this gets into a whole computer science discussion. But, you either store the data or you make it reversible. It can only be made reversible if no information is destroyed. Most simulation processes, such as system dynamics or cellular automata, destroy information. [...] Yeah, so there's a real engineering issue there. But it's not insoluble", and Pearce adds "with Moore's Law, very soon, you'll be able to have enough processing to do that" -- regardless of future possibilities, Wright's ideas about time and space are worth considering in relation to what I'm doing, in terms of dynamic (over time) visualisation.

The State of Church

In The State Of Church:Doug Church on the Death of PC Gaming and the Future of Defining Gameplay, by Justin Hall at Gamasutra, Doug Church talks about the PC versus the console in terms of experimentation, passive entertainment, and play. While the PC might allow for greater experimentation in gameplay, the console, being hooked up to a TV, is more geared as a passive entertainment device, and unfortunately, entertainment is taking over from play as the main selling-point for the majority of new games.

He went on to discuss the need for a game vocabulary, and mentioned that existing game genre terminology is the current "shorthand" for describing games. He sees the problem with this as new games are blending old genres, and the gameplay mechanics aren't properly described:

"Sometimes I think that genre is our shorthand to talk about play, and that's about as specific as we'd get, because when I show you imagery of a lot of games and the communication message, you know 'you're a powerful wizard', or 'you're going to defeat terrorists' or 'you're going to pilot planes', it doesn't really tell you anything about what you're going to actually do. Like: What are the verbs you have? What are the buttons you're going to use? What sort of mental action do you get? Why are you even there? Why isn't it just a movie?"

This illustrates my idea about gameplay being too tightly intertwined with visual presentation. You have a mental picture of what the general idea of the game is, but no explicit picture of exactly what the gameplay elements are in terms of the rules or environment you'll interact with.

Also, I think having a play vocabulary isn't as important as sharing the same mental model or concept of gameplay mechanics amongst game designers. Forget the vocabulary, the words, and concentrate on the actual things instead; "What's in a name? That which we call a rose By any other word would smell as sweet." --From Romeo and Juliet (II, ii, 1-2).

The game Katamari Damacy was mentioned as an example of "experimental play innovation". I don't think the gameplay was necessarily innovative -- you move around and collect items of increasing size -- just that the visual representation of the gameplay was innovative. Which is what my idea is all about.

What Games Aren't

In Gamasutra's cover feature, Book Excerpt: "A Theory of Fun for Game Design" - What Games Aren't, Raph Koster explains how game patterns (a.k.a. elements and mechanics) are wrapped in metaphor. "People tend to dress up game systems with some fiction", for example, in Checkers, "calling the über-checker a “king” [...] adds interesting shading to the game but the game at its core is unchanged".

According to the book's website, "It's about: What fun is, Why some games are fun and some games are boring, How different people respond to different kinds of fun, What makes a game fun or not, How games fit into the wider human culture, Whether games can be art, What degree of social responsibility game makers need to have, How games can develop. At its core, though, it is about why games matter".

The foreword of the book is by Will Wright, and on Slide 31 of the associated talk, Koster has "If I were Will Wright, I’d say that “Fun is the process of discovering areas in a possibility space.”" -- see Celia Pearce's conversation with Will Wright at Gamestudies for more about this.

Slide 41 mentions "communication" in terms of games ceasing to be craft and becoming art that is subject to interpretation. This relates to my idea, however I'm talking about visualisation as opposed to art, and I'm concentrating more on the visual representation of game elements more than defining what games are per se. The idea of fiction and metaphor are relevant to my idea insofar as they can provide visualisations (of games) with context or theme to engage an audience (players).