cool math games

In case you did not follow the comments on the last blog post with my views on the player as an artist, you might have missed that James Portnow from Extra Credits responded and he and I had a brief exchange. This discussion is now up on the Escapist in case anyone is interested in checking it out:

http://www.escapistmagazine.com/articles/view/editorials/misc/8976-Extra-Credits-Addendum-Discussing-the-Role-of-the-Player

The articles has lot of interesting responses from the readers. I think I have already said what I have to say, so I will not discuss it further here. However, do feel free to continue the discussion in the comments!

Introduction
In this week's Extra Credits it was argued that we should treat the player of a video game as an artist and co-author of the game. One major point was that other media can be said to be valid without an audience, but not so for videogames. In video games a player is needed for the work to fully come to life. The other point was that players have an artistic role in this co-operative creation and that understanding the feelings that drive an artist can be used to make better video games. You can watch the whole video here:
http://www.escapistmagazine.com/videos/view/extra-credits/3555-The-Role-of-the-Player

While I think that Extra Credits is an excellent show, I do not really agree with this. The hypothesis of player-is-artist sound quite plausible at first, but I think that if you take a closer look it does not really hold up. I also think that if we choose to design games with this mindset, we might be missing out on very important things that can be done in the medium.


Why an audience is crucial
Consider the information that a novel transfers to its audience. More than often a few words is all that is given as base for the reader to imagine a scene. It is assumed that the reader is able to fill out large informational gaps, make assumptions and to even retroactively dress earlier scenes as new information comes along. A novel also throws a lot non-trivial concepts at the audience. Just consider something simple as a person being labeled as "depressed". In order for the reader to understand the state of mind of this character, a personal knowledge in human behavior and psychology is necessary. The reader simply cannot understand literature without a certain amount of experience.

What I want to say with this is that books cannot be enjoyed in a void. They have to be processed by a human with a certain amount life experience in order to come alive. An alien would be completely unable to understand any literature even if it could decipher the language. There are just so many prerequisites needed that a deep study of humanity would necessary for full comprehension. And even then it might not be possible for the alien to understand; the very workings of our minds is probably crucial for a proper enjoyment of the work.

Further more, the mental image painted up in our mind is not merely an opinion-based interpretation of the written text. It is a full blown world, populated and maintained by ourselves. We are not just doing a trivial text to sensory input conversion when reading. We are doing an actual simulation, constantly adding and updating details based on the written words of the novel. The book helps us a long on the ride, but at the end the heavy work is done by the audience.

This mental construction work is not passive exercise either. We choose where to read, what to skim, where to put focus, if something should be skipped and so on. This is even more evident in other media. When watching a movie, interactions with other audience members can help shape the experience (a simple example would be laughing during funny moments). When listening to a piece of the music, the settings, our own movements, etc all change the way in which we experience the piece.

Enjoying a work of art is a very human experience and I argue that in this regard videogames are not different from other forms of media. What is so special about video games is that data flows in two directions and that the audience can help shape the output to an extent far beyond any other media. However, that does not mean that experiencing a videogame is totally different kind of activity.


Why players are not artists
Unless a game is incredibly linear (e.g. Dragon's Lair) everybody who play it will come away with their own personal experience. Because of this it is easy to imagine that videogames differ a lot from other media by essentially making the player a story-teller. And as being story-teller is essentially equal to being an artist, it leads to the conclusion that players are artists. I do not think this holds up though.

Consider doing a trek through the woods. Even if several people follow the same route in the same forest, each one will have with very different experiences. Some might take side-tracks, have some unique encounters, do the trekking at a different pace, etc. The possibilities are essentially endless. The person doing the journey shapes his/her experience in a unique way and has a big responsibility in how it all turns out. Still, this is not an artistic endeavor.

That is until the hiker decided to write, paint, talk, etc about the trek. Once a narrative, in whatever media, is created of the personal experience, an artistic process takes place. The art is not in living the journey, the art is in conveying it to other people; to create a work that expresses the very personal sensory input, actions and emotions evoked.

The same is true for videogames. Even though the player posses a great freedom in shaping their path through the virtual world, this does not equal the player to an artist. Likewise, even though readers of books create and simulate complex worlds, this does not make the reader into an artist. It is not until the personal experienced is expressed in some kind of medium, be that a written narrative of a game session or a painting from a scene in a novel, that art is made.

Sure there are videogames that give great opportunities for creating art, Minecraft being an good example. However, this artistic creation is a side thing and is not a requirement. The players can simply just let themselves be one with world, build a shelter, etc. It is not until the player simply sees the game as tool and foundation for their own work when the line between player and artist really blur.

With all this I am not saying that the activity of enjoying art is void of creativity. As explained when discussing how we read books, I made it quite clear that it takes a lot of effort to do it right. However, this does not mean that the act of reading a book is categorically equal to writing one. Instead it is more like the difference between solving a puzzle and creating the puzzle in the first place. Both activities are creative and challenging, yet quite different.


Why this matters
Why even have this discussion? Is it not just a silly debate over semantics? Well in part it is. But if we do not take care and use the words properly they will start to loose meaning. If we would say that the activity that players participate in during play is the same as artistic creation, then I think we simply stretch the concept of artistic endeavors too far, making far less usable.

A more important reason is that the way we see the relationship between players and videogames greatly shape the direction we choose we take the design of videogames. Even though video games have a very different voice from other media, we should not think of the activity of experiencing it as completely different. I fear that if we see players as storytellers and artists, we will miss out on a lot of opportunities at expanding the videogames medium. If the player is an artist, then our focus on game developers will be on creating brushes. This implies a bottom-up design, were short-term effects trump the bigger picture. If we want make games with a deeper meaning this is not the way to go. Instead we must focus on a higher level, something I think the player's role as outlined here greatly encourages.

Also, saying that the player is an artist and storyteller shift the burden in the wrong direction. If we grant the player a large artistic role, we make it easy to blame the player for any lack of meaning in a videogame and discourage the creators from trying to add it. A painting should not require a painter to enjoy it, a play should not require you to act for it to be engaging, and so forth. Like great works of art in other media a videogame can require a lot from the player. However, this does not mean it is up for the player to create meaning and depth, it should instead be there for the player to find and become immersed in.

Introduction
For a couple of months now I have, on and off, worked on some basic tech aspects for the engine. Everytime I was done with one of these I thought it was among the hardest things I would do for the new engine, yet the next feature as always proved more challenging. Terrain geometry was harder to implement than sun shadows, terrain texturing harder than geometry, and so on.

Implementing the script system is no different. It is easily the hardest thing I have done so far for our new engine - HPL 3. It has had this "perfect"sort of challenge: Difficult problems to solve, much basic knowledge to wrap your head around and awfully boring and monotonous parts. I really hope this marks the end of this trend of increasing difficulty, as another proportionally large step might make my brain to melt and my fingers to crack. At least it can wait for a lil while...

Enough complaining. Now that the scripting is pretty much implemented (some engine stuff still needs to be added and some more problems remain to be solved, but it is really minor), I am extremely happy with it. I think it will help us build better games, and to finish them faster. Now let's move on to how a boring scripting system accomplishes this.


Background
Before moving on the the actual scripting I need to explain what brought on the creation of the current system. It all started with our first commercial games, The Penumbra series.

When creating the Penumbra games our tools were primitive to say the least. All maps were made in a 3D modeling program, Maya, and then exported to Collada. The game engine loaded the Collada file and built the map from it. As a 3D modeling program is meant to create 3D models, it is not really meant to make levels in. With no ways of placing entities we had to use special naming conventions to tell the game where any non-static objects in the game were located. To be able to do this properly we had to make special "instance" versions of each model meant for the game, since without this you would not be able to see how an object was placed before the game started.

Lighting was equally annoying since Maya has no support for radius on lights. This mean that you could not visually see how far a light reached, but simply entered a numerical value and hoped for the best. As this was not enough, you also needed to place portals and group meshes in order for the engine to provide occlusion culling. This could be quite tricky at times, and often you could sit a day or two simply tweaking portal setup. Added to this was also the problem that Maya often failed to show any textures, and most editing was done on grayish levels. For more info, you can just check the wiki.

The problems do not stop here though. Everytime you meant a change to the game, you had to do a complete reload. So setting up lighting in the game could be quite the effort: change light position for two seconds, load map for 2 minutes, notice it is not good, repeat. As you might figure, we got quite good at batches tasks and the phrase "it will have to do" was uttered more often than not.

For scripting it was just a grueling. Every change in the script required a full restart of the game, creating the same sort of frustration present when modeling maps. And to make scripting even more frustrating, there was no syntax checking until the entire map was loaded! This meant you could wait two minutes of loading only to find out you had forgotten a semicolon or something else trivial.

As I write this, I actually have a hard time understanding how we could have gotten anything done at all. And unsurprisingly, even though we released mod tools and documentation, not a single user map for Penumbra was ever released.

For Amnesia we knew we wanted to fix this somehow. The first step we took was to simply make our own editor where all the maps are built. Since it rendered with the same engine as the game, it made is much easier and faster to tweak entity and light placement. We instantly saw that productivity rise with this change. For scripting it was pretty much the same, but we added the extremely simple fix of compiling the script before loading the game. This removed some of the time previously spent on, in vain, looking at loading screen.

Although we had new tools all was not good. You still had to reload all level data every time you made a change to the map or script. We did not think much of this though as we were so used to doing it this way, and happy that we had all the other improvements. However, a year and a half into the development we discussed if we really needed to reload the level. I cannot recall what sparked this idea, but anyhow we figured that we did not and I added a menu with a Quick Reload button. This cached all textures, models, etc and reloaded map very quickly (usually taking but a few seconds). This increased productivity and creativity tremendously and was one of the better decisions we made during the development of Amnesia. Another sign of how much these changes improved work flow are the over a hundred of user maps created as of today.

What is so strange about the reload-feature is that is something that we could have added during the development of the first Penumbra, but for some reason we did not. It is quite frightening how often you convinces yourself that there is no better way of doing a task, and never try to improve it. We did not want to make this mistake again and started thinking of what more we could do.


Taking script to the next level
In Amnesia and Penumbra, scripting is only used to control logic flow in the levels. How enemies spawn, how puzzles work and so on. All other gameplay is hard-coded into the exe file and written in C++. Normally when I write this kind of code, rendering for instance, I can do large chunks at a time and then simply see if it works as intended. This often in small projects that are fast to reload. However, when writing gameplay and UI code this is almost never the case. Instead you constantly need to fine tune algorithms and variables until you get the expected behavior and work in large projects. Not only does this mean a level restart (with full resource reload), but the exe itself also needs to be built from code, a process that can easily take half a minute, even if the changes are minor. This means that coding gameplay can be quite a hassle at times, on par with how map building was in the Penumbra days. With the lessons learned from Amnesia fresh in mind, this felt like the obvious area of improvement.

In order to make this happen we had to move as much gameplay code as possible into the scripting. What this meant was that we needed to do some large upgrades to our current script implementations. For example, right now we only supported the most basic types (bool, int and float) together with strings in script. This already caused some issues when exposing game/engine functions and when writing scripts, for example instead having a single argument for color, you had to have four floats (one for each color and one of alpha), making code ugly and writing it more cumbersome. So just this upgrade was worth doing.

We also needed expose all engine classes, so that the script could be used pretty much as if it was normal C++ code, and achieve pretty much the same things. I was not sure exactly how much to expose but knew that the more the better.

Finally, the most important feature was to be able to reload script at any point, so it would be easy to just change a line, click a reload button and then a few seconds (or less) later see the change in-game. To get this working was the by far the most important goal with the entire upgrade. This would not be as easy as the level script was in Amnesia though, since the script system would not only take care of the code, but of part of the data as well. This meant I needed to save the state somehow, a feat I was not sure yet how to accomplish.


Implementing Classes
Before tackling the problem of script reload, I first had to make sure to add engine types to the engine. And even before doing that I needed to be sure our current scripting middleware was up to the task.

The scripting system that we were using, and have been for a long time, is a library called Angel Script. It is actually the middleware that we have used the longest, ever since end of 2004 and the Energetic project. Even though we have used it for such a long time we never really used to anywhere near its full extent and now was finally the time to make up for that. I took a day to look through the documentation and found that AngelScript could support everything that we needed, and what was even better was that it was not that difficult to add.

AngelScript is a bit different from other popular script languages (like Lua) in that it is strongly typed and very closely connected to the underlying C++ code. For one thing this makes the script quite fast (although not faster, see end of post for more info). It also meant that AngelScript could link to the classes almost directly and I only had to declare the class and link to its different parts (whatever member variables and methods that I wanted to be exposed). It also supports pointers quite easily, but makes them more secure with a script specific handle type. This requires you to keep track of some memory management for the data, but you do not need to do it, and I could very easily and quickly add support for a vast number of engine resources (textures, meshes, lights, etc).

The only thing that was a little bit trickier was supporting inheritance (when a class can build upon another class). Basically you have to redeclare methods every time you add new class that inherits from something. You also need to specify to what other classes this class can be cast to. This might sound a bit of a hassle, but the result is that you can control so the script always has a very close mapping to the code it exposes, something that I was extremely thankful for when implementing the state saving (more on that below). Also, through some use of macros, adding the implemented classes become quite easy.


Basic script layout
The next thing to figure out was the basic structure of the script code. In Penumbra and Amnesia we simply added the functions directly in a script file and then allowed that script file to represent an object. I first thought that this would be a valid design this time again, until I started thinking about how to store the data (meaning any data that should be kept between executions of the script).

In Amnesia and Penumbra the only data that is saved is simple variables like an integer for the number of times the player had stepped on button or similar. This is done through using special functions for all these variables, for example:

SetLocalVarInt("ButtonPushNum", 1);

This function saves the data to the game, and when the script is reloaded (meaning destroyed and recreated) the script can easily reference the data again by doing:

int lX = GetLocalVarInt("ButtonPushNum");

However, this time the game had to save a lot more complex data, like pointers to resources, matrices and whatnot. At first I actually considered using system with functions like this, but I figured that it would just mean a lot of extra work, and just make things more difficult. Again I thought about the lessons from Amnesia's reload feature, and thought it was worth trying to find a better solution. What I ended up doing was to force each object to be contained in class, and then let all data to be members of that class. This allowed AngelScript to make copies (needed for enemies and whatever there will be more than one instance of) and also made it was easy to keep track of the members (you simply create an object in the C++ code and can then iterate all its data).

A problem that I realized now was that I needed to have C++ functions that only worked in certain types of classes and only on data for a certain copy. For instance, an enemy class might want a function like IsInLineOfSight(...) to see if it has visual contact with something. However, there were not any functionality in AngelScript for doing this. I could give the script class a template class that forces it to implement certain functions, but I could not let the C++ code act as a base and expose specific functions from it. To solve this I had do some hacking. I ended up using global functions, and to keep track of the currently active object. (Again with macros to the rescue.) The resulting solution was not perfect though, as the global function can be used in any class and not just the one it was meant for. I am still looking into some fix for this.


Saving the state
It was now time to save the state of the script. To start this off I took the naive approach and simply saved the script data directly by copying it. This works very well for stuff like vectors, matrices, etc where it is just a matter to make a copy of the data. But it does not work that well with resources like meshes, texture or even objects like physics bodies, billboards and lights. There is way too much data in those to copy. And if I simply save the pointer I need to make sure that no data has changed when the state is loaded again, or else the saved state will not work.

At the time I was only building the system to work with a script reload only, so these issues did not pose a problem. But a major obstacle popped up when I wanted to save classes defined in scripts code. These where not possible to simply save by copying, because when you rewrite a script they can change entirely. For instance, if the class when saved consisted of two integers, and when reloaded has one string and a matrix instead the data you saved is invalid. It gets even worse if a script class has another script class as member and even more so if script classes are saved in arrays.

I figured I needed to do some kind of drastic change if this was to work. I was sketching on a few systems that could save variables in a separate structure, when it hit me. The system I was working on could not only be used to save the state, but if I implemented it correctly I could also use it for saving. For Amnesia and Penumbra I use a special serialize system that can save classes to file if initialized correctly. It is quite cumbersome, but way better than writing load / save for every single variable. However, as I was to do most gameplay code through script (the code that pretty much contain everything that needs saving), I could actually get rid of rewriting the save code for every single gameplay update. Another huge benefit of using scripting: automatic saving. This is bound to save tons of work. (It did mean a lot more work though...)

Up to that point I had looked at the scripting as separate part of in the engine. A module that all other modules could work without if removed, exactly like how most other modules work. Thinking like this had colored a lot of the design decisions I had made. Now, I instead decided that since scripting will be basically what controls the engine, I started of thinking as something that was part of all other modules. This led me to do a major rewrite of the state saving.

There is quite a lot to say about this system (and serializing in general), but I do not really have the space right now, so I will just do a quick overview. First of all, the system defines a few basic types that all other structures are built up from. These are bool, int, float, vector, matrix, color, etc and I implement very specific create/save/load methods for each of these. In AngelScript all of these are also implemented as primitives (built-in types) or data objects (which is a type where AngelScript handle all memory management). When saved the actual binary data is saved.

All other classes are implemented as references, meaning AngelScript only manages their pointers. When saving the state of these classes, each class must implement a a special method that adds all of the member variables (either basic types or other reference classes) to a list. (Basically the explicit method described here). This list is then used when transferring data to a special save buffer that contains the data of all the basic types and/or sub buffers containing data for other classes.

For certain classes, its pointer is backed up in the C++ code. So when the save data for it is loaded, the pointer to the class is first searched for and if found it is updated with the saved member data. This makes it possible to reload the script code without having to recreate all the data.

Of course, all sorts of complications arise during this, and again it would take too long to go through them all. But as I hinted before one thing that saved me is that script objects are so closely connected to the engine data. When writing code that deals with serialization one of the biggest annoyances is that that class pointers with multiple inheritance can change their address when cast. Because the script language only returns void pointers (memory address with no type specified), you must be sure of the type it returns, something AngelScript allowed me to be.

Another fun thing with serialization are all the strange macros that you have to make. For example, this one was pretty fun:

#define ClassMemberClassOffset(aClass,aMember)
( size_t(static_cast(&(( (aClass*)1)->aMember)) ) -1 )

This one is used to make sure that class member offset address points correctly. It is one of the many things that make sure multiple inheritance problems, as explained above, does not screw things up. I got an entire header file just filled with fun stuff like this!


Closing notes
Once I had the system working it was quite an effort to add all the classes. There are countless classes that all need to be set up in specific way and everything to be saved needs to be initialized. Sitting several days in a row just coding stuff like this can really tear on the psyche. Fortunately, almost everything is added now, so the worst part should be over.

Another nice addition to the new script system is that it can auto generate the API function file for NotePad++. This allows NP++ to autocomplete any text that you write and you no longer need to keep function names or arguments list in your head (and skip the manual look-ups). What is annoying though is that it only works with global functions and NP++ cannot recognice what class a certain variable is in and show all its members (like visual assist can for C++). Does anybody know an editor that can manage this?

One thing that I was really eager to get working was threading. In Amnesia we use tons of special timer callbacks to set up events, and it would have been so nice to do something like this instead:

...
PlaySound("x");
Sleep(10)
PlaySound("y");
WaitUntilTriggered("a")
...

AngelScript does support this sort of thing, but there is currently no way of saving the state. So if the player where to save during a sleep, the game would be different when loaded. This is unfortunate, but I am looking into some other solutions instead. If anyone has ideas on this please share!

Right now I have only tried this system in some tests, but it still feels really good and works exactly as I want. It is just so great to be able to reload any code changes in a fraction of second. It allows for rapid iteration, makes one more productive and generally just makes it so much more enjoyable to code. It is also so gratifying to have an idea and then successfully implement it to the sort of quality you hoped for.


Addendum:
I earlier wrote that AngelScript was faster than Lua, which is not the case when it comes to code solely executed in the script (see here). So obviously my initial statement was not correct. However, I still think AngelScript must be faster in calling implemented c++ functions/types as there is pretty much no overlay involved. I do not have any data to back this up though, so do not take my word for it :)


Found this irresistibly interesting? We are hiring!

Road Rage!

Frictional Games is now looking for a talented programmer to join the company!

You will be working for a small team with a big focus on finding new and innovating solutions. We want you who are not afraid to explore uncharted territory and constantly learn new things. Self-discipline and independence are also important traits as all work will be done from home.

Some the things you will work with include:

• 3D math, rendering, shaders and everything else related.
• Console development (most likely Xbox 360).
• Hardware implementations (support for motion controls, etc).

All coding is in C++, so great skills in that is imperative. You should also be living in Sweden or a time-zone nearby. If you are fit in other areas we are of course prepared to be flexible.

If interested send CV to: jobs [at] frictionalgames [dot] com

What we are most interested in seeing is evidence, in form of things you have done, that you are qualified. Having interesting works to show off can make up for any gaps in knowledge.

Introduction
The main gist of this post is that we are not using the full narrative capability of video games. I believe we fail to take into account certain aspects that lie at the core of making artistic creations powerful and thus miss out on crucial strengths of the video game medium. To get to the core of these strengths, I will first have a look at other media (specifically film and literature), and then explore what lessons that can be applied to video games. What I end up with is a way of thinking that use basic elements of the film and literature experience, yet is quite different from these.

It is very easy to look at other form of media, see what they do well, and then try and copy this. I think this is a big problem for video games. Whenever a game focusing on a narrative-oriented experience is made, it is instantly compared to other media and judged according to their strengths. For instance it is very common praise to call video games cinematic, or to concentrate critique on their plot structure. Obviously, I do not think this is the right approach. Instead I think we need to take a step back, and consider what it is really in these other media that makes them work. We must then explore in what ways these concepts can (and if they can!) be applied to video games.

My suggestion for this "magical essence", which I will outline in this article, are empty spaces. The bits that require the audience's participation and imagination. Basically, the part of art that require us to be human.


The power of imagination
First of all let us take a look at literature. For this "The fall of the house of Usher" by Edgar Allen Poe will be used as an example:

"I know not how it was—but, with the first glimpse of the building, a sense of insufferable gloom pervaded my spirit. I say insufferable; for the feeling was unrelieved by any of that half-pleasurable, because poetic, sentiment, with which the mind usually receives even the sternest natural images of the desolate or terrible. I looked upon the scene before me—upon the mere house, and the simple landscape features of the domain—upon the bleak walls—upon the vacant eye-like windows—upon a few rank sedges—and upon a few white trunks of decayed trees—with an utter depression of soul which I can compare to no earthly sensation more properly than to the after-dream of the reveller upon opium—the bitter lapse into everyday life—the hideous dropping off of the veil. There was an iciness, a sinking, a sickening of the heart—an unredeemed dreariness of thought which no goading of the imagination could torture into aught of the sublime. "

This is an excerpt of a quite lengthy passage where the narrator describes the House of Usher as he approaches it. Even though it says a lot, it gives us very scarce information of how the house actually looked like. The focus is instead on the feelings and actions of the protagonist. The text tells us the response that the imagery evokes in the narrator and based on that it urges us to make up our own mental image of the scene. This is typical for literature. Descriptions are usually sparse and instead emotions and events are used as to paint a scene for the reader. A lot of responsibility is shouldered on the audience, certain knowledge is assumed and this (which I think is especially important to highlight) without the author losing any artistic control.

Next, let us consider movies. Normally one would think of movies as being very exact in their portrayal of a story, almost like a window to an alternate reality. However upon a bit of analysis it is clear that this is not the case. Film requires us to make non-trivial connections between sequences and invites us to read the minds of the actors. The Kuleshov Effect makes a clear case for this. Just watch the following video yourself and consider how your interpretation of the face changes depending on the context in which it is shown:



As we see a character on screen, we are meant to start imagining what that person might be feeling. Whenever a cut is made, it forces us to make up a casual relationship between the juxtaposed events. This can easily become quite complex as this short clip from the famous Odessa stairs sequence show:



Somehow we are able to make sense of this cacophony of imagery, constantly making connections between clips, weaving our own coherent narrative inside our minds. Just as books require readers to fill in the sensory details of a scene, a film forces the viewer to imagine the emotions and casual relationships portrayed. Both literature and film heavily depend upon the audience's imaginative interpretation and will lose its impact without it. I would even say that the greater this gap of imagination is, the more room for interpretation, the more powerful and artful the work becomes. By this I do not mean that the more obscure art is, the better it becomes. Rather, the ability to leave plenty of gaps for the audience to fill, without making the work incomprehensible and meaningless, is what makes great artists and great works of art.


Filling a gap
Even though this audience participation required in books and movies might not be obvious at first, it does not feel that strange once you realize it. It is quite easy to see that we make up worlds in our head when reading or that we construct a fluent narrative from edited imagery when watching movies. But viewed from the perspective of somebody who encounters this for the first time, I would say that is far from evident. It is really quite weird that we can count on the audience to build up whole scenes in their heads. This based on almost purely emotionally descriptive content. Dialog in literature is a great example of this, where the spoken words are alone at conveying the look, actions and sometimes even emotions of the characters involved. There are tons of background knowledge required to makes sense of this, and it would be extremely hard to teach computers the same tricks.

I bring this up mainly because I want to show that, even though all of this is now part of our everyday life, it is far from self-evident truths. For instance, film editing took a while before it was properly figured out, and its complex usages even longer. This should hint us that whatever there is left to figure out about the videogame media, we should not expect it to be self-evident or even seem like it would work when first encountered.

Another important reason for bringing this up is to show that all of these gap-filling has a retroactive aspect. For instance when connecting clips in a film, the whole meaning (ie the action that the clips portray) come together afterwards. Yet to us it seems like a continuous experience and in a way we actually inject false memories of an imagined event. This is basically how animation work, where we first see an object in one position, then in another, not until both event are experienced making our brain interpret the entirety as if motion occurred. However, we never experience it like that; we simple see it as a motion of an object from one point to another and do not notice the mental effort required.

This is even more evident in literature where descriptions of objects can come far after they were first introduced. Even though this may seem like a jarring discrepancy, it pose no problem to us and we can meld these new facts with the earlier event portrayed. For example, if we remember a tale the early happenings exist in our mental images with the detailed characters shaped during the full read-through. They are no longer the unknown entities they were when we read the passage for the first tine.

What this tells us is that we should not be afraid of giving the audience incomplete information or experiences. Not only does this "removal of facts" not pose a problem, but it actually seem essential in creating a powerful experience. It is actually as if something "magical" happens when we are forced to complete the work ourselves.

Side note:
Split-brain persons show a very extreme example of our human urge to, often unconsciously, fill these sort of gaps. For example, outlined here are some experiments where the subject effortlessly made up details from incomplete information without conscious knowledge about it. I think it clearly shows how the brain is hard-wired for this kind of behavior and that it is essential to what makes us human. This visual illusion found here also show how eager we are to create casual relationships, and how the context makes us change how these are made.


In search of the void
It is now time to take a deeper look into games and to search for an equivalent of the "gap filling" concepts found in literature and films. Instead of meeting this head on, I think it is important to discuss what it is that is especially distinct and descriptive (and thus not requiring the audience's interpretation) in games. I would say these things are:

• Details of the world. Not only are games extremely clear on what a scene looks like, they often allow it to be exploration and makes it possible to very closely examine the various parts of the world. This is something that is especially true for 3D games, where players can view objects from almost any angle they please.
  
  
• The fluidity and coherence of actions. As players are in direct control of the protagonist, there is never any doubt of what events are taking place. Because of the interactive nature of video games, a constant feedback loop of actions and consequences are required, forcing the events taking place to be exact. Video games are all about right here and right now.

Side note: I am aware that the above might not be strictly true for all game types and is more fitting for real-time 3D games. Although this should not disqualify any further conclusions, it might be preferable to think of the following discussion as focus on 3d video games in particular.

The above points mean that if we want to leave room for imagination in games, it cannot be the scene building from literature nor the connecting of events in film. With the level of detail of the world provided, little is left to the imagination. And the fluent events demanded leave very little room for players to fill in their minds. So what other gaps are there to be filled? To find this out, we need to take a look at a core feature of video games: interactivity.

So what exactly does interactivity encompass? I like Chris Crawford's definition (from this book):

"A cyclic process between two or more active agents in which each agent alternately listen, thinks and speaks."

What I like about this wording is that it makes it clear that interaction is not all about a user providing input. It is also about considering and then reacting to this input, and that the same applies for both sides (meaning both the human and computer). When it comes to finding opportunities for adding gaps of imagination, these are all of course on the humans side. Also note that the gaps might take place at any of steps: listening, thinking or speaking. With this in mind, I will make an attempt at some gap finding.

So where in this interactive cycle does there exist room for the imagination? The most obvious place is of course the "listening" (meaning any input). Even though we get a clear view of how the world looks like, there are still things left for our us to craft in our minds. This is something that is already present in some games and comes in the form of "environmental storytelling". Through exploration players can pull information from the world, gather details on past events and imagine emotional states of the world. Bioshock is a good example of this, where much of the attitudes and history of the sunken city can be found out purely by, the interactive process, of exploring the environment.

However, environments are lifeless entities, and while they can portray the aftermath of actions, they do not give us any feeling of agency. This greatly lessens the impact and diversity of imaginative gap-filling players can make. To take this to the next level, it is quite obvious that we need to included simulations of conscious beings. This allows us to construct mental "theories of mind", something that greatly increases the possibilities of expression. The problem is that we simply cannot do this with current technology, except at a very rudimentary level. While our techniques for facial expression is constantly getting better and better (L.A. Noir is a good example), this is only meant for prerecorded usage. When it comes to real-time procedural generation of expressive characters, we are at an extremely primitive stage. Because of this I believe that this can be very interesting to explore in the future, but not something that can be used right now.

So what else can be done? With expressive characters in real-time not an option, we must turn focus onto the actions themselves instead. As stated above, the events in video games do not leave any room for interpretation. But there still room for the imagination here though. What actions to make and why the are made.

Constrained role-playing
Imagination of the what and why of actions probably sounds a bit strange and needs some explanation. When players take control of an avatar in a video game, they are free to do what they please as long as it is accordance to the rules of the game world. This freedom might seem as the kind of gap that can be used to mimic the "magic" from literature and film. However not in the way actions are normally implemented: very specific and unambiguous. I reason so because there are two major problem with this approach.

The first is a technical one, namely is that it is pretty much impossible to give the player access the space of possible actions for any given situation. This means that there will always be events that the player can think of, but will be unable to carry then out. This limits the ability to role-play and might also leave, according to the player, the most intuitive and plausible action unavailable, breaking up flow and presence. The second problem is that the more events are added to aid role-playing, the harder it is to have artistic control, making the experience into an open-world simulation instead. As both of these problems work against one another, I think we have gotten pretty much as far as we can using this kind of design.

My suggestion for solving this problem is to have a limited number of actions available, but to lure players into imagining that the actual action performed was exactly the one they wanted to do. A very simple example of this can be found in games Samarost and Windowsill where the player can never in advance know what a mouse click will result in, yet when the action occurs it feels very intentional. This imagined motivation does not have to occur on a such low level though, and can include larger segments of the game. An example of this is The Path, where players are thrown into strange environment and forced to make up their own reasons for being there. Often this is something that is built up over a long time, yet greatly shapes how you view your entire session. I am not saying that these games are doing it the right way, only that they incorporate rudimentary versions of the ideas I am talking about, and hence can give one a basic hint of where to start from.

I bet that many will think of this concept as cheating. How can tricking the player be a proper design choice? If the whole interactive experience is an illusion, how can it carry any meaning? I argue that the same is true for other media. The events that you think happen on in film are in fact illusory too. Not only in the way that they merely consist of acting, set pieces and post production effects, but that many of the actions perceived was never filmed at all. They were instead conjured in the mind, by interpreting visual and auditory stimuli. The same is true for literature, were most of the mental images are never found in the text. Despite of this we do not describe the experiences these media give us as meaningless tricks.

Why "motivational imagination" sounds so strange has to do with the nature of interaction. When we watch a movie or read a book, this is passive experience where data only flow as input. But in the cycle of interaction, we are also part of creating output data. So when we create gaps of imagination for this kind of art work, we are unable to see it as a one-way stream of information, but have to include ourselves into it as well. The upside of it all, besides the solving the problem of role-playing, is that it fits neatly into same kind of concept that gaps in literature and film build upon. First of all, it contains a retroactive aspect to it, as players will need to digest a certain amount of data before settling on a certain motivation. It also forces us to make up a theory of mind, not for a fictional character, but for ourselves, inversely figuring out how we could come to a certain conclusion.

With this hypothesis I am not urging people to create games that are extremely linear and only require a single input. I still believe that we can have a wide palette of interaction choices, but that we might not want to be too specific about the exact actions that ought to occur. This is actually very closely related to the concept of player-avatar-symbiosis that have been discussed in an earlier post on this blog. I also do not believe that this takes away anything from the experience, but only adds to it, just like the same line of thinking does to other media.

End notes
This is far from a full theory at this point, and "environmental storytelling" and "imagined motivation" are most likely not the only imaginative gaps that can be used in games. Because of this I would be very interested in getting feedback and to hear your response on this work.

I would also like to point out that all of this awfully untested. It would be really interesting to see some Kuleshov-like experiments on the concept and see what kind of results can be made. It might be the case that this hypothesis does not work at all, or it might that it lead to wonderful and totally unexpected insights.

I also want to add that not all kind of experiences can be created like this. The same goes for literature and movies too, where leaving too much up to the audience simply does not work. Non-fictional books is one thing that comes to mind. Still, that is not a reason to not try this out. Before we try out all options that the video game medium provide, we will have no idea what can be accomplished with it.


Additional Notes: In Scott McCloud's book "Understanding comics" two similar "imaginary gaps" are explored in the medium of comics. One is the literal gaps between panels, that forces the audience to complete the missing information implied to be between. This is very much like what is found in books and movies, as it forces the reader to use external knowledge and also comes with a retrospective aspect. The second gap is one of cartoon symbolism, where simply drawn characters often can be more expressive than detailed ones. Again this requires quite a bit of interpretation from the audience.
I think this shows that the features discussed in film and book, apply to other media as well, making me more confident that they ought to play a big role in video games too.

Acknowledgments: This essay has been greatly inspired this post by Michael of Tale of Tales. If not for him I would probably have never started thinking in this direction and none of the above would have been written.