Archive for the 'programming' Category

Multiple Dispatch and Open Classes

July 13, 2009

In my last post I wrote that Scheme lets you extend the base language in powerful ways. In this post I’ll give you a concrete example of such an extension: TinyClos.

TinyClos is a minimal implementation in Scheme of Common Lisp’s object system CLOS. CLOS is different in many ways from the object systems of C++ and Java that most of us are familiar with. Two big differences are multiple dispatch and open classes.

Before I explain what multiple dispatch is I must explain what dispatch refers to. When you make a function call in any language, the compiler or language runtime must figure out which function the call refers to. This is called binding the function to an implementation. In some languages binding always occurs at compile time, in others always at runtime, and in still others sometimes at runtime and sometimes at compile time.

Some languages, such as C++, allow you to declare many functions with the same name as long as they have different parameter lists. Not only the name of the function but also the number of arguments and their types are used when binding. This is called overloading in C++ terminology. However, overloading is a compile-time mechanism, and as such only looks at the static types of the arguments. If you want to bind the call based on the run-time type of an object you have to make the function virtual.

However, virtual functions in C++ (and most other mainstream languages) only take the run-time type of the first parameter into account–the “this” pointer. This is called single dispatch, because it dispatches the call based on a single argument. As you can probably guess by now, multiple dispatch is when the run-time types of multiple arguments is used for binding. In other words, multiple dispatch is like overloading but with run-time types.

This is immensely useful in some situations, because you can decide what to do based on combinations of types, without having to sprinkle your code with instance-of tests or resorting to ugly work-arounds like the visitor pattern. For instance, to express that anti-aircraft units can attack air units but not tanks or other units types, I just specify a method can-fire-at? for the type combination (AirDefenseUnit, AirUnit) and another one for (AirDefenseUnit, Unit). The method with the most specific matching argument list will be selected, at run time.

Here’s another simple example in the syntax Bjarne Stroustrup wants for C++ (not included in the coming update of the standard):

bool canShareHex(virtual Unit u1, virtual Unit u2) {
  return true;
}
bool canShareHex(virtual GroundUnit u1, virtual GroundUnit u2) {
  return false;
}
bool canShareHex(virtual AirUnit u1, virtual AirUnit u2) {
  return false;
}
bool canShareHex(virtual SeaUnit u1, virtual SeaUnit u2) {
  return false;
}

In other words, GroundUnits can share hex with all units but other GroundUnits. This example also demonstrates the other feature I wanted to write about: open class extensions. As you can see above, the methods were not part of a class but added separately. This too is very useful, because you can’t always anticipate what people may want to do with your classes, and sometimes it’s impractical to cram everything into one class.

For example, let’s say you are writing a compiler and define classes for different kinds of expressions, like addition and multiplication and so on, reflecting the tree structure of arithmetic expressions. Since the set of operations (optimizations) you may want to perform on the tree is open ended, you wouldn’t want to make them all methods of the tree classes. It would be much cleaner to define each optimization separately, while still dispatching on the actual run-time type(s) of the tree nodes.

With Scheme, I could just download TinyClos and start using these features. They can be compiled efficiently too, though I’m not sure they are in Chicken. Knowing that it can be solved should the need arise is reassuring, though.

Further reading:

Scheme is not one Language

July 13, 2009

Considering all the hassle of using multiple languages in one project, was it really a good idea to use Scheme? I would say so.

Scheme is not like other languages. In a way, Scheme is not really one language, but a foundation for defining other languages. The semantics of Scheme were designed to enable programmers to devise their own control structures and the syntax was minimized to make it easy to parse, modify and layer your own syntax on top of. The language as a whole left so many other things unspecified that you must build your own environment on top of it unless you’re working on a small project.

One example of the power Scheme gives its users is unlimited continuations. A continuation is a point in the execution of a program which has been lifted up to the language layer, where it can be passed around or saved in variables, letting you execute it again later should you want to. That was pretty abstract so I’ll give you an example:

As you may know, functions in C programs push their return address on a stack before they are invoked. When a function returns, it pops the return address from the top of the stack and jumps to that address, which is where the call came from. The return address is the continuation of the caller (I’m simplifying here), but it doesn’t exist as a first-class entitiy in the C language, so we cannot see it or refer to it. In Scheme you can. Moreover, you can call a continuation as many times as you wish, and you can ask for the current continuation at any point, not just at call sites.

This is not terribly useful in day-to-day coding, but it is essential if you want to make your own language. Using continuations, you can implement exception-handling mechanisms like try-catch without resorting to low-level hackery (and things like loops with complicated exit logic and light-weight threads, too).

What all this comes down to is that Scheme is a very good environment for language experimentation, which has resulted in a lot of people hacking on these kinds of things. Consequently, you can go to the web page of your Scheme environment and download language extensions others have written that add or alter fundamental parts of the language; They may for instance add object-oriented constructs and run-time dispatch. Mainstream languages are much less malleable.

In summary, I think the ability to download language extensions and, so to speak, assemble my own language from pieces written by others has been a huge advantage. That, and the ability to make changes to the program as it’s running has been more than enough to justify the hassle.

Extended Scheme

July 13, 2009

I have now started to use the preprocessor on a small scale. All it does currently is translate dotted field expressions into the syntax TinyClos expects. I actually found out you could type  (slot@ foo x y) instead of (slot-ref (slot-ref foo ‘x) ‘y), but foo.x.y is better still, so I don’t regret writing the preprocessor. Besides, this way I can easily add more “macros” that don’t conform to Lisp’s grammar later.

This is not really a new language, just an extended variant of Scheme, but I still need a name for it to be able to talk about it.

Extended Scheme is too long a name, so I thought about shortening it to Extreme, but that doesn’t send the right message. Instead I’m calling it Steme (pronounced like steam), which you can interpret either as Ekstended Scheme or Stefan’s Scheme. If this ever grows into something serious I need to find another name for it.

Proactive Air Defense

June 19, 2009

Anti-aircraft units are now much better at defending their area, since they get to shoot first–during the enemy’s turn. Things are a bit confusing now, though, because the chain of events isn’t made manifest to the player. If you lose concentration and accidentally move a plane into the range of an hostile anti-aircraft unit you may be surprised to find your unit suddenly half as strong as it used to be. If you’re really unlucky the whole unit may disappear, with no indication of who fired at it.

In short, this change makes animation a requirement, and not just a feature that would be nice to have. I probably won’t take care of that right now, though.

Automatic Reactions

June 14, 2009

I haven’t completed any new functionality yet, but I’m working on a framework for managing automatic reactions. This is needed by anti-aircraft guns for instance, since they must be able to react immediately to defend their area effectively. Before an attacking plane is allowed to drop any bombs, all air defense units in range will get a chance to shoot it down.

To make this work I need to decouple the request for an action from actually performing the action, so that other units get a chance to intercept. When the user tries to attack an enemy unit, I now send an attack request to an arbiter object running in another (lightweight) thread. The arbiter asks enemy units nearby if they want to counter the action, and schedules the resulting actions. The schedule is then executed and the results broadcasted to everyone who can see them.

These changes not only enable automatic reactions; They also bring me closer to having networked play and fog of war (which refers to the inability to see where all the enemy’s units are).

The Perils of Mutable State

June 7, 2009

A few weeks ago, I spent more than a week debugging parts of a very large software system at work. I will not go into detail about the software, but I want to write down what I learned from it so that I will remember it better, and share my conclusions with you.

Actually, I started thinking about similar problems months earlier when I was writing new code that gradually became much more complicated than I had expected. Why did I end up in a morass of special cases and conditions?

Both problems involved updating the state of several different data structures that had to be kept synchronized. The basic problem was that changes to the data were spread out over many different functions and that early changes could invalidate later changes or affect the control flow in an undesired way by changing the outcome of a conditional branch.

When I was working on the first problem I was writing new code and had free hands when it came to design. After realizing I would have to scrap my first design to make further progress, I decided I would partition the problem into two distinct parts: One would figure out which changes needed to be made and encode them in a class called a ChangeSet; The other would apply the changes. The idea was that it would be easier to debug and understand the algorithm if you could print out the ChangeSet and check invariants before it was applied. This worked well and simplified the code enough that I was able to make progress again.

The other problem had another important ingredient: randomness. The iteration order of several central datastructures (such as sets) is random in our system (a side effect of our generational garbage collector’s decisions) and caused the (very large) algorithm to update the data structures in a different order from one run to another. Most of the time everything worked fine, but every now and then things failed with random symptoms.

It has slowly dawned on me that these two problems were essentially the same: Many changes had to be made to many different data structures, and those very changes could cause the algorithm to trip itself up.

My solution to the first problem (using a ChangeSet) worked well not only because it made following the changes easier, but also because it concentrated all state changes to one phase, letting the first part of the algorithm discover what needed to be done in a temporarily immutable world. I think there is an important and general principle here: Changes to mutable state should be confined to as few points in both space and time as possible. This goes for any language with mutable state. Object-oriented principles do not seem to cover this problem, since hiding the state inside a class does not make the state go away; It is just changed via a method instead.

An interesting parallel is that the solution I described above is similar to what database systems do with transactions, and databases are specialized on managing mutable state in unpredictable environments.

Crosshairs

May 24, 2009

Finally, after a lot of internal changes, you can tell if a unit can be attacked by holding the mouse pointer over it. A red crosshair will appear over it until you move the pointer away again. Here is a screenshot:
screenshot-090524-1

The mouse pointer doesn’t appear in the screenshot for some reason, but I was holding it over the crosshair. The selected unit is indicated by a sharp border around its hexagon, and the hexes it can move to are highlighted, just like before. The tank isn’t really supposed to be able to cross the strait in one turn, but I haven’t yet bothered to update the terrain graph completely after I made changes to the hex grid.

Another change that can be seen in the screenshot is an anti-aircraft unit, near Copenhagen. I haven’t written any code for it yet though, so it behaves like a tank :)

To accommodate the crosshair functionality I wrote a simple quadtree implementation which can map mouse clicks to scene graph nodes quickly. The new code is about 700 times faster than the old one, which searched through the entire scene graph including terrain and all. Working at this made me realize a lot of things about scene graphs and how they should interact with other parts, which will probably help when I set out to speed things up later. Currently there’s an annoying lag when you do many things that require updating the window, because I redraw too much stuff.

Progress and Plans

April 13, 2009

I have improved some more interface details. Now, when you select a unit, the borders of its hex become thicker until the unit is deselected again. Another change is that units now turn in the direction you move them, so a tank that has just been moved from west to east will face east, for instance.

I’ve been thinking about announcing an early preview version of the game on some Internet forums for feedback, and which features should be included in that case. I don’t want to send out a crappy game with a bunch of half-finished features, but I don’t want to wait until everything is perfect either. Right now, I’m leaning towards trying to get something simple out late this summer.

The features I think must be included in the preview are:

  • One relatively large and good-looking map, of let’s say Denmark, with shifting terrain etc.
  • There must be some visual indication that a unit can be attacked by the currently selected unit. Panzer General put a cross hair around it if I recall correctly.
  • Deployment and movement of lots of units must work flawlessly since you will be doing it a lot.
  • The bombers must be counterbalanced by for instance air defense units.
  • I must either adjust the hex grid to the terrain or the terrain to the hex grid so that it is always clear if a hex is a land or a sea hex, for instance.

That is it, I think. The other stuff either works already or is not essential. I may be forced to include zoom too when I increase the size of the map if it becomes too hard to get an overview of the field.

The preview version probably wont be much fun to play without sound, animations or surprises, but at least I’ll have something to show. There won’t be anyone to play against anyway, since there is no AI and no server :) You will have to sit next to your opponent.

Bombers can now Bomb

April 10, 2009

I’m back to hacking on the game again. I’ve made the bombers able to attack and polished some small things, like how units and flags are arranged inside hexagons.

As you can see, the map is also bigger now, and the panel is placed at the top of the window and contains more buttons. The “toggle view mode” button changes between air mode and ground mode, ie which unit is scaled down when an aircraft shares hex with a surface unit. I know the buttons look awful but fixing that can wait.

screenshot-090410

Layout Language

February 8, 2009

To my surprise, I didn’t get any error messages any longer after I stopped using multithreading. All I had to do to fix it was catch all exceptions from the evaluation loop and call a function to print an error message when an exception was caught, but I didn’t realize that at first. Instead, I wasted many hours fighting with Chicken. I got tired and frustrated and didn’t get as much work done as I had hoped.

However, I did start hacking on a sublanguage for automatic layout of user interface components. The idea is that you should be able to specify a layout using a clean declarative language, much like HTML or Mozilla’s XUL. That way you don’t have to write as much code to arrange buttons and the like, and the specifications will be much more readable and easier to change than imperative code.

Here is an example of the kind of declaration I’m aiming to support:

(hbox (width: 100 %)
  (hbox (align: left) ship tiger stuka)
  (hbox (align: right) end-button exit-button))

A hbox is a container that arranges its components horizontally, from left to right. In this case, there is a hbox that contains two other hboxes. One of them is left aligned and contains the components ship, tiger and stuka; And the other one is right aligned and contains end-button and exit-button. In other words, it means something like the following pseudo-HTML:


<hbox width="100%">
 <hbox align="left"> ship tiger stuka </hbox>
 <hbox align="right"> end-button exit-button </hbox>
</hbox>

Box nesting does not work yet, but alignment does. Automatic layout management actually appears to be quite easy to implement, at least if all you want is something HTML-like. You just make two passes over the tree–first, a bottom-up pass that calculates the size of all components, and then a top-down sweep that positions them.

After writing some code for arranging components horizontally I asked a friend of mine who is very proficient in functional languages if he knew a general way of expressing the iteration that could replace the ad-hoc code I had written. He came up with this neat piece of code and its brilliant name:

;; Return a list of results from applications of ‘termfun’ to the
;; elements of ’sequence’ and a carry.
;; ‘carryfn’ is used to calculate the next carry from the previous one
;; and the current element of the sequence.

(define (map/carry termfn carryfn carry sequence)
  (if (null? sequence)
    ’()
    (cons (termfn (car sequence) carry)
      (map/carry termfn carryfn
        (carryfn (car sequence) carry)
        (cdr sequence)))))

Using map/carry I could express a function for horizontal distribution of components without using any explicit iteration. Beautiful! Thanks, Tord. I considered many names for a similar iteration construct that remembered earlier results, but none of them was as succinct as “map/carry”.