Solving Get1000 continued…

In one of my previous posts, I laid out a plan to solve the Get 1000 game. It turns out that plan was wrong.

My expectiminimax based solution works well for a game with random elements and perfect information, but it is not very useful for a game with imperfect information. Get1000 is played simultaneously by the players, and the opponents choices are hidden until the end of the game.

This meant I had to go back to find a new strategy for solving the game. I decided to try and find the correct brute force way first, and then see if that could be made faster in some way.

Exploring brute force

A solution to the game involves finding a Nash equilibrium from all the pure strategies of the game. A brute force solution could be done by creating a matrix where all pure strategies are pitted against all the other pure strategies.

The full payoff matrix needed for a normal form brute force solution.

A strategy here refers to a function S \rightarrow Pl which given any game state S gives a Get1000 placement Pl . Since there are at most 211248 - 27648\cdot3 = 128304 states in the game, each which at most have 3 choices, there is an upper bound of M = 3^{128304} unique pure strategies.

This is of course not that helpful, since a 3^{128304}\times3^{128304} matrix is is enormous, and for each cell in the matrix all possible 6^9 games would have to be played to find the payoff P for the pure strategy pairs. On top of that, the best mixed strategy would then have to be calculated.

Subgame perfection and backwards induction

Modeling this in normal form as above seemed to get me nowhere, I therefore turned to extensive form, and something called subgame-perfect nash equilibria, and backwards induction. In the normal form solution I need to look at all possible strategies. Using subgame perfection, I hoped to get away with only looking at a very small subset.

While this sounds straighforward in theory, I found it quite hard to figure out where my information sets are, and whether I could consider each choice node in Get1000 a subgame. After struggling for a while, I ended up with an extensive form structure looking like this. Players are P1 and P2, and “move by nature” is the dice roll.

Extensive form of a game with the same structure as Get1000. As players do their choices, the information sets get larger and larger. Since the “moves by nature” are known by all, they do not increase the information sets.

This structure means that only the roots of the tree are subgames, since all other nodes are part of larger information sets.

Attempting backwards induction

The above structure means that it is not practical to use subgame perfection and backwards induction to solve the game, but taking inspiration from it could still be useful to get a good strategy.

The algorithm for subgame perfection goes like this:

  1. Consider the final subgames (those with no further subgames), pick a Nash equilibrium as solution there.
  2. When considering the next subgames up the tree, the payoffs in the subgames already considered are used to create the payoff matrix.
  3. Iterate step 2 until the root node of the extensive form tree is reached.

To get something working, I pretend that the other player is at the same state as me always. This means I can need only focus on the branches below that state. To keep memory in check i also recalculated payoffs instead of storing the result for each combination of states and games. The final algorithm I ended up with works like this:

  1. Consider the final subgames and pick a Nash equilibrium as solution.
  2. When looking at subgames higher up the tree, I use the choices (not payoffs) computed in 1, and use those choices to play out the game. Then I compare end results to get the payoff matrix for that subgame.
  3. As before, I iterate step 2 until I reach the root node.

This seems intuitively pretty reasonable.

Experimental results

The above method gives me a strategy that partly takes the imperfect information nature of the game into account. At many states it detected mixed strategies that had much higher payoffs compared to the pure versions. The strategies smashes all my previous best strategies by winning 1.75% more games.

The mixed strategy seems to play even more aggressively for results close to a 1000, and allowing heavy overshoot.

At this point, I was not really sure how to approach the game in a better way. In fact I was pretty ready to admin defeat for quite some time. Of course, immediately after i wrote that, I found this thesis, and this report.

Lots of new concepts to learn!

Scape – a very ninja scripting language


I made a small scripting language that runs in the browser. It is very ninja. To see the ninja, first open Javascript console and write:

function recur() {recur()};recur();

Hopefully it blew the stack. Then type this into the Scape REPL:

def recur() recur(); recur();

When you are convinced it will infinitely loop without blowing the stack, hit ctrl-c to stop further processing.

Rincewinds rave, that is black magic! Also called tail call elimination. Scape code is not evaluated by snarfing functions from Javascript (JS functions do not have tail call elimination before ECMAScript 6), but instead is compiled to its own set of instructions, which are then run on a stack machine (running in the Javascript VM). During parsing Scape functions are checked for whether they can use tail call elimination. If they can, they get different instructions that reuse the existing stack frame.

More magic

Scape has forward mode automatic differentiation as a language feature. Automatic differentiation allows you to compute the derivative of a function, without having to define the derivative explicitly.

Without automatic differentiation, this would be the way to compute the partial derivative of the function f(x,y) =  x^{2}y^{2} for x and y:

def fun(x,y) * (* x x) (* y y);
def diff_fun(x,y) [* (* 2 x) (* y y),* (* 2 y) (* x x)];
diff_fun(4,5);
[200, 160]

With automatic differentiation in Scape, this is how it is done:

def fun(x,y) * (* x x) (* y y);
diff(fun(4,5));
[200, 160]

This is very useful for a number of numerical methods involving derivatives. The feature is currently experimental, it might interact with non-double types in funky ways.

Wai?

Mostly just for fun. I also started toying with the idea to make a safe scripting language for use in networked games. A language and runtime that would allow the player to define custom logic during gameplay without being able to ruin the experience for other players.

A dream would be a personalized Starcraft where it is you and your custom control scripts versus the other player and his scripts.

I hope to create a simple real time multiplayer game to show how I imagine it working. For now, playing with the Scape REPL is the only way to try the language.

Sayōnara

Pear pram construction

My 3 year old daughter’s current favourite book is Jakob Martin Strid’s Den utrolige historien om den kjempestore pæra. In the story a giant pear is made hollow, and eventually turned into a boat. I decided to try and make a similar upright floating pear-boat out of a normal pear. One that my my daughter could play with once finished. Off I went to buy a pear and tools, the pear I ended up with have these specs:

  • Weighs around 238 g.
  • Displaces approximately 225 ml of water. This means it is slightly more dense then water since the weight of that water would be approximately 225 g.
  • Judging from the pears available in the shop, the one I chose was slightly more symmetric then most pears.

In addition to the pear, the tools I used are shown in the image below:

pear-tools
Pear, knife, melon baller (the MVT in pear pram construction), coins, thread.

The first step in pear pram creation, is to cut the windows using a knife. Once that is done carve out the inside using the melon baller. I recommend making the windows close to the top of the pear. If you make them too low they will result in water intake when launched. It is better to create the windows high initially and expand them downwards once you have a feeling for how the pear floats.

After carving out the core and flesh.
After carving out the core and flesh.

Another great reason for making the windows high up is lowering the center of mass. Pears have not yet been cultivated to float upright, and the center of mass is way too high for that. Having a too high center of mass on a ship is catastrophic. In its unmodified form a hollow pear will most likely perform worse then the Vasa on its maiden voyage.

To lower the center of mass even more, it is important to carve out as much as possible of the “roof”. Since the stalk extends into the pear this is difficult. It is easy to ruin the pear if you use too much force; be careful.

Even with the windows high up and a very light roof, the center of mass is still too high. Initially I added coins on the inside as ballast, this worked if I put the pear very carefully into the water, but it was still prone to capsizing.

Pear in bowl
Coins on a thread prevents capsizing. The pear is finally approved for transport of Lego men

To remedy this I moved the center of mass even lower by adding ballast on a thread below the pear. This was done by creating a knot on the thread and using a needle to get it through the bottom of the pear. Then the coins were added as shown in the picture. To ensure that water would not leak along the thread into the pear, I greased the entry points.

In its finished form the pear is quite stable, it could hold a surprising amount of Lego men without capsizing. I never got around  to test adding a sail, but I think it might be stable enough to support a small sail.

Sadly rot is an inevitability facing all pear prams. If you have a great idea for prolonging pear pram life, please leave a comment.

 

 

Superb toy!

This toy is incredible. It is reasonably easy to construct things, while still providing a challenge once you get the basics. It is suitable for nearly all ages. It says 3+ but if you remove the smaller parts it is suitable for much younger children even though they might just enjoy tearing down your constructs. The absurd thing is that when they do, you wont hesitate to make another one, since building these are way too much fun.

WEDGiTS
WEDGiTS