Man, this is a good cup of tea. Feels good, letting the warmth soak through my hands. You know the best thing about tea? When fresh it’s too hot to drink. You’re forced to be still for a moment, enjoy the anticipation.

Here, pour yourself a cup and pull up a chair - this is the perfect moment to let our minds go wandering.

Have you had enough about AlphaGo and Lee Sedol? Yeah, it’s been everywhere. I like writing AIs for games as much as the next guy - well, probably more. That stuff is like crack! Used to play Go, too. Mankind against his own creation! What a story!

No I don’t worry about the Rise of the Machines too much. I think the future will be more interesting than that - it puts me in mind of the whole oil thing.

Yeah, oil. Listen - mmmaaah, that first sip is the best, don’t you think? Anyway, where was I? Oh yes: oil is like artificial muscle.

On Artificial Intelligence and Oil

Thousands of years ago all we had was muscle power. So when someone started to become wealthy and powerful they could only get more work done by getting other people (or animals) to do them. You want to build a granary? Better be able to pay at least enough food for the workers who will build it for you. Oh, and for the hired muscle to assert that it’s yours. And so this concentration of wealth you created, earned or stole is parceled out to some extent amongst those around you.

Want to rule a kingdom? Build a cathedral? Live like a prince? You’re going to need to pay for a lot of people, because people are the fundamental unit of work. Yes, you’re right - I think Adam Smith did write something like that. Even slaves were paid with food and basic shelter.

So when the industrial revolution comes around things start getting crazy, because we have oil! Artificial muscle power! Suddenly we can do more work with fewer people. A lot more!

We need not dilute our wealth amongst so many other people when employing it to the same effect. And the longer the revolution goes on, the more pronounced the effect becomes! A couple of hundred years later and a roomful of people can create a billion dollar business from almost nothing - isn’t that amazing?

Wow, empty already. I always reach the bottom of the cup sooner than I expect. Oh yes thank-you, I will have another. Milk, no sugar please. Oh go on then, just one.

As Gods amongst Men

Mmmh, thanks. Anyway, I think we’re standing at the cusp of another industrial revolution. We don’t need to wait for the mythical strong AI and the singularity or whatever. Even weak AI is just the perfect complement to oil! Oil allows the wealthy to employ artificial muscle. AI allows them to employ artificial intelligence to control that artificial muscle!

See, when AI can direct machines to extract oil, produce goods, repair and replace other machines - and you must agree this is ultimately within our grasp, right? When we do this, for the first time in history we can employ our wealth without giving a single cent to another human being! Isn’t that a crazy thought? But it’s the only logical consequence!

It won’t happen overnight, of course. The replacement will be gradual. An ever-shrinking elite of the wealthy and powerful and a circle of others around them fulfilling our wishes. But ultimately we won’t even need other humans to protect our wealth. Mere humans won’t pose a threat to our autonomous guards and drones!

We lucky few who have the most resources when the revolution comes will become like gods amongst men - capable of anything, the world bending itself to our every whim. The rest of the human race will struggle and squabble over the whatever resources we leave out of pity or compassion, or that are simply not worth claiming. To them our capricious whims and wars will be as incomprehensible as they are dangerous and their opinions won’t matter a damn.

We won’t need to overthrow governments, they’ll collapse by themselves as the tax revenue from the increasingly impoverished and unemployed masses disappears, our wealth safely beyond their desperate grasp.

We can reestablish the feudal system if we like, allowing the lesser peoples to pay rent on our properties whilst working the land in occupations that we find… suitable. We’ll never need them of course, but it’s so fashionable to have an estate with a few surrounding villages, don’t you think darling? The admiration of the people! I have mine stand aside and cheer whenever I pass, it always brightens one up after a good day’s hunt.

But Seriously

Oh, was that really the last of the tea? Back to the grindstone, I guess. Places to go, things to do.

What? No, no, don’t worry. I wasn’t really serious about all that. There’s no chance at all that you or I will be one of the wealthy elite! No, our only hope will be to join the hacker insurrection…

Anyway, I really do have to get back to work - these deep learning models aren’t going to train themselves, you know!

This little Nosgoth strategy and tactics guide offers a way to improve at the game quickly enough to enjoy the surprising depth of play at mid and high levels.

Update: Nosgoth is now in Open Beta - if you’re not playing yet then you should! You can start with a free booster using my friend referral link - see you in the game!

As a new player Nosgoth is pretty punishing. I still remember the confusion and slaughter of my very first game and perhaps you still remember yours.

Rather than hand out specific yet somehow unhelpful tips, such as “learn to time the charge duration on your warbow so the shot is ready when you need it”, I’ve chosen to write about a method for improving efficiently. It will help your first 100-200 hours of play up to level 40; after that you will understand more than enough to choose your own onward path.

The guide is divided into three sections:

1. Basic advice and skills - aiming, listening, dying
2. Playing to learn - set round goals, flow of play, learn from the masters
3. Mid-level play - class counters, yomi, teamwork

Basic Advice and Skills

This guide assumes you already know the basics of play - what zoom does, how to dodge and so on. There are plenty of other guides that cover those kinds of basics. Instead I want to call out three things about Nosgoth that reward closer attention.

Aiming

Depending on your choice of class Nosgoth can be enjoyed without perfect twitch skills, but being able to aim accurately under duress will always make a big difference. If you play a lot of FPS shooters feel free to skip this short introduction.

If you don’t play a lot of FPS shooters and don’t know what inches/360 means then you need to know only one thing:

Your mouse sensitivity is too high.

Let’s fix that now before you get used to it. Edit My Games\Nosgoth\config\BCMPUserProfile.ini and make these changes:

1. Change bMouseSmoothing=True to bMouseSmoothing=False
2. Change LookSensitivity=whatever to LookSensitivity=1
3. Change MouseSensitivity=25 to MouseSensitivity=15

Make changes #1 and #2 in BCMPUserProfile.ini too for good measure. I actually play with MouseSensitivity=9, but 15 is a good starting value if you’re used to oversensitive mouse movement. Once you’re used to it, try reducing it by 2 or so each time and notice how much easier aiming becomes!

You can read up what all these do on this forum post, but these values are fine to start playing with. Also turn off “improve pointer accuracy” in Windows’ mouse settings while you’re at it and put the sensitivity slider onto 6/10. Direct movement is easier to learn.

On the subject of aiming: many people find it more accurate to hold the mouse with their whole hand and move the arm/wrist to aim. Try this too instead of pushing the mouse around with your fingers!

Much of early Nosgoth play as a human is long periods of tension followed by extremely intense periods of action. This is a difficult environment to get used to aiming in.

Instead, spend a couple of minutes per session in the tutorial. Run through to the first health station then turn and find a marker on the wall behind the flags. Hover your crosshair over a flag, then rapidly move it to the marker, shoot it and move it back. Repeat this with many different flags, ranges, zoomed and unzoomed, moving and stationary for your 120 ammo them exit the tutorial and start a match.

After a few sessions you should find it quite easy to keep a vampire in your crosshairs during short range and melee combat. If the action seems too confusing, check your frame rate by going to the video settings during a match. Anything below 30fps will be a problem; turn every graphics option to low or off, reduce your resolution and if you are playing on a laptop check the right graphics card is used and your CPU’s turbo boost is disabled. While you’re changing graphics settings you might as well increase your gamma, too.

As a vampire rapid aiming is less important but do use dodge-melee-dodge exclusively for movement on the ground in your first few matches as it is faster and less predictable than running and is worth making second nature. Also force yourself to only use charged attacks for a couple of rounds to get a feeling for how long they take to reach full power and how far and fast you move. You can actually turn corners while diving forwards like this!

Listening

The aural soundscape of Nosgoth is more revealing and important than most other games. Wear headphones and turn the volume up enough to hear footsteps clearly. Often hearing an audio cue is the only thing that separates you from life and death, so learn to recognize the distinctive sounds each vampire makes before they attack.

When you unexpectedly die, take a moment to recall the sounds you heard leading up to that moment.

Dying

Every time you die, which will be a lot, a ten second or so timeout gives you a natural moment to pause for reflection. Cultivate the habit of using this time well. Often, particularly at the start, you’ll feel you were killed unfairly, with no chance to do anything. Maybe you were rammed and pounded by two Tyrants and unable to move or react at all until pasted across the floor. Instead of raging at “lame tactics” reflect on the life choices that led you to this moment.

There are no lame tactics in Nosgoth. Everything can be countered. Nosgoth is a high-skill game. A good player will reliably destroy a worse player. Random chance and luck play a much smaller part than it sometimes feels at first.

Often the lesson to learn from a death is Do Not Let This Happen. By the time the first Tyrant hit you it was all over, but the audio cue for a stampeding Tyrant is very distinctive and gives plenty of opportunity to dodge out of the way. Were you standing in a particularly charge-worthy place, too close to your teammates or trapped in a tight space? Were you even aware that two vampires were playing Tyrant?

Improving at Nosgoth efficiently means asking these questions every time and trying different behaviour as a result.

The hardest thing to do is not to blame your team. You will play most of your low and mid level games in public servers with random groups of inexperienced players. You cannot expect your team to always have your back, but you can learn to make it easier for them by standing where two or three of them can clearly see you, for example.

Always remember you are not playing to win yet, you are playing to learn. Dying is an important part of that. Quietly give thanks to your opponent for showing you a way to improve your play.

Playing to Learn

The greatest barrier to improving early on in Nosgoth is trying to win. Whether you want the highest individual score or for your team to win every match, these goals are largely out of your control in a randomly-filled public server.

We are not playing to win, yet. First, we are playing to learn.

Set Round Goals

Spend the time waiting for a match to start contemplating what you want to learn and improve in each round of this match. Pick a goal that is as much in your control as possible. For example, perhaps you are learning to play as a Reaver. Equip Savage Pounce and make your goal to perform four Savage Pounces and escapes in a row without dying.

This has two benefits. Firstly, you can be entirely satisfied and happy at the end of a losing round in which two teammates rage quit because you had the opportunity to work on more stealthy and surprising approaches. Secondly, it forces you to develop a whole range of skills and way of understanding the game and level environment that you otherwise wouldn’t have.

In our previous example, to maximize pounces before death you will quickly learn to sneak carefully around the map, to select targets who, momentarily, are not being watched by their teammates, to cover your entry with a distracting smoke bomb or choking haze, to find just the right psychological moment in your team’s attack in which the humans are fighting every man for themselves. More than continually trying to get in fast and LMB-spam inexperienced alchemists would ever teach you.

When the round ends consider your personal goal score, not the round score. Did you achieve your goal? Did your approach work? Noscam is a great way to review rounds that just didn’t work out. Often although you felt “unlucky” during the game a careful review from your opponent’s perspective will highlight how their style of play made things particularly difficult for you. Again, our opponents are teaching us. What friends we have in them!

The Flow of Play

If your match goal is WHAT you want to achieve, the flow of play is HOW you want to achieve it. This is best explained by example. Let’s take a flow of play for learning how to play as a Sentinel:

Goal: Kidnap, finish and drink from four humans in a row without dying.

Flow of play:

1. Locate the humans
2. Watch them, choose a target and a time to attack and an ideal place to drop them
3. Dive in, pick up and get out with minimal damage
4. Land and finish your target
5. Safely drink from them, if possible, then repeat

You wouldn’t play a competitive match with a script like this, but breaking down the messy complexity of a round into distinct tasks and phases is an incredibly powerful way to learn faster.

By playing with this plan in your mind, you will begin the game fully focused on the best way to find the humans without getting sniped out of the sky. This leads to checking which classes they have, learning the common spots for humans to gather and the best approaches and observation areas. There’s no excuse for taking damage in this phase and you will quickly learn to accomplish it without being seen or shot at all.

Here’s another example that I used when learning how to snipe as the Scout:

Goal: Snipe four vampires with the warbow without dying.

Flow of play:

1. Choose a good sniping spot close to my team’s location and defend it with a trap
2. Watch for opportunistic targets and take early shots
3. Move to a more hidden spot further from my team where I can cover them just before the vampires arrive, perhaps using Camouflage
4. Snipe vampires in the melee, disrupting Tyrant jumps, Reaver pounces and Sentinel dives
5. Either finish the fight quickly with LMB spam or move under cover to another vantage spot before too many vampires decide to come looking for me
6. Heal safely and repeat

This flow taught me so much about the strengths and limitations of Camouflage vs just standing in a bush or dark doorway, about timing my draw, about vampire psychology and the right moment to choose a new spot (protip: in an extended fight, roughly 15 seconds after my first kill a vengeful respawned vampire will come to the spot I killed them from. A different flow includes “Find a spot overlooking the first one and kill them again for bonus points”).

You won’t use the same flow forever, only until you feel you have mastered it. New ideas and adjustments will come all the time and it’s good to try these out! Just remember to separate how well you played from how well-suited your flow was to this match when looking at the results. Understanding this difference is the first step towards mid-level strategic play.

Learn from the Masters

Watching high-level teams battling it out with each other in tournaments is fascinating, but most of what is happening and why will be opaque to you during your early levels.

A more efficient way to learn from experienced players is by watching their twitch streams and reading their guides. I learned everything I know about dealing death from above from Warmonic’s excellent guide and videos and the guides from high-ranked EU team Dead Sun.

When an experienced player talks about their understanding of the game, pay attention not just to what they say but to the way they are thinking about the game. A great example is Saturnity’s reply to a question about “stupid” stunlock:

Your stunlocking point directly ties in with spacing and dodging. High level players can roll out of the way of every vampire and potentially every human CC in the game.

Sent pickup gives audio cues you can use to dodge without even looking at the guy. Good tyrants do a wiggle charge, but you can predict how he’s going to try to swerve into you and get out of the way.

Think yomi. Reavers are the hardest to dodge but you have multiple ways to stop a pounce. You can shoot them down for fall damage, CC them out of the air, or just roll.

You can out-think humans when it comes to dodging their CCs. Did you just charge a melee attack through a human? Did they just roll backwards in a fight? Are they waiting for you to use a gap closer? In any of those situations, they’ll immediately want to hit you with a CC.

Either space+aim your attack so you slide past them or roll through them afterwards. If the human will predict that and aim the CC behind them, you can counter their counter by sneaking in a melee attack before dodging since they’ll wait a sec for the dodge. More yomi.

Tyrant grab can be interrupted by your teammates, use good positioning and spacing and don’t let them grab you at bad times. Don’t stand near a corner if you predict a tyrant will attack from there. He’ll grab you. Get away from jump landings or cancel the jump by airshotting with a CC move. He could grab you if he lands on you and your team is busy.

One of the top players, chriZor, has uploaded videos of the competitive ESL matches with German commentary. If you can understand the language, watch them and listen to his advice! If you can’t, watch them anyway and see how top players use your favourite class effectively.

Attempting to emulate a master has been a tried and tested method of learning for millenia, and with videos and Noscam we have more opportunities for this than ever. Use them!

Mid-level play

Class Counters

Players in the level 10-30 range often get the idea that class stacking (e.g. 4 Tyrants or 4 Scouts) is overpowered and unfair.

This is not true.

Competitive teams play with a mixture of classes because it is more effective. 4-stacks are rather rare in competitive play - so far in 9 ESL tournaments I’ve only seen three matches in which one team used a 4x class stack. The second half of this match is an example of a 4x Reaver team being taken apart by 1x Scout, 2x Hunter, 1x Prophet.

So why does class stacking in Nosgoth feel overpowered to relatively new players? There are two reasons.

Firstly, stacking classes is the simplest kind of team-level tactic to implement in a pub. “Hey let’s all go Tyrant and keep charging them lol” is all it takes. The hard counter to that would be something like a team of Alchemists, but low level players don’t necessarily have all the classes available to them or aren’t willing to change.

Secondly, matchmaking sometimes puts a much better team against a much worse one. In this case the better team will often just mess around. A fun way to mess around is to e.g. stack Sentinels and try to catch each others Kidnap drops. It would get slaughtered against a team of decent Scouts or Hunters but they don’t care because they’ve quickly realised they’re all better than you anyway. They’d beat you even more if they were playing properly.

So when you see a stacked team, embrace the challenge. Switch to a counter class and encourage your team mates to do so too. Learn the strengths and weaknesses of your opponents.

Here’s a rough-and-ready guide to class-based counters:

4x Hunter: mixture of classes - e.g. 1x Tyrant for CC, 2x Reaver, 1x Deceiver or Scout. Infect on the Deceiver can be fun because new players will tend to bunch far too close together as humans.

4x Prophet: also a mixture of classes; I’d field 1x or 2x Sentinels as the Prophet lacks any good way to get them out of the air (but beware of their accurate and powerful heavy pistols).

4x Scout: 4x Deceiver. Scouts hate deceivers. It’s easy to get in close to them without being hit by a charged shot but watch the ground for traps and be ready to disengage if they all throw their volleys down.

4x Alchemist: 4x Sentinel. Sentinel is such a hard counter to Alchemist that it isn’t even funny. Enjoy soaring through the skies with no danger of being shot at all. Hover above and throw air strikes to your heart’s content - take your time and try to stick them to a player, it’s hilarious.

4x Sentinel: Scouts and Hunters. It’s so easy to knock a careless Sentinel out of the sky with a warbow and on the ground they’re basically dead. Hunters can also practise their bola aiming.

4x Deceiver: a mixture of Hunters, Prophets and Alchemists. If the Deceivers are using illusions fire explosive shot to dispel them immediately. If they’re using shroud, the Prophet’s banish/hex shot will light them up like a Christmas tree for your team to focus fire.

4x Tyrant: at least 2x Alchemists and whatever else you fancy. If the Tyrants are using Leap, 1x or 2x Scouts with warbow will change their minds very quickly. It’s so easy to shoot them out of the sky, land them in a volley and then stun them with knives to keep them in it.

4x Reaver: a mixture of classes. I’d want to include at least 1x Scout with volley to create a protected area for the rest of the team to stand in and fight.

All of which leads us nicely on to the next section: yomi.

Yomi

David Sirlin has written such a good explanation of yomi that I won’t repeat it here. Go and read it now, then come back.

As you hit the 20s you will have a good feeling for your favourite classes and flows of play. You’ll often hit the top score on your team even if that isn’t what you were trying to do. You will find, now, that you CAN play to win, using the skills and techniques you have learned so far.

But sometimes it just won’t work. Sometimes there’s that one deceiver who always gets behind you as a scout. That one scout who always snipes you as a sentinel. That one alchemist whose firewall is always exactly blocking your approach. That one prophet who hits you with such inhuman accuracy you can’t believe they didn’t know you were coming.

Yomi level 0 is playing your favourite class in your favourite way. It’s what, given random opponents, pushes you to the top of the scoreboard when you play to win. Mid- and high-level players will recognize your effectiveness and will switch to a different class, loadout or positioning deliberately to counteract you. This is yomi level 1. To continue being effective you need to watch for this and to do it yourself.

Regularly checking the opponents’ classes is a one way to spot a yomi change coming. As soon as you spot it, you can react. Switch from a sentinel to a deceiver and stab those newly-spawned scouts in the back while they vainly scan the skies. Switch from a scout to an alchemist and tear that team of lumbering tyrants apart. Yomi level 2, bitches!

The best yomi moves come not from pure class changes, which are coarse and immediately visible to all attentive players, but from loadout and tactics changes. How will you play differently as a Scout, knowing that Deceiver is coming for YOU? Stand in front of a teammate? Always place a trap behind you in the bushes? Move across your team every 15 seconds? The choices here are both endless and extremely entertaining. Let your creativity soar!

As you reach higher levels you’ll start to feel yomi in individual fights as well. As a Sentinel you’ve just dropped a Hunter, landed behind him and hit him with Puncture. He 100% wants to dodge-roll away from you and then bola you in the face. So you want to dodge roll through him (vampires roll further than humans) so you’re still behind him when he stands up. But if he’s expecting that, he’ll roll through you instead and now you’re out of auto-attack range and have a bola around your waist while your snack disappears around the corner to rejoin his teammates.

Yomi. Find the most effective technique and exploit it until your opponents reactions to it become predictable. Then exploit that instead.

Teamwork

By this point most of the people you are playing against have a similar understanding of the flow and intricacies of a game of Nosgoth; the ever-shifting balance between the teams based on their respective health, distance, clip ammo, cooldowns and attention.

Now it is possible to play as a team. Basic teamwork such as “cover each other”, “drag a spare corpse to keep it fresh for a teammate” and “attack together” will already be well-ingrained. To go further you have to communicate. Is it better for the humans to stay in this camp or to keep moving to new health packs? Should the vampires rush the humans now while they are on the move or regroup and AoE them at the next health point? Do we need to change our classes and loadouts to counteract this team combination?

Some basic advice about team-level tactics:

1. Humans: Positioning - stand far enough away from each other that a vampire who has killed one of you has a long and painful walk through a hail of bolts and bullets before they get into range of the next one, but not so far away that you can easily be too cut off from each other by scenery or smoke grenades. Watch chriZor’s ESL match videos to see good positioning at work.

2. Humans: Defend your teammates - scouts should throw down a volley as soon as the vampires engage so the rest of their team can use it as a powerful place to stand and pick off the enemy. Hunters should use explosive shot to clear out deceiver illusions and blast reavers off teammates. Blinding shot and sunlight vial can be life savers for an outnumbered teammate and give you all an edge.

3. Vampires: Coordinate your attack - plan in advance the order you will engage in. For example, have your sentinel throw an air strike to force their Scout to dodge and lose his drawn shot and your Reavers to lay down smoke bombs. Once the smoke hits have your Tyrant charge into at least 2 of the enemy and ground slam them to keep them stunned while the Reavers close the distance to melee range and your Sentinel swoops in to pick off the Scout before he can throw his volley down.

4. Vampires: Focus fire - attack 2v1 or 3v1 to quickly down a human and then move onto the next one. The only exception here is the Sentinel, who will take one human out of the fight and then come back with e.g. a second air strike to help finish off the remaining humans.

Learning the depth of team-based tactics and strategies will take you way beyond the level of this guide and into clans and competitive play. You have already learned more than this little guide can teach you. Enjoy the rest of your journey towards mastery of Nosgoth!

CecilSunkure’s “How To Improve At Starcraft II 1v1 Efficiently” made a big impression on me when I first encountered it. It’s much better written and more complete than this and I heartily recommend reading it. The title pays homage to that great work; thank-you, Randy.

Also thanks to RazielWarmonic, whose Sentinel guide inspired me to write this in the first place, and to SilentVirtue for taking the time to teach me about positioning and whose competitive Sentinel play continues to provide inspiration.

Nosgoth is now in Open Beta - if you’re not playing yet then you should! You can start with a free booster using my friend referral link - see you in the game!

Writing AI to play games is a special kind of crack for me. Seeing my AI face off against other people - or better yet, AIs written by other people! It’s so… I don’t know how to describe it. I cannot resist its siren call.

I once lost several months of my life writing python-based AI for an open source game called The Battle for Wesnoth. What started as a fun way to learn python quickly turned into a several month-long obsession of unhealthy proportions. Towards the end of it I had an AI with an ego-inflating 98% win rate versus the default AI across all in-game factions and a selection of the smaller maps.

Then, on the very edge of victory, it all went horribly wrong.

Pride comes before a fall

My downfall begins on a hot summer’s night in Munich. I can’t sleep and I’m thinking about Wesnoth. I long to capture the last 2% and have an AI to share that wins every match, but I’ve exhausted all of the small tweaks and improvements I can think of.

Tonight I decide the time has come for a significant rewrite. It will put the AI on a much more strategically sound basis and practically guarantee total domination! I refil my glass, sit down and get started.

It take several evenings to finish tidying the changes up, but finally the moment for a full run-through comes. I hit the button and send my newly revamped AI into a gauntlet of battles against its soon-to-be-vanquished opponents. But something is wrong. It’s losing a match! Now another! And another! After thirty minutes or so the final figure comes back: 72%. SEVENTY. TWO. PERCENT.

The road from 72% to 98% took me almost a month of occasional evenings the first time around. What do I do now? Revert the changes and try something else? Or keep working on them, possibly throwing my time away for nothing?

Questions whirl around in my sleep-deprived mind, taunting with their unanswerability. Are these changes fundamentally worse than the previous strategy? Or are subtle bugs causing my AI to make stupid mistakes? If I spend two more weeks fine tuning this version, will it ever overtake my previous plateau or am I wasting my time?

In one especially lucid moment I feel a sudden connection to the hopeless fate of the hill-climbing algorithm, never aware whether it is stuck in a local minimum or not, always caught between tenacious hope and the terrible, crushing despair of futility.

Charlie Brown says that the tears of adversity water the soul. I learned that a single number just isn’t enough feedback to meaningfully understand the performance of a complex system. I also developed a deep sympathy for anyone who spends their life A/B testing website changes.

A few weeks later the Wesnoth team discontinue the python AI interface and I am rehabilitated back into normal life, but the experience makes a lasting an impression on me. How can we write better AI? How can we test and understand the behaviour of code that interacts with an overwhelmingly complex, stochastic environment in unpredictable ways?

Back on the wagon

The seasons change and years pass peacefully by. Purely by chance I stumble upon the Wesnoth website again and notice they have quite a decent lua-based AI subsystem in place now.

Unable to resist, I clone and compile the latest version and start learning how to use the new lua interfaces. It’s not long before I have a skeletal AI - in every sense of the word - managing a 30% win rate with lots of low-hanging improvements still unexplored. Yet I am uneasy. I can feel the same problem out there, an undefeated nemesis lying in wait.

I live by the sea now, so one evening I take my notepad and pen down to the beach, write down the problem as clearly as I can and stare helplessly at the shimmering waves, waiting for inspiration.

Despite vicious sea wind’s tenacious attempts to strip the flesh from my bones I manage to sketch out an idea for a workflow that feels like it has potential. As it turns out - to my own surprise and joy - it’s absolutely brilliant. Sometimes the Feynmann algorithm is the only one you need.

Machine learning, or is it machine teaching?

The problem I wrote down is that I’m drowning in an ocean of raw data without an easy way to understand the stories within it. Each automated match consists of hundreds of moves from a wide range of different units and to get any meaningful comparison I really need to run dozens or even hundreds of matches per change. Abstracting all of that as a single percentage throws too much information away but watching fifty replays each time I tweak a number is never going to happen.

At the beach I realized I need a machine to watch them for me.

This is how I built one.

Step one: visualize a single battle

Realizing I needed something between clicking through an entire replay and a single win/loss figure I started looking through battle logs picking out other statistics that seemed strategically relevant.

In Wesnoth there are 5 key metrics that together give you a pretty good feeling for which way the battle is going. I logged just these over time for each battle and added some unicode sparklines for readability:

yt_simple vs ai_default_rca  |  winner: 2 
units: ▄▄▄▄▄▄▄▄▃▃▃▂▂▂ 49 53 49 45 49 49 49 46 42 38 30 21 19 18 
cost : ▃▄▄▄▄▄▄▃▃▃▂▂▂▂ 40 48 45 43 44 44 44 41 38 31 27 21 22 18 
gold : ▄▁▄▄▄▄▄▅▅▃▃▄▃▃ 49  0 57 54 52 46 54 60 69 40 38 53 41 40 
vills: ▁▁▄▄▄▅▅▅▅▄▄▃▃▃  0  0 55 52 52 63 61 59 61 55 49 30 33 33 
inc  : ▃▂▄▄▄▅▅▅▅▄▄▃▃▃ 33 24 54 52 52 61 60 58 60 55 49 33 34 34 

Even so, reading through fifty of these reports at a time isn’t very illuminating, even when they are sorted. Sorting is my favourite cheap visualization trick but here I needed something more.

Step two: the unreasonable effectiveness of normalized compression distances

Over the past year I’ve been fascinated by the usefulness of the normalized compression distance (NCD) as a similarity measure. Basically it states that the degree of similarity between two objects can be approximated by the degree to which you can better compress them by concatenating them into one object rather than compressing them individually.

You can use this to detect the similarity between music, images, DNA and all sorts of arbitrary things wth surprisingly good performance. It’s one of the coolest and most implausibly effective hacks I’ve come across and I thouroughly recommend reading Vitanyi, Cilibrasi and Cohen’s papers on the subject.

NCD was perfect for this problem. I built up a similarity matrix by compressing my visual output summaries then used that to cluster them into an unrooted binary tree. A small python script injects some extra details to dot’s SVG output and suddenly I am looking at this:

The nice thing about having these in SVG files is they are mildly interactive - moving the mouse over any particular node shows the run summary in the bottom-left, as you can see for run 11 in the image above. The NCD clustering does a really great job here – the clusters really do seem intuitively similar and semantically meaningful even after deeper investigation.

Of particular interest are the battles that were lost but are clustered close to several wins. These scream “everything was fine until something unexpected happened” - a bug or unforseen situation the AI has blindly wandered into again. Watching the replay for one of these battles almost always shows a (sometimes rarely-occurring) bug at work. Just fixing these improved my AI massively - machine assisted debugging at its finest!

Step three: profit! My new machine-learning boosted workflow

When I sit down to improve my AI it now looks like this:

1. Code up whichever improvement or brilliant/stupid idea I’ve been dying to try out
2. Hit a key to run 50 battles in parallel using 8 cores, cluster the results and produce a prettified SVG file
3. Get a feel for the overall shape of the results. Are there lots of quick wins, or are matches more often drawn out? Are there interesting clsuters of wins and losses?
4. Tap on a few nodes to understand what is represented then pick one to look at in more detail, e.g. a loss surrounded by wins, or an example from the center of an interesting cluster of losses.
5. Type in the number of that node into my terminal to launch Wesnoth and view the replay of the run from that node. I can click forwards and backwards through the match, watching for mistakes both obvious and subtle in the behaviour of my AI. Sometimes I look at a couple of runs for comparison, such as a similar win/loss or two similar lost runs.
6. Sometimes I can see what the AI did wrong but don’t understand why. In this case I can pull up the AI debug log for that numbered run, jump to the turn I’m looking at in the replay viewer and read in depressing detail the reason my thief decided to go toe-to-toe with a mounted Lancer in waist-deep water instead of e.g. nipping onto that nice safe village and stabbing him in the back for 2x damage.
7. Come up with a new idea to improve the existing code and repeat from step 1.

Working like this is so beautiful because there’s a smooth multi-level mapping from the complex emergent behaviour in the game all the way back to the actual lines of code that evoked it and it’s so frictionless to move between levels of detail that it’s joyful in and of itself! It’s also extremely effective.

Some Promising Results

Within a week my lua AI has reached a 98% win rate and I haven’t even brought out my best tricks yet.

Every time a change decreases the win rate I’m able to quickly find the group of replays demonstrating the negative impact and follow through from there to the debug logs to the code, where I can fix it. It’s a whole new world compared to the way I worked last time, with just a single figure telling me “better” or “worse”.

I would be surprised if writing AI is the only application for a workflow like this.

A/B testing might benefit from a similar setup – take the logged or recorded traces of each visitor’s interactions with the site and cluster them so that when a test decreases signups you can look at the clustering and see which behavioural subgroup is failing to sign up, then go and watch a few sample recordings to understand why they might have changed their behaviour.

I don’t write a website analytics package any more, but if you do and you’d like to experiment with this I’d be interested in working with you. This stuff is a lot of fun!

There must be a lot of other situations that would benefit, too. Do you know of any others? Let me know, I love learning new things!

Update: I’ve added a few more links and technical details to the Hacker News discussion.

I kept wanting to check this on my iPad so went ahead and hacked together a cron job and static Google AppEngine site for it. It’s the daily probability that any given team will win the world cup, visualized over time.

Enjoy!

I’ve been having fun looking at the betfair market odds of a team winning the world cup over time. The interesting thing isn’t the wins and losses, although those are very clear, but how they affect the other teams chances.

The vertical axis is the % chance of winning, as predicted by the betfair market after the end of play each day.

In some cases a team wins but their performance was so unconvincing that their chances of winning the title barely increase. Relative to the points they received, they might actually decrease - witness Brazil’s early matches.

Germany goes from strength to strength, the only team that’s consistently impressed so far.

It’s also surprising how few teams are considered to have any worthwhile chance by the market - and these are getting fewer all the time. Judging by the size of the increase seen when the Netherlands thrashed the reigning world champion, the market’s really really convinced that one of the favourites will win.

Of course, I’m hoping England will have found the perfect way to build Rooney into their forward four and will put in a blazing performance tonight. Come on England!

Whether the market is rational and efficient is also interesting, especially if you like writing trading bots. Some of the movements suggest it might not be, but I really need to annotate the graph with match results to make them obvious.

I hacked this together a few nights ago using Python 2.7 (oh, python 3, I’ll start using you one day), Selenium+ PhantomJS (who has time to sign up for APIs?), Chart.js, Chart.js.legend, Linode, Ubuntu, GNU screen, Chrome (I like the profiling chart) and my beloved Vim.

Truly we stand on the shoulders of giants!

Anyone interested in seeing this online?

P.S. If you’d like to rewrite the graphing using something more interactive (d3.js would be my choice) then drop me a line, it’d be fun to collaborate!

demand-progress:

The NSA “is gathering nearly 5 billion records a day on the whereabouts of cellphones around the world.” (Washington Post)

Join us in protesting the National Security Agency’s wide-ranging invasion of privacy.

Take action →

This is not really just about the NSA; our whole system of government and national competition is set up to make pressure towards mass surveillance irresistible to many parties.

What only national-level budgets can do today, corporations and extremist groups will be able to do tomorrow. Or next year.

That’s why this is the right time to stand up for privacy.

Hi Edg3e,

I hadn’t looked into using Voiceover to activate app switching and so on - that’s… interesting. It takes away one of the few things I preferred about the Surface, whilst adding some of its own new annoyances. Progress, I guess?

I sympathize with the copy and paste - often I wanted to take some text from the console and put it into a bug ticket; I ended up writing a small script that uses our bug tracker’s API to add a comment containing text from the command-line, and then extending it to automatically include the last stack dump and log file output. Surprisingly, this made me more productive than I had been with manually copying and pasting things all the time. That’s something I found a lot while working with the iPad/Linode combination - things that were too uncomfortable to do the ‘normal’ way ended up being shoehorned into a 'console’ way which often resulted in quite a bit of time-saving - and, crucially, attention-saving.

Typing a short command at the console gives me fewer temptations to check my email or notice a support ticket has been updated than switching to the browser does.

Mark

“I never thought I’d write a blog post in Microsoft Word, but here I am doing just that,” begins my first blog post on the Surface. Written lying on a rabbit-shaped bench overlooking the sea, with a gentle breeze cooling the heat of the sun on my skin, all is well. A week later when I come to edit and upload it, I discover that Word has lost it[1].

Act One: All good things come to an end

This story begins back in January. I’d been curious about trying a Surface + Linode combination instead of my 2 year-old iPad + Linode setup. The allure of a full-fat browser that can run Gmail and iPython Notebooks, combined with a keyboard-centric OS has long tempted me. I hadn’t switched, though. Somehow, I had been waiting. At the end of that fateful January amidst the snow and the cold, everything changed.

I was promoted into a management role.

Overnight I find my entire interface to the rest of the company changes. What used to be IM and Mercurial is now emails with attached Word and Powerpoint documents. Within a fortnight the iPad + Linode setup collapses; neither iOS nor Linux nor Google Docs offer credible alternatives for reviewing, editing and returning Office documents and I grit my teeth, install Windows 8 and Office on my laptop and get to work.

Fast forward to April, and I’m wandering through a cherry-blossom filled Washington, exasperated by two months of being tied to a laptop, with all the power, size, screen, weight and fragility restrictions that go along with one. I’ve mostly stopped working in cafes and outdoors, but I miss it. As I walk I weigh up the alternatives. A 13" Air would be a massive step back towards the freedoms I’ve grown used to, but the fanless, robust silence of a tablet continues to attract me, and I don’t want to go back to managing local storage any more than I can help.

Reaching a decision, I drive to a cathedral-like mall and walk out again a few minutes later with a Surface RT and type cover. The time to try a Surface has come. If anything can let me work as flexibly and freely as my iPad in this new corporate world, the Surface can.

Act Two: Flashback

In the following glorious three months of summer and sunshine at my new home by the sea, I get to know the Surface, its glory and its shame. It works, I use it every day. Yet something is missing, something isn’t right. I just can’t fall in love with the Surface.

Back in primary school, my class made Christmas cards for our parents one year. My artistically-gifted friend, who knew the difference between a B and HB pencil, drew out a simple, bold cracker design with a simple twist - you can pull apart the cracker to see the message. No more, no less, but somehow he managed to spend the whole hour working on it.

I thought the pop-up idea was great. My card had a cracker too, and when you opened it there was a fireplace and tree inside and there were extra tabs to pull to make the fire go out and Father Christmas come down and you could open one of the presents under the tree to see my name and I’d fitted most of the words in without squashing them and when I showed the resultant sticky, thumb-printed mess to our teacher he asked me what on earth that was and told me to throw it away and make another.

I didn’t make another, but I did take a long, hard look at my friend’s card. Mine did more; why wasn’t it better? In that moment, I first learned about the relationship between elegance, simplicity and beauty, and my card with all its features and messy edges had none of these things.

Using the Surface gives me the same feeling. I can do more with the Surface, but it is not beautiful, nor do I enjoy using it - or being seen using it. It’s so obviously a business device that putting it on the table in a cafe feels… awkward. Like an imposition, doing office work in a leisure place.

The experience is often disjinting. Rotating the device results in a weird “pop” animation that often leaves desktop windows or even the start screen itself distorted and out of place. It persists in referring to itself as my “PC”. The keyboard doesn’t auto-appear when text fields have the focus in desktop applications. Metro IE sometimes decides to reload all the old open tabs when clicking a link in a mail switches to it, stealing the focus from the new page I just tried to visit. The kickstand is at just the wrong angle for everything and the keyboard, while fine on a table, is spongy and occasionally misses keypresses when used on a lap. Word doesn’t save your work unless you click on a 3.5" disk icon regularly. There are a hundred small things that make the whole experience feel fragile, confused and unsatisfying.

No, I cannot love the Surface. But neither do I discard it. Despite its failings, it does what it claims. It works, and it lets me work, flying through my Gmail with keyboard shortcuts, effortlessly opening, reviewing, changing and reattaching Office documents to emails. Sometimes, it even feels incredibly cool - using an iPython Notebook running on my Linode to graph and explore data from the tablet is absolutely great, even if it’s only possible thanks to the single-click jailbreak.

Act Three: Surprise

You see, there’s something else, something that changes things in ways I never expected: I can read the Surface’s screen in direct sunlight.

While I can just about work on my iPad in the shade of a tree, the Surface screen is clear and bright lying in direct sunlight in the hammock on my balcony, on a pier at the beach, outdoors in a café or on a rug in the garden. This changes the game again. Working in a natural environment can be both beautiful and liberating. It’s just a shame it’s coupled to a device that’s otherwise somehow disappointing.

Visiting my friends in Munich is typical of my mixed feelings for the situation. It’s a 7 hour train journey and a 7 day trip. I take the Surface for working on but also pack the iPad for playing games, so now everything weighs as much as a small laptop would anyway. My friends in Munich consistently tell me how healthy I’m looking. I put it down to my suntan from all those mornings in the hammock with the Surface.

I prefer the experience of working on the iPad. The amazing, unexpected mental freedom of having all my state on an automatically backed-up, always-on Linode coupled to a simple, essentially stateless window onto that world that I could toss around like a paperback book. The ability to open and close it at will, let the battery run flat, leave apps open with the contents unfinished and never give them a second thought.

The Surface doesn’t persist application state. Every week it threatens to restart itself within 2 days to install updates. If I haven’t manually saved work when it does that, the work is lost. Some things are saved on SkyDrive, some on the local disk, which isn’t automatically backed up either. So now I am forced to be aware of this leaky abstraction, to manage the remote and local state and at this point I might as well just have a MacBook Air and be done with it.

It lacks a zen-like calm.

And that’s what this comes down to. If you’re mostly doing non-web development, go for an iPad+Linode. If that combination can’t do the work you’re doing - like the MS Office editing I need to do now - then it’s a choice between the Surface and a laptop, not an iPad. Do I recommend the Surface for working like this? I don’t know. It’s been months and I still don’t know.

I would miss working outside if I had to go back to my laptop full time. Cycling into nature and working in a beautiful spot is fabulous. But these are things the screen lets me do, not the things the Surface lets me do. I expect to see similar screens on other devices soon. A good screen is not fundamental to the form factor or the manufacturer. There are probably options out there already that I haven’t seen because I never thought to look before.

I do know that I almost never use the Surface as a tablet. I use it like a little laptop. Maybe I’ll find the perfect combination of powershell scripts and metro apps to recreate an elegant workflow, but I’m not really motivated to because while I could use shell scripts and python programs on the Linode to work around the iPad’s email limitations I can’t do anything work around the Surface’s deeper usability flaws.

Every time I use it I miss my MacBook a little more.

Update: HN discussion here (including my thoughts on Microsoft Phone).

Update: I forgot to mention my Working in the Cloud mailing list. Articles like this appear there first, sign up if you’re interested in how things go from here!

[1] Actually it autosaved a file, and when I next tried to open an attachment during a conference call I ignored the annoying little restore blah blah tab. I’ve since learned not to do that. But I shouldn’t have to.

Note: this is (most of) the content from the first post to my Working in the Cloud email list - more are prepared and will go out soon, so if you don’t want to miss the rest you can still join the list here.

About a week after I published the iPad + Linode: One Year Later post, my employer decided to buy me a shiny new laptop for CUDA (GPU) development - a Dell XPS 15 with 8GB RAM, a quad-core i7 CPU and a 512GB SSD drive. It was shiny!

And it was fast. Compile times dropped from 53 seconds to just 20. Test suites ran faster. With 8 GB memory I could run larger tests, or more concurrently. Disk space - always a limitation on the Linode 512 - was no longer a problem.

I’ll be honest: after the first day, I began to have doubts about my future in the cloud.

There were one or two cracks in the armor; a laptop keyboard just isn’t as nice as the Apple Wireless one. It gets unpleasantly warm and it’s just a bit too high off the table. I tried an external keyboard, but then I’m sitting too far away from the screen.

I also missed touch. I never expected to say this, but moving the pointer around with a touchpad or even an external mouse just feels slightly awkward now. It’s less immediate; one more thing for my subconscious to work on.

Having a desktop-class browser is really, really nice though. GMail is more responsive and I can use Google docs like a Real Person again. I can even attach things to emails! Hallelujah!

The one thing I don’t really notice while programming is the screen. Everybody asks me how I can work on such a small or low-res screen with the iPad. Really, I don’t notice the difference unless I’m doing a 3-way merge and need the extra horizontal space.

I spent a whole week working on the new laptop. It was a busy time at work and I was multitasking like an over-caffeinated rabbit, flicking back to my emails every few minutes to clear the next dozen that had arrived since I last looked. Meanwhile, my iPad lay in the corner quietly discharging its battery into the void. 

Until Thursday afternoon. It was a splendid autumnal day and as I stepped outside at lunch to pick up some fresh bread from the bakers something in the air called to my soul. You can’t sit inside on the last fine day of autumn, it whispered to me, come with me and be free.

I went back inside, packed up my iPad, some water and a rug into my rucksack and set of, into the great outdoors. Twenty minutes later I was lying in the dappled shade in the vastness of Munich’s English Garden, feeling the breeze playing in my hair as set up my 3G tethering. This is right, I thought. This is the life! And yet at the back of my mind was a guilty question: am I wasting time out here? Would I be more effective sitting back at home chained to The Machine?

I logged in and carried on hacking away at the same bug I’d tried and failed to track down the day before; a subtle problem that spanned several concurrently-running services in our system. I’d grown used to the rapid build/retest cycle of the new laptop and was finding the slower pace of the Linode increasingly frustrating. The air was beginning to turn chill and suddenly this wasn’t where I wanted to be or what I wanted to be doing.

I queued up another build and a set of offline tests to run and checked the time; it’d take a good five to ten minutes before the results were in. Time to move. I told the system to ping my phone when it was done then packed up and got back onto my bike, heading South to find a nice warm cafe to finish the afternoon in.

Back on my bike, gently coasting along the tree-lined paths, my muscles warmed up and my mind relaxed again. My thoughts turned back to the problem; not wanting to waste any more time I wondered how I could most effectively narrow down the scope of the bug. As I was crossing a small bridge I felt the buzz of the notification in my pocket; the builds were ready. Nearby was a small sunlit glade; I settled myself down between a small tree and the stream.

Before logging in on the iPad, I lay back, watched the leaves fluttering against the darkening blue of the sky and finished thinking about the tests I’d run; what the various outcomes might actually mean, and what my next step could be. Once I knew exactly what I wanted to try next, I opened the iPad, logged in and did exactly that - no more, no less. Thirty minutes later I’d solved the problem.

Riding back home, I realized I’d spent the entire afternoon thinking about and working on the problem. Not checking my emails, not writing a quick reply in some barely-related feature discussion. Not reading Hacker News. Just focused on the most important problem of the day.

The same thing happened a week later when I had to write a press release. I’d spent all week putting it off, finding ‘more urgent’ things to do in my emails, attacking small development problems. By Friday afternoon I’d run out of time. I closed the laptop and took my iPad to the window ledge, where I sat with a cup of tea and looked out across the grass, letting my thoughts come to rest.

Then I started typing. Typing on the touch screen is slower than a keyboard, just as compiling on a Linode is slower than on a quad-core i7. Yet again, here the bottleneck wasn’t my typing speed, but my thoughts. The slower pace became meditative, the lack of distractions absolute. An hour later I was done. Later that night our CTO replied saying that he loved it, calling it “quirky, refreshing, and extremely likely to get the coverage we need.”

I’m coming to the conclusion that the speed and generality of a laptop or desktop system is optimizing the wrong thing - it’s increasing the rate at which my brain receives distractions from the true bottleneck in my work: measured thought, repeatable inspiration.

Yet I have one more test left to try, and one that I think might interest you: what if I try to get the best out of both worlds and use the iPad to remotely connect to my work laptop? This is something you could try, too - after a little bit of setup you could take your tablet down to the building cafeteria, or library, or outside, or home, and get a taste for what working on it is like without having to sign up and set up and entire remote server.

I’ve now set this up myself using an iPad and Ubuntu 12.04 laptop and will send detailed instructions on how to do it in the next email. If you want to receive it, click here to join the list.

Thanks, and until then, happy cloud working!

A year ago I said goodbye to my trusty MacBook Pro and started working exclusively on an iPad + Linode 512. It was an experiment at first - one that I never thought would last.

Twelve months later and I find I’m still working like this. A combination of Vim and GNU Screen for development, Pages for writing, Keynote for presentations, Jump and VNC for unavoidable X windows work, Mobile Safari for web apps and a hefty dose of python scripts to smooth off all the edges. I use it for development, for presentations, for my side projects, for everything. 

I really mean everything, by the way. A high-speed baby bottle demolished my MacBook Pro screen months ago and I still haven’t got around to getting it fixed yet.

I love this setup, but it isn’t perfect. I’ll be making some changes for in the coming months. Before I get into that, I want to share what this extraordinary year has been like.

In my original post - just one month after I started - I was still using the setup as a light, silent laptop replacement. Now I use it in ways and places I never even considered using a MacBook. It’s no exaggeration to say that, this year, the cloud has set me free.

Freedom

It’s a crisp summer morning. The clear blue sky promises a hot afternoon, but there’s a hint of freshness on the breeze that ruffles my hair. I smile in the dappled shade of the tree and lean back against the rock. My fingers lazily stroke the screen, scrolling through the feature spec, but my mind is a thousand miles away weighing up design decisions for our new product.

I’m constantly surprised how readable the iPad screen is outdoors. Despite being glossy, it’s much better than the anti-glare MacBook Pro screen. It makes working outdoors not just possible, but enjoyable.

A fine mist of tiny droplets falls over the screen and I casually wipe it clean; working next to the fountain is refreshing but I doubt I’d have risked $2000 of hardware here. The iPad’s proven to be a rugged little thing; it doesn’t seem to care if it gets wet and as my files are remote neither do I.

A buzz from my iPhone tells me my build and tests are finished. I pull it out and glance at the push notification, courtesy of Prowl. New test failures. I finish jotting down my thoughts so far into the Pages document I’m using for the spec then swipe back to iSSH. The tethered 3G connection from the iPhone is great, there’s no noticeable lag as I flick to my Vim screen and hit a shortcut that opens and formats the test results.

Hm, the “quick fix” I came up with on the ride here this morning has caused problems elsewhere. I dive into the code for a while, effortlessly navigating around our million line codebase with an instant code search web app I hacked together using python and flask. Happily, iSSH allows me to forward ports through to the iPad, making it easy to securely use HTML for my scripts and helper tools without exposing any HTTP ports to the outside world.

I finish correcting the fix and push it to our main repository. Jenkins will tell me if it passes on the test cluster in due course. I glance at the time and realize I’ve been sitting here for two hours now. I find I can work almost anywhere for two hours at a time, but to stay in one place for longer I need more comfort than a jumper to sit on and a rock for a backrest! Our mid-morning devteam meeting will be starting in fifteen minutes so it’s a good time to go somewhere quieter for the call.

I throw the iPad and keyboard into my rucksack, wade back across the stream and hop onto my bike. A brisk ride later and I’m back on foot, strolling through trees and flowers with my hands-free headset on in a vast landscaped garden that cuts right through Munich. The call is taking some time to set up. As I’m on 3G, I asked the office team to call my mobile number, but one of our developers is at home and doesn’t have a landline in his study, only Skype. An unexpected hiccup - I feel bad; if I were in an office this wouldn’t have happened.

While waiting for him to start the call with SkypeOut, I discover a rose bush and spend a moment enjoying their rich scent. It feels almost like cheating, working in such a beautiful environment, combining meetings with summer walks through the park. And yet, it doesn’t seem to have affected my productivity at all.

I wish the rest of our team could enjoy walking and talking about the last and upcoming week out here surrounded by nature instead of being stuck in a drab air-conditioned box the best part of a thousand miles away.

Then it strikes me: they could. And yet I worked in that office for six years and I never did. There’s a beautiful castle near the office set in vast landscaped gardens; I could have switched to an iPad + server (one of the office servers would have been fine) and spent a day working there, or any of the other beautiful places I loved to be in.

Of course, back then I used a graphical IDE and the iPad didn’t exist, but surely it would have worked with a laptop, too. Well, unless I ran out of power after an hour or two of multi-core compilation. Or it got too sunny. Or it started to rain. And then I’d have had to worry about finding a nice cafe that also has a free space on a large enough table near a power outlet. So many conditions. So much to fear. 

The 10-hour battery life, 3G connection and small form-factor of the iPad + wireless keyboard combination frees me from so much; today I can work wherever I can sit.

A chime from my phone snaps me out my reverie and we begin the call. The wind has picked up, so I mute myself when not speaking and cup the microphone in my hand when I am. It’s not perfect, but the fresh air filling my lungs and the rustle of the leaves in the trees overhead are worth it.

After so much nature I feel like sitting down with a proper seat and table when the call is done, so I cycle down through the park until I reach the Chinese Tower - a beer garden with fantastic benches in the shade. The perfect place to finish my morning’s coding.

At 1pm I leave the park and cut into the city, spending 45 minutes chatting with Matthias from MunichBeta over lunch in a tiny little Chinese place he knows, followed by a visit to Munich’s best ice cream parlour across the street. Matthias lives much as I do, flitting around the city trying out new places to work with his trusty Macbook Air.

This afternoon I’m giving a webinar to prospective customers; I need a quiet conference room and a rock solid internet connection for Skype. My location of choice: Combinat56.

I find a change of scene and some exercise are perfect to ward off any post-lunchtime lethargy and when I arrive at the stylish Combinat56 offices fifteen minutes later I’m feeling refreshed and ready to go.

Running the webinar from the iPad works just fine - Cisco have an iOS app and in any case we’re only sharing slides and dialling in. Usually I just email people the deck and call them on the phone, though; it’s simpler for all concerned.

After the call I catch up with my friends from the Combinat over a cup of tea in the lounge area. You can’t explore on your own all the time; working from home and in cafes is isolating after a while and I enjoy the community here as much as the decor.

While I work, afternoon becomes evening and I sign off from our team chat servers. On a whim, I kick off another round of data analysis on the million song dataset for a side project, then close the iPad and drop it into my bag. Knowing the Linode will keep working away at it, uninterrupted, while I cycle home and enjoy dinner with my wife and children gives me a warm, fuzzy feeling - my tireless robotic assistant, working away out of sight.

Trouble in Paradise

And yet not everything is perfect. I love having my data on a remote server and I’m deeply happy with my indefatigable Linode. Surprisingly, the weak link in all of this has become the iPad. And not just because of what it by design can’t do, but because the internet is moving towards richer and richer web apps and Mobile Safari still feels like a toy browser. 

Using Google Docs has been a pain all year and it isn’t getting any better. 

Having used the fascinating LightTable on the desktop, writing Clojure without it feels like wading through treacle, but Mobile Safari doesn’t have what it takes. I’ve even looked at hacking the engine out of the source and integrating it with the console, for goodness’ sake.

Luckily, splitting my devices between the server and a client means that I am free to change devices whenever I wish. I can work just as well on the twisted remains of my Macbook, a 3-screen office desktop or anything else with a browser, a terminal and a VNC app. My switching costs approach zero. And switch I will.

You see, a surprising alternative has appeared: the Microsoft Surface.

Surface + Linode 512

For a while now I’ve been telling people that Microsoft will become the new cool, the inventive underdog, and I still believe that. Windows 8 may be a huge gamble for Microsoft, but Windows 8 RT is a clear win for me.

Microsoft understands the keyboard. I can start, switch and control apps without leaving the keyboard. The device even comes with one.

Sometimes it’s really nice to have two windows open, especially when using video output to a larger monitor. I think the Windows 8 side-dock idea will suit me very well.

Love or hate Internet Explorer 10, I have every expectation that we’ll see the real rendering engine on Windows RT. I don’t care what they call it, if I can run LightTable and Google Docs without gouging my eyes out, I’ll be happy.

Last but not least, the Metro vibe feels fresh and new and I’m intrigued by Microsoft’s choice to make Javascript + HTML5 a first-class way to develop for the system. I’m already looking forward to hacking my own tiles together to smooth my workflow and simplify my day.

Perhaps in a year’s time I’ll be switching to another client. It doesn’t matter, and that’s the beauty of this setup - its flexibility. For me, though, this coming year will be the year of the Surface.

The Experiment is Over

Last year I started this as an experiment, but it stopped being that a long time ago. Today the entire city is my office - its parks, its countryside, its cafes and its workspaces. I have worked on river islands, half-way up trees and on exclusive rooftop terraces.

My side projects are simpler, more flexible and more exciting now my development machine is connected to the internet by a big fat pipe, 24/7. Want to run statistics over 5 years of Gmail messages? Just do it! No need to worry about how long it takes or whether someone will reset the wireless router part way through. It’s simple. It’s pure.

Feeling free to set off into the unknown, assured that I will find a place I can work, to explore and enjoy my surroundings… no, this isn’t an experiment any more.

It’s liberating, rejuvenating. It’s a way of life.

I will not give it up.

Postscript: Intrigued and looking for more? Tempted to try it out one afternoon? I’ve decided to start a mailing list - I plan to write one or two emails a month about my experiences; the highs and lows of this desk-free lifestyle. I’ll also share more practical tips such as setting up a VPN or keeping in touch with the team. You can join the list here.

Breaking news: Today Allinea ordered me a high-end laptop with a massive SSD for doing CUDA development on. Stay tuned to see how the iPad+Linode combination stacks up against top-of-the-range hardware!