Sunday, 25 December 2011

Dude, Where's My Car?


I needed a new job. The IT business was booming over in the US and relatives were beginning to ask me when I would be opening my own startup and raking in those easy VC millions. This was December of the year 2000. I didn't want to start up on my own, but I'd been to the US a couple of times and I wanted to go there. The US was the real world.

I was bored with Tigerland. I was bored with database programming; I felt like I'd been there and done that. What I really wanted to do was graphics programming, so I bought an OpenGL textbook and started to teach myself some new tools. 3D was a lot easier with real libraries than it had been at University. I took my 3D projects with me to work and I worked on them at lunch time, much to Meggs' chargrin. It was quite obvious that the code I was playing with had nothing to do with Tigerland's business.

I had also started looking into what it would take to get me over to the US. An H1B skilled worker visa seemed to be the trick. I found an ad in the IT section of the Age newspaper (in a few years' time this section would be discontinued, and IT jobs would vanish almost entirely from Melbourne's biggest broadsheet newspaper). They were looking for programmers who wanted to work in the USA, so I sent in a resume. A couple of days later I received a phonecall inviting me to interview.

I called in sick on the scheduled day, put on my suit, and went in to the temporary office from which the Americans were conducting interviews.

The receptionist gave me a questionnaire to fill out. The only question I remember from it was "What is the minimum salary you are prepared to work for?" I knew roughly what that should be, so I answered it truthfully. Stupidly.

The Americans turned out to be a pair of Indians. The interview turned out to be piss easy. A C++ test that was much easier than the Tigerland test and some verbal long-answer questions about broader topics, like multiple inheritance. I breezed through it.

The company was a big contracting firm based in Silicon Valley. The Americans sold it to me pretty hard--the lifestyle, the location. "Oh, you like SCUBA diving? There's amazing diving in California." They told me all about the benefits: medical, dental, a relocation bonus... I was pleased, but in the back of my mind, there was doubt. There was a used car salesman vibe to the whole thing. I just didn't trust them.

The Americans' offer letter went into the mail the next day. A complete package: contracts, visa paperwork, the whole nine yards. The offer was for that minimum salary I had indicated. There was no relocation bonus in the contract and a few other promises were also missing. The Americans called me and asked me if I would accept it. Salary was not negotiable. They claimed that the missing contract items were in fact there. I told them I was thinking about it.

I rechecked the contract. The missing clauses were definitely not there. I had, meanwhile, been reading about the cost of living in Silicon Valley and I was becoming worried about it. One article spoke of a how the mayor of one of the municipalities there had had to resign because he couldn't afford to live there any longer. A friend of mine who lived in the States who and had some experience with budgeting crunched some numbers for me, and it looked as if I could scrape by on my non-negotiable salary... barely.

I wanted to go, but I didn't trust the company and I really couldn't afford Silicon Valley. I told them no. They tried hard to change my mind, but not so hard that they offered me more money. I walked away from the table... (my willingness to do this has been at various times an asset and a liability, across the intervening span of my career)... but now I had a plan.

It had been  ridiculously easy to get that job offer, I reasoned, and I hadn't even left home. How many more opportunities would I find if I actually went over there in person?

I was going to do it, early in the new year. I figured how much money I thought I would need to go over there prospecting and calculated a departure date based on that. I remember doing that quite vividly. I was by myself, walking down the street at lunch time. The sky was blue and I would soon be free.

Amos was back in town and the talk was that he was thinking about coming back, if there was room for him. I went to the technical director and I said told him that I was planning to leave, in case it had any bearing on whether he would hire Amos back or not. He did not seem surprised.

I went back to work. The days passed. I worked hard to close off everything that was assigned to me. Meggs didn't talk to me any more than he had to, and that was fine.

A few weeks later the MD invited me to the pub at lunch time and told me that Meggs had instructed him to dissuade me from leaving. He offered me money. I told him that my decision was made. I was going to the US. I also mentioned that the money he had offered wouldn't have been enough, either. He was shocked when I told him what the going rate for a developer with a couple of years experience was, but I knew what I was talking about and I had statistics to back me up.

We had drinks down the pub on my last day. I got monstrously drunk and Guthrie told me I could crash at his house, which was a lot nearer to the pub than my own. I threw up in the taxi. The following morning Guthrie offered to drive me home... whereupon we discovered that his car had been stolen. The film "Dude, Where's My Car?" was then playing in the cinemas and I thought he was playing a joke on me.

Still, Guthrie told me that despite all of the trouble, it was a very good week for him indeed: the directors had paid attention when I told them what kind of money we were worth and he found himself on the receiving end of a very large payrise once I was out the door.

As for me, I was bound for the United States. By then, of course, there was a new phrase in the air: dotcom bubble. One of our own ministers boasted that we hadn't been wiped out because we'd failed to catch the wave in the first place. But I was already on my way.

I would find out for myself just how much blood was in the water.

Monday, 7 November 2011

Riding the Tiger

Amos left at the start of my second year at Tigerland and Meggs took over his role, as well as continuing with his own duties. The first thing that he needed to do was to replace Amos.

Meggs decided that he needed someone pretty serious to replace his compadre; somebody both skilled and experienced. And so he set about interviewing candidates for the role of a senior developer.

I was not present for any of the interviews, but I heard about what had happened from the technical director and from Meggs himself. Meggs reduced at least one candidate to tears during the interview process. For other candidates, if they were not doing well enough in the testing, he would get up and leave the room without saying a word.

Eventually, Meggs settled on a developer named Ivan. He didn't particularly like Ivan, but he was the only candidate who had actually passed Meggs' testing and soon enough Ivan joined us in the fishbowl. Ivan was very pleased to be in the role, and he expressed to me his delight about how much he was learning from Meggs, about how great a pleasure it was to be exposed to an intellect like that. No matter how badly Meggs treated him, Ivan seemed to be okay with it. I have never seen anything like it before or since.

Every day, for a solid half an hour just prior to lunchtime, Meggs would stand over Ivan and bellow at him;  insulting his work, his integrity, his experience and his intelligence. I remember Ivan's work being clean and tight and clear. He met all of Meggs' style guidelines and everything I ever saw of it looked good. Ivan was the only candidate who passed Meggs' screening; I'm pretty sure the problem was not actually in his court. Meggs liked to be feared, and without his friend Amos to snicker about it with he was probably lonely as well.

I started having my own problems with Meggs. His desk was abotu a meter and a half from my own and he started watching me over my shoulder. Meggs decided that I was compiling the source code a bit too often, which was costing me a valuable half minute or two each time I pressed F9. "Jared. You only need to recompile the module you're working in. Be more productive." Meggs' own productivity was tailing off, because, for one thing, he was spending so much time watching me when he wasn't yelling at Ivan.

Meggs assigned me a fairly complicated piece of logic to do involving persistent locking. My first attempt was a failure; I hadn't understood the idea of it properly and it didn't work when there was more than one client connected (which defeated the purpose of the exercise). Barely controlling his impatience, Meggs explained it in more detail, and even drew  me a picture on a scrap of paper and told me to build it. This was the first time I'd ever been shown a design drawing for anything at Tigerland... and, let me be honest, through the rest of my career, I've rarely seen a design drawing for anything that I didn't author myself. I any case, it made an impression on me.

This time I understood what was going on properly. I looked at the design carefully and I noticed what looked like a bad side effect. "No, that's the behaviour I want," said Meggs.
"Well, okay, then," I said.

I went and set up the locking the way he had asked. When the task was done he code reviewed it and I checked it in... and then I threw away the drawing.

About two weeks later, I found myself on the receiving end of a tongue-lashing on the same scale as the ones Ivan was subjected to. "Pikeman! This locking system you built is rubbish! Have you seen this side effect?"
"Meggs, I told you that was going to happen."
"I would never have agreed to that!"
"But... it was your design! I built it exactly the way you asked me to! You even drew me a picture!"
"Where's this picture, then?"
I've never thrown away a design drawing since.

By now Meggs had stopped contributing any code to the product. Ivan was building a special new feature, but it was Guthrie and I doing 80% of the work fixing bugs and migrating across to the new design. But it wasn't just the architecture changing. Meggs was making ad hoc changes to the feature set and functionality as  we went.

Meanwhile, I was becoming more and more interested in the design of the new infrastructure. I suggested to Meggs that it would be really nice if you could tell the framework what the cardinal relationship between different kinds of entities was and have it generate queries based on that. Meggs agreed that it was a good idea and he added some code that did indeed identify the relationships... but it was cosmetic only. None of the smarts I wanted to build were present. In hindsight, I think this actually half-assed attempt at an apology. I didn't criticize it but he could see I was disappointed.

The technical director could see that things were going badly. We'd barely enough money to cover costs during the year and the software was badly behind schedule. This latter problem was entirely due to the fact that Meggs kept moving the goalposts; making the design more and more elaborate and forcing us to constantly rework what we had done. He hadn't stopped coding himself, but he was coding a hobbyhorse project that had nothing to do with the work we were putting to market. Aware that something was amiss, the technical director instituted weekly development meetings.

These only made it worse. Meggs began to use the forums to taking credit for other people's ideas. Most usually, those ideas were mine. Sometimes Meggs would get the ideas wrong and miss the point... and if I would pipe up to correct him he would repeat what I said back to me, in front of the whole room, as if I was too dense to understand them.

Suddenly I was no longer afraid of Meggs anymore. I had learned a lot from him, but I knew that he was no longer this infallible font of technical wisdom. He had taught me that the design was the most important part of an engineer's job and I was going to stand by that principal, regardless of his sensitive ego.

Meggs' new design ideas were all to do with preventing programmers from making mistakes where his framework became unintuitive or failed to handle errors well, and I was assigned the task of making the little classes that would enforce this... classes that soon be scattered through the codebase like a bucket full or rabbit pellets. I didn't complain, but I became sly about it, and I started making these turd classes more and more elaborate; decorating them with superfluous template parameters and inheritance hierarchies. These exquisitely-polished turds passed through Meggs' formerly-rigorous code reviews without comment. I turned one of these projects (a particularly bad mixed-metaphor turd class) into a three-week-long project which to this day is one of the most conceptually-complicated and difficult pieces of technology I've ever worked on.

Meggs was even less attentive than ever, and Ivan was copping worse and worse abuse every day. In a bid to get me out of his hair, Meggs assigned me to port one of the biggest modules in the product across to his new framework, with a number or revisions to the functionality that he laid out in an email.

It was a disaster. If we'd ported the module as it was that would all have been fine, but the functionality changes were going to take a module that was compact and well featured and useful and turn it into something no customer would ever have a use for. Usually I wouldn't have known or cared about the customer's end of the experience, but we used that module internally to manage our development database, so, unlike some other areas of the application, I knew it inside out. The development team used that module on a daily basis in the same way that our customers did.

I told Meggs it was a bad idea. We argued loudly about it for a full day, but I refused to give ground. We left it over the weekend and took it to the development meeting the following monday morning.

Words got heated, but there was no yelling. Eventually the technical director threw up his hands and said "Pike, just do it the way Meggs wants it."

I went back to my desk and went to it like a man possessed. I was determined to get the work done as quickly as possible so that Meggs could see what a disaster it was before he forgot the arguments and blamed me for it. I didn't speak a word, I don't think I took a lunch break. I don't think I have ever produced as much code as I did in the following forty eight hours.

Once I had written most of the code and fallen once more into the sleepy cycles that followed which involved running the regression test suite and writing new tests... but mostly staring at the screen while the tests executed... once the anger started to fade... I realized that it was a futile exercise.

I have a vivid memory of going to a camera store with my mother when I was a boy. There had been some problem in developing a roll of holiday snaps and my mother was furious. The man in the camera store was an old Indian man. He stayed calm and my mother settled down. Once they had reached a peaceable agreement, he told her that he was glad. "You came in here riding the tiger," he said. "But don't you feel better now that you have climbed off?"

That was what I had to do. It was time to climb down from the tiger.


Thursday, 13 October 2011

Sugar Fixed

... and we're back. Sorry for the disruption, folks.

I've begun working on the next entry now and it should be up over the weekend or early next week.

-- Pike

Sunday, 25 September 2011

Sugar Fix

Hey, guys and girls,

I realize that it's been a couple of weeks since the last sugar fix. I have been moving and it's curtailed the amount of time I've had to write on here. It may be another week or two but, rest assured, the blogging will continue!

Pike

Wednesday, 7 September 2011

Tiger Stripes

Guthrie and I settled in at Tigerland fairly quickly. We were given a number of small problems to solve, and these would be rigorously code-reviewed by the senior developers before we could check them in. Sometimes they'd send me back two or three times before I solved the problem to their satisfaction.

I subjected Meggs, who sat nearest to me, with a constant barrage of questions. How should I implement this? What did it mean? What was the best approach? What impact would it have? He answered all of them with patience, and I sucked the information up like a sponge.

Guthrie and I had a few problems when I came to procedure. Neither of us had worked in a commercial coding environment before, and with a home-grown version control system things had to be done a particular way. I actually broke the version control system a couple of times, until Meggs made me fix the holes I found by using it incorrectly. That's right: I had to actually code part of the version control system, which was a horrible mess of batch files and command-line C applications that called into each other recursively.

Meggs and Amos had more stringent procedures of any place I've worked since. Eventually they set down a formalized work procedure, and Guthrie and I were told that failure to comply was a 'sackable offense'. It went something like this:


  • Code until the problem is solved.
  • Execute the automated regression test scripts to make sure that nothing has broken.
  • Write new scripts to test what you have just written or fixed.
  • Merge down any changes that other developers have checked in while you completed your task.
  • Code review.
  • Adjust solution as per review.
  • Merge.
  • Run scripts.
  • Code Review.
  • Checkin.

It was a laborious way to work, but it did have certain benefits. We didn't have 100% coverage in the test scripts, but it meant that we could cut a release straight from version control and be assured that the greater portion of the software still ran properly. And we did have Jing writing scripts full time, so the coverage was always improving.

I learned a lot of things in that first year, particularly from Meggs. Meggs very smart and very volatile guy, but certainly it the best programmer I had ever met at that stage. He had a lot of ideas and techniques that I didn't see put into practice elsewhere for a good five years. He was very quick to leverage advanced features of the language and very keen for me and Guthrie to learn about them. Despite being the rawest of junior coders, I was exposed to the Standard Template Library well in advance of the majority of programmers in the field.

Meggs also taught me the primacy of design; of keeping the big picture in mind no matter how far down the rabbit hole of implementation went. No design is perfect, but it's better to have a design that's wrong than to have no design at all. You can fix a design that is obsolete, inappropriate or invalid, but if there's no design the chances are that you will have to throw out the codebase and start over. That was the best lesson about software development that I ever learned.

I'll go further: I learned more about programming from Meggs in that first year than I did from anyone else, at school or in the workplace.

Senior management decided that they wanted the software to start running on top of a third database engine. Meggs took some meetings with a consultant from the DB vendor and, when the vendor delivered us software, I asked if I could have a copy to play with on my work machine. Meggs ignored my request and assigned me a different task: to upgrade the database connectivity to be in compliance with the ODBC 3.0 standard. It wasn't an easy task, but it gave me a good look into the guts of the application and I took to it with relish. I didn't realize that it was a test.

Once I had successfully completed that task, Meggs gave me the CDs for the new database engine. "You asked for it, you got it," he said. "Your new task is to the database port."

I spent two solid months on it. The new engine was quite unlike the two we already supported and getting it to work sent me chasing all through the codebase. I learned a hell of a lot about how the software really worked, under the covers. Eventually, I cracked it. There was one case where the test scripts didn't report the same information due to an in-built behaviour in the database engine, but the behaviour was still correct (just a little different) and Meggs agreed that everything was honky dory. I modified the test scripts to allow it and checked it in, happy as Larry.

I'm pretty sure that not even one client ever bought the product for use on the new engine. Certainly, the support documentation was never updated to reflect the third engine and none of the project managers ever came to ask me how to set one up. There was a fourth engine that customers did want to use, and which had long been promised to them, but that was simply not on the schedule. The most important thing, as far as management was concerned, was that we could put the logo of the new database vendor on the packaging. We could boast some kind of a formal partner relationship to them.


But I didn't give a hoot. I knew I'd earned my stripes. I had about one year of solid experience and for the first time I felt as if I was a competent programmer. I couldn't have asked for much more out of my first year on the job, and I was looking forward to seeing what new challenges would arise.

I had no idea those challenges would be social, rather than technical.

Sunday, 28 August 2011

Enter the Dragon

When I rolled up at Tigerland on my first day, wearing my brand new suit, there was another lanky young guy hanging around outside the front doors looking uncomfortable in his office clothes. Being razorminded deductive prodigies fresh from our nations shining institutions of tertiary education, we quickly worked out that they'd hired two graduate programmers, and we were both starting on the same day. My new colleague told me his name was Guthrie.

We went up to the office. Amos came to meet us at reception. He was wearing jeans stained with battery acid and a t-shirt. "You boys won't be needing those suits around here," he said. I had my tie off before he'd even shown us to our desks.

Guthrie and I would be sharing an office with Amos and Meggs. There was no door, and the wall facing the corridor was all glass. It was like being in a very small and poorly-lit fishbowl.

In the office next door was another programmer, Jing, who Meggs told us was also a programmer. "But she only writes test scripts,"  said Amos, dismissively. There was one other programmer, Robert, who occupied the next office along, but he wrote small applications for clients and was not part of the team. The four of us in the Fishbowl were the team; everybody else was inconsequential. Marketing, we were told, was the Enemy.

The first task that Guthrie and I were given was to unpack and set up our computers, which were delivered not long after we arrived. Guthrie was quite excited about it. I was less enthusiastic. Before long we had the  computers put together, and Amos instructed us to install the latest version of Windows NT Workstation.

I'd never used NT before and had no real idea of how it was different to Windows 98. I didn't much care either, until we started having problems. Neither Guthrie nor I could get the damn OS to install properly. After a day of stuffing around, Meggs gave us a different set of CDs to use and told us to install NT Server. Server went on with no problems.

I immediately had another problem. My network card didn't work. I had installed it myself and I was worried that it was my fault, but Amos just gave me a new one. The new one worked fine. Two days had passed and I'd barely managed to get my computer up and running.

My heart wasn't in it. The Dragon had promised they would call me that week with an offer and I was fretting about how I would be able to break it to them that I was already going to quit.

I came down with a flu. I hadn't been sick in the five preceding years when I had been at University, but less than a week in the office and I felt as if I'd been run over. Guthrie and I were given access to the source code and instructed as to how the version control system worked. Neither of us had used version control software before, but, we were told, even if it had, the experience would have meant little. The version control software in use at Tigerland had been written by Meggs and bore only superficial resemblance to the solutions in use throughout the rest of industry.

In the mornings, Meggs would give us long show-and-tell lectures about the design of the application, the transformative path he was taking it down, so it could go from being a simple client/server system to a multi-tiered enterprise system, "just like a real one." I was sick that I wasn't taking much of it in; it was all I could do to stay awake. Guthrie claims that one lunchtime he caught me crashed out from exhaustion in an empty office, but I have no recollection of the event.

Guthrie and I were not allowed to program just yet. We were set exercises in using command line utilities to search and replace in the code base using grep and a variety of homegrown tools (again, most of them authored by Meggs). The sort of tasks that are probably familiar to any UNIX programmer of the day, hacked and kludged into a windows environment. Amos decided that these exercises would be competitive, although I think we would have both learned faster if we'd been allowed to work through the problems together.

The cumulative solution to all of these command-line problems was intended to reformat the whitespace in the source code, which Meggs had recently decided had to be absolutely regular. Guthrie and I each reformatted half of the codebase (alphabetically, by file name). After making sure  that everything still compiled, this was the first time that Guthrie and I were permitted to check any code into the repository.

"Congratulations," said Meggs. "You're now officially programmers."

Neither of us had actually written a line of code, but we had certainly had the importance of procedure and code- cleanliness drummed into us.

Meanwhile, the Dragon still hadn't contacted me. I didn't have a mobile or private access to a phone in the office, so I wound up sneaking out of the office a couple of times during our morning coffee breaks to use the phone box on the corner. It was a week before I managed to raise my contact there. She couldn't tell me why, but there was no offer in the pike. I needn't have worried about how Tigerland would take such a quick resignation, because it looked as if I would be staying there indefinitely.

It was less than a year until Y2K and its accompanying crash. In the fallout from that, the Dragon would shut down its Melbourne offices entirely. In later years, I learned that nobody knew exactly why the Dragon had built the Usability Lab that had failed to employ me in the first place. Apparently it never saw a single day's use.

I was fated for a career in the coding trenches, it seemed. I was disappointed that I wouldn't have the Dragon on my resume, but at the same time, I think I was a bit relieved. I was a programmer with a lot to learn, and it seamed as if that Meggs was going to be a good person to learn from.

Now, finally, I thought I knew what would happen next.

Sunday, 21 August 2011

Lions and Tigers and Bears

Out of college with my degree, I had everything going for me except for a job.

I didn't know what kind of job I wanted, but I did know the kind I didn't.

Before I had graduated I'd gone to a career information session at one of the big banks. I listened to about an hour of corporate doubletalk before they trotted out a pretty young white girl who'd been in the previous year's graduate program. She told us about her exciting job maintaining legacy software written in an obsolete languages running on on ancient mainframes. She was bright and perky and enthusiastic and I walked out of there ready to slit my wrists.

I had zero experience, so there wasn't much to look at on my resume besides my education and the time I'd spent stacking boxes in a warehouse. With three majors, two of which were not computer science or engineering related, the most common reaction I got was one of confusion. "Just what do you want to do with all this stuff?" "Uh, the job I applied for is in software development," was never a good enough answer. One place looked at my university career and said "Well, you're obviously not much of a programmer, so we're not even going to let you do the test."

The most rigorous interview I did was for a position as a tech writer. Half a day on site, four interviews, and a writing sample. It was forty degrees, the air conditioner was broken, and I was stuffed into a suit while the rest of the staff wore shorts and t-shirts.

I went to an interview in a building that was still under construction, where I had to push through plastic sheeting and climb up scaffolding, when upon arrival they looked at my resume and said "Oh, so, as a graduate, you don't actually have any experience?"

I was interviewed for a graduate role by a man who put both fists on the table and demanded "What makes you think that you're ready for this job?" He followed this question up with "Do you have a girlfriend?"

I did a couple of interviews with a company that was on its way to becoming one of the country's biggest contracting firms, Lion Consulting. The second of these included an hour of formal testing. The first two sections I was fine with: computing and mathematics. The third section was management-related, and I had no idea what to write. The four section was supposed to assess language skills: in addition to answering the questions,  I corrected spelling and grammar errors I had noticed in the questions themselves.

I applied for a job where they wanted someone with good tech and language skills and required a portfolio showing graphics work. At the interview they asked me chemistry questions. They didn't know exactly what they wanted from the person they were hiring. I lost out to a candidate with a degree in graphic design degree and a portfolio. 

Lion called me and told me that I was on the shortlist, and they'd be doing a third round of interviews in December. I was going to be overseas in December, so I asked if they could schedule the interview for another time. When they told me they would pencil me in for their next graduate intake, the following December, I decided that I didn't want to work at Lion Consulting after all.

I did an interview with a medium-sized tech company, Bear Technologies. I didn't really understand what it was that they did at Bear, but it sounded high tech and interesting. There were three guys in the room: two suits who talked and asked questions, and a bearded guy who sat off to one side listening. The beardy guy would occasionally interrupt, ask me a question, and then nod. I didn't impressed to suits much, but I felt like I had some kind of unspoken rapport with the grumpy old beardy guy.

I wascontacted by a Dragon-sized overseas corporation who had somehow discovered my combination of different majors and were keen to hire me to work in the new Usability lab that they had just built, and which they had no idea what to do with. The interview went well, and they said "We'll call you. Soon."

The Dragon didn't, but the Bear did. They were considering me and one other guy and they will be in touch about a second interview... one day.

I did another interview with a small software company called Tigerland. Three guys interviewed me: a manager and two engineers who introduced themselves as Amos and Meggs. Meggs wore a beard and reminded me a lot of the guy at Bear Technologies. The interview went well, until Meggs produced the test.

The first part of the test was easy. They showed some simple procedural C code to me and I had to work through it, explaining what it was doing. The second showed some object-oriented C++. I worked it through as best I could, but I did not know the answers to Meggs' questions about  inheritance, composition, and the stack. I figured I was out of the running, but perhaps I could learn something from the interview, so I relaxed and settled in. It was nice to be talking about programming instead of how confusing my resume was. "I don't know," I said, when faced with a question I couldn't answer. "How does it work?"

I think that attitude is what got me the offer. Amos called me within a few days and said, "Are you interested in this job? It's a hardcore C++ job; it's not for pussies. If you are interested, it's yours." I told them I would think about it. Amos told me to call him back the next day.

I called up Bear Technologies, who still had me shortlisted. They told me that they couldn't make a decision for another month. There might not be a job going there, after all. I told them I had another offer on the table, and they advised me to take it. So I called Tigerland back and accepted their offer. I would start work in two weeks' time.

The following week, the Dragon called me. Things were moving ahead. They liked me, they wanted me to work there. They would be in touch with me again really soon... but they had no paperwork and no actual offer yet. I'd already signed with Tigerland.

The following Monday was my first day at work. My first day full-time employed, in any capacity.

I was now a real person.

Sunday, 14 August 2011

Robot Monster

I had finished my undergraduate degree and I didn't know what the hell to do. I knew I wasn't ready to join the workforce. So I started looking at post graduate programs.

One of the universities was offering a postgraduate diploma in Robotics. Anybody with a comp sci or an engineering degree could apply for it. So I collected up my transcripts and sent them in. I received an acceptance letter in fairly short order... followed by a second letter informing me that the course had been cancelled, because there was only one person enrolled in it.

My dreams of robotic world domination dashed, I went an enrolled in a one year arts program at a different University. It was fantastic. Lots of great subjects, and not a career in sight. I asked if I could overload my course and do some extra units, but the course coordinator adviser refused: these were Arts subjects an it was  unthinkable that a student should want extra work. I went from having 28 hours of classes a week to eight, with no big assignments and no exams. It was easy and rewarding. At the end of the year, my supervisor asked me where I was going to have the work I'd done for his class published.  It didn't have robots in it, but it did have space ships and quantum physics.

Once I had graduated, I needed to get a job. This wasn't so easy: most of the companies I talked to were confused by the fact I had qualifications in Computer Science, Psychology and English, and I wasn't getting very far. My old university asked me to show up and enrol in subjects for my deferred honours year, and even though I'd had never intended to go back there I went and did it. I had nothing else to do.

We had to choose a thesis topic before the year began, but I was so disinterested in it that I did nothing about it. At the last minute I went through the handbook advertising which topics were being offered by the various faculty members, and I found one professor who had five topics I liked. I went to talk to him.

It turned out that Professor Gupta was the head of the department. I had no idea, even though I'd been in the department for three years already. He looked at my background, raised an eyebrow at the second major in psychology, and agreed to let me do one of the thesis topics I was interested in. He chose the topic. And he told me to write a much more general topic heading on the form than the one we discussed. I should have known that something was awry.

At the start of the year we were told that honours level subjects would be much more difficult than undergraduate subjects, but I found the opposite to be true. The classes were easy, the assignments were easy, the exams were easy. I wondered what I was supposed to be learning. How was this preparing me for the real world?

My thesis was another matter altogether.

The first thing that Professor Gupta did was tell me that I would be co-supervised by another lecturer, Dr Victoria  Wong. This, I was later informed, meant that Gupta wanted me out of his hair unless I produced something worth publishing, so that he could take the credit. Vicky was kind and supportive, so I didn't care. I did care that Gupta had decided to change my topic. He had some students building a piece of software for him that would measure certain aspects of human behaviour and he needed somebody with a psychology background to develop an experiment around that. I was unhappy, but it was easier than the thesis I had planned. I was given an alpha of the software to play with and told to begin my literature review.

The lit. review was easy. I read all of the research and found it patchy, unscientific and generally a load of hot air. Being the angry young man that I was, I proceeded to tear it apart. This actually impressed both Gupta and Vicky: they were unused to seeing students who were a/ critical, and b/ able to frame a coherent argument. They rewarded me with good marks and told me I was PhD material.

The alpha software I was supposed to be testing, however, was a problem. It did work... sometimes... but it failed to harvest the data correctly, and it would freeze or crash frequently. There was no way I could use it in the state it was. I was not allowed access to the codebase or the team of students who were developing it for Professor Gupta, but he assured me that it would be ready.

I went away and designed the experiments I would perform with the software. I framed my argument, developed my hypothesis, worked out the metrics I would use. I got the whole thing ready to go.  Eventually Gupta took an interest, and invited me to a few meetings in his office. During those meetings he'd make me sit while he read what I'd written. Then e'd explain my ideas back to me as if they were his own... and I was a turnip. I just nodded my head and agreed with whatever he said. After every meeting I'd ask "How is the software coming, Professor?"
"It will be ready. Don't worry about it."
After a few weeks Vicky called me into her office.
"The software is broken. The students who are writing it have gone on holiday. Professor Gupta says you won't be able to use it."
"But... my whole thesis is about this software." Gupta had forced the topic on me to begin with. It wasn't Vicky's fault. "What does Professor Gupta say?"
"Professor Gupta says that you should just do some manual testing instead."
"My thesis topic is 'automated testing'."
"Not any more."

Vicky was sympathetic, but she couldn't do anything... or say anything. The faculty, I had discovered, were paranoid about Professor Gupta finding out that they'd said anything bad about him. The ears had walls.

"I'm going to go and speak to the Professor."

"Um... Professor Gupta is actually at a conference overseas for the next month."


My thesis was gutted. I performed some manual tests and collected some results, but I had nothing to write about anymore and I had to pad to reach the wordcount. I thought I would bring some of the scientific rigour to my thesis that the studies I had criticized had not: I would perform some statistical analysis on my data to see if it was any good or not. When Professor Gupta returned from his junket and I told him about this, his face darkened. "No statistics," he said.
"But..."
"This is computer science. We don't do statistics."

I did the analysis anyway. That was as close as I came to expressing my frustration at the way the thesis had turned out.  My marks fell from an A to a C, but I didn't care. I was glad to be out of there. I was glad to be leaving this hell of Academia, where everybody was paranoid that their seniors were out to get them; where incompetence was rewarded, innovation was stifled, promises were broken, and ideas were stolen. Out in the Real World,  I reasoned, when there was money on the line, none of that would be permitted to happen.

Yeah, I know. You can stop laughing now.

Friday, 5 August 2011

The End of the World as We Know It

I wasn't concerned about finding a new job after I was done with Fjord Systems. Chucky had been hassling me for a resume; they needed someone like me at his new gig. Two of Fjord Systems' rivals had also approached me. I was pretty sure I could find work without any trouble, and I had my entire three month notice period to do it in.

About two weeks later, a Global Financial Crisis was declared and suddenly all of those opportunities went away. But it was October, and I was tired, and I still wanted to see the product out the door.

First thing I did was tell the team that I was no longer available after hours, and I was not working any more overtime. The immediate ramifications of this were that I was not going to conference into development meetings and I would not be returning to Sweden for the company 'conference'. The conference was a marketing gimmick; a holiday party that would last a week and would see customers from all over the globe descend on the offices in Sweden. I also knew, from the previous year, that almost nobody would be doing any work whatsoever for the entire month of December and half of January. I that wanted to leave quietly.

My work was pretty much done. Chucky had finished the drivers before he left, and I had finished integrating them when I gave my notice. Mostly my job consisted of writing unit tests and fixing bugs that arose because Anders starting messing around with the drivers. His changes were minor and really served only to break the interface or the unit tests. I suspect that this was a way of taking credit for Chucky's work by putting somebody else's name all over the version control system, but I stopped caring. Chucky had agreed to do any urgent maintenance that was needed in his drivers for six months  after his resignation (he did not give the required three months' notice), but there weren't any issues beyond the new ones Anders introduced, and then demanded that I fix in the interface. I complained the first time, but after that I gave up. They were minor changes. Many of the changes simply showed that the team had not understood the design as I had laid it out for them, but the software still worked well enough... it just meant that many of the features I had written sat dormant.

About a month later Martin sent me an email saying that he wanted to talk to me. He said he'd been trying to call me on my office line, but I never answered. I had been turning the phone off after 6pm, and there was no voicemail set up. I had said in my letter of resignation that I would no longer be available after hours. It took Martin six weeks before he thought to email me.

We arranged a time and I got on the phone with him, but I found that I wasn't really listening to whatever it was he was filling my ear with. I sat there remembering him making fun of my friends behind their backs, and I wondered what he'd said about me. I told him that I was tired of the work. I told him that I'd already done this same project when I had worked at ATB Software, and that the team's refusal to honour their promises meant that this product would not be as good as the one I had worked on two years prior. Even if it had been, it wasn't the work I had signed up to do. That work had departed when Jacob had been let go.

The product was released in January. It was substantially better than the prior version, since the new design, even compromised as it was, was at least cleanly-written and fairly efficient. The drivers worked and integrated into the software seamlessly. Despite my reservations, the UI team produced a very nice
new front end for the application from scratch and the product looked quite polished. Certainly, it presented a much shinier facade than the product I had built at ATB Software...

But it just wasn't as good. My product for ATB still rated in the top three of its kind when it came to performance and efficacy, and the new product from Fjord Systems stayed well below it.

But that wasn't my problem. I had quit my job at the middle of the biggest economic downturn in decades, and I was feeling fine.

Wednesday, 27 July 2011

Ars Moriendi

A couple of days before the end of my vacation I received an email from Andy, the Project Manager at Fjord Systems.

"Pike, I hate to ruin your holiday, but everything's gone wrong. We're in deep shit."

Andy wouldn't tell me anymore. I arrived back in Sweden a couple of days later expecting to find the common area festooned with intestines and severed heads in the fridge. It sounded to me as if the project had died hard, but what I really found was... nothing.

The team had told me that they were good to go when I had left for my vacation, three weeks prior. They had all agreed to do the work. They knew who was supposed to be doing it, and how. That was what they told me.

Apparently, once I had left the office, instead of buckling in to do the work they had panicked. As best I could tell, the team had spent three weeks running in circles around the office, screaming and throwing their own faeces at each other. As far as the art of dying went, these kids were a bunch of amateurs.

Anders had finally asserted himself a little bit. There wasn't enough time to undertake the full scope of the project I had laid out, so they would be cleaning up the old core engine (the part that he'd told me made him feel sick to his stomach) and reusing it more-or-less as it was. The so-called 'core engine' was the piece of the product that actually did the majority of the work of the app, but it was not necessarily a complicated piece of technology. It sprawled a bit, but I did not believe it would have been difficult to rewrite it, or simply to break into chunks in preparation for a rewrite... as per the designs we'd agreed upon. Nonetheless, the team lead had finally made a decision and I decided to go with it. He agreed that the core engine would be integrated as per my designs so that it could later be unplugged, piece-by-piece, when we had finished this initial refit. Fixing foundations and the plumbing was the main thing.

I left the office with a bad taste in my mouth. Despite the fact that everybody had agreed to stick to the design, I'd heard them muttering. They didn't believe we could make it work. They didn't believe that we could make the deadlines. I was frustrated. I'd done this all before, with less people, and I knew it would work if they'd stop bitching and start coding. But I couldn't say that to them; that was Anders' job.

I went home and I tried to get to work on the driver integration with Chucky, but it wasn't that easy. My working hours in Australia would end right when the team's would begin, and I would finish a full day's coding with hours of emails, realtime chats and conference calls with the team. Every morning I would get up, sync down the source code, and find that the team had broken what I was working on the day before. Half of the productive part of my day would them be lost fixing it. I don't know what the team was actually doing, but it was clearly not working, since the project wouldn't even compile after their checkins.

I asked them nicely not to break the build. Weeks went by and I still found myself having to fix it on a daily basis. I tried a sterner approach, but this didn't help matters. I didn't yell down the phone, but once I abandoned politeness, the build breaks became less frequent... and meetings became commensurately more surly. I could hear them muttering at me in Swedish.

Joseph was granted a patent that he had filed. Nobody seamed to care. "Software patents don't mean anything," I was told.

Srinith and Sven complained to me that nobody else was working. Anders and the senior guys sat around in their office with the door closed. Tyko was wrapped up in his research. Nobody knew what the UI team was doing. I told that they were the A-team. They were the most productive guys and they were on board with my desire for progress. If we hasd to do this with a team of three, plus Chuck,y then so be it--I'd done it before, back at ATB Software, and this time I didn't have interference from management and marketing to contend with. I had to trust someone, so I trusted them. I asked them to look after the integration with the core engine and the UI team while I pulled my head in and finished the driver interface, and they agreed. This was a mistake.

My A-team introduced some new code libraries to instrument the source in order to catch memory leaks. This library was incompatible with my test apps, but it was cancerous: once it was in the application it proliferated everywhere and could not be removed. I complained: the new framework did not leak memory and the overhead of the instrumentation was huge. It threw lots of false positives and it made debugging a nightmare. Only the core engine needed to be instrumented, and there needed to be a way to turn the whole thing off. But they were adamant, and I had bigger fish to fry. I wrote new test apps and got on with it.

My A-team took the very simple structure that I had created to be a unit of currency between all of the subsystems and wrapped it in layers and layers of macros, so that they would act in an object-oriented way without being truly object-oriented... or debuggable.

My A-team changed all of the error handling routines so that, instead of passing error information up, they would throw assertions that would crash the application.

At this point I decided that I needed to rein them in, and I think they simply decided that the would instead replace me. They started to pull apart the infrastucture I had written, stripping out the inheritance hierarchy that would permit us to drop in new versions of the core engine. If they needed to make a small change to a class that I had written they would push the entire code module into an 'attic' repository and then create a new one... containing almost exactly the same code, but minus the history of changes and of course with his name on it as the original author.

I just took it. The design was all that mattered. If we could stick to the design for this release I would be able to steer the team back on course after the release date, I thought. I was up to my eyeballs in the driver interface and I didn't want to fight about it. But I knew that they were gradually stripping away all of my design and replacing it with Special Magic.

I start to get bug reports from the communications layer. I'd forgotten about the comms layer, with all of the other nonsense, but I didn't think it was a major issue. I couldn't duplicate the bugs, so I asked for log files. No matter how often I asked, nobody would furnish them to me... but the reports kept coming.

I did get a number of feature requests. "This comms layer is useless," Srinith would say. "It doesn't do X."
"It was designed so that could be built on top of it. It's easy enough."
"It should be handled inside the comms layer."

Like an idiot, I would then extend the comms layer to handle that behaviour, rather than insisting that the behaviour should be handled externally. Instead of thanking me, Srinith would then say "This comms layer is useless. It doesn't handle Y."

I did that three times, until Srinith ran out of new features. The comms layer was still buggy, and still a source of complaint, but by then Chucky had finished the driver and I had finished integrating it. Once I was able to focus my full attention on it, I was able to duplicate the underlying bug in the comms layer... and it was a serious one. Srinith jumped on it.

"Alright," I told the Andy, Anders and the A-Team. "I found the bug and I can fix it, but I need two weeks. Ten business days."

I showed them a design document, and they agreed to it. I got on it, and made good progress. Srinith threw some new feature requests at the comms layer for me, which he 'needed urgently', so I build those as well: twice each, once for the old comms layer (so that Srinith could continue programming with it), and once for the new one. On the ninth day I had finished all of the features and I was chasing the last remaining bug when Andy called me up. "We're pulling the plug on your comms layer. Srinith is going to write a new one."

I was on schedule. I would have the layer ready for them, bug free, on the day I had promised. It had the same interface as the old one; nobody would even notice when the new library went in, except that the bugs would disappear.

"Sorry. Anders told him to go ahead." In other words, Anders needed me to fail.

"What's he going to build? What's the design? Is there a document?"

"He's just going to do whatever."

I could see the death of my baby looming.

Joseph was laid off. Martin sent me a note to assure me that Chucky and this had nothing to do with Chucky and I, it was just that they didn't think Joseph's research had a place at the company.

A week later, Chucky quit. The drivers were done and he'd been offered something that paid better and gave him health coverage. Also, he said, he was tired of all the sniveling. I decided that I was, too. I was required to give three months notice, so I held out a few more days before I handed in my notice. My last day would coincide with the release date.

I wanted to make a graceful exit.

Friday, 22 July 2011

Misfits of Science

University was a bit of a shock. I had a very heavy courseload in first year... more contact hours than I had in my last year of high school... and it fell to me to make sure that all of the prac classes and tutorials I would have to attend were timetabled correctly. I didn't actually know what a 'tutorial' was, and I wasn't much interested in finding out.

The other thing was the maths.

I had never enjoyed maths, but I was good at it. At High School, maths was one of my best subjects, at least in terms of grades. But apparently the university wasn't happy with the standard of maths from incoming students, and we were all subjected to a preliminary test before the first term began to prove our competence. And it was just as well. I scored 12 out of 15 on the test, but all three of the questions I got wrong were all the advanced trigonometry questions. My class at high school had done a different unit when others had done trig, and I hadn't learned any of the advanced stuff. So, on top of the heavy courseload, during first term I had to take extra classes in high school level trigonometry. But it paid off: when I resat the trig test and my marks went from 0 to 100%.

But all the classes were dull. They were teaching Pascal, which I was already bored with, and I didn't learn much in the first semester. My most difficult class was Deductive Logic, administered under the philosophy faculty by a lecturer, Hermann, who was no longer teaching any comp sci because he'd failed most of his students in his subject area in prior years.

Hermann taught us formal reasoning, and it was a lot more difficult than I had expected. They were small classes and I on one occasion fell asleep sitting in the front row, directly in front of him. By then I was starting to perfect the art of only working as hard as I had to, so I got through it with reasonable grades.

I soon found in the following semesters when we took boolean logic in maths and computer science units that I had already covered the material in much greater depth, and those subjects proved easy. A valuable subject, as much as I had disliked it.

But then I pretty much hated everything. I had made a few friends, but I wasn't enjoy  my subjects. I wanted to drop out every single semester. Later in first year we were taught about dynamic memory... pointers... and this was something at least new to me. I quite distinctly remember the a lecturer telling us that pointer arithmetic should be done on paper; it was too difficult to do it in one's head... which any commercial programmer working in a native language (at that time, the vast majority) will tell you is patent bullshit. But it was true that pointers were something many students couldn't master. Pointers, I think, are the first big conceptual leap you need to make in order to become a real programmer.

I did not, at that stage, appreciate that most of the people in my classes would never be able to be effective programmers in the real world: even if they could master the technical aspects, the ability to sit down and solve difficult problems all day long is not one that most people are wired for. For my part, I enjoyed the problem solving and I flat out just liked making things, but I felt like I'd already done all of these things before. I wanted to build Skynet, but nobody else was interested in that. It was data structures and algorithms I mostly already knew, employed in the service of meaningless tasks that had already been solved a thousand times. At the end of the day I was promised a semi-lucrative career maintaining ancient software on obscure hardware that would likely be used only by banks, for exciting banking purposes.

I wanted out, but I didn't really know what else to do.  I made the Dean's list in first year despite my misery. Second year my courseload was lighter. No more maths, no more deductive logic... but the comp sci classes were less interesting. We did a lot of hardware and operating systems subjects, the only one of which interested me was the brief unit we did in assembly programming. For the third time I found myself studying boolean algebra. I was still bored and I still wanted to drop out.

In third year we got to choose subjects. I knew I didn't want to take anything to do with networks; I was terrified that it would lead me to a career as a system administrator. Sysadmin, I reasoned, was the most miserable job in the world. If the network is running fine, nobody notices. If the network goes down... which they frequently did, and do... even if it's no fault of your own... everybody hates you. Not for me. I wanted to get out of university with as little work as possible, so I tried to sign up for a bunch of easy subjects... but those all had Databases as a prerequisite. I signed on for Databases and somehow then found there was no room for the easy subjects. Databases proved to be the second-most useful subject I took, although it was a long way from being my favourite.

I enrolled in COBOL. I wanted to do C++, but, owing to the strange degree I had enrolled in, I didn't have the prerequisite year of C programming. Third year had a lot less contac hours than first, so I went to the C++ lectures anyway. Within two weeks I decided that I had to find a way out of COBOL and into C++. If I learned COBOL there was the horrible possibility I would one day have to program it in the real world. By the same token, I knew that C++ was a viable language. COBOL was for retirees; C++ was for powerful young men. Besides: at that time the only language I was any good with was Pascal, and I knew that there were no careers in that.

I had to get into C++. There was nothing for it but to try it on, and see what I could get away with... but in the end it was no difficulty. The professor had seen me in his lectures and he just signed me into his subject, without even asking if I had the prerequisites. I dropped COBOL as quickly as I could.

This was the single best thing I ever did in my five years at University.

I liked C++. I had a bit to learn, but I was able to pick up most of it out of the book. I knew it would be valuable and I paid attention and that, more than anything, is the basis of my career in software. Naked C++.

The other big challenge of third year was the Software Engineering team project. I wound up on a team with one friend and four strangers. The project itself was dead boring... an inventory management database app... but we divvied up the work and I buried my head in my area, which was the User Interface. I took on board my task and decided to let everybody else get on with their own. This was a bad idea.

For all the time we spent in documentation, we really had no plan for how to integrate all of the pieces, and ground zero for this was my UI. When the time came, and I saw how inconsistent and flat-out terrible my teammates' work was, I had to get up and leave the lab. I wanted to punch someone. Did they have no pride int heir work? What the hell was I going to do with that mess? The team lead had flown off to China to look for a wife. The programmers who'd made the mess had no idea that they'd done anything wrong. That left me and the 'chief coder', Adrian, whose job it had been to oversee everybody else's work and make sure it would all integrate. I'd done my bit; I decided that it was on him.

This was very selfish of me, but I had big projects due in other subjects that the rest of the team did not. I went off to work on those and I left Adrian to it. Luckily, he rose to the occasion. Adrian worked long hours, day and night, and he got it done. He made it all work and I think our project scored better than anybody else's. I felt bad about having left Adrian to all the work and resolved to never do that again... and karma has since paid me back many times over.

I didn't make the Dean's list in third year, but I didn't go to many classes, either. All the same, I completed my degree with good enough marks that they offered me an honours year, which I accepted... and then deferred. I couldn't handle another year on that campus, with those subjects. Computer Science? What does that even mean? We don't say 'physics science' or 'chemistry science' or 'mathematics science'. A computer is a device, it's not a branch of science.

And what exactly had been scientific about my degree, really? What sort of research did we do, what new discoveries were we making? I had taken classes in other science disciplines, and even the 'soft' sciences were more research-oriented than CS. What I'd learned was really a kind of engineering, no matter how my degree was named. Since leaving I have never once had the word 'scientist' in my job title (although I would, for a time, hold the title of Advanced Researcher).

Did I want to be an engineer? Did I want to go and work for a bank? No, I decided.

I wanted to build killer robots.

Wednesday, 13 July 2011

Paintball

There were some new faces on the team at Fjord systems: Lars, Srinith and Sven. Lars was quiet and earnest. Sven was a punk rock hacker just out of high school. Srinith was the only one on the entire team who gave any feedback on my documents or my code. The code looked really good, he said, but he asked me to justify some of the most basic assumptions that went into the design. I was pleased that somebody was thinking about it and I was just as pleased that I was able explain them to his obvioussatisfaction: a young, talented coming into the project without any preconceived ideas about how it should work.

Sven was now in my chair in the office with Tyko (formerly seat of Åke the Genius), so I was shunted to a spare desk in a room with some marketing guys. That was fine; my main business would be conducted in the conference room, which I requested for an hour every day.

Finding the correct hour was a problem. Not because the room was booked, but because the team took the notion of 'flexible hours' to extremes. We couldn't start before 10:30 because that was the earliest some of the team members would arrive. We had to wrap the meetings up by 1:001, because that was when some members liked to go to the gym. And so on. Nobody was willing to budge.

I got on with it. I presented the essays I had written, which had been available to everyone for feedback for months. It got the usual cool reception, but everybody was nodding and agreeing and nobody was shaking their head. The main thing that bothered me was that the senior guys... Anders and his crew... were not attending most of the meetings. They were working hard on the current release, they said, which was indeed due out right about then. I remembered Tyko's explanation that they were the only ones who actually did any work, so I left them to it.

I didn't see or hear much from Tyko at all, but someobody reported to me that he'd had a look at the infrastructure I had written and declared it to be 'quite good'. "That's great," I said. "Pike, you don't understand. That's huge. Tyko never says anything good about anybody's code." I remembered Tyko's admiration for Åke the Genius and decided that I would let the matter drop.

I started to present the code I had written formally; showing how everything worked and how the pieces would fit together. The version that Anders was working on shipped and they started to join the meetings in time for me to demonstrate how the system would run, end to end. The code was there for anybody who wanted to play with it; I wanted to keep no secrets.

There was no dissent, so it was time to divvy up work for the functionality build-out; to design the components that would sit inside the framework. I had laid out some guidelines for this, but I wanted Anders to step up and do this part: it was his team and he had the most experience with the existing code, which we would be cannibalizing for this purpose. He'd said he was in, but I wanted to see him put his keyboard where his mouth was. I wanted to let him show that he was indeed the team lead.

These meetings went as before. Nobody contributed much and I found myself working through fresh designs on the fly, on the whiteboard. Anders would say nothing at all. But once I called an end to the meetings, the room would suddenly fill with discussion in Swedish. I was the only person in the room who could not speak the language, and the Swedes had actively discouraged me from trying to learn any.

Chucky arrived with his wife, and they were set up in a hotel suite a few doors from mine. I had companions to dine with now, people to hang out with on the weekend. I was excited. Chucky and I conducted extra sessions for the team so that they could learn about how the drivers would work and discuss what they would do. It was decided, as expected, that I would do the integration with the drivers. That was fine by me: it was delicate work and I'd done it before.

I met with the UI team. They had been building some kind of a fancy framework upon which they were going to build a brand new interface for 'my' version of the product. They'd been working on it for years and did not at that point have a single line of code in production. I asked for a demonstration of what they had been building and they showed me a form with a single, very beautifully-rendered button on it.

I asked Anders if we could count on them to deliver a full-blown UI and he shrugged. Nobody paid attention to the the UI team and they had no idea as to how capable they were or were not. As far as I could tell, they'd spent years inventing a button that I could have had in 15 seconds using the existing Microsoft framework. I decided that if the worst happened, we could whip up a UI quickly, so long as the team understood my messaging system. So I devoted some time to that.

Meanwhile, Anders and his cronies began to have closed door meetings. After a few of these they called me into one of these and told me that they wanted to offer me an 'alternative design' to the one I had been working on for six months and had now spent weeks bringing the team up to speed on. They'd put probably three hours of discussion into it, and they had no diagrams or documents.

This 'alternative design' was a giant black box that looked exactly like the old design, which Anders had earlier told me that he hated. When I asked how he would solve the specific problems I had been trying to address with the redesign he waved his fingers dismissively. "Oh, we'll just do some special magic," he said. Every time I raised a specific problem, he offered Special Magic as a solution.

What he meant was, he and his trio would hack it all together in way that would lead to exactly the same disaster I was now trying to fix. I argued as politely as I could, and then a bit more savagely when it was clear that they weren't listening. In the end they had to concede that my design offered solutions and theirs did not. I took Anders out for lunch to try to smooth over any hurt feelings and I believed that I managed to do so.

Chucky got to see all of this firsthand. "I just work on the drivers," he said. "And I'm happy to stay in my box... but I'm glad it's going to be you who integrates them. These guys are idiots."
"They're smart guys, they're just not used to being able to..."
"Pike, they're idiots. This is going to end badly."

Chucky went back home to the States to work on the drivers. My stress levels were high. I couldn't sleep. I was bored and lonely, shut up in the hotel room by myself, but I found that most nights of the week there was some crew from the office (seldom from the development team and never from Anders' crew) going out for drinks. I went with every chance I had. Suddenly I was drinking as much or more than I had been during my last year at ATB Software.

We started to nail down some designs for the new components. We assigned some of the work to the humps that Tyko had warned me about and found that they actually delivered some good ideas. Anders and his guys didn't do a damn thing, as far as I could see. Tyko was wrapped up in some research projects of his own that were supposedly far too technical for anybody else to look at. Nobody wanted to talk about Jacob, working by himself in Vegas.

Sven had become very interested in the infrastructure I'd written. I wanted somebody to help me maintain it as it came under strain from having real components integrated with it, so I spent a bit of time training him up. Then one day he came to work one day with a new haircut. The same haircut that I was wearing. I had an apprentice.

Sven reported some bugs in the communication system that I had written. I looked into them, knowing that the comms layer needed work, but I found flaws in Sven's code that fixed all of the bugs I could duplicate with my testbed apps. Sven claimed there will still problems, and I was sure he was right... but I was running out of time. I would have to sort them out when I got home.

Somebody organized a paintball session for the final weekend of my trip. I showed up hung over and ragged, but I figured that I had to. Sven demanded to be on my team, but I was finally running out of patience with him. "Sven, you don't have to always be on my team."
"Dude, it's paintball, and you used to be in the Special Forces."
Again with the special forces. Where was this coming from? "Sven, I was never in the Special Forces." "Well, you know. The Marine Corps."


"Sven, I'm from Australia. We don't have marines. I was never in the army."

He went to the other team, but it was clear that he didn't believe me.

Hangover paintball was absolutely no fun at all. I came out of it exhausted and covered in bruises. In fact, it felt like just another day at the office, chasing people around and dodging pot shots fired from under cover. But that was okay.

I was done in Sweden for the moment, and I was about to take four weeks of holiday. The team had all agreed to their various tasks and we had the roadmap all sorted out, right up to the release date. The worst was behind me, I thought. Now we just had to put our heads down and work.

Thursday, 7 July 2011

The Pit and the Pendulum

I got straight to work when I got home, jumping right into every programmer's favourite pit: documentation.

I wrote documents explaining the current system and what was wrong with it. I wrote a document about how to proceed. I wrote design documents for all of the back-end of the application: how the pieces would fit together, physically and logically. Everything was to be data-driven, and based on a really clean interface. I wanted to be able to really break the application into pieces which we could be replaced or re-engineered without risking the entire system. We'd scavenge what we had to from the old app and build as much new as we could.

The integration of third party software was becoming very important to the business and I wanted to be able to drop these new external pieces into our own product with only the thinnest of of shim interfaces. Likewise, I wanted the components to be easy to re-license individually or as a whole: that, too, was of increasing importance. I tried not to get too bogged in detail: I was the architect, not the team lead, and I knew that I would not be able to supervise the whole work effort. I wanted the other people to be able to have their say. As long as they understood the philosophy of it and they built code that conformed to the interfaces I felt that my job was done. If anything broke it would be easy to fix it in isolation.

It looked great on paper.

I think Jacob was the only one who responded to the docs, and I'm almost as sure he's the only one who read them. I took that for agreement and I got on with it, so I went of to build the infrastructure the app would hang on. This would the startup and shutdown sequence for the various business components; thread marshalling within and between those components; system maintenance for updates and upgrades; and of course communication with other elements that were outside of the main process or distributed across the network. I was running out of time, so I rushed through this last piece and it was flawed. I would come back and clean it up, I decided: more important that I be able to demonstrate how the system would run, from end-to-end. I wanted to show the pendulum swinging a full arc in both directions.



While this was going on, Chucky was beginning to design the new drivers and we consulted with each other about how that element would be integrated into the system. We'd both worked in these roles before and we knew what was needed. I didn't feel like I was part of the team in Sweden, but it definitely felt as if Chucky and  were working together on something cool.

I built dummy components to plug into the infrastructure I'd been working on and I found that all the pieces fit exactly as planned. I was as proud of that effort as I have been of anything in my career  in my career.

I got ready to fly to Sweden to present what I'd done. The plan was that  I would clean up the comms system  and supervise integration while team built out the actual components, shoehorning as much as they could scavenge out of the old system into the new architecture. I would additionally built the component that connected to Chucky's work. All of the conceptual stuff was done.


Jacob was not coming this time. He wasn't really part of the refit, and he had now officially been relegated to individual research by Martin. Jacob would stay home and work on the matrix project we'd worked on in Vegas, as well as some other ideas he'd had backburnered at the time. Nobody was much interested in talking about Jacob or his work and I was worried for him, but I had my own problems to deal with.

I arrived in Sweden after thirty-six hours, odd, went tot he hotel, caugth a shower, and showed up at work right before lunchtime. The desire to get things done, I think, was the beginning and the end of my troubles at Fjord Systems.

Tuesday, 28 June 2011

Brazil

After Vegas I returned to Australia with not much to work on. I proofread some of Jacob's patent apps, I went through his new matrix idea, I started building some of the stuff that we'd talked about back at ATB Software... but I didn't have anything formal to work on. I was still waiting for network access, but the company more or less shut down in the first week of December and most of my queries went unanswered. Occasionally I'd get a chirpy reply from someone in the IT department saying: "Done! There you go!" but no passwords or network details were forthcoming.

New Year passed, and so did the first couple of week of January. A full month since I had returned. Very slowly the logistics guy, Wils, organized me some flights out to Sweden to meet the team, to coincide with Jacob's next visit. The first itinerary he sent me was mental: the numbers on the flight times didn't add up and he had me routed through Atlanta. Once I got back in I had to look at it twice to discover that the flights he had booked originated in Melbourne, Florida.

Eventually, after I started to get a bit shirty about my network access, somebody in IT realized what had happened. They had indeed set me up with VPN access to the network and an internal email account, and then they sent all the information about it to that internal email. Several times. Once they put the information where I could find it I found that I could log into the network, no problem, but I had no access to anything beyond the intranet newsletter.

I flew to Sweden. Wils met me at the airport and took me back to the apartment that Jacob and I would be sharing  for my stay. I showered off my 35 hours of airports and aeroplanes and went straight to work. I was in the office by about lunchtime.

I was given the seat vacated by Åke, sharing an office with Tyko. Tyko was no problem; I knew him already, and I discovered that I had a greater tolerance for the techno music that pumped out of his computer all day long than I had expected. Åke, however, was another story, even though he was gone.

Åke's desk told a lot about the man and his work. It was filthy; covered with derelict CRT monitors, cables, mouldy coffee mugs, wastepaper, dust, and a layer of grease that I still cannot explain. I spent ninety minutes cleaning before I was satisfied that I wouldn't contract tetanus from sitting there. Åke's computer was on and unlocked, so I turned my attention to that next. He had left a thousand windows open, each one of them showing a gorgon's nest of ugly code. I carefully saved the contents of each window and closed them until I could again see the desktop of the PC. Then I shut the machine down, wrapped it in plastic, and buried it in a 12 foot hole. If we ever found a sudden need for whatever Åke had been working, on we were beyond hope.

(Alright, I lied about the hole. I put the machine in a corner of the room in case I might one day need something to accidentally-on-purpose put my foot through.)

I met the team. They seemed like a nice, if reserved bunch of guys. No overt social problems, although a couple of them were definitely capital-P Programmers. The guys Tyko said were good shared one office, the guys he said did not were in another. There was a third team, the UI guys, who lived in their own world and nobody was sure what they did... but it involved lots of code churn and without any visible results. Which is odd, you might think, for a team whose business is entirely wrapped in the visual element of the product. But they were not my problem.

I was finally granted access to the source code repository. I checked the code out of CVS, but I couldn't work out how to build it. When I asked where the main solution was... the build scripts, the make files... I was told that there was none: everybody just did whatever they had to in order to make it build on their local machine.

Which is to say, nobody knew what the fuck was happening, and if they did, they were keeping that information to themselves. It was like being in Terry Gilliam's Brazil.

I spent a day making the code I had build as best I could, but there were pieces missing. When I enquired after them I learned that they were in a different repository. I had to stand over the IT guys in order to get them to give me access. This happened three before I had all of the source code. I was shocked.

A source code repository is EVERYTHING to a team. If the code constitutes the raw ingredients of your product, the repository is the refrigerator. Even if the raw produce is low quality, you at least try to keep the fridge clean... because if the fridge goes you've got nothing. At Fjord systems, it was like they were keeping the ingredients in a series of buckets,w hich they'd stashed in the outhouse. I'd seen poor code and CM before, but this was rancid.

I spent most of the week getting it all to build in one place. Harder than it seemed: each of the four repositories had some overlap with the others, but they were not kept properly in sync. Once I was done I got IT to set up a new repository, I checked everything in and I said "Okay boys, this is the one place where you will get your code from, and the one place you will put it back." They seemed pleased with it, but it was hard to tell. In hindisght, I should have added "Try not to break it."

The code was horrible. Spaghetti and shitballs. It couldn't be saved. I said as much to Tyko, who gloomily agreed. I told Jacob, who took it pretty well. But our biggest immediate problem was the drivers. It was dangerous, letting our customers use those half-finished, barely functional drivers and we needed to fix them, stat.

I told Jacob that I thought Chucky was free... or at least he had been a couple of months earlier. Jacob remembered Chucky from ATB Software and so we got in touch with him. With Jacob on hand and in the office to hassle HR, we got Chucky sorted for a contract in days, not months, and we got his network access hooked up properly. We got him access to the new repository and turned him loose on the drivers. He worked sixteen straight hours from the minute his access was hooked up. The drivers, he said, were rubbish, but he could at least make them stable. Which he did. He did us proud.

Monday morning Jacob called a meeting of the development team. As soon as he had all of us in there (excluding those who came to work after 11am) he surprised me by saying "Aaaaand.... take it away Pike."
"Uh, what do you want me to say, Jacob?"
"Whatever needs to be said."

I took it by the horns. The product needed an overhaul and a redesign. I went through what was wrong with it, and how to fix it. I roughed out new designs on the whiteboard. I estimated about a year's work.

The team took it coolly. I was a stranger, the only non-Swedish speaker in the room, telling them that what they had been working for all of these years was wrong and needed to be done over. I worried that I was offending them. I worried that I was stepping on Anders, the new team leader... but something had to be done, and nobody else was prepared to step up. Nobody said much of anything when I asked for feedback beyond a noncommital "That looks okay." After each meeting I made an effort to talk to the other developers one-on-one, and the feedback they gave me was invariably good: "This seems like the sort of thing we need," said one of the better devs. Anders told me "We need to start over. I get a bad feeling when I look at the old code."

I started feeling good about it. I figured if this was Brazil, I was Harry Tuttle: renegade air conditioning specialist, suspected terrorist and all-round bad ass. When the going gets tough, Harry swings in through the window and fixes the problem, permit or not. He's played by Robert De Niro, so you know he is not-to-be-fucked-with.

So we made a plan. I would go back to Australia and start documenting up the new system and building  prototype infrastructure while the team finished off the new release. Chucky would begin work on a new set of drivers. Once the release was out and the infrastructure was ready we would reconvene in Sweden and bring the whole team onto the new project for build out. 

It was going to be a lot of work, and there was and obvious pessimism about whether we could accomplish any of this, but I was confident: I'd managed it before with a team a third of the size and a quarter as experienced, and this time I was going in knowing exactly what I was doing. In the eight or so years I had been a professional developer I'd yet to encounter a technical issue that I couldn't solve. It might take time or study or a shitload of pestering questions to somebody more experienced, but we'd solved every problem that came our way, every place I'd been. What I failed to realize was that it wasn't the technical problems that had proven to be insurmountable int he past; it was the people problems. And we already had people problems.

It was clear that Jacob wasn't going to be able to move to Sweden. Although I was sharing the apartment with him, I didn't know exactly what was going on until later in the week. I didn know that Jacob had been suffering from terrible migraines and that they were clearly work-related. This wasn't the easy-going, goofy Jacob I knew.

Anders and I were invited to a dinner with the CEO, the COO and another guest, Martin, who Jacob old me would be the new CTO. Since he couldn't make the move Jacob was stepping back into a similar role to the one he'd held at ATBSoft: that of a chief research consultant.

Martin was a charismatic, likeable guy from Sydney who impressed me as somebody who knew nothing at all about software development and who lacked even the barest interest in it. When they'd been recruiting a new CTO the executive staff had originally chosen Jacob over him. Now he was coming on board and Jacob would be working under him. That did not seem like a healthy position for Jacob to be in, and, with me as his chief henchman stirring up shit inside the development team, that didn't seem like a good place for me to be, either. It was clear that if shit went down... and it always does... Martin wasn't going to be much of an advocate for anything.

It was just like Brazil. The irrelevant and incompetent machinations of people way beyond my pay grade were going to be a problem for me in ways I didn't understand. It was just like Brazil, but I wasn't Harry Tuttle, the A/C fixit terrorist... I was Sam Lowry, the bureaucrat who, while trying to fix an administrative screwup, becomes an enemy of the state.

Sunday, 26 June 2011

Storm Lord

I didn't think I was going to make it to Vegas.

Even getting the new laptop proved to be an ordeal. I bought it online from a Sydney retailer and, in order to expedite the process, I said I would pick it up from the depot at the airport as soon as it arrived. The individual who designed the traffic infrastructure around Melbourne Airport was surely a Genius Programmer in a previous life: the only way to get to the airport  is via the Tullamarine Freeway, and if there's an accident on the road you're shit out of luck.

There was an accident Laptop Day; a jack-knifed semi trailer. I sat in traffic for three hours until I finally reached an exit, whereupon I turned around and went home.

I went again the next day and picked up the laptop. I spent the entire following day setting it up, and then I got on the plane and flew to Vegas. Amount of code I had to demonstrate to my new boss: zero. I did have plenty of reading for the flight, though: two expensive textbooks.

Jacob collected me at Vegas airport and dropped me at my hotel. I had a shower an we copped some lunch, and then we got to work.

Jacob was--and is--an ideas man. I don't mean that in a derogatory way: he has a list of patents as long as my arm and he has a genuine talent for finding new angles on the problems of the sector. Jacob can write code and will often built proof-of-concept prototypes or small routines to  speed up difficult tasks, but he is not an engineer or an architect, and he'll tell you so. That was what he needed me for.

It didn't matter that I couldn't demo anything yet, because Jacob had a new idea of his own. If it worked, it would massively reduce the memory footprint and increase the speed of the software in a particular process that has always been an industry bugbear. We spent most of the first week working through the maths of it. Could it work? What were the limitations? The concept was great, but would it scale?

While I was there I also got to sit in on Jacob's daily phonecalls from HQ in Sweden. Management told him that the development team was in chaos. They had a deadline in March (this was just prior to Christmas) and nobody even knew what work was going into it. The team leader stepped down. Blood was raining from the sky and the Frost Giants were waking from their sleep beneath the ice.

I suggested to Jacob that, if nobody knew what needed to go into the release, there shouldn't be a release. He  himself was still coming to grips with the organization and it spoke volumes about his predecessor that the team had no idea what they were supposed to be working on. There had to be a release; but management didn't know or care what was going to be in it. Jacob very sensibly set the bar low: some cleanup and a couple of small features.

As for the supposed 'chaos'... well, the team leader had, indeed, stepped down, but the rest of it was hysteria. Jacob told me that the team had been fine when he'd seen them, two weeks ago. He appointed a new team lead and set them to work. It would take ma about a year to fully understand how the team could seem calm and organized to Jacob while the rest of management saw them as a pack of insane chimpanzees flinging faeces around the office. Both views were correct.

The second week I was in Vegas, an engineer named Tyko flew in to join us. Tyko was young, although it wasn't obvious due to the ZZ Top beard he wore (in retrospect he was probably going more for a Storm Lord kind of a look). He was quiet and thoughtful and clearly very smart. Tyko was a dyed-in-the-wool hacker, but, judging by the books he was reading, he was building an excellent engineer's skillset as well. Jacob had selected Tyko to join us for a good reason.

Tyko wasn't much interested in Jacob's new idea, but we were all interested in improving the product. I grilled Tyko on the codebase and I got more than I bargained for.

The drivers didn't work, and hadn't been maintained in years. The main application service was a resource hog, and nobody quite understood how it worked. The core of the product was a nightmare kluged together out of seaweed and snot. The database was a set of flat files that were loaded up into STL containers without any concern for paging or caching, and these themselves were copied all over memory over and over again.

Into the bargain, Tyko didn't think much of the team. He named three good developers, but he claimed that the rest of the group were humps. The smartest guy on the team, in Tyko's opinion, was Åke--who had just turned in his notice and was going across to work for the competition. Tyko said that Åke was not a good engineer, but he was apparently a genius hacker who was brilliant at solving tough but discrete problems. The 'g' word again: without having met him or seen his code, I was glad that Åke was leaving.

Tyko's words boiled down to this: the product was a pile of shit, and the team didn't know what to do about it.

I said I would take a look at it if somebody would give me access to the source code. The competing product I'd built at ATBSoft was reviewing really well, usually ranking in the top 3 products of its kind, so I was confident that I knew what I was doing.

My hope then was that I could suggest some quick fixes that the team could then refine into a plan to fix the product. My job was to be Jacob's research adjutant; I wasn't really a part of the team and I didn't want to overstep myself... but  if the product was as bad as it sounded, the brilliant and fancy ideas that Jacob and I was supposed to be building for Jacob were worthless.

At the end of the week Jacob took me and Tyko back to the airport to catch our respective flights. Jacob wasn't sure exactly what would happen next: he was trying to sell his family home so he could pick up his things and relocate to Sweden, but the US housing bubble had burst and there were dozens of brand new, unoccupied places in his town going cheap. We'd meet again in Sweden the following January and really get things moving.

Once Jacob was gone, I walked Tyko to his departure gate. My flight wasn't for a couple of hours yet. Tyko looked at me and said: "It's not going to work out."

Tyko and I had been staying in the same hotel, and we'd gone drinking in the casinos a few times after dinner. Tyko, I think, was trying to set a new benchmark for Scandinavian impassivity, but he was a smart guy and I came to trust him. This was not what I had expected to hear from him.

"You mean Jacob's matrix ideas?"

"No, I don't think Jacob is going to work out. I think one skilled developer is all he has to bring to the table." I was a bit stunned. Jacob was somebody I admired and trusted him. I knew that he wasn't much of an engineer or a leader, but he was a very sharp predictor of technology and he knew the industry inside out and he was well-respected for it.

There was a strain of naked pessimism in Tyko's tone that I'd never heard before, but which I would become very familiar with over the coming months.