Thursday, 28 April 2011

Fear and Circuses

The acquisition went ahead and the owner of our company took his payout. He remained a part of the organization, but I think that basically meant that he continued to draw a salary and occupy the most comfortable corner office--a process that had really begun about the time that Bashir was hired.

The man who had mastermind the acquisition by our new corporate overlords, Lester Gould, was technically in charge of our group, but since he was located off site at our new head office this really mean was that Bashir, who served as the primary interface between our group and the parent company, was in now large and in charge.

While our network was refitted to make us a part of the larger company, we had some downtime and all of our ongoing work came to a stop. "Read the manuals," we were told. "You'll be back online in two or three days." Still smarting from nearly being sacked, I decided that this was the opportunity I had been looking for to show how much I really did bring to the table.

When I was first hired on by the company my skills as a database programmer were a key selling point. Our product was built on a flat file database and, as is usually the case, a decade later it had become apparent that this wasn't going to serve our needs very well. Customers wanted multiple-user access to each project, and that was difficult with a flat file system... and in fact impossible without some kind of client-server architecture. Early in my tenure, at my manager Eric's request, I had drawn up some design documents and some UML pictures for this. After some consideration, Eric had then decided not to include me in the meeting where he presented them to the then-owner of the company and nothing ever came of it. A proper back end was something our customers desperately wanted, but it was a huge job and the company simply didn't trust me, a junior programmer with a bad attitude and a shitty track record, to do it. So they did nothing.

I decided that I would spend the three days of downtime building a prototype. It was hacked together quickly, but I did manage to get it working to the point where I could demonstrate what I still think is a quite compelling picture. I could show data migrating into the database, and an object-oriented middle tier capable of reading that data back. I could then show those middle-tier objects adapted back into our usual data-structures, pushed through the graphics pipeline and displayed on the screen with very similar performance to the old system. It was looking good.

I called Bash and Eric into my office for a demo and ran the prototype side-by-side with the release so that they could compare the results. Then I showed them how easy it was to build out the middle tier objects and how to do the migration. It was rough and I did not have that middle tier running as an external server yet, but it certainly worked. I built the prototype on Access, because I had no access to anything else with the network down, but I made it clear that we'd have to find a real database engine. It wouldn't scale up, but it did the job for a proof of concept. We had already learned the worth of Access with the Development Database, and I actually believed they were listening to me now. Bashir's eyes lit up and he congratulated me on my work. Sincerely, I thought. I was pleased with myself for the first time in a year, and I resolved anew to make the most of this adventure. This lasted for almost an entire week.

Word came down from head office that a part of our application needed to be made to work with an external application, which was code named 'Tarzan'. This, of course, was only partially true, but I had no idea how these games were played at the time. Some engineering heavyweights came down from head office to look at our code, and, Rhaj later admitted to me, they had laughed his face.

In order for our product to become part of Tarzan's workflow, we would have to work with Tarzan's database... although the Heavyweights had not decided which database engine to use. I was not allowed to participate in those meetings, but, when Bashir later reported what had been discussed in those meetings to the rest of us he quite deliberately refused to look at me when he revealed that the Heavyweights had weighed up all of the options... and chosen to use Access.

Let me be clear on that. Access would not scale well enough to be useful for our tiny internal development database, let alone for our actual product... and our product was only a tiny part of Tarzan. The Heavyweight engineers wanted to use it for the entire product. Suddenly I was glad I had been excluded from the meetings.

Bash came to me later in the week with a new task: to create a complete data model for our application. I estimated a week for the task, because our project was huge: in addition to the flat file database we maintained a huge amount of data in other formats including, but not limited to INI files, the registry, various rendering formats, and of course, good old Access. It was, frankly, a disaster, and it was difficult to document because I was always finding new bits of persistent data. At the end of the week, when it wasn't ready, and Bashir was unhappy with me. Apparently he had promised it to the analysts for Tarzan could work out how best to proceed with the integration.

I turned the data model in the following week, and within a couple of days Bashir and Rhaj came to see me--both of them angry.

"We showed this to the Tarzan guys and they're appalled!" said Rhaj. "It's a huge mess!"
"That's right," I said. "It's full of duplicate data, the keys are poorly defined, there's been no attempt to normalize it.."
I could see that none of this meant anything to Rhaj. In reply he spluttered: "So why did you write this garbage?"
"Uh... because Bash asked me to document our data model, and that's what I did. I didn't design it... you did."
Rhaj had no reply to this and they left. I'm quite sure that they went back to the analysts and told them that it was the fault of a junior developer (that bloody Pikeman) and that of course it wasn't the real data model.

That was the last I ever directly heard about the database project, although I did overhear Rhaj and Bashir talking about it in the kitchen. It had been given a codename of its own and a satellite office in another city had been tasked with building the entire back-end for Tarzan. Again. They had already spent two years trying and failed, and now they had been authorized to start over.

The tiny part of the mysterious 'Tarzan' that I had witnessed seemed like a circus, replete with clowns and jugglers and jungle beasts but completely lacking in animal wranglers or a competent ringmaster... and it was 3 years over schedule even before we had started doing our part.

That was about the time that the ringmaster, Lester Gould, decided to put down his whip and quit.


  1. Just letting you know that I'm reading this and enjoying it. It helps me accept my own job better, and appreciate my husband's situation (he's also a non-US-citizen software engineer). :) (Here from your comment on Scalzi's blog mentioning it, fyi.)

  2. Hey, thanks, Anon!

    I've been through about 12 years of this lunacy and sometimes it gets really tough out there. So far I've only covered about 18 months in the life of, so... lots more to come, I hope.