The MurkyGrey blog

Millions of people have already heard this story: Dave Carroll, a talented musician, flew with United Airlines. He checked in his guitar as luggage and then watched helplessly as United baggage handlers tossed his precious instrument around. Reunited with his guitar upon landing he discovered that as he feared, it had been broken. Dave tried in vain to get United to cover the cost of repair, but after a few months he received a final negative response from the airline stating that since he did not report his loss within 24 hours of collecting his luggage he is ineligible for any compensation. In response Dave wrote a song titled “United Breaks Guitars” and published it on YouTube:

The song became an instant hit on YouTube and on the iTunes music store; it quickly proved to be a PR nightmare for United. In addition to the sweet taste of revenge, it landed Dave a nice chunk of change from iTunes sales, two new guitars from the manufacturer, a serious boost to his popularity and a second career as a sought-after public speaker. It’s a great song: well-written, nicely performed, funny and catchy. Give it a click every time you feel like poking fun at United, the airline industry at large or poor Ms. Irlweg, but if we actually want to learn something from Dave’s guitar’s misfortune, let’s try to unravel the secret to the song’s popularity and the underpinning of the guitar incident.

A mountain of frustration

The millions of listeners who fueled the song’s popularity don’t just enjoy a catchy tune, they also relate to the songwriter’s pain and frustration on a very personal level. It is more than a song — it is a battle hymn for the hoards of frustrated airline passengers and for anyone who ever struggled with an uncaring and indifferent big-company “service” organization. Indeed, BoingBoing dubbed it “the complaint anthem”. Admittedly, airlines are the worst: you trust them with your life and your property while they seal you up in a metal tube and instruct you to sit down quietly while they abuse your trust. There is a lot of frustration built up there, and the song easily taps into it.

Who’s at fault?

Pointing fingers is fun, who should we blame for the guitar’s demise? Obviously the baggage handlers who tossed the guitar bear the most direct responsibility, they should have been much more careful. The trouble with accidents though, is that they tend to happen no matter how careful we are. Sure, baggage handlers can significantly reduce the rate at which baggage is damaged, but ensuring the complete safety of each and every checked item is entirely impractical.
When accidents happen, customer service should pick up the pieces. Three Chicago service agents (the ones “who showed complete indifference”) were perfectly positioned to avert the catastrophe and they dropped the ball gloriously. These service agents could have been more sympathetic, they could have asked for the claim check, tracked the item and told the concerned passenger where his luggage was, but most importantly they could have explained the process for reporting damaged luggage and emphasized that it must be done within 24 hours of landing. Just explaining the process would have prevented the PR disaster. Dave probably would not have become a fan of the airline that broke his beloved guitar, but he would have been compensated for the damage and likely would not have written “United Breaks Guitars”.

Who’s not at fault?

Ironically, the one person that Dave chose to name and shame in front of millions of YouTube viewers is hardly at fault. Poor Ms. Irlweg, the bearer of bad news, was wrongfully singled out. You see, second only to big-company customers in their helplessness are big-company employees, often rendered equally powerless by rules and procedures. By the time Ms. Irlweg received the claim the only thing she could do was note that the claim was not submitted within the 24-hour window and reject it.
This notion is not lost on Dave Carrol himself or on United. Dave all but acknowledged Ms. Irlweg’s innocence in his sequel song and United spokesperson Robin Urbanski stated that one of the steps they took following the incident was to “provide agents with a better way to escalate and respond to special situations”.

It’s all about process

Guitar tossing and careless service agents aside, what made this incident truly torturous is rules and procedures; a passenger who was not made aware of the rules in time and an employee who was rendered powerless by process.

And a lesson for acquired startups

Every now and then, a perfectly good startup turns around and gets itself acquired by an industry giant. When this happens, you (a perfectly happy startup employee) might wake up the next morning to find out that you have become… Ms. Irlweg; you try to get things done but find yourself suddenly bound by rules and procedures to the point of complete helplessness. It’s frustrating, and to make things worse all around you long-time employees of the industry giant seem to get their way with ease by invoking just the right rule every time.
Simply put, if you want to be successful in your new big-corporate job, you are going to have to take some time and learn the rules. You might want to:

• Start by finding reliable and current information about rules and procedures. HR should be able to help you with that, if only you could figure out the steps to reach the rights person in HR…
• Take the time to familiarize yourself with the rules (this may take a while).
• As an interim solution, align yourself with someone who’s been around for a while (if you can find such an ally).

Do you have a hair-raising tale of horrific process or terrifying big-company acquisitions? Leave a comment

I recently commented on a blog post that started with a tale of a lousy supervisor doing a truly horrific job of relaying negative feedback to a subordinate. It got me thinking about some of the miserable bosses I’ve had over the years. Not wanting to linger too much in negativity, I tried to conjure up some of the better managers I reported to. A good manager is hard to find, but when you have one, you often get to learn valuable lessons. Here are some things I have learned from one good boss.

Maintain a blame-free work environment

Finger-pointing is a toxic, counter-productive behavior, but there is more to it; in a software startup it is essential to move forward at a fast and efficient pace. A no-mistakes pace is simply too slow. To survive, we have to move at a speed that guarantees a certain rate of error. We must accept the fact that mistakes will be made and corrected on the fly, simply because a pace that yields no mistakes will not bring us to takeoff before the end of the runway. I once spent a few weeks at a client site in a foreign country where a culture of “no mistakes” prevails; as you may expect, everything was gold-plated to death. Time was regularly wasted on unnecessarily perfect performances, and on cover-ups when things didn’t go according to plan.
“No blame” does not mean “no accountability”. Anyone who’s ever worked for me has heard me say: “It’s OK to make a mistake and you will not be judged for it. Making a mistake and not learning from it is a different story”.

Don’t confuse “urgent” with “important”

You plan your day, week and month. You focus your efforts in a calculated effort to achieve very specific goals. Then someone rushes in screaming that the sky is falling and all progress is put on hold until the oh-so-urgent issue is resolved. There’s a hero’s aura about riding to the rescue and saving the day, but when the day is done, you are still a day (or a week, or a month) behind your schedule. The fires you are putting out may be real, or they may be artificial emergencies conceived to manipulate your priority list. Don’t let the moment’s glory distract you from executing your plan for too long. It may not be as urgent, but it is far more important.

Manage your personal productivity

Developers’ productivity is a complicated issue. First off, the disparity between individuals is huge. It is not uncommon for a star developer to be ten times as productive as an average, good developer. You will not find this kind of distribution with say, athletes or steel mill workers. On top of that, there are many subjective and even random factors at play; the estimates that we use to measure productivity are always partially subjective, and if Alice took a day to resolve one specific bug, while Bob took a week to resolve another, can you really say that Alice is more productive? Or was she just lucky?
Improving your personal productivity is a great way to get better at what you do, but as a developer you are the only one who can tell how productive your day is. You can expect your manager to measure your productivity over time, but when it comes to your day-to-day personal productivity — you’re on your own. The CEO of a company I worked for used to say: “It’s tough with you developers, even if I see you wandering around aimlessly all day, as long as you have that thinking expression on your faces, you could be working”. So watch your own personal productivity every day. You’re the only one who can do it, and you’re the one to benefit from it.

Have you had a boss worth remembering? Leave a comment.

* In case you were wondering: the mythical boss I am referring to is Alex Shapira. Alex has done wonders for many software companies, he’s an executive consultant now.

Like any other tragic comedy, a software startup is not worth much without a good cast of characters. Here, for your reading pleasure, are some of them:

The Entrepreneur

The founding father (or mother). The whole thing was his idea, he got it started from nothing, it is his baby and he will see it to maturity. Naturally, he feels like he owns the place.
Strength: Relentless faith in a bright future
Weakness: Relentless faith in a bright future
In his defense: The whole thing really was his idea

The sales guy

He’s the one reeling in the big deals. He’s slick, he’s smart, he’s the cool kid. He can demo a product that requires five massive production servers on his laptop in mid-flight from a middle seat. He can drink any prospective customer under the table. He appears utterly unperturbed no matter how desperate the situation (perhaps because he knows that there will be plenty of time for him to cry alone, in his hotel room, when there’s no one around). His sales are the only source of future revenue so naturally, he feels like he owns the place.

Strength: Handles rejection very well
Weakness: His geek-cred leaves a lot to be desired
In his defense: Perhaps more than anyone else he’s the one assuming a significant personal financial risk. His base pay is way lower than it would have been in an established company and his commission hinges on an unproven product.

The coder

Super-geek is here to build the most amazing product the world has ever seen. Can solve any problem given sufficient quantities of time, coffee and Fritos (in warm climates may substitute Mountain Dew for coffee). The coder believes that the company starts and ends with the code, all of the other characters are just milling about and basking in the glory of the incredible product. Without a doubt the work-horse in the house, nobody puts in more hours than the coder and nobody has less of a social life outside of work. The code is the company’s main asset so naturally, the coder owns the place.
Strength: Can make the impossible possible
Weakness: Does not understand people, may need gentle reminders concerning personal hygiene
In the coder’s defense: It really is a complicated job and not very many can do it well

The VC folks

While the rest of us are busy with important tasks the VC folks take care of such minutia as, you know, paying the bills. They put their money where our mouths are and if they don’t see that money back with a nice return they will not be pleased. Unless you attend board meetings they will, for the most part, stay out of your way. Feel free to whisper a “thank you” every time you get a paycheck. Oh and also, they have a piece of paper that says they own the place.
Strength: The ability to turn promises into funds
Weakness: Do not seem to maintain a steady core temperature like other mammals
In their defense: Have we mentioned that they pay the bills?

Marketing

Nobody can really define what their job is, but it is an important job and they do it well. They write collateral and debate the weight and finish of brochure paper. They design, build and staff trade show booths. Sometimes they do market research and even contribute to product design, yet the ambiguity that surrounds their job robs them of any glory. They try to comfort themselves by acting like they own the place.
Strengths: Versatility and flexibility
Weakness: As bright as they may be they can’t shake a bit of an inferiority complex
In their defense: Everyone else has a well-defined job, why can’t Marketing?

Which is your favorite character? Leave a comment.

The difference between amateur and professional software development is, I am told, process. Tedious, endless, creativity-stifling, headache-inducing process. A bunch of rules that developers hate and ignore, while development managers bite their lips and try to enforce.

Software development, like many other technical undertakings, requires a combination of creativity and discipline and it is rarely successful unless both are present. You may choose to only hire highly creative and extremely self-disciplined developers. If you do, please let me know how that is working out for you… I’m not saying that these people do not exist, they do, and they are spectacular, but they are few and far between and if you hold out for them your staffing efforts may take longer than expected. For most of us there is a conflict between being creative and being disciplined and we struggle for balance. So there you are with a team of creative developers who are not crazy about rules.

Early in my programming career I was part of a maintenance team. Our job was to fix bugs in the live product while the product development team worked on future releases. I learned a lot about good and bad code, and about how good code can be turned bad by a series of undocumented changes. When it’s new, good code is coherent and consistent, it is readable and it makes sense. But code that has been around the block and back often becomes a jumble of changes made by different people unaware of each other. This degradation can be mitigated by documenting changes. Back in 1993, the rules stated clearly that each change must be documented in the source file’s header. It’s good process, but it was almost never followed. Enforcement (via code reviews) helped some, but the team did not seem to have the degree of self-discipline required to eliminate the problem completely. Then one day the problem was eliminated, not by better enforcement but by the introduction of a tool: a new source control system was put in place that required a comment on each change and inserted the comment into the source header. People still could write a non-comment such as “NA”, but nobody did. Instead, every change was accompanied by a good, descriptive, useful comment.

Process was followed at last. Changes were documented and code integrity was protected. Best of all, our manager didn’t have to harass us to do it anymore; we felt more independent and he had no complaints about the new order of things either. Code reviews, now free from process enforcing micro-management became positive educational experiences allowing us to learn from each other.

The tool (in this case the new source control system) did two things:

1. It provided people with a ‘just-in-time’ reminder to follow process. When you were done making your change and you wanted to push it to integration testing you had to check it into the source control system. That was part of your job, you had to do it anyway. While you were there, the dialog popped up and asked you to put in a comment and you did. The burden of remembering the rule had been lifted from your shoulders.

2. It took away the tedious, mindless task of taking your comment, adding date and time and pasting it into the right place in the headers of several source files. You were left with the meaningful task of writing a paragraph to describe your change, the rest was taken care of automatically.

I’ve seen the same thing happen with other process-related tasks. Collecting timesheets was always a painful and frustrating task until I had a system in place that sent email reminders with a “report your hours” link in the message. Logging hours was never hard, the painful part was remembering to do it. With an automated reminder that puts you a click away from completing your report, the pain is gone and process is followed smoothly and effortlessly. Similarly, testing and verification procedures were a real PITA until issue tracking systems with built-in workflow management came along.

No matter how lacking in discipline you think we developers are, what we really hate is having to remember tasks that are not directly related to coding. We also dislike mindless repetitive tasks that we know should be automated. Micro-management is bad for us and bad for our managers, give us a tool for every rule and process nirvana will follow.

Do rules drive you crazy? What do you do to cope with process? Leave a comment.

Suppose we arrange the world’s software along an axis. On the one end of the axis is the ultimate shrink-wrapped product; sitting on the shelf, waiting to be picked up by a user who will rip it open, install and use it happily, never knowing (or caring) who built it or how it was built. On the other end is the definitive tailor-made solution; written from scratch to suit the needs of an individual user, commissioned to a master developer of the user’s choosing who must have spent many hours with the user to learn of his pain and design a unique custom-made solution.

It is this axis that yields the renowned divide between product companies and professional services companies. Product companies sell software, and they sell the same software product to as many clients as they can. Professional services companies sell their time and expertise developing or customizing software for specific clients, one project at a time.

Having started my career in product companies I grew accustomed to the many advantages that a product environment provides for software developers. In a product environment you can always bring an exciting technological idea to the table, a discussion of potential markets for the new idea will follow and if it makes business sense you get to build it. Product companies take technological risks and innovate; the rewards associated with bringing a new technology to a broad market justify the risk. If you are ever asked to build something that does not make sense to you (and we developers always know best…), you can put your negotiation skills to the test in a discussion with product management. With a reasonable product manager, you can impact product direction quite nicely.

In a professional services environment, a new business idea usually starts with a new project for a specific client. The project managers rule the roost, and each project is expected to be profitable or it is not worth undertaking. Much care is taken in gathering and refining requirements and design, but this process is centered around the client, not around technology. The entire project is funded by the client and sometimes the client owns the work-product completely. Naturally it is the client that impacts requirements the most, not the developer, not even the project manager. In a professional services environment developers have much less of a say, or at least so I thought.

If all had gone according to plan I would have spent my entire career working for product companies, but then there’s this pesky little thing called “opportunity” that always seems to get in plan’s way. When I made my opportunity-driven move to professional services I wasn’t sure what to expect. What I found surprised me, and eventually helped me become a better product developer than I ever was. The first thing I learned was that professional services companies are not as bad as I thought they were. They learned a long time ago that their profitability soars with repeatable projects. They don’t really take it one project at a time. If a process or a tool can be developed to serve multiple projects they dive right in and build it, in essence developing their own mini-products. The next thing I learned was quite annoying at first. It turns out that clients who fund a custom development effort are not inclined to keep paying for the promise of a product in the making. Instead, they want tangible deliverables for their continued support. Those tangible deliverables are artifacts that we are all know and love: requirement and design documents, project plans, test plans, and other such paperwork.

I was never opposed to reasonable documentation. In fact, I can rightfully be accused of having been a bit too fanatic about internal and external documentation of code, but I couldn’t help but feel that creating the level of documentation clients required was a waste of time. We have a perfectly good design that the developers understand very well, why spend more time polishing it so that some bureaucrat can read it and stamp an invoice? We test thoroughly and efficiently, why publish an executed test plan? We already know that the product is tested. These artifacts do not help us build the product, and they do not help the end-user use it, they are simply irrelevant. I was exhibiting a bias common in the product world: The product is important, anything outside the bounds of the shrink-wrap (or the tarball for that matter) is not important. Despite this minor annoyance, life in professional services was fun and rewarding and some three years later I returned to the warm embrace of a small product company.

Guess what? I’ve come to appreciate those annoying little artifacts. You see, even in a product environment you probably have important stakeholders outside the development team, prime examples are: non-technical founders, sales teams and marketing people. These stakeholders might not sign your invoices, but it is in your best interest to keep them engaged. Just a bit of effort to gather high-level requirements and design and turn it into a human-readable document will go a long way in sharing your vision of a future product or feature. Throw together an executed test plan or other evidence of testing and their confidence in your latest release will skyrocket. My foray into professional services turned out to be a rounding experience. I learned a thing or two about engaging non-technical stakeholders and lost some old biases.

Which environment do you prefer: product or services? What do you do to engage your non-technical stakeholders? Leave a comment.

Related Link: Giff Constable, When it comes to startups, products and services don’t mix

Somewhere out there is a CS student or a recent graduate who read my “Why work for a startup?” post and vowed to make a career in software startups. Well my friend, I love your style but be warned: the startup life is not all fun and games. Here’s what you should brace yourself for:

Risk and uncertainty

The fact is that startups fail at a much higher rate than older companies. It’s a risk you choose to take and you have to be prepared for. Work for startups long enough and you are guaranteed to experience painful layoffs and cutbacks, if not a complete collapse of the company. Can’t handle it? Don’t work for a startup! But remember: while startups are more risky than other companies, the illusion of complete job security anywhere is just that — an illusion.

Limited resources

Unless you plan to take my advice through a time machine back to the year 1999, the startup you work for will have limited resources. Do not expect glorious office space (that is, if you’re out of the founder’s garage and actually have an office). You’re equally unlikely to find unlimited expensive equipment or first-class travel. In a well-funded early stage startup you will have the tools you need to succeed, but in a “no frills” kind of way. You can expect a somewhat more luxurious setting once revenue from sales starts trickling in.

Stressful times

There are not too many of us and we’re running on a tight schedule with limited funding, it’s going to get stressful at times. Expect bouts of long hours, night and weekend work (but don’t let that become your long-term standard, you’re no good to anyone if you’re burned out).

What sacrifices have you had to make for your startup? Leave a comment.

The question is obviously one that I never need to discuss with colleagues, but people outside the software industry and those who found their success in large corporations often find it strange; why would anyone choose to stake their career on a string of tiny companies, each with limited funding and an uncertain future? Well, I love the startup life and I wouldn’t have it any other way. Here’s why:

Atmosphere

Ah, the startup atmosphere! The camaraderie! The sense of purpose that surrounds a small group of people dedicated to a common goal! The feeling that you hold your fate in your own own hands! You matter and you can make the company a success by doing the best job you can do. There is nothing quite like it.

Financial opportunity

If you work for a startup, you have equity. Whether it is a fat stack of founders’ stock or a modest package of incentive stock options, it means that if and when the company is successful, you will get your piece of the pie.

Variety

If you like some variety at work, you’re in the right place. With a small team, many of the business functions are performed by any team member who has some free bandwidth. That’s how as a developer, I got to do sales support, on-site customer support, collateral writing and more trade shows than I care to remember. It’s fun.

No nonsense

There are not too many of us, we’re running on a tight schedule with limited funding and a clear goal. There is simply no time for nonsense. We don’t sit through 3-hour meetings, we don’t do TPS reports and we don’t play office politics. The only reasonable thing to do is to focus on meaningful work.

Unusual perks

Startups are teeming with original thinkers and the flow of unique ideas does not stop when it comes to employee perks. While you should not expect to find on-site daycare or a company gym, other, more quirky perks are common. Think free snacks and soft drinks, gaming consoles in the break room, paintball outings and beer in the fridge for Friday afternoon happy-hour in the conference room (try to squeeze that one past HR in a Fortune-500 company), and there is excellent coffee, always.

Long-term prospects

As strange as it may sound, there are long-term prospects to be found in a startup. Regardless of whether your startup ends in an acquisition or fire-sale, one thing is guaranteed to outlive the company — the people you work with. The “startup people” will move on, most likely to another startup and if you play it right you will move right on up with them. Some of my friends have been moving around together as a group since 2003 and they are now on their third startup, having successfully sold the first two.

Also, not all successful startups get gobbled up by industry giants. While IPOs are not what they used to be, some continue to grow and evolve into privately-owned midsized companies that maintain many of the advantages of startup life for their employees.

What do you like best about the startup life? Leave a comment.