The MurkyGrey blog


Talking to people about technology

Blast it like Google!

A tiny case study

Start talking about email blasts and people get very emotional. Some marketers swear by them, they will tell you that if you do it right an email blast is a great way to reach a whole lot of people and track their response. Other marketers hate them, saying that blasting thousands of emails is a waste of time, most people don’t read them and worst of all: many will consider it spam and hate you for it.

When Google blasted a Google Ads email promotion last month to their customers I got an unusual glimpse at their blasting strategy. You see, I own no less than seven Google Apps domains and so, over the course of the day, more and more emails from the same blast found their way into my inbox. It looked like this:

3 emails

The thing that should jump out at you is that although we’re looking at three emails from the same campaign, each offering is different. We’ll get back to that in a second, but first let’s talk about the mailing list.

The mailing list, obviously, makes or breaks an email campaign. Blasting a bunch of random strangers is probably the best way to waste your time and money, and ruin your reputation all in one go. Targeted lists of people you’re not directly affiliated with are a grey area, but the sure bet is your list of existing customers. Your customers expect to hear from you, they should be happy to hear from you (unless they hate you, in which case you should not worry about email marketing — you have much bigger problems!). Google is making a very safe bet by emailing current customers and you can do it too. If you’re uncertain about email marketing and you’re looking for a place to start, your list of current customers is a safe, natural choice. By the way, if you’re looking for an easy way to increase this list, offer something for free!

Now back to content. It is no accident that three emails from the same campaign contain three different offers. In our case the three offers were:

  1. Spend $25, get $100 free
  2. Spend $50, get $150 free
  3. Spend $100, get $300 free

The clear trend here is “the more you spend, the more you get for free”. This is true in absolute terms while in relative terms it is actually the lowest offer that yields the highest bonus (400%) while the other two are slightly lower (300%). What does this mean to us? Not much, except the knowledge that someone is playing with the numbers and trying different options. Which one of these is the most attractive? Who prefers the lower offer and who prefers the higher ones? We don’t know and neither did Google, until they ran the campaign. But you can be sure that responses to the campaign were tracked and analyzed to death and the marketing folks at Google now know exactly which offer appeals to different segments of their target demographic.

And that is the real takeaway here: It is good to craft your message carefully before you deliver it to your audience. But when delivering a marketing message to a broad audience, it is always better to have at least two variations of your message, deliver both and measure the response you get with each one. Choose the one that worked better as the base for your next campaign and repeat, or conclude that different demographics respond better to different messaging and split future campaigns accordingly. Just be sure to follow the rules for statistical significance before you make your decision. (If you need help designing a statistically valid test drop me a note, I’d be happy to help.)

Alright, maybe there ARE dumb questions

(but that’s completely beside the point)

It happens almost every time: I do a seminar, a panel or some other speaking engagement and it’s fun all the way down to the Q&A section. Then I see a hesitant hand raised from the back row and I already know that before asking their question, the person in the back would say something like “I have a bit of a dumb question”. And I answer, of course, but not before I add the automatic counter-preemption “there are no dumb questions” (usually stopping short of becoming the cliched grade-school teacher by omitting the “only dumb answers” part).

Recently, I took some flak from a co-presenter for this old habit. “Don’t say that” he said after the crowd has dispersed “you know damn well it’s not true, there are tons of dumb questions”. I did not think this was a fair comment, but it was kind of hard to argue with, and possibly true.

Let’s start with the “true” part. Are there really tons of dumb questions? What is a dumb question anyway? Perhaps if you ask a question that shows you completely failed to understand what I’ve been talking about for the past hour it could be considered dumb? I think not, more likely it shows that I failed to explain things properly or that I wasn’t targeting you (which means that it was a mistake to invite you, not your fault).

But forget all that. The actual dumbness of the question is completely beside the point! The person in the back row is not really saying “I think my question is dumb”. They are saying: “I’m not very comfortable speaking in front of all these people, so please don’t mock me or my question”. And what I’m trying to say to them is: “Don’t worry, I would never do that. Speak up, I am really interested in what you have to say”.

And so, my dear accidental critic, I fully intend to keep saying “there are no dumb questions.” I’m sorry it bothers you so much. And may I suggest you stop taking everything so literally? That might help too.


The unexpected truth about LinkedIn endorsements

LinkedIn endorsements are not just the selfless, friendly pats on the back they seem to be. They are also a self-promotion tool that allows you to exploit the networks of your connections including colleagues, friends and competitors.

LinkedIn launched the “endorsements” feature in September of 2012, touting it as “a great way to recognize your 1st-degree connections’ skills and expertise with one click”. This easy-to-use (some say “lazy”) feature sees a lot of use. Endorsements pop up in my own LinkedIn activity feed on a daily basis.

Many LinkedIn users embraced endorsements and now, more than five months after launch, the stream shows no sign of slowing down. However, the new feature has also drawn its share of criticism. Many argue that it adds nothing to existing LinkedIn recommendations, that recommendations require more effort on behalf of the endorser and are therefore more meaningful. Mashable went as far as calling all LinkedIn endorsements “meaningless” in a recent op-ed while others describe the slew of endorsements as “noise”.

But critics and supporters alike miss a very important aspect of LinkedIn recommendations: It is a free and easy way to get your name on the activity feeds of your second-degree connection. Furthermore, it is the only way to gain exposure to the connections of those who elected not to share their network with you.

Here’s how it works: John is a Southern California marketing professional working for White Goose Marketing Inc. One of John’s LinkedIn connections is Liz, who works for Golden Eggs Marketing Inc. across town. John and Liz met at a conference a couple of years ago and connected on LinkedIn. Liz, as you may expect, is connected to virtually everyone at her office. John is ready to make his next career move and he would like to get noticed by hiring managers at Golden Eggs. John endorses Liz for an existing skill on her LinkedIn profile. Shortly afterwards, a message pops up in Liz’s activity feed, visible to of all of her contacts, (including many at Golden Eggs): “Liz is endorsed by John”. A link to John’s profile accompanies the message. Possibly, one of Liz’s co-workers or managers will be intrigued enough to click through and view John’s profile — mission accomplished! If it doesn’t work the first time, John can repeat the process, endorsing Liz for a different skill a few days later. If he does it too frequently he risks annoying Liz, but every endorsement is another shot at the exposure John seeks.

And here’s a slightly more deceptive example: Pete and Holly are executive recruiters. They often find themselves competing over talent and clients. As recruiters, their respective networks are critical assets they closely guard. You can be sure that both their LinkedIn profiles are set so others cannot see their connections. However, they left “who can see your activity feed” at the default setting, allowing their connections to see it. In the spirit of 21st century co-opetition Pete connected with Holly on LinkedIn, knowing she cannot see his list of connection. Holly would love to add some of Pete’s top candidates to her talent pool, so she ‘innocently’ endorses him for an existing skill. Here too, with her name on Pete’s activity feed there is a chance that some of Pete’s candidates will click to view Holly’s profile. (In fairness, the endorsement will show up on Holly’s feed as well, exposing Pete to her own network.)

Think what you will about John and Holly’s moral conduct, they utilize LinkedIn endorsements in a powerful and creative way. If you are concerned about others utilizing your own activity feed in this way, you can remove endorsements from your activity feed  (update: Erika Napoletano wrote an excellent tutorial explaining how to do that).

And finally, to my many LinkedIn contacts whom I have endorsed over the past few months, and to the few that I have endorsed recently while researching this post, I really did mean each and every endorsement. You guys are awesome!

The Geek at the Trade Show

Jeffrey is and old colleague, a brilliant engineer, and a geek to the core; I ran into him on my way out of a particularly interesting session at an industry trade show. 

MurkyGrey: “Hey, Jeffrey! I didn’t know you were here!”
Jeffrey: “Yeah, they sent me this year. The sessions are quite good.”
MG: “I’m heading back to the exhibit hall, want to tag along?”
J: “No thanks, I’m trying to stay away from the exhibit hall, it’s so annoying.”
MG: “Annoying?”
J: “Yeah, a bunch of sales guys, none of them understand half of what they sell, and they all smile at you and latch onto you like leeches. They want me to buy their stuff but I don’t do any buying, that’s my boss’s job.”
MG: “Yes, it’s a jungle out there. But from what I’ve seen yesterday there’s some stuff there you might like; some new versions of tools that you already use and some new tools you may not have heard of, including two that launched this week.”
J: “But if I walk up to them, they’ll start asking me all these questions, it almost feels like they are trying to interrogate me. By the time they finally let me go I’ve told them a whole lot about my job and haven’t learned anything new about their products.”
MG: “That’s just an old sales trick, some sales people believe that by asking questions they can demonstrate their own knowledge, even establish themselves as an authority in your eyes while maintaining control over the conversation.”
J: “That’s ridiculous! Does it ever work?”
MG: “Surprisingly it does, it works very well when you sell commodities to consumers. Ask anyone whose job is to sell cars or appliances and they’ll tell you. It does not work when you’re selling complicated technology to experts, which is why it does not work here.”
J: “You see, it’s a waste of time”
MG: “It does not have to be. If I find myself under interrogation I just reply to their questions with my own. Nobody likes to look clueless so after two or three tough technical questions they stop and go get their own tech guy, that often leads to a meaningful conversation about the problem space and their solutions. I often learn new, interesting stuff this way.”
J: “That actually makes things worse for me, once I’ve caused someone to spend time with me I feel awful walking away from them without buying anything”
MG: “That’s understandable but you really should not feel bad. Nobody expects you to buy anything right away, and even in the long run they should know that a 5% conversion rate is very high for a trade show. Hey, they got to spend a few minutes talking to a handsome guy like you, they have nothing to complain about! So, what do you say? Are you coming with me?”
J: “Sure, maybe… in a minute. Hey, look! Free coffee! I’ll see you in a few!”

I made my way into the exhibit hall by myself, wondering what it might take to get more Jefferys to shed their inhibitions and step inside.

“United Breaks Guitars” and a lesson for acquired startups

Millions of people have already heard this story: Dave Carroll, a talented musician, flew with United Airlines. He checked in his guitar as luggage and then watched helplessly as United baggage handlers tossed his precious instrument around. Reunited with his guitar upon landing he discovered that as he feared, it had been broken. Dave tried in vain to get United to cover the cost of repair, but after a few months he received a final negative response from the airline stating that since he did not report his loss within 24 hours of collecting his luggage he is ineligible for any compensation. In response Dave wrote a song titled “United Breaks Guitars” and published it on YouTube:

The song became an instant hit on YouTube and on the iTunes music store; it quickly proved to be a PR nightmare for United. In addition to the sweet taste of revenge, it landed Dave a nice chunk of change from iTunes sales, two new guitars from the manufacturer, a serious boost to his popularity and a second career as a sought-after public speaker. It’s a great song: well-written, nicely performed, funny and catchy. Give it a click every time you feel like poking fun at United, the airline industry at large or poor Ms. Irlweg, but if we actually want to learn something from Dave’s guitar’s misfortune, let’s try to unravel the secret to the song’s popularity and the underpinning of the guitar incident.

A mountain of frustration

The millions of listeners who fueled the song’s popularity don’t just enjoy a catchy tune, they also relate to the songwriter’s pain and frustration on a very personal level. It is more than a song — it is a battle hymn for the hoards of frustrated airline passengers and for anyone who ever struggled with an uncaring and indifferent big-company “service” organization. Indeed, BoingBoing dubbed it “the complaint anthem”. Admittedly, airlines are the worst: you trust them with your life and your property while they seal you up in a metal tube and instruct you to sit down quietly while they abuse your trust. There is a lot of frustration built up there, and the song easily taps into it.

Who’s at fault?

Pointing fingers is fun, who should we blame for the guitar’s demise? Obviously the baggage handlers who tossed the guitar bear the most direct responsibility, they should have been much more careful. The trouble with accidents though, is that they tend to happen no matter how careful we are. Sure, baggage handlers can significantly reduce the rate at which baggage is damaged, but ensuring the complete safety of each and every checked item is entirely impractical.
When accidents happen, customer service should pick up the pieces. Three Chicago service agents (the ones “who showed complete indifference”) were perfectly positioned to avert the catastrophe and they dropped the ball gloriously. These service agents could have been more sympathetic, they could have asked for the claim check, tracked the item and told the concerned passenger where his luggage was, but most importantly they could have explained the process for reporting damaged luggage and emphasized that it must be done within 24 hours of landing. Just explaining the process would have prevented the PR disaster. Dave probably would not have become a fan of the airline that broke his beloved guitar, but he would have been compensated for the damage and likely would not have written “United Breaks Guitars”.

Who’s not at fault?

Ironically, the one person that Dave chose to name and shame in front of millions of YouTube viewers is hardly at fault. Poor Ms. Irlweg, the bearer of bad news, was wrongfully singled out. You see, second only to big-company customers in their helplessness are big-company employees, often rendered equally powerless by rules and procedures. By the time Ms. Irlweg received the claim the only thing she could do was note that the claim was not submitted within the 24-hour window and reject it.
This notion is not lost on Dave Carrol himself or on United. Dave all but acknowledged Ms. Irlweg’s innocence in his sequel song and United spokesperson Robin Urbanski stated that one of the steps they took following the incident was to “provide agents with a better way to escalate and respond to special situations”.

It’s all about process

Guitar tossing and careless service agents aside, what made this incident truly torturous is rules and procedures; a passenger who was not made aware of the rules in time and an employee who was rendered powerless by process.

And a lesson for acquired startups

Every now and then, a perfectly good startup turns around and gets itself acquired by an industry giant. When this happens, you (a perfectly happy startup employee) might wake up the next morning to find out that you have become… Ms. Irlweg; you try to get things done but find yourself suddenly bound by rules and procedures to the point of complete helplessness. It’s frustrating, and to make things worse all around you long-time employees of the industry giant seem to get their way with ease by invoking just the right rule every time.
Simply put, if you want to be successful in your new big-corporate job, you are going to have to take some time and learn the rules. You might want to:

  • Start by finding reliable and current information about rules and procedures. HR should be able to help you with that, if only you could figure out the steps to reach the rights person in HR…
  • Take the time to familiarize yourself with the rules (this may take a while).
  • As an interim solution, align yourself with someone who’s been around for a while (if you can find such an ally).

Do you have a hair-raising tale of horrific process or terrifying big-company acquisitions? Leave a comment

Five lethal mistakes in offshore software development

As I’m sure you’ve heard, there are far-away lands where the best and brightest youths flock to science and technology schools. They emerge a few years later in hoards of brilliant and eager software developers. Why not put them to work for you? They are smart, talented, easy to find and hire and they work for very reasonable pay. The big guys have been doing it for years and recently, smaller and smaller startups are incorporating offshore development into their plans.
As a development manager I have, over the years, utilized development teams in India, China and Russia. Here are some lethal mistakes to avoid:

Downplaying the culture gap

Communicating with your offshore developers may seem easy; in all likelihood they speak decent English. A friend of mine, a European microbiologist who hosts in his research lab science grad students from all over the world, commented that in recent years the level of English spoken by young foreigners has improved dramatically. I think he’s right, and I believe that what we’re seeing is a generation of young people, now in their twenties, who speak good English because they grew up with Internet access and were exposed to a lot of English content.
But beware: culture is much broader than language and communication has a lot more to it than words alone. Intonations, gestures, idioms, slang, vocabulary, unspoken assumptions, acceptable and unacceptable behavior vary wildly across the globe and these variations can lead to catastrophic failures in communication. The fact that good English is spoken on both sides might lull you into thinking that you understand each other when in fact you do not.

To avoid this mistake:

Encourage offshore participants to explain back to you their understanding of a conversation (especially if the conversation is one-sided in nature, e.g. you are assigning tasks or briefing them on a new initiative).
Document and share meeting minutes; writing gets the non-verbals out of the way.
Mind the gap; keep reminding yourself of the cultural differences that you must bridge.

Managing offshore developers remotely

The whole point in offshoring is to save money, right? Excellent offshore developers are sometimes easy to come by, but hiring good managers is never easy and not nearly as cheap. You might be tempted to skip hiring a manager altogether, tap into the startup spirit and “do it yourself” (that is: manage your offshore developers yourself, remotely). My advice: don’t!
Managing people is hard, managing developers is harder. A software development team’s manager needs to be there with the developers, meet with each one on a regular basis, get a first-hand account of their activity, successes and challenges. Unless you are an expatriate offshoring back home you already have a cultural gap standing between you and your remote team. Add to that the time zone differences and the loss of non-verbal cues that come with long-distance telecommunication and you have yourself a recipe for failure.

To avoid this mistake:

Hire a local manager for your offshore team. If the team is small (less than 5-10 developers) you can hire a “working manager”: a senior developer that also happens to have management skills. If your team is larger than that, hire a full-time manager. In any case you must have a single point of contact and accountability on the ground, close to your developers and on the other side of the geographical and cultural gaps.

Skimping on travel

International travel is expensive, time-consuming and uncomfortable. With video conferencing and web meetings readily available you feel like you are virtually there, but “virtually” does not cut it. If you don’t travel to visit your offshore team you might never interact directly with some team members (usually the more junior ones), you will never get a feel for the atmosphere in their office, you might never know what is working well and what isn’t. If you never visit your offshore team you are relying completely and blindly on your remote manager, creating a dangerous single point of failure in your organization.

To avoid this mistake:

Every six months or so clear some time on your calendar, make sure your passport and visas are in order, get a plane ticket and pack your bags.
To make things a bit easier, alternate between traveling to visit your offshore team and inviting your offshore counterpart to visit you. This will give your remote manager an opportunity to interact with other members of your local team. Travel can be a burden, but in some cases your remote manager may see these trips as an added benefit.

Failing to protect Intellectual Property

Coders handle one of your company’s most valuable assets: the code. At home you are protected from code theft by non-disclosure and confidentiality agreements, ethical norms and the legal system. Personally, I never encountered a developer that stole source code and sold it to a competitor. Both the developers and your competitors have too much to lose in terms of reputation and legal liability (both civil and criminal).
In a foreign country, do you know what the social norms are? Will the social norms protect you? Can you enforce a non-disclosure or confidentiality agreement? Can you count on the legal system? If you fail to answer these questions before trusting your offshore developers with critical parts of your code you might find yourself up the creek without a paddle.

To avoid this mistake:

Get sound advice, specific to the location of your offshore team. Do not share critical portions of your code with developers in locations with a questionable confidentiality record.

Placing marketing functions away from the target market

This mistake is common among foreign companies that start out with a development center in a low-cost locale and develop a product for the US market. Once product development is complete they would open a sales office in the US and leave product development,QA, tech writing and other functions back home where things are more reasonably priced. It hurts to spend big bucks on a US sales force when you have grown accustomed to much lower costs, but you know there is no escape from it.
But what about marketing functions? Do you really have to hire super expensive US-based product managers? Lead generation is all done via telecommunication, it’s not quite the same as sales, is it? And what about collateral writing, does it really have to be done at $80 and more per hour when your brightest developers work for less than a third of that?
The answer is yes, you have to spend the money and keep your marketing functions close to the target market. If you don’t, you are wasting your money on your US-based sales team. Instead of results, they will deliver a constant flow of complaints about the quality of leads, marketing materials and product features.

To avoid this mistake:

Once you commit to the US market, hire a local sales and marketing team, don’t blunt your own competitive edge with a marketing team that is not in touch with its own market.

Need Help?

Do you need help defining, building, marketing or selling your software product? drop me a note, I’d be happy to help.

Things I have learned from my boss

I recently commented on a blog post that started with a tale of a lousy supervisor doing a truly horrific job of relaying negative feedback to a subordinate. It got me thinking about some of the miserable bosses I’ve had over the years. Not wanting to linger too much in negativity, I tried to conjure up some of the better managers I reported to. A good manager is hard to find, but when you have one, you often get to learn valuable lessons. Here are some things I have learned from one good boss.

Maintain a blame-free work environment

Finger-pointing is a toxic, counter-productive behavior, but there is more to it; in a software startup it is essential to move forward at a fast and efficient pace. A no-mistakes pace is simply too slow. To survive, we have to move at a speed that guarantees a certain rate of error. We must accept the fact that mistakes will be made and corrected on the fly, simply because a pace that yields no mistakes will not bring us to takeoff before the end of the runway. I once spent a few weeks at a client site in a foreign country where a culture of “no mistakes” prevails; as you may expect, everything was gold-plated to death. Time was regularly wasted on unnecessarily perfect performances, and on cover-ups when things didn’t go according to plan.
“No blame” does not mean “no accountability”. Anyone who’s ever worked for me has heard me say: “It’s OK to make a mistake and you will not be judged for it. Making a mistake and not learning from it is a different story”.

Don’t confuse “urgent” with “important”

You plan your day, week and month. You focus your efforts in a calculated effort to achieve very specific goals. Then someone rushes in screaming that the sky is falling and all progress is put on hold until the oh-so-urgent issue is resolved. There’s a hero’s aura about riding to the rescue and saving the day, but when the day is done, you are still a day (or a week, or a month) behind your schedule. The fires you are putting out may be real, or they may be artificial emergencies conceived to manipulate your priority list. Don’t let the moment’s glory distract you from executing your plan for too long. It may not be as urgent, but it is far more important.

Manage your personal productivity

Developers’ productivity is a complicated issue. First off, the disparity between individuals is huge. It is not uncommon for a star developer to be ten times as productive as an average, good developer. You will not find this kind of distribution with say, athletes or steel mill workers. On top of that, there are many subjective and even random factors at play; the estimates that we use to measure productivity are always partially subjective, and if Alice took a day to resolve one specific bug, while Bob took a week to resolve another, can you really say that Alice is more productive? Or was she just lucky?
Improving your personal productivity is a great way to get better at what you do, but as a developer you are the only one who can tell how productive your day is. You can expect your manager to measure your productivity over time, but when it comes to your day-to-day personal productivity — you’re on your own. The CEO of a company I worked for used to say: “It’s tough with you developers, even if I see you wandering around aimlessly all day, as long as you have that thinking expression on your faces, you could be working”. So watch your own personal productivity every day. You’re the only one who can do it, and you’re the one to benefit from it.

Have you had a boss worth remembering? Leave a comment.

* In case you were wondering: the mythical boss I am referring to is Alex Shapira. Alex has done wonders for many software companies, he’s an executive consultant now.


Like any other tragic comedy, a software startup is not worth much without a good cast of characters. Here, for your reading pleasure, are some of them:

The Entrepreneur

The founding father (or mother). The whole thing was his idea, he got it started from nothing, it is his baby and he will see it to maturity. Naturally, he feels like he owns the place.
Strength: Relentless faith in a bright future
Weakness: Relentless faith in a bright future
In his defense: The whole thing really was his idea

The sales guy

He’s the one reeling in the big deals. He’s slick, he’s smart, he’s the cool kid. He can demo a product that requires five massive production servers on his laptop in mid-flight from a middle seat. He can drink any prospective customer under the table. He appears utterly unperturbed no matter how desperate the situation (perhaps because he knows that there will be plenty of time for him to cry alone, in his hotel room, when there’s no one around). His sales are the only source of future revenue so naturally, he feels like he owns the place.

Strength: Handles rejection very well
Weakness: His geek-cred leaves a lot to be desired
In his defense: Perhaps more than anyone else he’s the one assuming a significant personal financial risk. His base pay is way lower than it would have been in an established company and his commission hinges on an unproven product.

The coder

Super-geek is here to build the most amazing product the world has ever seen. Can solve any problem given sufficient quantities of time, coffee and Fritos (in warm climates may substitute Mountain Dew for coffee). The coder believes that the company starts and ends with the code, all of the other characters are just milling about and basking in the glory of the incredible product. Without a doubt the work-horse in the house, nobody puts in more hours than the coder and nobody has less of a social life outside of work. The code is the company’s main asset so naturally, the coder owns the place.
Strength: Can make the impossible possible
Weakness: Does not understand people, may need gentle reminders concerning personal hygiene
In the coder’s defense: It really is a complicated job and not very many can do it well

The VC folks

While the rest of us are busy with important tasks the VC folks take care of such minutia as, you know, paying the bills. They put their money where our mouths are and if they don’t see that money back with a nice return they will not be pleased. Unless you attend board meetings they will, for the most part, stay out of your way. Feel free to whisper a “thank you” every time you get a paycheck. Oh and also, they have a piece of paper that says they own the place.
Strength: The ability to turn promises into funds
Weakness: Do not seem to maintain a steady core temperature like other mammals
In their defense: Have we mentioned that they pay the bills?


Nobody can really define what their job is, but it is an important job and they do it well. They write collateral and debate the weight and finish of brochure paper. They design, build and staff trade show booths. Sometimes they do market research and even contribute to product design, yet the ambiguity that surrounds their job robs them of any glory. They try to comfort themselves by acting like they own the place.
Strengths: Versatility and flexibility
Weakness: As bright as they may be they can’t shake a bit of an inferiority complex
In their defense: Everyone else has a well-defined job, why can’t Marketing?

Which is your favorite character? Leave a comment.

A tool for every rule

The difference between amateur and professional software development is, I am told, process. Tedious, endless, creativity-stifling, headache-inducing process. A bunch of rules that developers hate and ignore, while development managers bite their lips and try to enforce.

Software development, like many other technical undertakings, requires a combination of creativity and discipline and it is rarely successful unless both are present. You may choose to only hire highly creative and extremely self-disciplined developers. If you do, please let me know how that is working out for you… I’m not saying that these people do not exist, they do, and they are spectacular, but they are few and far between and if you hold out for them your staffing efforts may take longer than expected. For most of us there is a conflict between being creative and being disciplined and we struggle for balance. So there you are with a team of creative developers who are not crazy about rules.

Early in my programming career I was part of a maintenance team. Our job was to fix bugs in the live product while the product development team worked on future releases. I learned a lot about good and bad code, and about how good code can be turned bad by a series of undocumented changes. When it’s new, good code is coherent and consistent, it is readable and it makes sense. But code that has been around the block and back often becomes a jumble of changes made by different people unaware of each other. This degradation can be mitigated by documenting changes. Back in 1993, the rules stated clearly that each change must be documented in the source file’s header. It’s good process, but it was almost never followed. Enforcement (via code reviews) helped some, but the team did not seem to have the degree of self-discipline required to eliminate the problem completely. Then one day the problem was eliminated, not by better enforcement but by the introduction of a tool: a new source control system was put in place that required a comment on each change and inserted the comment into the source header. People still could write a non-comment such as “NA”, but nobody did. Instead, every change was accompanied by a good, descriptive, useful comment.

Process was followed at last. Changes were documented and code integrity was protected. Best of all, our manager didn’t have to harass us to do it anymore; we felt more independent and he had no complaints about the new order of things either. Code reviews, now free from process enforcing micro-management became positive educational experiences allowing us to learn from each other.

The tool (in this case the new source control system) did two things:

1. It provided people with a ‘just-in-time’ reminder to follow process. When you were done making your change and you wanted to push it to integration testing you had to check it into the source control system. That was part of your job, you had to do it anyway. While you were there, the dialog popped up and asked you to put in a comment and you did. The burden of remembering the rule had been lifted from your shoulders.

2. It took away the tedious, mindless task of taking your comment, adding date and time and pasting it into the right place in the headers of several source files. You were left with the meaningful task of writing a paragraph to describe your change, the rest was taken care of automatically.

I’ve seen the same thing happen with other process-related tasks. Collecting timesheets was always a painful and frustrating task until I had a system in place that sent email reminders with a “report your hours” link in the message. Logging hours was never hard, the painful part was remembering to do it. With an automated reminder that puts you a click away from completing your report, the pain is gone and process is followed smoothly and effortlessly. Similarly, testing and verification procedures were a real PITA until issue tracking systems with built-in workflow management came along.

No matter how lacking in discipline you think we developers are, what we really hate is having to remember tasks that are not directly related to coding. We also dislike mindless repetitive tasks that we know should be automated. Micro-management is bad for us and bad for our managers, give us a tool for every rule and process nirvana will follow.

Do rules drive you crazy? What do you do to cope with process? Leave a comment.

I’ve seen the future, brother

Twice before I was granted a crystal-clear glimpse into the future. I claim no credit for it, the circumstances just happened to align that way. Here’s how it went:

Summer 1999

The summer of 1999 was a time to relax and celebrate. The company I was working for was swimming in investors’ cash, we had just delivered version 2.0 of a very exciting and innovative product and we were getting ready to conquer new markets. I was just weeks away from marrying the woman of my dreams, life was good. One afternoon I was taking a break in the CEO’s office. I enjoyed telling him about the wonders of the new release and he was happy to share his adventures in the wonderful world of venture capital.

I was impressed, the numbers from the recent round of funding were excellent. A handsome sum was raised and the company’s valuation was head-spinning. And then there was one more number; apparently the monthly burn rate (in other words: the rate at which we were losing money) has also “risen nicely”. I was puzzled, “you say that as if it is a good thing” I said to my CEO. “But it is a good thing, a higher burn rate will lead to a higher valuation. If you want people to believe that you are worth five billion Dollars you should be spending millions every month” he said. “That does not make any sense” I thought, “losing millions does not make you more valuable, it’s completely backwards!”. And so, for a few seconds I saw the dot com bubble burst right in front of my eyes, months ahead of when it actually happened.

Spring 2001

By the spring of 2001 I was running technical operations for the same company out of Bethesda, Maryland. We were working on a deal with Yahoo and I was in the process of setting up some of our servers in a Yahoo hosting facilitiy in Dallas, Texas. One beautiful spring morning I was making my way to Dallas again from Reagan National airport on an American Airlines jet. It was a northbound takeoff from runway 1 and I was seated at a window seat on the port side of the aircraft. During takeoff, I could see the Virginia highways rushing past my window. Up in front, I could see a patch of blue sky through the open cockpit door.

Then the Pentagon drifted into view to my left and in an instant the air defense person in me (who had seen active reserve duty only two years earlier) was awake and screaming “this is bad, bad, bad!” I knew the drills, “airliner used as a weapon”, “airliner used as cover for hostiles” and I could see the scenario unfold before my eyes: a passenger gets up during takeoff, rushes the cockpit, knocks out the flight deck crew and grabs the yoke to bank hard left and crash us all into the Pentagon. This time it was the 9/11 attacks that I saw, months before they took place.

What you see is one thing, what you do with it is another

Those two amazing moments of clarity allowed me to foresee, almost in detail, the loss of thousands of lives on 9/11 and the loss of untold amounts of dot-bomb dollars. What’s not so impressive is what I did with those moments of clarity: I did absolutely nothing. I was alarmed for a minute, then I looked around; everyone else seemed happy enough with the situation, nobody was screaming that losing money to make yourself look big is stupid, or that cockpit doors should be secured, so I got with the program and kept quiet. I can only hope that if another moment of clarity presents itself I will have the wherewithal to recognize it and the courage to act on it.

When did you experience your moment of clarity? What did you see? What did you do or not do? Leave a comment.

[YouTube video of runway 1 takeoff from DCA, with the Pentagon at 1:00]