Until I spent the better part of 50 hours writing this guide about measuring developer productivity, I had little appreciation for how far we’ve come in the past four years. If you’re an engineering manager, there’s a growing body of data that suggests you can make a blanket increase in engineering efficacy. If you’re a great developer, you now stand to earn your due. These are exciting times to be programming.
Additional topics covered in the (admittedly a bit too epic) guide:
- Better data, better policies. Examples of how measurement can improve retention, morale, and amount of work completed.
- Best tools for measuring developers in 2019. Three tools available to measure developer performance, and how they differ.
- What makes code measurement possible? What changed that made it possible to measure developer output, when it was long assumed impossible?
- On securing developer buy-in. Addresses the first question asked by many companies pursuing developer measurement: how do I get the team on board?
- Transparency always wins on a long enough time horizon. What can be learned from the past about those who embrace and resist increased transparency?
If you manage developers, would love to hear what you think of this effort? Is it relevant to you? Anything that could make it better?
[This post was originally sent to the TBTTY list (4), but I thought others might enjoy it too so it’s reposted here]
My favorite thing this year is February.
Last year, Bonanza started a tradition of swapping Seattle for somewhere tropical every February. We call it “Freedom February.” Its birth came from asking ourselves some pointed questions:
- We’re an Internet company, we’re small-ish (1), why do we have to work in the same place every month?
- How will it impact productivity if we give our team the freedom to work on their own schedule?
- What was that bright, glowy thing that seemed to hang in the sky all day, making things warm and pleasant last Summer? (2) Can we hunt it down?
Curiosity got the better of us, so we booked some plane tickets, a month of lodging in Costa Rica, and we did it. It was more fun than I’d hoped for. There were monkeys.
In response to our first question, we learned that there was really no reason we couldn’t get away with doing this every February. All we need is a connection to the Internet and this business can run itself fine. We were all friends already, but this trip reinforced our friendships and our combined sense of mission. To the second question, we got more code checked in that month than we did the month before or after, in part because of a collective spirit of working nights and weekends, even though we worked less during daylight hours. As a nice kicker, our sales also jumped about 15% that month. There was a lot to like about this idea.
This brings us to February 2014. Even before we departed, it had the makings of a TBTTY email when this victory dropped into our lap:
In their annual survey of more than 12,000 sellers (the biggest/only third-party survey on marketplaces), Bonanza beat out Etsy, eBay and Amazon in whether sellers would recommend it to a friend! We had scored in the top four of this survey during previous years, but this was the first year we won both “Most Recommended” and “Top Score Overall.” Etsy, eBay and Amazon finished 2nd, 3rd and 4th to us in cumulative score. February was off to a fine start.
And then, the trip. This year, since Bonanza has a new tyke in tow (our developer’s criminally adorable daughter, Kira), we decided to visit the north shore of Oahu, where we’d have easy access to modern health care if anything important came up. Here’s Kira enjoying Freedom February 2014:
In terms of Februarys, this year was even better than last year. Bonanza has grown since 2013, so we had more people = more opportunities for epic BBQs = more fun. The availability of our own hot tub this year was a nice touch too.
Enjoying BBQ dinner
A good time was had by all this year. But my favorite part of the trip was seeing how happy my parents were to be in Hawaii. Until I started dragging them around the world with me a couple years ago, they hadn’t been on a vacation since their honeymoon (25+ years ago). My mother in particular has more energy and enthusiasm than any of my 30-something friends, and her happy-go-lucky spirit becomes infectious in a group setting. Here’s her with my pa in front of our house:
Even when I was sitting upstairs with ocean surf drowning out most sound, I could still hear my mom’s laughs echoing above everyone else’s yapping downstairs. It made me realize that I probably have her to thank for much of my willingness to fail (in jokes, in crazy ideas, in business). When you grow up with someone that thinks most everything you say is hiiii-larious, I think there’s a bit of irrational confidence that sticks to you, failures be damned.If any of you run a small-ish company and would like to give this idea a try, DM me and I’ll share what I’ve learned from my two years experience. Generally speaking, it’s less work than I had expected (3). I think “inertia” was the biggest reason it took us a couple years to try it out, but now I think that this is exactly the sort of benefit a startup deserves to compensate for all the hard work we put in throughout the year.
(1) About 15 people, many of which are remote, some part-time. So far the participants have just been folks working out of our Seattle office, but I hope to keep slowly growing the trip as long as we can employ hard workers who GSD with minimal oversight.
(2) I’m not telling. You’re just going to have to try this yourself or wait a few months and hope.
(3) The hardest part is finding the big enough+affordable lodging that’s conducive to work (i.e., has a view of the ocean). Besides that, everything else tends to fall into place, especially if you get your team involved in the planning process.
(4) Still hasn’t been posted to the list though after posting it there yesterday, so not sure exactly how “alive” the list still is
I’ve read two blog posts in the last day that remind me about my favorite part of the Internet, from a consumer perspective: the opportunity for justice to be done to companies whose draconian policies make our lives as consumers worse.
The first post was Joel’s indictment of Circuit City (spot on in my experience… the only store I had ever gone to where I couldn’t make a return, with a receipt in hand for an unopened product purchased less than a week earlier). The second was Lo Toney’s scathing assessment of some random IT company.
As a consumer, there is no more frustrating feeling than the one that comes when you’re dealing with a company that believes it has little to gain by making you happy. In addition to Circuit City, I’ve felt this way dealing with Staples (wouldn’t let me make a totally reasonable return), and Chase+Paypal (pages take 10+ seconds to load on a regular basis). Comcast has made me feel that way quite a bit too, although less so in the last couple months.
What is frustrating as a consumer is even more infuriating as an entrepreneur, because I understand that companies like these often have millions of dollars they could spend to improve service. But when Powers that Be sit in their boardroom to discuss how to dole out the bounty of revenue, they want to find an equation that describes why they should invest in an improved customer experience. When they can’t find one, the question becomes “Who cares if our unreasonable return policy upsets a few thousand people? Our customers number in the mieeeeeellons. ”
But, by faciliating frictionless communication between a huge body of consumers, the Internet has proven to be the great equalizer for these anti-customer companies. And it seems that they are increasingly meeting the justice they deserve. Circuit City is now gone, Comcast has been spending money in hopes to repair its image. Just as telling, companies that do care about their customers, like Costco, Amazon, and Nordstrom have thrived in the “Communication Age” we now live in.
It seems that the cost of leaving your customers frustrated is increasing with every new Facebook, Twitter, or LiveJournal account that gets registered (not to mention Yelp). As an avowed lover of justice, this trend has my vote as one of the best developments of the last 10 years. I hope & believe we’ll continue to see movement toward customer-friendly policies, as the communication pathways afforded by the web lay waste to old-guard companies that still don’t “get” that frustrated customers cost a lot more than that one customer.
Progress continues to lurch forward in fits and starts as we settle on the personnel configuration to lead us to launch. Having watched a handful of similar revelations occur to many of our previous team members, it has dawned on me that the same factors that make Bonanzle so exhilarating for me are the factors that cause others to turn tail and head for the highway. My conclusion is that, until you’ve experienced the atmosphere before, it is easy to over- or under- estimate how difficult it is to be a part of creating something big from scratch. As usual, Paul Graham has insightful observations on the topic, but reading his poetic account of “hard work” makes it sound more romantic than I think it is. For my time, and the time of our future potential applicants, I think it is vital to accurately describe the most important differences between the startup and non-startup company.
I don’t think that work at a startup is most accurately described as “harder” than work at a large company. One of the “hardest” jobs I ever had was keeping my brain busy while I did nothing for 8 hours a day as a web programmer at the University Bookstore. A better point of comparison between small and large company is the degree of chaos and compromise you experience on a daily basis.
Specifically, these here are my five biggest contrasts that I think startle people who haven’t been immersed in a startup before:
1. The roadmap is drawn as you go. Well, technically, the roadmap is drawn at the beginning, but the more time gets spent drawing that original roadmap, the more time was wasted when that everything-you-know-is-wrong moment happens. Startups are about doing, not speculating.
2. Despite the best intentions, things will be broken. Sometimes with no easy solutions. And it will take creativity to work around it.
3. You are beholden to deadlines. No matter what excellent new service pack is available; no matter what important features from the next milestone one would rather work on. Of course, sometimes that excellent service pack absolutely does need to be installed, so you have to figure out the relative degree of necessity.
4. You are beholden to deadlines. Items only get checked off the schedule if each and every team member is 100% productive with their time. Working at Microsoft it might well be weeks before somebody notices you’ve been spinning your wheels over a certain problem. At a startup, spinning your wheels for 3 days will show up on the schedule.
5. You are your manager. And you are your everything else. Even in a company that attempts to create specialized roles, there usually isn’t time to send an email to the manager to get a task clarified, then get ahold of your web designer to create HTML, before finally working on the original bug that had been assigned to you. Instead, each person must often use their confidence (and common sense) to guide them to a sensible solution when a task has not been well-defined (see also item #1).
Look down the list, and there it is: compromise, chaos, compromise, chaos, chaos. Is it intrinsically harder to deal with chaos and compromise than a lack thereof? I doubt it. But it does take a special personality to have the confidence, patience, and foresight to see how the decisions they make on an everyday basis might seem like chaos, but when the dust settles, suddenly something amazing stands where moments ago there was nothing.
That is the payoff that awaits those with the grit to make something big happen.
I think I’ve hit upon a pretty apt analogy for the ebb and flow of getting one’s business rolling. It is the very indicator used by millions of business the world over. It is the stock market.
First of all — and you won’t hear me admitting this again at any point in the near future — a lot of what comprises “success” is stupid luck. I have spent hundreds of hours recruiting our team to this point, and the best people we have all 1) came from different sources 2) did not find out about Bonanzle through any of the numerous postings I’ve made to sites like Jobster and the UW Career Center. People routinely ask me (and I routinely ask other people) how to find the best people for a project, and the generally accepted answer is that nobody’s got a clue. You just keep talking to people and eventually get lucky. The stock market analog is the (fairly common) incident where a lifelong financial analyst is beaten by the S&P 500. Even seasoned analysts can’t generally compete with luck.
Second, no single day is very indicative of the overall trend. I think this is one those principles you hear a lot when talking about entrepreneurialism without really understanding it. It’s often worded as “you should expect a lot of adversity and challenges to overcome,” but when you actually experience these “challenges” (or less nicely: failures) on a daily basis, it is easy to get discouraged and lose track of the overall upward trend. What it feels like is that every time you get traction with a new idea or new recruit or well-executed maneuver, it gets negated by the Looming Unforeseeable Obstacles. But, viewed objectively, a business only needs to have slightly less failures than it has successes to win. In the stock market (and in my stock market, fantasy basketball), the same is true: trend trumps daily blips.
And the correlate to both the first and second principle? That the best you can do for either your business or stock is to put yourself in the best possible position to succeed, cross your fingers, and pray to the law of averages for a break. Oh dearest law of averages always comes to the rescue of the worthy. But eventually.
Is there any literature yet devoted to the subject of “sleep management?” I have heard plenty of unnuanced opinions that lack of sleep causes diminished energy, herpes, death, or what-have-you; but I can not recall having read much literature about how one can step past those generalities and make their sleep schedule work for them.
From what I can tell, there is a pattern of trade-offs that correspond with different sleep frequencies. I’d roughly summarize my own trade-offs as follows:
The major takeaway being that, except in extreme (< 5 hours) cases, it is usually fine to queue up sleep during the weekend and use it during the week. This is the driving principle behind how stuff gets done here at Bonanzle. But it’s astounding to think how much more could get done by those that are gifted by only needing to average 6 hours or less per night. It’s a real shame that science hasn’t come to the rescue with the insta-rejuvinate pill. Sleeping is so old fashioned. Productivity enthusiasts shudder at that wasted 33% of all time every day.
I concluded my regularly scheduled meetings with Karrie Kohlhaas this evening. As always, there was much ado and many great questions raised for me to consider.
But my favorite question of the night goes to the imaginary businessman that Karrie impersonated, who asked, “Well, this looks like a good enough idea I suppose, but let’s be honest: you’re 27, you’ve never done this before, so what the hell do you know about anything?”
She got me. I think I’m cool no matter the line of questioning, but it was a little startling to pretend as though I were being asked that question in real time. In that sort of situation, even the smallest pause before your answer can be interpreted as doubt. If you do pause, the answer you give must answer the original question and reassert your confidence. I think the trick to answering “tough questions” like this is to know they are coming, and when they do, to give a straightforward answer that belies confidence.
When I think about how to go about answering tough questions with confidence, I think back to the first couple seasons of The Apprentice (before I re-learned to ignore Trump). With a few exceptions (ahem, “Street Smart” Chris), I was consistently impressed by how well each of those contestants could quickly and cooly respond to intense personal attacks by each other and Trump himself. As season 3 Alex said, in one of my all-time favorite interviews,
“People know how to argue more or less from their upbringing but the biggest thing being an attorney did was to prepare me to go into the boardroom and not take arguments personally.”
I thus engage the process of assimilating quick logic robot.
Talking to the fabulously intriguing Mr. Nathan Rohm this evening, I found myself referring to my previous venture, Spek, as “a vaccination against quitting.” The more I think about it, the more I believe that is the most simple and accurate description I could give of the experience.
For those not in the know, Spek was the Xbox Live Arcade video game company that some friends and I started to start a little more than a year ago. It began in a flash. Within a week of having the idea to start the company, I had teamed up with two hard-working and dedicated friends to fill the roles of designer and artist. Within another week we had our programmer, and away we went.
By the time the ride came to an end, we had worked on it about six months and had the trappings of a working prototype, complete with an engine and a decent amount of the art assets. We had the game fully designed, and had entered serious talks with Microsoft about publishing the game on XBLA. I believe that, had we not been derailed, we stood at least a 50% chance of getting the game published, and from there, taking the company in whatever direction we wanted.
But we didn’t. Ostensibly, this was due to NCA provisions in my current employment contract (i.e., my lawyer said we could justifiably be sued and our company taken), but at the same time, I was tired by the time the party came to an end. Here’s what I learned:
1. One person can not be the business planner, project manager, and lead programmer while working 40+ hours a week at their day job.
2. Any attempts by one person to be all of the above roles will swallow you whole and make you suck at everything you do, which will in turn kill motivation.
3. Starting a business is learning to embrace ebb and flow. On one day you might win a client. On the next day you can lose a partner. On the day after you can be admired. On the day after that you can be forced to revise the whole business plan.
4. Once you commit to a particular idea, many other good opportunities will present themselves. In the case of Spek, it was the opportunity to be the lead developer on the title that was going to be the highest-budget project our studio had ever taken on. By a factor of two.
5. And despite all of this, your commitment must be absolutely unwavering.
Though I’ve avoided reliving lessons number 1 and 2 with Bonanzle (where I’ve yet to do a lick of programming (I miss it, but Bonanzle’s success to this point would have been completely unattainable any other way)), lessons 3, 4 and 5 have been just as applicable to this project as they had been to Spek. The difference is, this time I saw it coming. The longer I can prove to myself that I have that unwavering focus on what the end result is, the more fortified becomes my ability to lead any type of business, or really, realize any type of goal. What is it that Thomas Edison says on a notecard tacked to my wall?
“The three great essentials to achieve anything worth while are, first, hard work; second, stick-to-itiveness; and third, common sense.”
I knew the words, but now I know the depth of their truth.