Until I spent the better part of 50 hours writing this guide about measuring developer productivity, I had little appreciation for how far we’ve come in the past four years. If you’re an engineering manager, there’s a growing body of data that suggests you can make a blanket increase in engineering efficacy. If you’re a great developer, you now stand to earn your due. These are exciting times to be programming.
Additional topics covered in the (admittedly a bit too epic) guide:
- Better data, better policies. Examples of how measurement can improve retention, morale, and amount of work completed.
- Best tools for measuring developers in 2019. Three tools available to measure developer performance, and how they differ.
- What makes code measurement possible? What changed that made it possible to measure developer output, when it was long assumed impossible?
- On securing developer buy-in. Addresses the first question asked by many companies pursuing developer measurement: how do I get the team on board?
- Transparency always wins on a long enough time horizon. What can be learned from the past about those who embrace and resist increased transparency?
If you manage developers, would love to hear what you think of this effort? Is it relevant to you? Anything that could make it better?
Xmarks (nee Firemarks) has been a reliable companion to address my cross-platform bookmarking needs over the past 10 years. As such, I was saddened to receive word that parent company Lastpass has chosen to discontinue Xmarks as of mid-2018. This sent me on a quest to find the best Xmarks alternative, but the first Google result I was presented (alternative.to) when searching “Xmarks alternatives” contained a spate of services that had precious little relevance to users like myself who simply want the means to save bookmarks in a cross-browser compatible extension, with as few other bells and whistles as possible.
After considerable Googling, here are the top three recommendations I’d present to other Xmarks users being forced to abandon ship:
There’s good reason that this is the most-cited service you’ll find (alongside Xmarks) when Googling terms like “cross browser bookmarks.” It supports all the major browsers/platforms (including my current laptop, running Chromium on Arch Linux). It has been doing this long enough to have a quaint (read: “embarrassingly outdated”) little website that imputes the difficulty of building a business with via bookmarking extension.
Eversync’s web site welcomes you to the 90s
Most importantly, it’s got impeccable ratings. As of March 2018, it maintains a 4.5 star rating on the Chrome app store with about 3,000 ratings. I consider this an incredible feat, given that greater volume of ratings typically drive a product’s average toward the 3 stars.
The biggest gripe I have with Eversync is that its creators package a collection of junk like “Speed Dial.” If you’re a power user with more than 500 bookmarks, Nimbus (who appears to have purchased the extension from Everhelper at some point) will try to charge you $45/year for the extension (plus Nimbus bloat). Woe that it does cost money to maintain a web service.
Ok, technically this isn’t a cross-browser solution. But if you’re like me and you only stray outside of Chrome on occasion (usually to test a page I’m developing on another browser), then the simplest way to sync bookmarks is to simply use the browser’s built-in capability. No extensions to download, no potential that you’re going to have to change managers again when software maker abandons their product.
If you choose this option, you will most likely want to visit your browser’s Settings -> Sync settings and disable the properties that you don’t want or need synced on all of your workstations:
This option “only” manages 4 stars on 300 ratings in the Chrome app store, so it’s a half-step below Eversync in terms of its user reputation. I’m including it here anyways because its home page inspires me to believe that its developers are actively working on it, and thus its functionality may be more likely to improve compared to Eversync. Further, it’s list of features including “Duplicate finder” and “Broken link finder” indicate a level of product dogfooding (creator using own product = creation of features to maximize user satisfaction) that was largely absent in Xmarks. For what it’s worth, the Pro version of Raindrop is also a few bucks cheaper than the Pro version of Eversync ($5/month for Eversync vs $3/month for Raindrop).
If anyone has firsthand experience with Raindrop, I’d be much obliged to learn your satisfaction level in the comment section below.
From the files of “endlessly frustrating problems that Google wouldn’t help me with.”
When I bought my iPhone 7, I couldn’t open links in new tabs in Safari for the first six months I owned it. Whenever I tried to hold a link to open it, Safari would sometimes ignore me, sometimes it would start showing a preview of the destination for that link. Which behavior it chose to do seemed to depend on the whether Mercury was in retrograde.
After much searching, I eventually learned this behavior isn’t a “bug.” According to Apple, it is the “feature” that they have long been marketing as “3d touch.” That is, your iPhone is now paying attention to how hard your touch your Safari links. If you hold a link “hard,” then, instead of popping up the context menu that allows you to open the link in a new tab, you instead get the obnoxious popup preview of the next page.
The solution to being able to pop the standard iOS Safari context menu? Hold on a link in Safari, but do it lightly, tenderly, as if you were stroking a baby dove.
This is what “intuitive UI” has become on the new iPhone.
Turning off 3d touch
If, like me, you find 3d touch to be more of a nuisance than a feature, you can disable it as follows: open Settings, search for “Touch,” and pick what should be the first option in the list:
Then click into the 3d touch option, which can be turned off for a 32% less aggravating mobile experience:
PS bonus fix! Have you had trouble moving your iPhone’s icons around since upgrading to iOS 10.1? Guess why. That’s right, 3d touch. As with the above, to move around app icons on a 3d-touch device, you have to touch them ever so lightly.
I will continue waiting for this to feel like a “feature.”
Over the past 5 years, I’ve probably spent more time in Evernote than most any other productivity app. I’ve created thousands of notes across Evernote’s OS X, iOS and web clients, becoming intimately familiar with the limitations of each. Over the years, I’ve periodically tested alternatives, mostly when Evernote obliterates my changes via merge conflicts with unsynched notes. I had high hopes for Onenote when I gave it a few weeks’ test run recently, but ultimately determined it was even more fatally flawed than Evernote (not to mention how difficult they make it to import past work from Evernote).
A couple days ago I discovered Quip. It is a multiplatform note-taking app that apparently started as a collaboration platform, and was acquired by Salesforce for $750m in 2016. It sports a reasonably effective built-in Evernote importer, plus an impressive suite of free features. Based on my initial impressions, I’m giving it even odds to replace Evernote as my note-taking app of choice. Yet, like Evernote, it too comes with a set of warts.
Given how powerful-yet-flawed both products are, I’m going to attempt to enumerate, from a note-taker’s perspective, where each shines. Hopefully this list (ordered roughly by impact on my everyday productivity) helps others decide which set of drawbacks is most forgivable to them.
- Notes can be displayed in grid while ordered by recency of update or other criteria (Quip only allows user-defined sorting in list view…and only on desktop clients at that. C’mon dudes, allowing notes ordered by recency of update is utterly essential feature)
- Can navigate from note to note with single click, since list of notes remains visible in left pane even when a note is being edited
- Better UI for text formatting: resides permanently at top of each note (in Quip, text formatting options are hidden behind a sometimes temperamental popout in left gutter)
- Support for searching phrases (not just individual words)
- Essentially instant search
- Not as aggressive about trying to spam all my contacts during install (unlike Quip — forcing me to manually uncheck every Google contact during iOS install. Not a great approach to build goodwill with a new user, Quip)
- Notes are saved in real time as they’re created (!). The note’s text is automatically synchronized to all devices that have the note open. Not only does this fix Evernote’s most fatal shortcoming (merge conflicts), it also allows multiple people to edit the same document concurrently, a la Google Docs
- Less janky checkboxes for todo lists. They don’t slow down the note when there are lots of them, they can be copy-pasted without introducing weirdness, and their line gets crossed out when you check off your todo item.
- Spreadsheets with full Excel-like functionality (functions, etc) are natively supported, and can be inserted inline into any note
- Code blocks can be inserted into any note (!)
- Variety of attractive pre-built font themes can be selected and set as default for notes
- Can use across more than two devices for free (Evernote now charges for this functionality)
- Note’s edit history can be viewed without paying monthly fee
- More robust feature set for collaboration (collaboration seems to have been the initial purpose behind Quip’s existence) than Evernote. Can add inline comments to any line of any note.
- More intuitive interface for working with multi-level lists (arrows to move back and forth between list level, rather than needing to use tab+shift)
- Can organize notes into subfolders within a notebook
- No upsells as yet in free version
Both apps allow chatting (because everyone wants to chat in their note taking app…right?), and both allow some degree of collaboration on notes (Quip much stronger here). It’s my goal to use Quip as my primary app for the next couple weeks before I make a final call on the victor. If you have experience with either app, would be delighted to hear your power user tips in the comment section below!
It starts simple. Create a note in Evernote webview. List some fruits. If you’re anything like me, you probably make a couple lists like this per day. All seems well. Later that day, you remember this list, and have a burning need to add “Starfruit” to it. All looks well?
That’s right, my fruit list has split itself in two lists with edits from desktop and mobile. Similar problems persist as I try to use the document.
Unless one is fastidious enough to sync every note when finished, and then manually sync when resuming on another platform, you are liable (at rate inversely proportional to speed of your internet connection) to soon find yourself with something like this:
Every time I switch devices, I must remember “ensure save,” then pre-sync on the new device (and remember to confirm sync finished). Granted, there are a number of trigger points for syncs, but it’s impossible to be sure which, if any, have triggered/completed. One too-fast move, and I’m back in the ghetto of Mergetown.
I usually don’t even know I’ve been losing list items (to duplicated versions), until I notice the note lags in load/save time. And then I realize there are 10 copies of the list in my note, below the (already long) version I was treating as the “real list.”
I sympathize that getting sync to work reliably across disparate platforms is a hard problem . So why not, instead of fully fixing the sync, just spend a few days fixing the merge?
If you could automatically add new lines to both documents without creating duplicate copies, that would be a good start. If you kept the note differences in the area they were found (not bottom of document), that would help. Source control tools have spent 20+ years building great systems to merge two documents. You can probably crib most of your implementation from a plugin/library in that domain? Or at least mark “duplicate documents” so I know I’ll need to hand merge them?
Thanks for considering these improvements. I take the time to write this because I still believe you’re the best solution in your space. If you could just address this fundamental stuff, I’d reap benefits greater than the (sizeable) time it took to create this report.
 Lists featuring fruits.
 Albeit the hard problem you ostensibly built your business to solve
Ugh, just spent an hour traveling from forum to forum trying to figure out why I couldn’t CTRL-C in Rubymine and CTRL-SHIFT-V into terminal. As many forum posters were eager to point out, it is possible to use CTRL-SHIFT-INSERT or middle click to paste into terminal, just not CTRL-SHIFT-V. Unfortunately, those workarounds were not OK since my insert key is in the arctic circle of my keyboard, and I don’t want to touch the mouse.
Luckily, the folks at Rubymine helped me figure out the answer where Google couldn’t.
The problem is that when running Rubymine via OpenJDK, the clilpboard used is not compatible with the clipboard used in terminal. The solution was to run Rubymine with Oracle Java instead. In Rubymine, this is a matter of first installing Oracle Java (I’ll let you Google that one, you have to add a repository) and then adding the following lines to the beginning of your Rubymine startup script:
After that you should be golden. In my hour of Googling I read that many other IDEs (Netbeans in particular) seem to be subject to the same problem with CTRL-SHIFT-V not working. I’d reckon that if these users were to change their application to use Oracle Java it would probably resolve the problem in other IDEs as well.
What is programming perfection? While the qualities that comprise “the most productive” developer will vary to some extent based on industry and role, I believe there are a number of similarities that productive developers tend to share. Understanding these similarities is a key to improving oneself (if oneself == developer) or differentiating from a pool of similar-seeming developer prospects. In my assorted experience (full time jobs in Pascal, C, C++, C#, and now Ruby (damn I sound old)), I have repeatedly observed developers who vary in productivity by a factor of 10x, and I believe it is a worthwhile exercise to try to understand the specifics behind this vast difference.
I do not profess to be kid-tested or mother-approved (let alone academically rigorous or scientifically proven) when it comes to prescribing what exactly these qualities are. But I do have a blog, an opinion, and an eagerness to discuss this topic with others. I realize this is a highly subjective, but that’s part of what makes it fun to try to quantify.
</ OBLIGATORY DISCLAIMER>
As a starting point for discussion, I hereby submit the following chart which approximates the general progression I’ve observed in developers that move from being marginally productive to absurdly productive. It works from the bottom (stuff good junior programmers do) to the top (stuff superstars do).
And here is my rationale, starting from the bottom:
Level 1: Qualities possessed by an effective newbie
Comments on something, somewhere. A starting point for effective code commenting is that you’re not too lazy or obstinate to do it in principle.
Self-confident. Programmers that lack self-confidence can never be extremely effective, because they spend so much time analyzing their own code and questioning their lead about their code. That said, if you’re learning, it’s far better to know what you don’t know than to think you do know when you really don’t. So this one can be a difficult balance, particularly as coders grow in experience and become less willing to ask questions.
Abides by existing conventions. This is something that new coders actually tend to have working in their favor over veterans. On balance, they seem to be more willing to adapt to the conventions of existing code, rather than turning a five person programming project into a spaghetti codebase with five different styles. A codebase with multiple disparate conventions is a subtle and insidious technical debt. Usually programmers starting out are pretty good at avoiding this, even if their reason is simply that they don’t have their own sense for style.
Doesn’t stay stuck on a problem without calling for backup. This ties into the aforementioned danger with being self-confident. This is another area where young programmers tend to do pretty well, while more experienced coders can sometimes let themselves get trapped more frequently. But if you can avoid this when you’re starting, you’re getting off on the right foot.
Level 2: Qualities possessed by an effective intermediate programmer
Understand the significance of a method’s contract. Also known as “writes methods that don’t have unexpected side effects.” This basically means that the developer is good at naming their functions/methods, such that the function/method does not affect the objects that are passed to it in a way that isn’t implied by its name. For example, when I first started coding, I would write functions with names like “inflictDamage(player)” that might reduce a player’s hitpoints, change their AI state, and change the AI state of the enemies around the player. As I became more experienced, I learned that “superfunctions” like this were not only impossible to adapt, but they were very confusing when read by another programmer from their calling point: “I thought it was just supposed to inflict damage, why did it change the AI state of the enemies?”
Figures out missing steps when given tasks. This is a key difference between a developer that is a net asset or liability. Often times, a level 0 developer will appear to be getting a lot done, but their productivity depends on having more advanced programmers that they consult with whenever they confront a non-trivial problem. As a lead developer, I would attempt to move my more junior developers up the hierarchy by asking “Well, what do you think?” or “What information would you need to be able to answer that question?” This was usually followed by a question like, “so what breakpoint could you set in your debugger to be able to get the information you need?” By helping them figure it out themselves, it builds confidence, while implicitly teaching that it is not more efficient to break their teammates’ “mental context” for problems that they have the power to solve themselves.
Consistently focused, on task. OK, this isn’t really one that people “figure out” at level two, so much as it is a quality that usually accounts for a 2x difference in productivity between those that have it and those that don’t. I don’t know how you teach this, though, and I’d estimate that about half of the programmers I worked with at my past jobs ended up spending 25-50% of their day randomly browsing tech sites (justified in their head as “research?”). Woe is our GDP.
Handy with a debugger. Slow: figure out how a complicated system works on paper or in your head. Fast: run a complicated system and see what it does. Slow: carefully consider how to adapt a system to make it do something tricky. Fast: change it, put in a breakpoint, does it work? What variables cause it not to? Caveat: You’ve still got to think about edge cases that didn’t naturally occur in your debugging state.
Level 3: Hey, you’re pretty good!
Thinks through edge cases / good bug spotter. When I worked at my first job, I often felt that programmers should be judged equally on the number of their bugs they fixed and the number of other programmers bugs’ they found. Of course, there are those that will make the case that bug testing is the job of QA. And yes, QA is good. But QA could never be as effective as a developer that naturally has a sense for the edge cases that could affect their code so don’t write those bugs in the first place. And getting QA to try to reproduce intermittent bugs is usually no less work than just examining the code and thinking about what might be broke.
Unwilling to accept anomalous behavior. Good programmers learn that there is a serious cost to “code folklore” — the mysterious behaviors that have been vaguely attributed to a system without understanding the full root cause. Code folklore eventually leads to code paranoia, which can cause inability to refactor, or reluctance to touch that one thing that is so ugly, yet so mysterious and fragile.
Understands foreign code quickly. If this post is supper, here’s the steak. Weak developers need code to be their own to be able to work with it. And proficient coders are proficient because, for the 75% of the time that you are not working in code that you recently wrote, you can figure out what’s going on without hours or days of rumination. More advanced versions of this trait are featured in level four (“Can adapt foreign code quickly”) and level five (“Accepts foreign code as own”). Stay tuned for the thrilling conclusion on what the experts can do with code that isn’t theirs.
Doesn’t write the same code twice. OK, I’ve now burned through at least a thousand words without being a language zealot, so please allow me this brief lapse into why I heart introspective languages: duplicated code is the root of great evil. The obvious drawback of writing the same thing twice is that it took longer to write it, and it will take longer to adapt it. The more sinister implications are that, if the same method is implemented in three different ways, paralysis often sets in and developers become unwilling to consolidate the methods or figure out which is the best one to call. In a worst case scenario, they write their own method, and the spiral into madness is fully underway.
Level 4: You’re one of the best programmers in your company
Comments consistently on code goals/purpose/gotchyas. Bonus points for examples. You notice that I haven’t mentioned code commenting since level one? It is out of my begrudging respect for the Crazed Rubyists who espouse the viewpoint that “good code is self-documenting.” In an expressive language, I will buy that to an extent. But to my mind, there is no question that large systems necessarily have complicated parts, and no matter how brilliantly you implement those parts, the coders that follow you will take longer to assimilate them if the docs are thin. Think about how you feel when you find a plugin or gem that has great documentation. Do you get that satisfying, “I know what’s going on and am going to implement this immediately”-sort of feeling? Or do you get the “Oh God this plugin looks like exactly what I need but it’s going to take four hours to figure out how to use the damned thing.” An extra hour spent by the original programmer to throw you a bone would have saved you (and countless others) that time. Now do you sympathize with their viewpoint that their code is “self-documenting?”
Can adapt foreign code quickly. This is the next level of “understands foreign code quickly.” Not only do you understand it, but you know how to change it without breaking stuff or changing its style unnecessarily. Go get ’em.
Doesn’t write the same concept twice. And this is the next level of “doesn’t write the same code twice.” In a nutshell, this is the superpower that good system architects possess: a knack for seeming patterns and similarities across a system, and knowing how to conceptualize that pattern into a digestible unit that is modular, and thus maintainable.
Level 5: Have I mentioned to you that we’re hiring?
Constant experimenting for incremental gains. The best of the best feel an uncomfortable churn in their stomach if they have to use “find all” to get to a method’s definition. They feel a thrill of victory if they can type in “hpl/s” to open a file in the “hand_picked_lists” directory called “show” (thank you Rubymine!). They don’t settle for a slow development environment, build process, or test suite. They cause trouble if they don’t have the best possible tools to do their job effectively. Each little thing the programming expert does might only increase their overall productivity by 1% or less, but since developer productivity is a bell curve, those last few percent ratchet the expert developer from the 95th to 99th percentile.
Accepts foreign code as their own. OK, I admit that this is a weird thing to put at the top of my pyramid, but it’s so extremely rare and valuable that I figure it will stand as a worthwhile challenge to developers, if nothing else. Whereas good developers understand others’ code and great developers can adapt it, truly extraordinary developers like foreign code just as much as they like their own. In a trivial case, their boss loves them because rather than complaining that code isn’t right, they will make just the right number of revisions to improve what needs to be improved (and, pursuant to level 3, they will have thought through edge cases before changing). In a more interesting example, they might take the time to grok and extensively revise a plugin that most developers would just throw away, because the expert developer has ascertained that it will be 10% faster to make heavy revisions than to rewrite from scratch. In a nutshell, whereas most developers tolerate the imperfections in code that is not their own, experts empathize with how those imperfections came about, and they have a knack for figuring out the shortest path to making the code in question more usable. And as a bonus, they often “just do it” sans the lamentations of their less productive counterparts.
This isn’t to say they won’t revise other people’s crappy code. But they’re just as likely to revise the crappy code of others as they are their own crappy code (hey, it happens). The trick that these experts pull is that they weigh the merits of their own code against the code of others with only one objective: what will get the job done best?
Teach me better
What qualities do you think are shared by the most effective developers? What do you think are the rarest and most desirable qualities to find? Would love to hear from a couple meta-thinkers to compare notes on the similarities you’ve observed amongst your most astoundingly productive.
Alright, I admit it: I didn’t seriously write tests for Bonanzle until a couple months ago. But I had my reasons, and I think they were good ones.
Reason #1 was that I hated everything about fixtures. I hated creating them, I hated updating them every time we migrated, and I hated remembering which fixture corresponded to which record. Factory Girl was the panacea for this woe.
Reason #2 was that it took an eon to run even a single test. When trying to iterate tests and fixes, this meant that I ended up spending my time 10 parts waiting to one part coding. After much digging, I eventually determined that 90% of our test load time was attributable to caching all our classes in advance. Of course, my first inclination was just not not cache classes in our test environment, which actually worked reasonably well to speed tests the hell up, until I started writing integration tests, and found our models getting undefined and unusable over the course of multiple requests. Then, I found the answer:
This line basically says, even if you set config.cache_classes = true, Rails should not try to pre-load all models (which, in our case is more than 100).
Adding this line allows us to cache classes in test (which fixes the integration test problems), while at the same time getting the benefits of a configuration that doesn’t take 2 minutes to load.
(Of course, also key was configuring our test rakefile such that we could run single tests, rather than being obligated to run the entire suite of tests at once. If anyone needs finds this post and doesn’t yet know how to invoke a single test, post a comment and I’ll get unlazy and post the code for that)