Is Gmail secure from its employees?

The recent essay from Paul Graham, “Don’t Talk to Corp Dev,” reminded me of a question I’ve wondered from time to time over the years: what would stop Google from accessing a startup’s Gmail (or Google Apps) if it were advantageous to their business negotiations? A part of me always cringes at this thought; it’s too paranoid. It goes against their culture. Still, given how bad an internal breach could be, I was curious to see what language Google would use to assure its users that their Gmail was in safer hands than, say, an Uber itinerary.

The first result I came upon in my Googling was a guy (Christopher Nguyen [1]), who on Quora who gives a clear and explicit description of what Google has done to protect Gmail from internal intrusions. The author describes a tough privacy policy that inspired several upvotes, not to mention at least two separate articles from tech sites quoting his answer as proof of “Google’s policy”. Nguyen concludes: “… ultimately, an internal culture of respecting users’ privacy helps keep [us] in check.” It sounded pretty good, like the sort of approach I’d expect Google to take.

My problem with the answer? It’s not from Google. Dropbox is explicit about how employees access data; I find their policies well-thought out and comprehensive. Evernote’s security is moderately explicit [2] that it has more lax policies [3]. Either way, I can evaluate the security of my accounts from an internal employee who might be bored some Wednesday night.

Meanwhile, Google itself is mum on who internally can read my Gmail, and for what purposes. “Gmail security”-related queries yield results ranging from how to secure your account against external entities to how Google keeps us safe from open wifi. They also assure us that they aren’t manually reading every one of our emails to serve us ads (Really?? So do they contract hamsters to pick all those ads? [4]). But nothing in my Googling can locate a Google-authored document that describes their internal Gmail security policy at all.

Does this lack of explicitness matter? I’m torn. Google’s generic privacy policy paints the picture of a company that cares about security. They seemed to get pretty upset when the NSA spied on their users. And yet, I’ve probably had more conversations in Gmail over the past 10 years than I’ve had in the real world. If there were any service I’d like to see a clear, tough, explicit policy on all aspects of data security, it would be Gmail.

I think their policies are probably as good or better than Dropbox, but at this point they are forcing me to assign an awful lot of credibility to that guy on Quora.

 

[1] Christopher Nguyen says he worked at Google between 2003 and 2008.
[2] See the “Customer Account Access” section
[3] “This [admin access] tool allows our customer service and platform administration teams to resolve customer issues.” Which customer support personnel are accessing? What do they access?
[4] Blog sarcasm

Copy Ubuntu (Xubuntu) to new partition with minimum pain

Every time I upgrade computers or hard drives I have to re-figure out how to get my Ubuntu and Xubuntu OS reinstalled in the minimum number of steps. The following is the sequence I used to get the job done here in 2013 on Raring Ringtail.

  1. Create a Live Ubuntu Startup Disc. This has consistently been more painful than I think it ought to be. Last time I tried this, usb-creator-gtk wouldn’t recognize my USB drive. This time, usb-creator-gtk would crash with various Segfaults before it completed copying my iso to my flash drive. Eventually I discovered UNetbootin and all was well. It can grab the OS installs for you, or you can give it the path to an ISO you want to burn. The only trick with it for me (perhaps because of my balky flash drive) was that I had to plug and unplug the flash drive from my computer a few times before UNetbootin (or Xubuntu) would recognize it. As far as OS goes, I put the most recent Ubuntu LTS on my flash drive since I figured it would have the best toolset for modifying partitions.
  2. Boot from the Live Startup Disc. In the menu, pick “Try Ubuntu.” Click the Ubuntu icon (upper left) and search for “Gparted.” Fire it up. If you’re lucky, your new drive is larger than the partition you want to copy. If not, you’ll have to resize the partition you’re copying such that it can fit onto the new disk.
  3. Copy the partition. Since you live booted, neither partition should be mounted, so you should be able to click “Copy” on the old partition and “Paste” in on the new drive (if it has an existing partition, you’ll need to delete that first). Apply change. Wait an hour.
  4. Run some esoteric crap to change UUID of new partition. Start terminal then
    sudo blkid # shows the list of all your drives, observe that the new and old drive have same UUID, that won't do!
    sudo tune2fs -U random /dev/sdXX # XX is the letter+number of the new partition you copied to. This will assign it a new UUID
  5. Open the file manager. The top left choices should be your various mountable drives. You should recognize the UID for one of them as the UID you randomly created in the last step. Click on that drive to mount it (UUID of drive may not show until you click it, that’s fine. Just keep clicking until you find the drive with the new UUID).
  6. Run some more esoteric crap to update your grub and fstab config. Grub is the bootloader. It lives in /boot/grub/grub.cfg on the drive you just mounted (should be /media/UUID/boot/grub/grub.cfg). Do a find-and-replace of all the old UUIDs with your new UUID. Also change the line “menuentry Ubuntu” to “menuentry UBUNTU” so you can be sure that you’re booting into the right grub after step 7. Save that file. Then open /media/UUID/etc/fstab and update the UUID there as well. More detailed (longer-winded) version of these instructions can be found in step 5 here.
  7. Ensure drive is bootable. Still in the Live Ubuntu Trial, go to System -> Administration -> Disk Utility. Pick your new disk and unmount it. Then click “edit partition” and choose the “Bootable” checkbox.
  8. (Optional) Update your MBR. If your new partition is on a new drive, you can just setup your BIOS to try to boot off the new drive first and you should be GTG. Otherwise, you can follow the instructions in Step 6 of the aforementioned link to update your MBR. If you don’t see the upper-cased UBUNTU after you point to the new drive, that’s probably a sign you need to update MBR.

After that, reboot and you should be GTG. Seems like a lot of steps for something that ought to be simple, but the tricky bit is to get your Grub/bootup stuff able to disambiguate between two drives that look identical on a byte-for-byte basis.

Install Balsamiq (and Air) on Ubuntu/Linux Pangolin 64 bit

In my continued effort to ween myself off VMware (slow start speed, many gigs to copy every time I get a new computer) I decided to invest some time this afternoon toward getting one of my essential tools, Balsamiq Mockups, to work on 64-bit Precise Pangolin. I had assumed this would be impossible, so was stunned to find that it’s not only possible, but pretty darned easy. Here are the steps that I cobbled together from a few sources to get it working fast:

Install Adobe Air

wget http://airdownload.adobe.com/air/lin/download/latest/AdobeAIRInstaller.bin
chmod +x AdobeAIRInstaller.bin
sudo ln -s /usr/lib/x86_64-linux-gnu/libgnome-keyring.so.0 /usr/lib/libgnome-keyring.so.0
sudo ln -s /usr/lib/x86_64-linux-gnu/libgnome-keyring.so.0.2.0 /usr/lib/libgnome-keyring.so.0.2.0
sudo ./AdobeAIRInstaller.bin

The middle bit makes some symlinks so that Adobe Air can access gnome keyring (required for install). If you’re running 32-bit Pangolin, those two steps are slightly different:

sudo ln -s /usr/lib/i386-linux-gnu/libgnome-keyring.so.0 /usr/lib/libgnome-keyring.so.0
sudo ln -s /usr/lib/i386-linux-gnu/libgnome-keyring.so.0.2.0 /usr/lib/libgnome-keyring.so.0.2.0

Install Balsamiq

wget http://builds.balsamiq.com/b/mockups-desktop/MockupsForDesktop64bit.deb
sudo dpkg -i MockupsForDesktop64bit.deb

Or if you’re running 32-bit:

wget http://builds.balsamiq.com/b/mockups-desktop/MockupsForDesktop32bit.deb
sudo dpkg -i MockupsForDesktop32bit.deb

Yay For Easy

Thanks to Linux.org for the guidance on installing Adobe Air. The steps on Adobe’s own site are an awful mess.

Ubuntu, Dark Side of Simplicity

The following is my take on how the current passion for UI “simplicity” may be to blame for Unity, the downfall of Ubuntu as I’ve known & loved it. It’s wide-ranging, so I won’t be able to fully substantiate its every point, but if you simply click the area you’d like to know more about, I will wish that you were being shown detailed examples in support of my point. Many of the newly introduced usability issues in Unity are shared by Gnome3, so it seems that now is as good a time as ever for people who care to try to persuade some Linux distro to retain the high standard of developer usability we’ve become accustomed to.

#—————————

Steve Jobs broke my OS, and I don’t even use a Mac. It began about 10 years ago, around the time Jobs had re-joined Apple, and the software industry was smitten with building UI that had every button and link you could need. “If they might need it, why not include it?” seemed to be the effective rationale.

good-ole-days

Windows XP represented the pinnacle of the art; complete with a “Start” button, that, when clicked, exploded into two columns. These columns in turn had menu options like “All Programs” that could themselves balloon out to several more (overlapping) columns. In the case of the “All Programs” specifically, the user was treated to an unordered list of every program they had ever installed (often hundreds). It was so…terrible…yet quick to an advanced user (e.g., those that figured out how to sort it). For new users, well, you could probably figure out some of the basics within a week.*

But soon people began to decide this was arrangement was not ideal. Or even OK. I noticed this in full force with the release of the first iPhone. It was a device that was so stripped down that it didn’t include a feature (secure network access for business email) that would could have increased its user base significantly. It launched anyway, was quickly dubbed the “Jesus Phone,” and has managed to sell a couple gajillion units since.

Gone forever were the days of “the most commercially successful” products were “the most feature-full” ones.

This evolution, which I’d pin as starting around 2006 (first iPhone) has continued expanding its base of believers to present day. Now, in addition to the set of Apple devices, the default aesthetic for web consumer products has become “huge buttons / huge text / nothing that requires reading.”

In the context of the web, I think that this growing obsession with simple UI is usually a great thing. Like it or not, our attention is fragmented and life’s too short to read the manual for a product that I simply want to entertain me (see: Twitter, Instagram).

The problem is when the momentum for this trend** pushes it out to use cases where it makes no sense. Sometimes, a detail-rich interface is required to get the job done efficiently. In the case where an app is used by novice and sophisticated*** users, a balanced curve of “level of detail shown to user” vs “user expertise level” might look something like this

balance_complexity1

That is, this balanced approach would dictate that novice users were exposed only to the most essential 10-20% of all UI complexity. The UI should appear very basic to them. As the user’s needs become more sophisticated, the UI reveals contextual choices and/or configuration menus that accommodate their needs as power users of the product. Novice users are happy because they don’t see the complex pieces. Sophisticated users are happy because they can use it with maximum productivity so long as they’re willing to read a handful of configuration menus.

Products rarely end up this balanced. Windows XP threw the user in the deep end, both in terms of the learning complexity, and the vast sea of choices/links presented to even the notice user. OS X freed users from this soup of links and options, though before they got smart about context sensitivity, it often came at the expense of more clicks.

Ubuntu, pre-Unity, was arguably even worse than XP to the poor novice:

balance-pre-unity

No oversized buttons or contextual UI reveal here. The reason the project thrived was only because the Ubuntu audience is made up largely of users who have advanced expectations for their OS. Many are programmers. They have to juggle IDEs, web browsers, web browser tools, and a smattering of terminal shells. Usually across multiple high resolution monitors, over multiple workspaces. To them, if complexity is the price that must be paid for configurability, then it shall be paid****.

This isn’t the sort of thing a novice will understand, let alone feel comfortable with, but the software did meet the needs of it sophisticated user base.

Then came tablets, the momentum of simplicity, and Ubuntu’s loving ode to it all: Unity.

balance-post-unity

Because this blog needs to get finished before tomorrow morning, I am forced to gloss over a detailed analysis of the functionality lost between pre- and post-Unity versions of Ubuntu. A couple salient examples include: well-integrated dual monitor support, multiple/configurable taskbar support, desktop customization, and ability to keep program menus within each program. For the quantitative-minded amongst you, the compared market share of Ubuntu vs. Mint makes the point more compellingly than my mini-list.

If it’s not already obvious, I love the trend toward simplicity. It was one of the main points of opportunity I saw in starting Bonanza back in 2008 — we sought to build a version of eBay that would be usable to busy people and non-experts. Simplicity continues to be something that I push for as often as anyone on my team, and I think it continues to be a big difference between our platforms.

But I don’t believe that “simplicity” should be the same thing as “dumbed down,” and I wonder if Unity’s pared down featureset could be a result of the Ubuntu designers mistakenly conflating “simple” and “feature-sparse”?

tl; dr

“Simple” and “effective” are closely related for novice users and for simple products. But they can be inversely related when “simple” gets in the way of “configurability,” which begets effectiveness for power users. In the case of Ubuntu, the users are largely geeks who use complex features to maximize productivity. Give the pared-down version of Ubuntu to novice users, but don’t let it rob the majority of your users the functionality they need.*****

Update: Finally inspired to post this to HN after reading Linus’ comments about Gnome3 being a detriment to usability. Given that Gnome3 has traveled a very similar path to Unity in terms of degrading the user experience (for sophisticated users) with its newest release, I am hoping that perhaps a sympathetic designer of Unity or Gnome3 might find this.

Footnotes

* Though as a computer repair guy, I often saw the concept take far longer to sink in. And don’t even get me started on trying to teach my grandparents exactly what a file system was and “what it did”

** Regarding use of “trend” to label the simplicity movement: I mean only that it is influencing all corners of design (web, native apps, mobile, and beyond) — not that it is ephemeral or irrational.

*** “Sophisticated” here means “more advanced,” or “more demanding,” not somuch the “better looking” or “more expensive” connotations of the word.

**** Of course, something doesn’t need to be complex to be configurable. Progressively revealed / contextual UIs can often deliver much of the best from both worlds. But it’s also easy to get implement rather intricate revealing schemes incorrectly and be worse off than if you had simply built a cluttered but static interface.

***** What makes it doubly insulting is that until Ocelot, we could get the functionality we needed by choosing the “Classic Ubuntu” login. What explanation is there to chop a feature that’s already been built…and provided the main lifeline to advanced users after Unity’s release?

Fix it: Ubuntu CTRL-SHIFT-V Won’t Paste Into Terminal

Ugh, just spent an hour traveling from forum to forum trying to figure out why I couldn’t CTRL-C in Rubymine and CTRL-SHIFT-V into terminal. As many forum posters were eager to point out, it is possible to use CTRL-SHIFT-INSERT or middle click to paste into terminal, just not CTRL-SHIFT-V. Unfortunately, those workarounds were not OK since my insert key is in the arctic circle of my keyboard, and I don’t want to touch the mouse.

Luckily, the folks at Rubymine helped me figure out the answer where Google couldn’t.

The problem is that when running Rubymine via OpenJDK, the clilpboard used is not compatible with the clipboard used in terminal. The solution was to run Rubymine with Oracle Java instead. In Rubymine, this is a matter of first installing Oracle Java (I’ll let you Google that one, you have to add a repository) and then adding the following lines to the beginning of your Rubymine startup script:

export JAVA_HOME=/usr/lib/jvm/java-6-sun
export JDK_HOME=$JAVA_HOME
export RUBYMINE_JDK=$JAVA_HOME

After that you should be golden. In my hour of Googling I read that many other IDEs (Netbeans in particular) seem to be subject to the same problem with CTRL-SHIFT-V not working. I’d reckon that if these users were to change their application to use Oracle Java it would probably resolve the problem in other IDEs as well.

Linux Mint Firefox & Chrome :: Remove Search Branding

I’m all for Mint Linux making some bucks via their Chrome and Firefox searchers, but not if it comes at the expense of basic usability. <quickie rant> If I were the Mint maintainers, I’d take a long look at whether it was desirable (let alone essential) that they hijack my CTRL-K functionality and replace standard Google results with their poorly formatted, functionality-impaired substitute.</quickie rant>

Anyhow, if you are here, you’re probably trying to figure out how to remove the Mint branded search from Firefox and/or Chrome. And I’m here to tell you how.

Remove Search Branding from Firefox

  1. Click on the Google search icon in your title bar
  2. Click “Manage Search Engines”
  3. Click the link to “Get more search engines”
  4. Choose a Google, any Google, from Mozilla’s choices. I chose Google SSL, which worked nicely.
  5. After you install your Google SSL (or other version of Mozilla version of Google), click “Manage Search Engines” again, and move your new Google Search to the top of the list.
  6. Voila!

Remove Search Branding from Chrome

There are probably an assortment of ways to accomplish this. I chose to Google “Chrome deb package” which led me to Google’s official distributions of Chrome, which can be found here. After following Google’s instructions to install my Chrome package, all was well (though that meant that I was running “Chrome” rather than “Chromium.” Whatevskis.)

Other than the annoying search stuff, so far Mint Linux seems to be an easy-to-setup iteration on the developer utopia that Ubuntu was built as, before it decided to go the way of the mandatory Unity.

Likes & Dislikes: The Product Edition

Anyone who has followed my blogs over the last couple years knows that I’m a very big fan of the like/dislike list. But I generally try to exclude products from my lists since they don’t have that “essence of life” quality that I’ve strived for in my lists.

But products are important, too. So here you have it: a like/dislike list dedicated to the products I use or have used. I’ll actually split this particular list into four levels of like because I can quantify more precision when it comes to products.

Love

  • VMWare. Being able to seamlessly run Ubuntu & Win7 side-by-side (and have both of them performant) still feels like the coolest thing ever, even after doing it a year.
  • Rubymine. See: favorite Rails tools
  • New Relic. See: favorite Rails tools
  • Quora. Today it is very good. And if they don’t mess it up, next year it is going to be the oracle that has an intelligent answer for everything.
  • Google Search. So easy to take for granted, given how it is woven into every minute of our lives. But can you imagine a world without it? Try using any other search engine for solving programming problems if you want to remember why Google search deserves your love.

Like

  • Windows 7. Terrible for programming Rails, great for UI/usability and productivity.
  • Ubuntu. Great for programming Rails, passable for usability and productivity (Gimp and Openoffice sure as hell ain’t no Photoshop and MS Office)
  • Microsoft Onenote. I have found no better tool for mapping whatever arbitrary structure/idea from my brain into tangible existence.
  • Firefox 4. It has Firebug.
  • Google Chrome. Introduced to the world the realization that we were browsing at half-speed. Low memory footprint.
  • Github. The world of open source programming could accurately be talked about in terms of “BG” and “AG”. It is not an overstatement to say that, along with Git (see below), Github has completely and utterly revolutionized the world’s ability to collaborate on complex projects. The residual impact of that change is hard to grasp.
  • Stackoverflow. Opportunity for them to move to “love” if ever they could build a half-decent search… I still use Google to find answers that I suspect are somewhere on Stackoverflow
  • Amazon. Like Google Search, above, it is such a part of our daily lives that it’s easy to take for granted. But also like Google, imagine shopping for commodities without it. Not to mention their efforts to lead the cloud computing movement.

Deeply Divided

  • Git. The “deeply divided” category exists specifically for git. On one hand, I love what it lets me do (effectively manage source control). On the other hand, I despise how unnecessarily arcane the syntax is, and how the documentation feels like it was written by a seemly unbathed newsgroup

Dislike

  • Rhythmbox. Happy to remove them from this list if they can ever pull off the herculean feat of not considering the word “The” when sorting artists by name. Update: ho, shit! A hero!
  • Dell’s website. Inconsistent, buggy, hard to shop. Do like: Dell’s products, esp the pricing of them)
  • eMusic. Every company has a right to pivot and drop their early adopters — it is business. But the manner in which eMusic made this pivot (with an utter lack of advance notice and concerted effort to mask what was really happening) still rubs me the wrong way when I think about it.

Despise

  • Quicken. Everything feels like it takes 5x longer than necessary.
  • Microsoft Money. Try to use it sometime.
  • Playstation 3. Already hated their constant 45 minute system updates; and days after I wrote that blog post, they give away my CC and password to the Internet. Bang up job, guys.

Free Lock on Single Mysql Table Row

We ran into a problem today where a single row in one of our tables seemed to get stuck in a state where any query that tried to update it would hit our lock wait timeout of 50 seconds. I googled and googled to try to figure out a straightforward way to release this lock, but the closest thing I could find was a big assed Mysql page on table locks that lacked any specific solutions and this Stack Overflow post that suggests fixing a similar problem by dropping the entire table and re-importing it (uh, no thanks).

After some trial and error, I came up with two viable ways to track down and fix this problem.

The first way is to actually look at Mysql’s innodb status by logging into your Mysql server and running

show innodb status\G

This will list any known locks by Mysql and what it’s trying to do about them. In our case, the locked row did not show up in the innodb pool status, so instead I executed

show processlist;

This listed everything that currently had a connection open to Mysql, and how long it’s connection has been open for. In Rails it is a bit hard to spot which connection might be the one to blame, since every Rails instance leaves its connection open whether it is waiting for a transaction to complete or it is doing nothing. In today’s case, I happened to have a good hunch about which of the 50 connections might be the problem one (even though it was listed as being in the “sleep” state…strangely), so I killed it by restarting the server, and all was well. However, I could have also killed it using:

kill process [process id];

If you don’t happen to know which of your processes has done the lock, the only recourse I know would be to restart your servers and see which Mysql processes remain open after the servers have reset their connections. If a process stays connected when it’s parent has left, then it is your enemy, and it must be put down. Hope this methodology helps someone and/or my future self.

Rails 3 Autoload Modules/Classes – 3 Easy Fixes

Judging by the number of queries on Stack Overflow, it is a very common problem for folks using Rails 3 that they are not getting the modules or classes from their lib directory loaded in Rails 3, especially in production. This leads to errors like “NoMethodError,” “No such file to load [file in your lib path],” and “uninitialized constant [thing you’re trying to load].” From the information I canvassed trying to solve the problem this morning, there are three common reasons (and one uncommon reason) that this can happen. I’ll go in order of descending frequency.

1. You haven’t included the lib directory in your autoload path

Rails 3 has been updated such that classes/modules (henceforth, C/M) are lazy loaded from the autoload paths as they are needed. A common problem, especially for the newly upgraded, is that the C/M you are trying to load isn’t in your autoload path. Easy enough fix:

config.autoload_paths += Dir["#{config.root}/lib/**/"]

This will autoload your lib directory and any subdirectories of it. Similarly, if you have model subdirectories, you may need to add

config.autoload_paths += Dir["#{config.root}/app/models/**/"]

To your application.rb file.

2. Your C/M is not named in the way Rails expects

Rails has opinions about what the name of your C/M should be, and if you name it differently, your C/M won’t be found. Specifically, the namespace for your C/M should match the directory structure that leads to it. So if you have a C/M with the filename “search.rb” in the lib/google directory, it’s name should be “Google::Search.” The name of the C/M should match the filename, and the name of the directory (in this case “Google”) should be part of the namespace for your module. See also this.

3. Your application is threadsafe.

This was one that was hard to find on SO or Google, but if you have configured your application to be threadsafe (done by default in production.rb), then the C/Ms you have, even if they exist in your autoload path, will not be defined until you require them in your application. According to the Rails docs, there is apparently something not threadsafe about automatically loading the C/Ms from their corresponding files. To workaround this, you can either require your individual library files before you use them, or comment out config.threadsafe!

4. (Esoteric) You have a model in a subdirectory, where the name of the subdirectory matches the name of the model.

Yes, this is another error we actually had when upgrading from Rails 2 to 3: as of Rails 3.0.4 there is a bug wherein, if you have a directory app/models/item and you have the file “item.rb” in your app/models/item directory, then you will get errors from Rails that your models are undefined (usually it picks the second model in the item subdirectory). We fixed this by just renaming our subdirectories where appropriate. Hopefully in one of the next versions of Rails this will get fixed and no longer be a possible problem.

Had other reasons why C/Ms failed to load for you? Do post below and we can gather up all the possibilities in one tidy location.

Traits & Qualities of Best Developers on a Deserted Island

What is programming perfection? While the qualities that comprise “the most productive” developer will vary to some extent based on industry and role, I believe there are a number of similarities that productive developers tend to share. Understanding these similarities is a key to improving oneself (if oneself == developer) or differentiating from a pool of similar-seeming developer prospects. In my assorted experience (full time jobs in Pascal, C, C++, C#, and now Ruby (damn I sound old)), I have repeatedly observed developers who vary in productivity by a factor of 10x, and I believe it is a worthwhile exercise to try to understand the specifics behind this vast difference.

<OBLIGATORY DISCLAIMER>

I do not profess to be kid-tested or mother-approved (let alone academically rigorous or scientifically proven) when it comes to prescribing what exactly these qualities are. But I do have a blog, an opinion, and an eagerness to discuss this topic with others. I realize this is a highly subjective, but that’s part of what makes it fun to try to quantify.

</ OBLIGATORY DISCLAIMER>

As a starting point for discussion, I hereby submit the following chart which approximates the general progression I’ve observed in developers that move from being marginally productive to absurdly productive. It works from the bottom (stuff good junior programmers do) to the top (stuff superstars do).

Levels of Developer Productivity

And here is my rationale, starting from the bottom:
mario_1

Level 1: Qualities possessed by an effective newbie

Comments on something, somewhere. A starting point for effective code commenting is that you’re not too lazy or obstinate to do it in principle.

Self-confident. Programmers that lack self-confidence can never be extremely effective, because they spend so much time analyzing their own code and questioning their lead about their code. That said, if you’re learning, it’s far better to know what you don’t know than to think you do know when you really don’t. So this one can be a difficult balance, particularly as coders grow in experience and become less willing to ask questions.

Abides by existing conventions. This is something that new coders actually tend to have working in their favor over veterans. On balance, they seem to be more willing to adapt to the conventions of existing code, rather than turning a five person programming project into a spaghetti codebase with five different styles. A codebase with multiple disparate conventions is a subtle and insidious technical debt. Usually programmers starting out are pretty good at avoiding this, even if their reason is simply that they don’t have their own sense for style.

Doesn’t stay stuck on a problem without calling for backup. This ties into the aforementioned danger with being self-confident. This is another area where young programmers tend to do pretty well, while more experienced coders can sometimes let themselves get trapped more frequently. But if you can avoid this when you’re starting, you’re getting off on the right foot.

mario_2

Level 2: Qualities possessed by an effective intermediate programmer

Understand the significance of a method’s contract. Also known as “writes methods that don’t have unexpected side effects.” This basically means that the developer is good at naming their functions/methods, such that the function/method does not affect the objects that are passed to it in a way that isn’t implied by its name. For example, when I first started coding, I would write functions with names like “inflictDamage(player)” that might reduce a player’s hitpoints, change their AI state, and change the AI state of the enemies around the player. As I became more experienced, I learned that “superfunctions” like this were not only impossible to adapt, but they were very confusing when read by another programmer from their calling point: “I thought it was just supposed to inflict damage, why did it change the AI state of the enemies?”

Figures out missing steps when given tasks. This is a key difference between a developer that is a net asset or liability. Often times, a level 0 developer will appear to be getting a lot done, but their productivity depends on having more advanced programmers that they consult with whenever they confront a non-trivial problem. As a lead developer, I would attempt to move my more junior developers up the hierarchy by asking “Well, what do you think?” or “What information would you need to be able to answer that question?” This was usually followed by a question like, “so what breakpoint could you set in your debugger to be able to get the information you need?” By helping them figure it out themselves, it builds confidence, while implicitly teaching that it is not more efficient to break their teammates’ “mental context” for problems that they have the power to solve themselves.

Consistently focused, on task. OK, this isn’t really one that people “figure out” at level two, so much as it is a quality that usually accounts for a 2x difference in productivity between those that have it and those that don’t. I don’t know how you teach this, though, and I’d estimate that about half of the programmers I worked with at my past jobs ended up spending 25-50% of their day randomly browsing tech sites (justified in their head as “research?”). Woe is our GDP.

Handy with a debugger. Slow: figure out how a complicated system works on paper or in your head. Fast: run a complicated system and see what it does. Slow: carefully consider how to adapt a system to make it do something tricky. Fast: change it, put in a breakpoint, does it work? What variables cause it not to? Caveat: You’ve still got to think about edge cases that didn’t naturally occur in your debugging state.

mario_3

Level 3: Hey, you’re pretty good!

Thinks through edge cases / good bug spotter. When I worked at my first job, I often felt that programmers should be judged equally on the number of their bugs they fixed and the number of other programmers bugs’ they found. Of course, there are those that will make the case that bug testing is the job of QA. And yes, QA is good. But QA could never be as effective as a developer that naturally has a sense for the edge cases that could affect their code so don’t write those bugs in the first place. And getting QA to try to reproduce intermittent bugs is usually no less work than just examining the code and thinking about what might be broke.

Unwilling to accept anomalous behavior. Good programmers learn that there is a serious cost to “code folklore” — the mysterious behaviors that have been vaguely attributed to a system without understanding the full root cause. Code folklore eventually leads to code paranoia, which can cause inability to refactor, or reluctance to touch that one thing that is so ugly, yet so mysterious and fragile.

Understands foreign code quickly. If this post is supper, here’s the steak. Weak developers need code to be their own to be able to work with it. And proficient coders are proficient because, for the 75% of the time that you are not working in code that you recently wrote, you can figure out what’s going on without hours or days of rumination. More advanced versions of this trait are featured in level four (“Can adapt foreign code quickly”) and level five (“Accepts foreign code as own”). Stay tuned for the thrilling conclusion on what the experts can do with code that isn’t theirs.

Doesn’t write the same code twice. OK, I’ve now burned through at least a thousand words without being a language zealot, so please allow me this brief lapse into why I heart introspective languages: duplicated code is the root of great evil. The obvious drawback of writing the same thing twice is that it took longer to write it, and it will take longer to adapt it. The more sinister implications are that, if the same method is implemented in three different ways, paralysis often sets in and developers become unwilling to consolidate the methods or figure out which is the best one to call. In a worst case scenario, they write their own method, and the spiral into madness is fully underway.
mario_4

Level 4: You’re one of the best programmers in your company

Comments consistently on code goals/purpose/gotchyas. Bonus points for examples. You notice that I haven’t mentioned code commenting since level one? It is out of my begrudging respect for the Crazed Rubyists who espouse the viewpoint that “good code is self-documenting.” In an expressive language, I will buy that to an extent. But to my mind, there is no question that large systems necessarily have complicated parts, and no matter how brilliantly you implement those parts, the coders that follow you will take longer to assimilate them if the docs are thin. Think about how you feel when you find a plugin or gem that has great documentation. Do you get that satisfying, “I know what’s going on and am going to implement this immediately”-sort of feeling? Or do you get the “Oh God this plugin looks like exactly what I need but it’s going to take four hours to figure out how to use the damned thing.” An extra hour spent by the original programmer to throw you a bone would have saved you (and countless others) that time. Now do you sympathize with their viewpoint that their code is “self-documenting?”

Can adapt foreign code quickly. This is the next level of “understands foreign code quickly.” Not only do you understand it, but you know how to change it without breaking stuff or changing its style unnecessarily. Go get ’em.

Doesn’t write the same concept twice. And this is the next level of “doesn’t write the same code twice.” In a nutshell, this is the superpower that good system architects possess: a knack for seeming patterns and similarities across a system, and knowing how to conceptualize that pattern into a digestible unit that is modular, and thus maintainable.
mario_5

Level 5: Have I mentioned to you that we’re hiring?

Constant experimenting for incremental gains. The best of the best feel an uncomfortable churn in their stomach if they have to use “find all” to get to a method’s definition. They feel a thrill of victory if they can type in “hpl/s” to open a file in the “hand_picked_lists” directory called “show” (thank you Rubymine!). They don’t settle for a slow development environment, build process, or test suite. They cause trouble if they don’t have the best possible tools to do their job effectively. Each little thing the programming expert does might only increase their overall productivity by 1% or less, but since developer productivity is a bell curve, those last few percent ratchet the expert developer from the 95th to 99th percentile.

Accepts foreign code as their own. OK, I admit that this is a weird thing to put at the top of my pyramid, but it’s so extremely rare and valuable that I figure it will stand as a worthwhile challenge to developers, if nothing else. Whereas good developers understand others’ code and great developers can adapt it, truly extraordinary developers like foreign code just as much as they like their own. In a trivial case, their boss loves them because rather than complaining that code isn’t right, they will make just the right number of revisions to improve what needs to be improved (and, pursuant to level 3, they will have thought through edge cases before changing). In a more interesting example, they might take the time to grok and extensively revise a plugin that most developers would just throw away, because the expert developer has ascertained that it will be 10% faster to make heavy revisions than to rewrite from scratch. In a nutshell, whereas most developers tolerate the imperfections in code that is not their own, experts empathize with how those imperfections came about, and they have a knack for figuring out the shortest path to making the code in question more usable. And as a bonus, they often “just do it” sans the lamentations of their less productive counterparts.

This isn’t to say they won’t revise other people’s crappy code. But they’re just as likely to revise the crappy code of others as they are their own crappy code (hey, it happens). The trick that these experts pull is that they weigh the merits of their own code against the code of others with only one objective: what will get the job done best?

Teach me better

What qualities do you think are shared by the most effective developers? What do you think are the rarest and most desirable qualities to find? Would love to hear from a couple meta-thinkers to compare notes on the similarities you’ve observed amongst your most astoundingly productive.