Measuring developer productivity in 2019
Until I spent the better part of 50 hours writing this guide about measuring developer productivity, I had little appreciation for how far we’ve come in the past four years. If you’re an engineering manager, there’s a growing body of data that suggests you can make a blanket increase in engineering efficacy. If you’re a great developer, you now stand to earn your due. These are exciting times to be programming.
Additional topics covered in the (admittedly a bit too epic) guide:
- Better data, better policies. Examples of how measurement can improve retention, morale, and amount of work completed.
- Best tools for measuring developers in 2019. Three tools available to measure developer performance, and how they differ.
- What makes code measurement possible? What changed that made it possible to measure developer output, when it was long assumed impossible?
- On securing developer buy-in. Addresses the first question asked by many companies pursuing developer measurement: how do I get the team on board?
- Transparency always wins on a long enough time horizon. What can be learned from the past about those who embrace and resist increased transparency?
If you manage developers, would love to hear what you think of this effort? Is it relevant to you? Anything that could make it better?
Best interview questions to ask candidate interviewees
It’s too hard to find the best interview questions. Even though it’s widely acknowledged that “hiring right” is the make-or-break duty of every growing business, finding the perfect interview question requires sifting through a compendium of books, blogs, and brains.
As a boss, what I want is a broad list of great interview questions, organized by the quality I wish to evaluate (e.g., “hard worker,” “focused,” “team player”). So that’s what I’ve compiled, below.
Read more about the methodology at the bottom of this post, following the questions. tl; dr it involved selectively extracting from all the best books and blog posts I could find in a week of research.
Part one: the most sought-after traits in employees
The most sought-after traits have been broken into four tiers, based on the frequency with which they were mentioned in first-page search results. If you only have an hour for your interview, you probably don’t want to go very in-depth on more than 3 different areas. A popular strategy at bigger companies is to share different areas of evaluation between the team. It helps to ensure that you won’t miss a fatal weakness of the applicant.
If you are going to interview collaboratively, remember that some questions are duplicated between sections. Don’t be that interviewer who repeats a question from the previous interviewer.
Tier One
There is one talent that almost every blog & book regards as critical. Apparently, businesses don’t want to hire a candidate that causes drama and resists management?
- Team player. Are they friendly, agreeable and upbeat?
Tier Two
These qualities were cited by 75% of publications surveyed, which is to say they are important to virtually every company. “Honesty” is one of the hardest qualities to evaluate in an interview context, but I’ve found a handful of questions to penetrate the interview defenses.
- Adaptable. Are they flexible and eager to learn?
- Trustworthy. Shun the crooks.
- Culture fit. Does their personality match the company’s values?
Tier Three
Qualities that were mentioned by roughly half the sources surveyed. Many of the most critical talents Bonanza seeks are located here.
- Communicator. Do they know how to wield language and be tactful?
- Leadership. Does this candidate have the tools to be a leader?
- Ambitious. Every business wants a team that is self-motivated and hard-working
- Focused. Can they possibly eschew Facebook, Twitter, IM, Hacker News, et al. for eight hours?
- Problem-solving. Can they work autonomously, surmounting challenges and making decisions lacking manager intervention?
- Intelligent. Does this candidate have the raw smarts to be proficient at their job?
- Organized. Can they keep track of the details and prioritize among choices?
Tier Four
These qualities weren’t mentioned as consistently across articles surveyed. It is my personal judgement that they are still important to assess, depending on the role being filled.
- Passionate. Are they excitable? Do they love their work?
- Creative. Can they figure out novel solutions to problems?
- Stable. Loyalty and dependability are essential to team morale.
- Invested in us. Have they done the homework on the company they’re appyling to?
- Meat & potatoes. Miscellaneous questions not to forget.
Part 2: My favorite interview questions
This list is the culmination of a week spent combing the internet and popular books for great interview question. I have attempted to choose my favorite versions of the classic questions (e.g., “What’s your weakness?”) such that the version chosen gives the candidate the opportunity to talk in the realm of specific examples.
Do note: some questions apply to more than one category, so are listed more than once. Don’t make a fool of yourself in a collaborative interview by asking the same question twice. Use your communication skills to coordinate questions with your teammates.
Team player, friendly, agreeable, upbeat
- What do you think makes a person likeable?
- How would a co-worker who likes you describe you?
- How would a co-worker who dislikes you describe you?
- What qualities in your co-workers bother you most? Do you appreciate most?
- If I were to ask your current boss what your greatest strength is, what would he or she tell me?
- What do you have Strong Opinions about?
- Describe the boss who would get the very best work from you.
- What changes have you made in working with others to be more effective at work?
- What could you have done to improve your relationship with your worst boss?
- What’s a time you disagreed with a decision that was made by your boss? How did you handle it?
- Tell me about a time you had a conflict with a co-worker, and how you handled it?
- How do you deal with a difficult customer?
- What do you feel an employer owes an employee?
- Give an example of a time you misjudged someone
- When was the last time you were angry?
- Can you describe a time when your work was criticized?
Adaptable, flexible, eager to learn
- What do you think are the most important attributes of successful people? How do you rate yourself in those areas?
- If you could change one thing about yourself by snapping your fingers, what would it be and why? Do you have a plan for
- making that improvement?
- Tell me about competitor (or similar) businesses you kept up with in your past position. What did you glean from them?
- When have you failed? Describe what happened and what you learned from it.
- How do you take advantage of your strengths?
- How do you compensate for your weaknesses?
- What’s the most interesting thing you’ve learned in the past year?
- If you were starting college today, what classes would you take? (Alternative: What would you like to learn more about in the next year?)
- What books are you reading? (Alternative: What industry publications are your favorites?)
- Tell me about the last time you took a risk? Was it the right decision? What did you learn?
- How do you keep up with changes in your field?
- What was the most useful criticism you received?
- How do you measure your own success?
Trustworthy, honest
- Have you worked in a position where you felt you weren’t given enough to do?
- What was the most useful criticism you received?
- If I were to ask your current boss what your greatest weakness was, what would he or she tell me?
- Tell me about the last time you had to make an unpopular decision
- When have you failed? Describe what happened and what you learned from it.
- Tell me about a work incident in which you were totally honest, despite a potential risk or downside.
- What’s your greatest fear about this opportunity?
- Where does your boss think you are now? [Ed: answer doesn’t matter, smell of truthfulness in response does]
- If I looked through your web browser history, what would I learn that isn’t on your resume?
- Tell me about some of the gaps in your employment. How did these come about?
- May I contact your current employer? May I contact your references?
- Why shouldn’t I hire you?
Culture fit, compatible personality
- Describe the boss who would get the very best work from you
- What will make you love coming to work here everyday?
- What type of work environment do you prefer?
- What gets you excited outside of work?
- If I looked through your web browser history, what would I learn that isn’t on your resume?
- Assume that you come to work here. One year from now you go home one Friday evening thinking that accepting this job was the best thing you ever did. What happened during the year for you to feel that way?
- How much time per week do you spend working alone? Do you think it should be more or less?
- How competitive are you?
- Do you have a favorite quote?
- What unique experiences separate you from other candidates?
- What are the most important rewards you expect to get from your career?
- Where do you see yourself in five years?
- Which areas of your work are most often praised?
- Tell me about the funniest role you’ve had on a job/project?
- When you are on vacation, what do you miss most about work?
- How would you describe yourself to a stranger at a party?
- How do you incorporate fun into your day?
Communicator, tactful
See also: “Team player”
- Why are you leaving your current job?
- Have you given presentations to your peers, either at work or in other contexts?
- See that picture frame on the wall? Sell it to me.
- What’s a time you disagreed with a decision that was made by your boss? (Alternate: If you know your boss is 100% wrong about something, how would you handle it?)
- If you feel strongly about an idea, but a person doesn’t agree with you, what tactics would you use to persuade them?
- What’s your greatest fear about this opportunity?
- Describe a time where you put in extra effort to make sure a person with whom you were communicating had really understood your point.
- Describe a situation in which you dealt with a difficult customer
- Why shouldn’t we hire you?
- What do you think we could do better or differently?
Leadership
- How do you motivate people?
- What do you think are the most important attributes of successful leaders?
- Tell me about the last time you had to make an unpopular decision
- Tell me about the last time you fired someone
- How do you decide what tasks to delegate to whom?
- How do you bring other team members into your process? (Alternate: How do you get others on board with your decisions?)
- Did you inaugurate new processes in your previous position? Tell me about them
- Have you been in charge of a budget? How did you balance priorities?
- What’s the most difficult part of being a manager for you?
- What do your reports find most difficult about your management style?
- How do you set expectations with your reports or peers? How do you handle a situation where a report fails to meet expectations (e.g., consistently missing deadlines)?
- Describe a situation where you successfully resolved a conflict with a team member.
- Would you rather be liked or feared?
- Describe a crisis you faced at work. What was your role? How did you resolve it? What were the results?
- Do you have a management philosophy? No.
Ambitious, self-motivated, hard-working
See also: “Invested in us”
- What would your first 30, 60, 90 days look like in this role?
- What extracurricular activities were you involved with in college? Since?
- What side projects have you done? What was the biggest?
- Tell me something that happened at work in the last year that made it a truly great day.
- Have you been promoted in past jobs? How did these promotions come about?
- Describe a situation in your past job where you were proactive? (Alternate: Give me an example from a previous job where you’ve shown initiative)
- When did you first start doing [the job’s requirement]? [Ed note: if they started doing it before required in school, suggests self-starter]
- How do you keep up with changes in your field?
- How competitive are you?
- What are your short range goals and objectives?
- Give me an example of a goal you set in the past and how you went about reaching it?
- Where do you see yourself in five years?
- Do you take work home with you?
- How ambitious are you? No.
Focused, tenacious
- What are your strategies for making the most of your day?
- What do you do when things are slow? (You are lacking for projects)
- What do you do when you’re burned out to rekindle your spark?
- If you’re struggling with an idea, how do you break through and find a solution?
- How do you pass time while waiting for [job specific pause point, like compiling a program]?
- Describe a time where you put in extra effort to improve a business process to be more efficient?
- What percentage of your time is spent on each of your job responsibilities? [Answer matters less than whether they seem to keep track of it]
Problem-solving, self-sufficient, autonomous, decisive
See also: “creative.”
- Describe a time when you were asked to do something you weren’t trained to do. How did you handle it?
- Tell me about a time when you made a decision without all the information you needed.
- Can you tell me about a time when you discovered a more efficient way to do a routine task? Have you done this repeatedly?
- What are some of the Google tricks you use to find answers to your questions?
- How do you decide whether to message your boss when you have a tough problem?
- How many dentists are there in Poland?
- How would you build a toaster for a blind person?
- See that picture frame on the wall? Sell it to me.
- Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
- Can you think of a time where you were forced to take charge when your manager wasn’t available?
- Have you worked in a position where you felt you weren’t given enough to do?
- What was the most important task you’ve ever had?
Intelligent, proficient, competent
- What are you especially good at compared to others?
- Have you been promoted in past jobs? How did these promotions come about?
- What’s the one accomplishment you’re most proud of? Why?
- What is your greatest professional achievement?
- What’s your greatest fear about this opportunity?
- Tell me something that happened at work in the last year that made it a truly great day
- How would you rate yourself in terms of speed, quality, and self-direction? [Ed: Can they answer a multi-part question?]
- In what college courses did you get the worst grades? Why? How do you think it’ll effect your job performance?
- Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
- What do you consider to be the most important aspect of your craft?
- What do you consider to be the most misunderstood aspect of your craft?
- If I asked a co-worker friend what made you most valuable to the team, what would they say?
Organized, detail-oriented
- What would your first 30, 60, or 90 days look like in this role?
- Tell me about the first 60 minutes of your typical day
- What are your strategies for making the most of your day?
- Have you been in charge of a budget? How did you balance priorities?
- What do you do when you begin a project? How does this help the project go successfully?
- What tools do you use to maintain organization? How do you keep lists?
- Tell me about a project where you noticed a small detail that made a big difference?
- What percentage of your time is spent on each of your job responsibilities?
- Describe the top of your desk
- Do you always double-check your work? What’s your system for deciding?
Passionate, excitable
- What excites you about our company?
- What gets you excited outside of work?
- Describe the boss who would get the very best work from you.
- What are the most important rewards you expect to get from your career?
- If you had unlimited money, what would you be doing?
Creative
- Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
- Tell me about the last time you found a unique solution to a long-standing problem.
- Tell me about the last time you figured out how to save your company money? Did they heed your advice?
- Did you inaugurate new processes in your previous position?
- What improvements might you make to our business?
- Where do you draw inspiration for your craft from?
Stable, loyal, dependable
- Why are you leaving current job? Have you brought your concerns with your job to the attention of your boss?
- Is this the first time you’ve thought about leaving? What made you stay before?
- Why have you had so many jobs?
- How do we know you’ll stick around?
- Will your current job be surprised to hear you’re leaving?
- What aspect of our job appeals to you least?
- How does your ideal job stack up to what we’re offering?
- Have you been absent from work more than a few days in any previous position?
Invested in us
- What about this job most excites you?
- What about this company most excites you? (Alternative: What do you know about the company?)
- What do you think we could do better or differently?
- What advantage do we have over our competitors? (Who are our competitors?)
- How does your ideal job stack up to what we’re offering?
- Where do you see yourself in five years?
- What criteria are you using to choose companies you interview with?
Misc, administrative
- Tell me about yourself
- Which areas of your work are most often praised?
- Is there any question I haven’t asked you that I should?
- Salary range expectations?
- When can you start?
- Are you considering other offers?
Methodology
Settling on the definition for what makes a “great” interview question is a subjective affair. There were a couple rules I applied. I wanted questions that lead the candidate to talk about a specific example; questions like “what’s your management style?” (a common question amongst my sources [1]) are too vague to expect worthwhile insights in return. When in doubt, I considered whether I would be annoyed to get asked the question in an interview (e.g., “Give us one word that describes you“). If so, it’s not on the list. 80% of questions evaluated are not on the list.
Once I decided what I was after, I purchased a handful of Amazon’s top-rated interview books, and visited tens of Google’s top ranked pages on interviewing. Most of my sources are linked in the footnotes [1][2].
Building a dataset from my research, I sought to make two determinations:
- What are the top 15 qualities that employers seek in an employee?
- What are the most succinct questions to evaluate those qualities?
The first goal was tricky, because often, sources will refer to the same idea in slightly different ways (e.g., “adaptable,” “flexible,” or “willing to learn”). Some qualities, like “problem-solving” and “creative” feel distinct, but related. In these cases, I added links between the two categories to help the reader find related questions.
Still here? Hello!
I’d wager you would pass the “hard-working” and “tenacious” evaluation phases of the interview. So why not check out the many jobs Bonanza is hiring for at the moment? Everyone loves an interview where they know the questions in advance. 🙂
Also: if you have a favorite interview question that I haven’t listed here, please share it in the comments! Around 10% of the questions above were from my personal collection, indicating there are likely still more good questions lurking out there.
Footnotes
[1] Bibliography:
- Good friends: Alan Talanoa, Landon Schropp, James Spence
- How to answer 31 most common interview Qs (TheMuse.com)
- Twenty Five Great Interview Qs (AbcNews, Bob Rosner)
- 101 Great Answers to Toughest Interview Qs (Ron Fry)
- Winning Answers to 500 Interview Qs (Lavie Margolin and Rachel Miller)
- Personality Traits to get Hired (via CareerBuilder 2014 study)
- Seven qualities looked for in candidates
- What do employers really want?
- Top skills and values employers look for
- 15 traits employers look for
[2] I didn’t start documenting until around half way through my research, so the list isn’t quite comprehensive
Traits & Qualities of Best Developers on a Deserted Island
What is programming perfection? While the qualities that comprise “the most productive” developer will vary to some extent based on industry and role, I believe there are a number of similarities that productive developers tend to share. Understanding these similarities is a key to improving oneself (if oneself == developer) or differentiating from a pool of similar-seeming developer prospects. In my assorted experience (full time jobs in Pascal, C, C++, C#, and now Ruby (damn I sound old)), I have repeatedly observed developers who vary in productivity by a factor of 10x, and I believe it is a worthwhile exercise to try to understand the specifics behind this vast difference.
<OBLIGATORY DISCLAIMER>
I do not profess to be kid-tested or mother-approved (let alone academically rigorous or scientifically proven) when it comes to prescribing what exactly these qualities are. But I do have a blog, an opinion, and an eagerness to discuss this topic with others. I realize this is a highly subjective, but that’s part of what makes it fun to try to quantify.
</ OBLIGATORY DISCLAIMER>
As a starting point for discussion, I hereby submit the following chart which approximates the general progression I’ve observed in developers that move from being marginally productive to absurdly productive. It works from the bottom (stuff good junior programmers do) to the top (stuff superstars do).
And here is my rationale, starting from the bottom:
Level 1: Qualities possessed by an effective newbie
Comments on something, somewhere. A starting point for effective code commenting is that you’re not too lazy or obstinate to do it in principle.
Self-confident. Programmers that lack self-confidence can never be extremely effective, because they spend so much time analyzing their own code and questioning their lead about their code. That said, if you’re learning, it’s far better to know what you don’t know than to think you do know when you really don’t. So this one can be a difficult balance, particularly as coders grow in experience and become less willing to ask questions.
Abides by existing conventions. This is something that new coders actually tend to have working in their favor over veterans. On balance, they seem to be more willing to adapt to the conventions of existing code, rather than turning a five person programming project into a spaghetti codebase with five different styles. A codebase with multiple disparate conventions is a subtle and insidious technical debt. Usually programmers starting out are pretty good at avoiding this, even if their reason is simply that they don’t have their own sense for style.
Doesn’t stay stuck on a problem without calling for backup. This ties into the aforementioned danger with being self-confident. This is another area where young programmers tend to do pretty well, while more experienced coders can sometimes let themselves get trapped more frequently. But if you can avoid this when you’re starting, you’re getting off on the right foot.
Level 2: Qualities possessed by an effective intermediate programmer
Understand the significance of a method’s contract. Also known as “writes methods that don’t have unexpected side effects.” This basically means that the developer is good at naming their functions/methods, such that the function/method does not affect the objects that are passed to it in a way that isn’t implied by its name. For example, when I first started coding, I would write functions with names like “inflictDamage(player)” that might reduce a player’s hitpoints, change their AI state, and change the AI state of the enemies around the player. As I became more experienced, I learned that “superfunctions” like this were not only impossible to adapt, but they were very confusing when read by another programmer from their calling point: “I thought it was just supposed to inflict damage, why did it change the AI state of the enemies?”
Figures out missing steps when given tasks. This is a key difference between a developer that is a net asset or liability. Often times, a level 0 developer will appear to be getting a lot done, but their productivity depends on having more advanced programmers that they consult with whenever they confront a non-trivial problem. As a lead developer, I would attempt to move my more junior developers up the hierarchy by asking “Well, what do you think?” or “What information would you need to be able to answer that question?” This was usually followed by a question like, “so what breakpoint could you set in your debugger to be able to get the information you need?” By helping them figure it out themselves, it builds confidence, while implicitly teaching that it is not more efficient to break their teammates’ “mental context” for problems that they have the power to solve themselves.
Consistently focused, on task. OK, this isn’t really one that people “figure out” at level two, so much as it is a quality that usually accounts for a 2x difference in productivity between those that have it and those that don’t. I don’t know how you teach this, though, and I’d estimate that about half of the programmers I worked with at my past jobs ended up spending 25-50% of their day randomly browsing tech sites (justified in their head as “research?”). Woe is our GDP.
Handy with a debugger. Slow: figure out how a complicated system works on paper or in your head. Fast: run a complicated system and see what it does. Slow: carefully consider how to adapt a system to make it do something tricky. Fast: change it, put in a breakpoint, does it work? What variables cause it not to? Caveat: You’ve still got to think about edge cases that didn’t naturally occur in your debugging state.
Level 3: Hey, you’re pretty good!
Thinks through edge cases / good bug spotter. When I worked at my first job, I often felt that programmers should be judged equally on the number of their bugs they fixed and the number of other programmers bugs’ they found. Of course, there are those that will make the case that bug testing is the job of QA. And yes, QA is good. But QA could never be as effective as a developer that naturally has a sense for the edge cases that could affect their code so don’t write those bugs in the first place. And getting QA to try to reproduce intermittent bugs is usually no less work than just examining the code and thinking about what might be broke.
Unwilling to accept anomalous behavior. Good programmers learn that there is a serious cost to “code folklore” — the mysterious behaviors that have been vaguely attributed to a system without understanding the full root cause. Code folklore eventually leads to code paranoia, which can cause inability to refactor, or reluctance to touch that one thing that is so ugly, yet so mysterious and fragile.
Understands foreign code quickly. If this post is supper, here’s the steak. Weak developers need code to be their own to be able to work with it. And proficient coders are proficient because, for the 75% of the time that you are not working in code that you recently wrote, you can figure out what’s going on without hours or days of rumination. More advanced versions of this trait are featured in level four (“Can adapt foreign code quickly”) and level five (“Accepts foreign code as own”). Stay tuned for the thrilling conclusion on what the experts can do with code that isn’t theirs.
Doesn’t write the same code twice. OK, I’ve now burned through at least a thousand words without being a language zealot, so please allow me this brief lapse into why I heart introspective languages: duplicated code is the root of great evil. The obvious drawback of writing the same thing twice is that it took longer to write it, and it will take longer to adapt it. The more sinister implications are that, if the same method is implemented in three different ways, paralysis often sets in and developers become unwilling to consolidate the methods or figure out which is the best one to call. In a worst case scenario, they write their own method, and the spiral into madness is fully underway.
Level 4: You’re one of the best programmers in your company
Comments consistently on code goals/purpose/gotchyas. Bonus points for examples. You notice that I haven’t mentioned code commenting since level one? It is out of my begrudging respect for the Crazed Rubyists who espouse the viewpoint that “good code is self-documenting.” In an expressive language, I will buy that to an extent. But to my mind, there is no question that large systems necessarily have complicated parts, and no matter how brilliantly you implement those parts, the coders that follow you will take longer to assimilate them if the docs are thin. Think about how you feel when you find a plugin or gem that has great documentation. Do you get that satisfying, “I know what’s going on and am going to implement this immediately”-sort of feeling? Or do you get the “Oh God this plugin looks like exactly what I need but it’s going to take four hours to figure out how to use the damned thing.” An extra hour spent by the original programmer to throw you a bone would have saved you (and countless others) that time. Now do you sympathize with their viewpoint that their code is “self-documenting?”
Can adapt foreign code quickly. This is the next level of “understands foreign code quickly.” Not only do you understand it, but you know how to change it without breaking stuff or changing its style unnecessarily. Go get ’em.
Doesn’t write the same concept twice. And this is the next level of “doesn’t write the same code twice.” In a nutshell, this is the superpower that good system architects possess: a knack for seeming patterns and similarities across a system, and knowing how to conceptualize that pattern into a digestible unit that is modular, and thus maintainable.
Level 5: Have I mentioned to you that we’re hiring?
Constant experimenting for incremental gains. The best of the best feel an uncomfortable churn in their stomach if they have to use “find all” to get to a method’s definition. They feel a thrill of victory if they can type in “hpl/s” to open a file in the “hand_picked_lists” directory called “show” (thank you Rubymine!). They don’t settle for a slow development environment, build process, or test suite. They cause trouble if they don’t have the best possible tools to do their job effectively. Each little thing the programming expert does might only increase their overall productivity by 1% or less, but since developer productivity is a bell curve, those last few percent ratchet the expert developer from the 95th to 99th percentile.
Accepts foreign code as their own. OK, I admit that this is a weird thing to put at the top of my pyramid, but it’s so extremely rare and valuable that I figure it will stand as a worthwhile challenge to developers, if nothing else. Whereas good developers understand others’ code and great developers can adapt it, truly extraordinary developers like foreign code just as much as they like their own. In a trivial case, their boss loves them because rather than complaining that code isn’t right, they will make just the right number of revisions to improve what needs to be improved (and, pursuant to level 3, they will have thought through edge cases before changing). In a more interesting example, they might take the time to grok and extensively revise a plugin that most developers would just throw away, because the expert developer has ascertained that it will be 10% faster to make heavy revisions than to rewrite from scratch. In a nutshell, whereas most developers tolerate the imperfections in code that is not their own, experts empathize with how those imperfections came about, and they have a knack for figuring out the shortest path to making the code in question more usable. And as a bonus, they often “just do it” sans the lamentations of their less productive counterparts.
This isn’t to say they won’t revise other people’s crappy code. But they’re just as likely to revise the crappy code of others as they are their own crappy code (hey, it happens). The trick that these experts pull is that they weigh the merits of their own code against the code of others with only one objective: what will get the job done best?
Teach me better
What qualities do you think are shared by the most effective developers? What do you think are the rarest and most desirable qualities to find? Would love to hear from a couple meta-thinkers to compare notes on the similarities you’ve observed amongst your most astoundingly productive.
Hint for Job Seekers: Wake Up and Write!
Over the last three years I’ve spent at least 6 months hiring, which equates to more than 1,000 applicants reviewed. But even before I had seen our 50th applicant, I was stunned by the applicant apathy that pervaded our job inbox. At first I figured it must be us. When we were initially hiring, it was for the opportunity to work for peanuts at an unproven web startup. Surely this must explain why 95% of the applications we received were a resume accompanied by a generic cover letter, or no cover letter at all.
But now that we have proven our business, with ample resources to bring aboard top tier talent, I am baffled at the scarcity of job seekers who understand the opportunity that the cover letter presents for them to stand out from the other 49 applications I’ll receive today.
Think about it, job seeker. Every day, my inbox is flooded with anywhere from 25-50 applicants. Each of these applicants sends a resume, and each of these resumes detail experience at a bunch of companies I haven’t heard of in job titles that can only hint at what the person might have really done on a day-to-day basis.
If you were me under these circumstances, how would you weed out the applicants that are most interesting? What would wake up you from the torrent of generic cover letters and byzantine job histories?
P-E-R-S-O-N-A-L-I-T-Y.
When I am not paying close attention, it feels like the same guy has been applying for our job repeatedly for months, each time with a slightly different form letter to accompany his or her list of jobs titles.
The applicants that wake me up from this march of sameness are those 5% that demonstrate they have actually taken the 5 minutes to understand what Bonanzle is, what about the company gets them excited, and why they would be a good fit relative to our objectives and specific job description. (IMPORTANT NOTE: Batch-replacing [company name] with “Bonanzle” does not qualify as personalizing)
Interestingly, the applicants for business-related positions we’ve posted in the past tend to do a comparatively phenomenal job at this. If only these business people had design, UI, or programming skills, they would immediately ascend to the top of our “To interview” list. But the actual creators — programmers, designers, and UI experts — just don’t seem to get it. I suppose it could be a chicken-and-egg situation, where the minority of them that do get it are swooped up immediately by companies that crave that glimpse of personality, and the rest of them keep blindly applying to every job on Craigslist without giving a damn.
The other sorely underrepresented aspect to a good application? Decent portfolios. If you’re a designer, take the slight interest I’ve already expressed toward resumes, and cut it in half. Your value is much easier to ascertain by what you’ve done than what you’ve said, and you have the perfect opportunity to show us what you’ve done by creating a modern, user friendly portfolio. On average, I’d estimate I see about one modern, well constructed portfolio of these for every 20 designers that apply. (Personal bias: Flash-based portfolio sites load slow and feel staid; I might be unique in that opinion though)
I see a huge opportunity to awaken and realize how little effort it would take to create an application that shines. You want to be a real overachiever? Why not spend 15 minutes to sign up for an account and browse the site, and incorporate that experience into your cover letter? Amongst more than 50 applicants for our first hire, Mark Dorsey, aka BonanzleMark aka the best hire I’ve made so far, was the SINGLE applicant that spent the 15 minutes required to do this. In more than 500 applications since, I have yet to see it again.
The world is rife with creative ways to get your application noticed. All it takes is 15-30 minutes of your time (including time to personalize the letter) to rise into the 90th percentile. If it’s a job you care about, you’re earning a potentially $100k salary for 30 minutes of work = about $3-4k per minute. I know lawyers that don’t even make that much.
Must Love Chaos and Compromise
Progress continues to lurch forward in fits and starts as we settle on the personnel configuration to lead us to launch. Having watched a handful of similar revelations occur to many of our previous team members, it has dawned on me that the same factors that make Bonanzle so exhilarating for me are the factors that cause others to turn tail and head for the highway. My conclusion is that, until you’ve experienced the atmosphere before, it is easy to over- or under- estimate how difficult it is to be a part of creating something big from scratch. As usual, Paul Graham has insightful observations on the topic, but reading his poetic account of “hard work” makes it sound more romantic than I think it is. For my time, and the time of our future potential applicants, I think it is vital to accurately describe the most important differences between the startup and non-startup company.
I don’t think that work at a startup is most accurately described as “harder” than work at a large company. One of the “hardest” jobs I ever had was keeping my brain busy while I did nothing for 8 hours a day as a web programmer at the University Bookstore. A better point of comparison between small and large company is the degree of chaos and compromise you experience on a daily basis.
Specifically, these here are my five biggest contrasts that I think startle people who haven’t been immersed in a startup before:
1. The roadmap is drawn as you go. Well, technically, the roadmap is drawn at the beginning, but the more time gets spent drawing that original roadmap, the more time was wasted when that everything-you-know-is-wrong moment happens. Startups are about doing, not speculating.
2. Despite the best intentions, things will be broken. Sometimes with no easy solutions. And it will take creativity to work around it.
3. You are beholden to deadlines. No matter what excellent new service pack is available; no matter what important features from the next milestone one would rather work on. Of course, sometimes that excellent service pack absolutely does need to be installed, so you have to figure out the relative degree of necessity.
4. You are beholden to deadlines. Items only get checked off the schedule if each and every team member is 100% productive with their time. Working at Microsoft it might well be weeks before somebody notices you’ve been spinning your wheels over a certain problem. At a startup, spinning your wheels for 3 days will show up on the schedule.
5. You are your manager. And you are your everything else. Even in a company that attempts to create specialized roles, there usually isn’t time to send an email to the manager to get a task clarified, then get ahold of your web designer to create HTML, before finally working on the original bug that had been assigned to you. Instead, each person must often use their confidence (and common sense) to guide them to a sensible solution when a task has not been well-defined (see also item #1).
Look down the list, and there it is: compromise, chaos, compromise, chaos, chaos. Is it intrinsically harder to deal with chaos and compromise than a lack thereof? I doubt it. But it does take a special personality to have the confidence, patience, and foresight to see how the decisions they make on an everyday basis might seem like chaos, but when the dust settles, suddenly something amazing stands where moments ago there was nothing.
That is the payoff that awaits those with the grit to make something big happen.
Buy! Buy! Buy!
I think I’ve hit upon a pretty apt analogy for the ebb and flow of getting one’s business rolling. It is the very indicator used by millions of business the world over. It is the stock market.
First of all — and you won’t hear me admitting this again at any point in the near future — a lot of what comprises “success” is stupid luck. I have spent hundreds of hours recruiting our team to this point, and the best people we have all 1) came from different sources 2) did not find out about Bonanzle through any of the numerous postings I’ve made to sites like Jobster and the UW Career Center. People routinely ask me (and I routinely ask other people) how to find the best people for a project, and the generally accepted answer is that nobody’s got a clue. You just keep talking to people and eventually get lucky. The stock market analog is the (fairly common) incident where a lifelong financial analyst is beaten by the S&P 500. Even seasoned analysts can’t generally compete with luck.
Second, no single day is very indicative of the overall trend. I think this is one those principles you hear a lot when talking about entrepreneurialism without really understanding it. It’s often worded as “you should expect a lot of adversity and challenges to overcome,” but when you actually experience these “challenges” (or less nicely: failures) on a daily basis, it is easy to get discouraged and lose track of the overall upward trend. What it feels like is that every time you get traction with a new idea or new recruit or well-executed maneuver, it gets negated by the Looming Unforeseeable Obstacles. But, viewed objectively, a business only needs to have slightly less failures than it has successes to win. In the stock market (and in my stock market, fantasy basketball), the same is true: trend trumps daily blips.
And the correlate to both the first and second principle? That the best you can do for either your business or stock is to put yourself in the best possible position to succeed, cross your fingers, and pray to the law of averages for a break. Oh dearest law of averages always comes to the rescue of the worthy. But eventually.
The Slow Learning Advocate
What does experience buy an employee? It buys a body of knowledge to draw upon. Reflecting upon the nebulous “things that went wrong,” savvy veterans can provide great value to a company by using the lessons learned from past failures to steer clear of disaster on future projects.
But this is only in the perfect case.
In the real world, most people do not have the capacity to remember specific examples of what they’ve seen. Instead, most leaders that I have worked under develop a “gut feeling” that guides their judgement. If asked to substantiate the gut feeling, these leaders can be evasive, because without being able to remember the examples, one might not be sure what specifically led to that feeling being formed. Instead of the example that brought about the feeling, they will more often cite a recent-but-not-too-relevant example that comes to mind.
As a leader, I have found that I am as guilty of this as the leaders before me. I do not have the ability to remember every situation that has led to my gut feelings, and so I can sometimes find myself unable to substantiate even my “stronger” gut feelings.
Now, the fact that we tend to develop a gut feeling that we can not explain is not in itself a problem. If your gut feelings are correct within the context that you assert them, fine.
But here’s a little secret that I think is the key failing of many highly experienced leaders: their gut feeling is to avoid risk, and while that gut feeling is understandable, it is also stupid.
Think about it: indiscriminately avoiding risk is the natural culmination of years of collecting examples of things that go wrong. First you work on a project that was overly ambitious and got overscheduled, so you remember to keep your scope refined. Then you work on a project that used very elaborate systems that caused you to run out of memory, so you remember to avoid very elaborate systems. On a third project, you try a new 3d algorithm that ends up being thrown out during alpha because of low-level incompatibilities, so you remember to be wary of new algorithms.
Combine all of those lessons into one lesson and what do you get? At the most abstract level, all three of these examples suggest that by taking a risk (large scope, elaborate systems, new algorithm) the project suffered.
This is why I believe that many experienced people learn to avoid risks as a habit. And this is why I make it a point not to learn from my mistakes.
You can not make significant innovations without taking significant risks. Yes, you expose yourself to second-guessing, particularly if your risks have failed before. And yes, it is far more difficult to implement a nuanced risk-aversion plan that considers the specific reasons behind past failures, rather than simply avoid all possible risks that could beget failure.
But as an employer, I believe that the holy grail of maximal ass-kicking job applicants are those that have been through projects before, failed, but are just as willing to push the envelope as they had been during their first project. They need to have learned something from their failures so as not to be derelict, but learning to simply avoid risk is the recipe for perpetual sterility.
Profiteers, Continued
The philosophical side of me woke up a bit disgruntled this morning. It had processed that last blog post during my 6 hours of rest, and it concluded that what I had written ultimately spelled the unraveling of capitalism as we know it. “For you see,” says philosophical side, “if a company’s profit represents only the gap between the contributions of its workers and their actual pay, then any company that isn’t losing money is operating unjustly. And any company that is losing money won’t be operating for long.” Ah, guilty as charged. So how’s about I speak more precisely?
The root of that equation remains true. A company’s profit still represents the difference between the contributions of its workers and their pay. The crucial detail here is that in this definition, a company’s “workers” also includes its top brass — the people making the deals (the other crucial detail is that some of that profit is mitigated by taxes, risk, etc. boring!). In a perfectly just company, the gap between workers’ contributions and compensation represents the value that the company’s principal shareholders bring, because they are ultimately the ones that will reap that profit.
What had induced me to carelessly paint profit as an illustration that high performers were not fairly compensated was that it’s usually true. But only of high performers. This axiom can be witnessed at most any company, but it is perhaps easiest to illustrate at technology companies, if only because that’s where I’ve witnessed it. The gap in productivity between a company’s most productive programmer and least productive programmer is usually around 10x (This is not an exaggeration; if anything, it is an understatement. Unless they are damn good at hiring, a gap of 10x is usually observable by the time a company reaches 20 or so programmers. See Paul Graham’s earlier linked articles for some exlanation of how this is possible).
The gap in salary between a company’s most productive programmer and least productive programmer? Perhaps 2x, at most 3x. Blame methods of evaluation, blame economies of scale, blame the rain, but the reality is that a programmer who is 10x better than another programmer who makes $40,000 is not getting paid $400,000. They’re probably getting paid $70,000, and hopefully, working with a boss who lavishly praises their contributions on a regular basis.
But the bottom line is that $330,000 of that programmer’s productivity is being manifest as profit for the company (or $330,000 of that programmer’s productivity is paying for the 8 overcompensated crappy programmers, if you prefer). This is not a “good deal” for the highly talented individual. But I admittedly mispoke in attributing all profit to inequity. In actuality, some profit is absorbed by future investment, some by past and future risk, and some by the shareholders who will take that profit. The rest? That is inequity.
The Hunt Continues
I still need a bona fide “second in command”-type partner.
How much time to invest on this search and how to go about in on a day-to-day basis? I’ve gotten some interesting opinions on this from different people I’ve asked. When I started a thread about it on Biznik yesterday, I got one opinion (from the esteemed Mr. Pierre Leonard) that I should become a more active presence in community boards to make the people come to me, and one opinion (from the esteemed Mr. Kelly Hobkirk) that I’d be better off to just focus on building the business. Of course, both of these opinions were coming from people who don’t have an intimate knowledge of the business or its current state of affairs. But independently of my situation, they make excellent points to be weighed in the overall argument of how much time to invest, and where to invest it.
However, my prevailing opinion remains that, despite the small army of helpful consultants and rad programmers and able web designers that are already contributing to this project, having an equally-contributing, self-motivated co-founder is a critical path requirement to get where we’re going at the rate I want to get there.
The next frontier of exploration was actually proposed to me by none other than my girlfriend, the lovely KT. And it’s a damn good idea: students. Yeah, they aren’t going to step in with years of experience or a rolodex of industry contacts. But as I’ve found, those assets tend to go hand-in-hand with financial responsibilities and monetary expectations. I think there’s a reason that young co-founders flock together (Bill Gates and Paul Allen, anyone? Or their arch-nemeses, Sergey and Larry?): it’s that they’ve got fire and they’ve got freedom. Of course, more seasoned co-founders can make something work if they’ve got a cache of money and cred piled up between them, but whuppersnappers on step 29 of 800 don’t have such frivolities (observe: blog sarcasm) at hand.
In the next couple days, I’ll post a link on here to my newest student job listing. But in a nutshell, my pitch to students will be the same as my pitch to every bright person employed by a company they don’t have a share in: why do you think you’re getting paid? So few smart people seem to “get” that their salary is inherently less than their value. If their salary equalled their value, then “profit” for their company would not exist. I find it ironic that when most people hear their company is profitable they will rejoice in this fact, perhaps boasting about it to their friends; when in fact, this profit represents the difference between what the sum of the employees are earning and what the company is reaping from those efforts. The bigger the profit, the bigger the discrepancy in what people earn vs. what they produce (and in a perfect world, would be entitled to). So my pitch, quite simply, is that join us and you can be the benefactor of your excellence.
It’s the pitch that ultimately persuaded me to stop thinking like an employee.