I can't remember how I got interested in OKRs: Maybe from reading blogs from the west coast, or perhaps from conversations with my colleague Dan. Anyhow, however I first learned about OKRs, Dan and I had a lot of good conversations. It turned out that there was some interest from the C-Suite. A number of us read Christina Wodtke's book Radical Focus (my review). Her book focuses on a small startup; my company is still a startup (in my view) but has about 400 employees, a complicated business model, and a lot of functional areas that require coordination. Thus, we realized that this was going to be a big project, and we would need a coach. I called Christina, and she recommended Ben Lamorte. So, full disclosure: Ben was our OKR coach, and I can't imagine how we would have gotten the process launched without him. (The book has some sensible advice on whether you should engage outside help: see pp. 156-159.)
So what are OKRs? OKR stands for "Objectives and Key results." The Objective should be inspiring, directional, and represent a critical goal for a period of time, typically a quarter. A Key Result is usually (and should be) a quantifiable representation of how close you are to "done," but with a crucial twist: Accomplishing 100% of your KR should be a stretch - something that would mean you've truly blown it out of the park. More than that: You've won the world series. Getting to 30% of your OKR should be routine. Getting to 50% is great. Getting to 70% is awesome. And 100% . . . well, if you get to 100%, you probably defined your OKR poorly. If you're disciplined, you'll have only one or two O's, and each O will have maybe 3 KRs, tops.
Why do you do all this? To expose your company goals, and, as the Niven/Lamorte subtitle has it, drive focus, alignment, and engagement. My company needed all of these things because we have a tendency to want to do too much. For us, OKRs were and are a device to decide on what is really important, and lasso medium-term concrete goals to strategy.
Having said all that, if you're an early stage startup, you can get pretty far with Wodtke's book. But my company is different: As I've noted, it has a very complicated business model and many functional areas. To get OKRs defined right, you are going to need what the Romans called a vade mecum -- a handbook that will guide you. Objectives and Key Results: Driving Focus, Alignment, and Engagement with OKRs does this. For bigger companies, I recommend their book highly: It will help you accomplish some of the following tasks:
One thing I quite like about this book is that it provides some of the history of OKRs (pp. 1-6). OKRs have a significant archaeological substrate, going back to Peter Drucker's book The Practice of Management (1954). Drucker had an idea of "management by objectives" (MBOs). MBOs were about goal-setting. Niven and Lamorte note that Drucker saw MBOs as "a highly participative event"; if you turn MBOs into a "top-down bureaucratic exercise," you're doing it wrong (p. 3). Thus in the OKR prehistory, we see that there was a significant stress on the idea that MBOs/OKRs encompass thoughtful collaboration around defining the various means to an end; they are not merely or only the end. A second iteration of MBOs came from Andy Grove, the CEO of Intel, in his book High Output Management (1983). He extended the idea of MBOs in a number of ways, and re-formulated: (1) The Objective: an answer to the question, "Where do I want to go?" and (2) Key Results: Answers to the question, "How will I pace myself to see if I am getting there?" (quoted p. 4, my emphasis). Grove chose his words carefully. The objective is not quite the destination alone: It's involved with desire ("want"); if you don't really want it, it's not an objective. And answering the question about how to "pace" is articulated for oneself: It's something of a matter of personal enlightenment: How will I know that I'm making progress?1 Grove also told us that good MBOs help us think about saying no. This nugget from Grove is worth quoting in full:
Here, as elsewhere, we fall victim to our inability to say "no" -- in this case, to too many objectives. We must realize -- and act on the realization -- that if we try to focus on everything, we focus on nothing. A few extremely well-chosen objectives impart a clear message about what we say "yes" to and what we say "no" to -- which is what we must have if any MBO system is to work. (quoted on p. 4)
As I re-read this quotation from Grove, I am struck with the thought that so many of our current ideas about OKRs are implied by Grove's earlier insights. For instance, we are told that for OKRs, we should calculate the chances that we will accomplish our objectives. But this was already in Grove: "objectives should be set at a point high enough so that even if the individual (or organization) pushes himself hard, he will still only have a 50-50 chance of making them" (quoted, p. 5; see also in Grove's book, p. 165). Got that? MBOs should be hard and we should have a relatively low probability of getting to that stretch goal. This is something that is stressed in the current literature, but it was already there in Grove's work. Grove additionally agreed with Drucker that properly done, objectives are (in Niven and Lamorte's words) "a mix of top-down and bottom-up involvement" (p. 4). Having said all that, a significant departure of OKRs from MBOs are that for Grove, MBOs are exercised by individual managers, while in the Niven/Lamorte books, teams take precedence as the OKR unit (though there are OKR "owners").
Is an OKR merely the defining of an objective and the selection of KRs? No way. Properly done, it's a process. Without the process, you won't expose the tacit knowledge and conflicts that have been frustrating your company. The authors provide an example of the process in Chapter 2, "Preparing for your OKR Journey." First off, you are going to need executive sponsorship. Above I noted that my colleague Dan and I read and discussed OKRs long before it was implemented. However, without executive sponsorship, you are never going to get your executive team to define whole-company OKRs, and teams won't be able to align. The authors "give permission" for some things that will vex you: For instance, below the executive team, are OKRs only defined by individual teams? No, the authors say that teams may combine to deliver one set of OKRs, or you may find that you want to define squads that draw membership from different teams to satisfy squad-level OKRs (pp. 36-37). The meat of the process outline is given in a section called "An OKRs Development Plan" (pp. 37-60): Trust me -- the first time, do what the authors say; there are too many ways to screw it up. Once you've gotten through one or two quarters, tinker.
What I want to do for the rest of this review is pick out a few places where you should really listen to Ben; and I'm also going to tease out some points that I find particularly vexing. I'm hoping that these vexatious bits will be addressed in some 2nd edition of the book. I'm going to belabor little OKR details as presented by the authors: For my take on the basic idea, read my review of Wodtke's book, or possibly watch this video from Google Ventures.
I'm calling this "listen to Ben" because it is pretty clear that much of the nitty gritty of the OKR patterns in the book comes from Ben's consulting.
It can't be emphasized enough that the OKR process is not purely top-down. To be sure, the executive team must define the corporate objectives, based on the mission, vision, and strategy of the company. But even then, the executive team should designate some listening time and welcome feedback. Ben writes
OKRs are not a top-down exercise with goals handed down, as if on stone tables, to lower-level units and departments who are expected to dutifully execute, regardless of their opinion. In fact, an important distinction of this model is its focus on inclusivity. It is expected that individuals will have a legitimate say in the objectives and key results chosen, reflecting a mix of top-down and bottom-up goal setting. Having the opportunity to meaningfully contribute to what you will be held accountable for goes a long way in enhancing engagement. And later, when results are tabulated, the change to engage in a meaningful discussion, conducted in a spirit of inquiry, also boosts morale[.] (p. 24; compare pages 154 and 164)2
Sometimes the executive team objectives may be "correct" but they will be worded in a way that is not enticing or doesn't capture the OKR directive to be inspirational, attainable, and, perhaps most critically, doable in a quarter, controllable by the team (at its level), provide business value, and be qualitative (pp. 62-64). I think getting the wording right, and actionable, is a classic case where bottom-up feedback is going to result in OKRs that are actually effective.
I want to quibble a bit with the definition of an objective in this book. Here's how the authors put it: The objective should answer the question, "what do we want to do?" (p. 61). An answer to this question will help, but I want to note that such an answer will be something about "doing" the end result, and may not account for the progress that leads to the accomplishment. Let's go back to Grove. Andy Grove was on to something when he said an objective is an answer to the question "where do I want to go?" (The authors know this, and tell us a bit later that when you define your goal, you should start with a verb; and that the goal should "propel the organization forward in a desired direction" [p. 67]). Still, I think there is something distinctive in Grove's concision, especially it its setting. Grove's passage is worth quoting in full:
The idea behind MBO is extremely simple: If you don't know where you're going, you will not get there. Or, as an old Indian saying puts it, "If you don't know where you're going, any road will get you there."
A successful MBO system needs to answer only two questions:
- Where do I want to go? (The answer provides the objective.)
- How will I pace myself to see if I am getting there? (The answer gives us milestones, or key results).
To illustrate an objective and a key result, consider the following: I want to go to the airport to catch a plane in an hour. That is my objective. I know that I must drive through towns A, B, and C on my way there. My key results become reaching A, B, and C in 10, 20, and 30 minutes respectively. If I have been driving for 10 minutes and haven't yet made it to town A, I know I'm lost. Unless I get off the highway and ask someone for directions, I probably won't make my flight. (pp. 110-111)
I think there is some subtlety in this short parable. For one thing, the objective isn't to get on the plane, or even to drive. It's to get to the airport. Getting on the plane is more of a accomplishment; getting to the airport is more of a destination. Notice as well that in Grove's account, neither the objective nor the key results define precisely how you get there. Maybe you have access to a helicopter; that mode of travel might satisfy the objective and be measured by the key results. Framing the language game of OKRs through destinations and the notion of a journey may make a numerical KR pop out a bit easier because we think of journeys in terms of distance. When Grove gets to the numerical part of the MBO, he talks about "reaching A, B, and C in 10, 20, and 30 minutes" -- clever: he drops the idea that the means must be a car. Additionally, the idea that the KR is a means to test whether one is failing to make progress toward the objective seems to be somewhat lost in the OKR literature.3 Indeed, if you have taken 10 minutes and haven't gotten to town A, maybe you had better hire a helicopter! I think this is a powerful idea, and is very dependent on the idea that progress toward an OKR is about a journey.
If after reading this book and/or Wodtke's, you don't get the concept that a key result should be numerical and measurable, then reread the books. Still, coming up with great KRs is hard; really hard. In one of the case studies, an executive sponsor (co-founder and Chief Product Officer of his company) says: "The biggest surprise was how difficult it was to write good key results" (p. 189). Why? It is telling that the writer here is a Chief Product Officer, because I think product and engineering teams have it especially hard, especially given the KRs described in this book.
I am now going to flog the basics of KRs in order to tell you that there are two kinds of KRs and one practice that may not be good. The basic guidance for a great KR is that it must be . . .
I am going to boil this down a little bit. "Aspirational" means should be hard -- a real stretch -- the number should be "big" based on your understanding of where you are starting from. "Specific" means it should not use words that can be gamed. "Progress-based" means that a score of 10 should mean more progress than a score of 5. "Owned" and "vertically and horizontal aligned" means that you have to have control over it (and if you don't have control, team up with those who can share ownership). And "Drive the right behavior" means that the KR should measure the right thing: Something that, when resulting in a high score, means that you are getting to your objective. I'm going to boil this down to two rules:
- A KR must be a quantitative measurement of something factual that you control and where accomplishing it means you have gone very far in the direction of your objective.
- The KR must be collected automatically or be easy to capture.
Now let's talk about what I will call "degenerate" KRs. Niven and Lamorte allow two KRs that in my opinion simply do not follow the rules.
The first is what they call a "Baseline key result" (pp. 76-78). Here's the example: "since [a] strategy is relatively new, they've never measured [it]" (p. 78); so it must be measured, and that becomes the baseline. They say that this can then tee up a KR for the next quarter. The problem here is is that this quarter's KRs are supposed to drive the current objective ("drive the right behavior"). However, how can you know that this is even the right KR during the quarter, in anticipation of next quarter? I think baselining is a "pre-OKR" behavior because it has to measure with the next objective, and thus can't fit easily under an objective in the current quarter. Thus baselining is logically incompatible with alignment between an objective and its KRs. I strongly advise you that you baseline proposed KRs before the quarter has started and consider it to be a supporting ancillary process -- a pre-requisitve, in other words. Indeed, you might baseline many potential KRs so as to get some context; but they are not yet KRs.
The second is the "Milestone key result": "Occasionally, you may encounter things that despite your best efforts, are very difficult to translate into a a metric key result. This often occurs when you have a seemingly binary outcome at play: We either did it or we didn't" (p. 78-79). First off, Wodtke among others says that this is by definition a "task" and not amenable to support a genuine aspirational / directive objective. Having said that, I acknowledge the difficulty: On product and engineering teams in particular, we oftentimes are expected to deliver something specific in a quarter, and we'd like it to fit OKRs so we can get credit for it. However, you are simply not going to be able to measure incremental progress through this kind of KR; it's too lumpy. Some teams will also say: I have to use a milestone because there's nothing we can measure that "covers" the work. I have four answers for this:
Objective: Make the platform team exciting and a great rotation for everyone in engineering!
How do you measure this? We realized that until our systems are more consistent, taking a rotation on the team would not be exciting; indeed, it would be a horror. So we wrote a paragraph that stressed the idea that automatic rebuilds of systems can contribute to consistency. Additionally, we wanted to make things simpler, and sometimes lower costs can be a proxy for greater simplicity. And both of those ideas make the team more exciting. So our KRs are
- Key result 1: 100% of systems rebuilt automatically by the end of Q2.
- Key result 2: Reduce Amazon bill by 50%.
You're thinking that we cheated. Well, maybe. But one thing I love about these KRs is that I can automate them; we wrote a script to generate a spreadsheet that shows the disposition of all of our cloud systems, including freshness. Similarly, we look at the bill over time. By automating, we don't have to think about the quality of the KR, and we satisfy most of the other goals (specific / owned / progress-based / driving the right behavior).
I would like to draw this out as Norman's Axiom of OKRs: On balance, an automated KR is almost always better than a manual KR or one that requires laborious gathering and discussion.
For what it's worth, milestone KRs are even named as an anti-pattern by the authors: "with no metrics [for a milestone KR], to actually help you keep score, you're defeating the purpose of the OKR process" (p. 155). I'd ban them altogether.
My last caution about KRs: Scoring. Scoring goes back to Google and is repeated in Wodtke and this book. The motivation goes like this: You define a KR such as "acquire 20 new customers," but get only 12 (see p. 80). Is that good or bad? This book proposes that you define levels at the 1.0, 0.7, and 0.3 marks, where the 1.0 is the full monty (20 users), 0.7 means progress that is difficult, though obtainable, and 0.3 means "business as usual" (p. 81). I'm against it. I think you defined a bad KR. Suppose you had said, "acquire on average 7 customers per month" (there are three months in a quarter, so if completely satisfied this KR would generate 21 customers). You set that number on the principle that, say, getting 7 (about 0.3) is routine. I think you're done. The 0.7 is now about doubling routine business and the 20 is the crazy goal. Why are you dividing this up into levels like 0.3, 0.7, and 1.0? Let the KR do the work for you, and eliminate the levels. You're just creating work for yourself.
What if your average, great, and insane progress is more like 10, 100, 1000? You might need to think about an exponential scale. In any case, to me the categorization of progress in 0.3, 0.7, and 1.0 lends itself to unnecessary articulation and potentially the proliferation of milestone key results to fill in the levels.
Now I want to discuss the range of KRs presented in the book. Most of the KRs presented here are about customer acquisition and corporate goals around money. These are not very suitable to product and engineering teams. One category that does work somewhat are key results around conversion, and I'll be talking about that below.
How let's recall that for a good OKR, the objective must be controllable by the team, which I mentioned above; but here's more:
Whoever drafts the objective, whether it's at the corporate, business unit, department, team, or individual level, must be able to control the outcome. . . . when you create a new objective it is with the explicit understanding that you independently possess the means to realize it. (p. 64, emphasis added)
Wow. This is a big deal. This has huge consequences for KRs because it means that KRs have to measure something you control (completely). I don't know about your product organization, but if you think that by changing details in the product you can control outcomes, I'm a little skeptical. You are almost surely going to have to work with others, probably sales, marketing, or the user training group. Somehow you are going to have to collaborate with others to teach your users to use that new feature and/or re-educate your users on the change. And if you don't have control, most likely by collaborating with other groups, your KRs won't capture your progress. For instance, can a product tweak make more money, at least directly? I would say "no," because users are going to do whatever did they before: The power of inertia is incredible, whether your product is outwward or inward.
The other problem with the KR examples is that almost all of the KRs are about customer acquisition (visitors) and revenue. But trust me, product teams don't by themselves have complete control over these aspects of the business.
Additionally, the model for user behavior in many of the KRs in the book is pretty simplistic: Getting more visitors. Believe me, in modern products, many of the most critical feature changes happen after registration or the paywall, and involve changes that are not just measured at the entryway; indeed, for many internal corporate products, users are required to use what you've built, and "conversion" isn't really an issue; changing behavior is.
Here is a pretty complete list of all of the KRs in the book. What I want to stress here is that just about all of the KRs here deal with visitor counts, income. I'm going to strike through every KR that deals with visitors, customer acquisition, and revenue or income. (I am not saying these are bad KRs, just that the book needs radically more KRs that are about rich and complex product development.) For the ones that wouldn't apply to a product or engineering group, I will sometimes add a comment in brackets afterward naming the group that probably owns the OKR. I will bold the ones that I think are plausible for a product or engineering group.
Am I right? Here are the ones I put in bold that seem to work for a product / engineering team; this time I'm going to bold the word that are consequential for builders:
Now I am not surprised that there are only these few KRs that amenable to product/engineering organizations. Why? Because OKRs are really about corporate objectives, which is to say the relationship between the mission, vision, strategy, and tactics. In other words, the main "customer" for OKRs is the executive team.
Here are some lessons I draw from this list of KRs that might guide product/engineering teams.
A warning, though: As is evident from these KRs, some conversions and changes in behavior are going to require collaboration with marketing and/or your training organization. A lot of apps that are behind the firewall can get new awesome features delivered: But does anyone use them? Mere announcements may not work: Leverage your training and support teams and get a shared OKR going.
Alright, that's all I've got . . . well, there's one last thing, which is the relationship between OKRs and agile methodologies. This book mentions "agile" in a number of places (pp. 15, 19, 21). I think it can be done well, but I'm going to leave that for a later time.
Endnotes
1. It is worth noting that in Grove's book, MBOs are tied to self-actualization, i.e., doing one's personal best. See High Output Management, pp. 164-166.
2. On occasion the book does imply top-down work, for instance: "we would expect over-all company-level OKRs to be created, and once they have been widely communicated, business units or teams will create their own OKRs that demonstrate their alignment to overall goals" (p. 34). However, this is in a section about corporate and "business unit" levels, and seems to be about companies with genuinely different lines.
3. Though the authors of this book do say that an OKR is a "dynamic real-time learning aid" (p. 155); they just don't emphasize the relevance of good KRs to mid-journey failures.
comments powered by Disqus