4 Interesting Reads

February 2020

I received positive feedback and encouragement for my previous 4 Interesting Reads post. Here is the next one. Feedback is always welcome, please reach out.

Making Uncommon Knowledge Common by Kevin Kwok

https://kwokchain.com/2019/04/09/making-uncommon-knowledge-common/

Rich Barton has founded three companies valued at over a billion dollars: Expedia, Glassdoor, and Zillow. In the linked article, Kwok explains Barton's central strategy: take public knowledge that is hard to obtain for historical or structural reasons, make it easily accessible, and as a result, own customer demand. When looking for flights or hotels, you start with Expedia. When you're looking to sell your house, your journey begins at Zillow with its Zestimate. This is data that has been converted to content that people want. All this "content" substantially lowers customer acquisition costs (CAC), the bane of a startup's unit economics. The takeaway for product people: cheaply create high quality, personalized content, and use it to drive down CAC, and provide value.

The Limits Of High Speed Rail by Ivan Rivera

https://mappingignorance.org/2020/01/22/the-limits-of-high-speed-rail/

I am fascinated and genuinely interested in the engineering problems faced in building things that are world record holders - the fastest car or train, the tallest building, the largest dam. For this reason, I could not put down Adrian Newey's autobiography How To Build A Car, about his journey in the world of Formula 1 racing, and the nonstop engineering ingenuity to build some of the world's fastest cars. In a similar vein, Rivera's article goes into detail about the engineering required to break the world record for the fastest train — SNCF's TGV V150 (named because 150m/s == 540km/h, the target speed) clocked 574.8 km/h on April 30, 2007. He goes into a lot of fascinating detail around wheel-rail contact points, why diesel-powered trains can't really go beyond 240 km/h, the role of aerodynamics, and the evolution of the pantograph-catenary contact (for electrical power transmission). I firmly believe that we advance the state of all human inventions and knowledge when we push the limits of what we can achieve, and for this reason, studying record-breaking inventions is worthwhile.

The New Business Of AI by Martin Casado and Matt Bornstein

https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/

The stellar people at Andreesen-Horowitz write incredibly insightful commentaries on entire industries: find the thing you're interested in (SaaS? Marketplaces? Crypto?) and follow the relevant a16z partners. Casado and Bornstein's post on the fundamentals of AI startups is packed with insights and hard to compress further. They posit that AI startups have lower gross margins (50-60%) compared to traditional SaaS businesses (60-80%) because of two reasons. 1) High cloud costs due to training, retraining, and inference over large data sets. 2) Human-in-the-loop costs, both for high-quality data (cleaning, labeling) and for checking the AI's results. AI startups have a harder time scaling because edge cases are much harder to deal with - there is a very long tail of edge cases due to the sheer volume of high dimensional space. Finally, they have weaker defensive moats because a lot of model development is being done in academia, or is open-sourced, and data scale effects are weaker than people think. I believe we are still in AI frontier land: new discoveries, new tools, new rules. Hence, understanding the business of AI is as vital as grokking the technology, and makes this article worth the time.

Group Chat: Group Stress by Team Basecamp

https://basecamp.com/guides/group-chat-problems

One of the most significant challenges of today's Slack based tech work environment is that there is no time to think. I don't believe Slack is the root cause; it is somewhere in between an enabler and an accelerant. But, there are too many channels to follow, lots of FOMO about stuff that does not matter, shallow thinking, and fast responses, no preservation of context, and forced ephemerality. The linked article (obviously biased, Basecamp is a competing product), lists out all these problems in greater detail. The central point, which I wholeheartedly agree with, is that "chat attacks attention, and severely hinders deep work." If you're in a Slack based environment and have figured out a way to do deep work without constant interruption, kindly write a post about it and help out the world (and let me know!)

Bonus: The Man In The Arena by Theodore Roosevelt

Doris Kearn Goodwin's book Leadership is a fantastic read about how 4 of America's great leaders (Johnson is controversial) suffered and dealt with significant personal hardship, and how it shaped their presidency and their leadership. Theodore Roosevelt is my favorite leader because of his bias toward action, his values and morality centered around always doing the right thing, his boldness, and his courage. I come back to this quote often, whenever I'm feeling low about my work, or overly criticized, or when an idea that I thought was going to change the world turns out to be a dud, or when I am scared of taking a leap. I hope it helps you as much as it has helped me.

It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.

— Theodore Roosevelt, Citizenship in a Republic

3 Frameworks For Making Complex Decisions

Life is full of complex decisions: capital purchases, such as a car or a house, planning a vacation, choosing a new job, picking a product strategy, prioritizing roadmaps, hiring someone. Complex decisions have several shared traits: the list of options is often extensive, evaluation criteria are ill-defined, the outcomes are hard to predict, input data is unavailable or incomplete. Humans understand large systems by building mental models, which are more straightforward than the reality they represent. Mental models are a great thing: they allow us to make progress without getting bogged down in every little detail. But they also have their flaws. Most notably, human cognitive biases, our failures to think and communicate clearly, lead us to sub-optimal decisions.

Psychologists and behavioral economists have spent a considerable amount of time and mental energy dedicated to understanding the human decision-making process. By systematically understanding our cognitive biases and flaws, smart people have come up with frameworks to counteract their ill effects. Using these frameworks can lead to decisions with better eventual outcomes., how we fail, and how to make decisions that lead to better results. Amos Tversky, Daniel Kahneman, Richard Thaler, Dan Ariely, and Chip Heath have done seminal research in this field — and distilled their ideas into highly readable books. Thinking FastandSlowNudgePredictably Irrational, and Decisive are amongst my favorites. I strongly encourage you to read these to gain a deeper understanding of the field.

In this post, I present three practical frameworks to improve decision- making in different contexts. Frameworks are hard to understand in the abstract. Just reading theory leads to a shallow understanding of how to apply them in practice. To make things more concrete, I use two practical problems that I have solved using some combination of these three frameworks.

  1. How to buy a car: Large capital purchases, such as buying a car or a house, can make for challenging decisions. Some input data for our car-buying determination: I have a growing family with small kids. I have a short commute to work. I care about style and comfort. I don't intend to race my car on a track. I care about the environment and would like something efficient. I have a nominal budget of $50k in mind. A cursory examination of the car market should quickly reveal a broad spectrum of options. For the sake of this post, let's narrow that down to 1) a minivan (Honda Odyssey), 2) a hybrid sedan (Prius), and 3) a pure electric (Tesla Model 3).

  2. How to pick the next feature: We make many complicated decisions at work. Product Managers and organizational leaders often need to decide what part of their product they should focus on given their goals. This strategic choice is perhaps the most impactful, on par with perfect execution. Input data: our app is in the market. It is growing slowly. Churn is higher than we'd like. Research shows that the current set of customers like the app, but don't love it. Should we focus on acquiring new users, increasing lifetime value, or churning fewer users?

How Not To Decide

1. Gut Feeling

Listening to your gut is probably the most common approach to decision making. It's the way we make most decisions - if we did an exhaustive process to decide what to eat for lunch, we'd never get anything done or be able to make any progress. Instinct is your subconscious brain pattern matching inputs with what it has seen in the past and making a quick, shortcut decision. Our brains are fantastic at taking in vast amounts of data and making gestalt decisions; don't fight your instincts.

However, when it comes to highly complex decisions, the very brain that helps us make rapid decisions and move forward with life, deludes us into making bad decisions. Our fast decision-making process is often known as the "reptilian brain" or "System 1 thinking". It is the reason we survived on the savannah: when we thought we saw a lion, our brain didn't take its time working through whether it was a bird, or a blade of grass, or a zebra. It told us to climb the tree first. Our deep-thinking, thoughtful cousins were pruned out of the family tree by the lion. These instant reactions, fight-or-flight instincts, all the shortcuts our brains use, can show up as cognitive biases in decision-making.

Let's use the car buying example. Imagine we walked into the local Toyota dealership. It's a boiling hot day in the middle of summer. Salespeople are extremely busy, overworked, and slightly rude. They give us the keys to a car that's been baking in the sun. We test drive it, hate it, and pass summary judgment: it's a pile of rubbish. The car is way too hot, takes forever to cool down, drives like a sloth. Additionally, we're unhappy about not being treated like royalty and don't want to buy from that dealer anyway — hard pass.

We have attributed the rudeness of a particular salesperson to not just the entire dealership, but all the dealers of this specific car manufacturer. This mistake is called a fundamental attribution error. We have attributed the car's inability to cool down instantly to a manufacturing flaw. We have ignored the base rate: all vehicles sitting in the sun on that day were hot and would take time to cool down. As a result of these biases, we may have discarded a perfectly reasonable option thanks to our instant decision making brain.

2. The Giant Spreadsheet

I love spreadsheets. They allow me to organize my life (and I love organization) and view things at various levels of detail. It is very tempting to distill every decision to some formula and take the flawed human out of the loop. The formula can be straightforward: weighted sums seem to do the trick. Every decision now becomes so precise, so mathematically elegant. Don't like the outcome? You must have gotten the inputs or weights wrong.

Let's visit our product feature prioritization decision. We could build features that target acquisition, LTV, or churn. Each row has a cost and an impact estimate. Any Product Manager worth their salt will come up with a table of priorities, each of these features as rows, and drop columns to show potential impact. Some complex mathematical jiu-jitsu comes next, and the potential impact column has numbers and color coding from red to green. We must pick the greenest feature because our matrix just told us so!

This approach is problematic because it reduces humans to automatons and throws out all intuition. Moreover, it overly simplifies the model by diminishing highly complex information into a number. In the made-up example above, working on notifications seems to win over everything else, using the scheme I've put in. But that the model itself is biased - cost and impact estimates might be completely bogus, our gut might tell us that focusing on acquiring new users is more important, or that the cost of doing notifications is probably higher.

Intuition is essential — our brains are pattern-matching against past experiences and predicting the future. Numerical models create a false sense of precision and delude us into trusting the models. Our minds are excellent at translating vast amounts of information into decisions, and we should trust them while finding ways to correct their shortcomings.

Decision Frameworks

The next section outlines the three decision frameworks that I have used in some shape or form. None of these frameworks are mine - I have merely adapted them for my purposes and found them to be applicable and relevant.

Framework 1: Reducing Dimensionality

The credit for this idea goes to my friend and colleague Josh Williams. The principles are easy to understand and apply on the fly, require little formal work, and help break through a decision making logjam.

Complex decisions are often challenging because they contain an overwhelming number of dimensions. Decomposing the problem results in a large number of smaller choices along each dimension. However, dimensions are not orthogonal — changes in one affect another. Trying to optimize all dimensions at the same time quickly gets overwhelming.

Take the car-buying example: we need to make individual decisions about passenger capacity, gas mileage, styling, manufacturer, safety features, cargo hauling, maintenance, buy vs. lease, and so on. In the example above, we need a car that can carry five humans, is efficient, stylish, safe, easy to maintain, and costs less than 50k. A Porsche 911 is stylish and safe, but doesn't cost less than 50k or carry five humans. A minivan fits most of the requirements but is on the lowest end of the style spectrum. A Prius is in the "meh" range on most things but does excellent on efficiency. The perfect car simply doesn't exist. What do we do?

A good approach in such circumstances is to reduce dimensionality. If you magically cared only about your budget and passenger capacity, the answer would become much more apparent. We can reduce dimensionality in 3 ways:

  1. Aggressively ignore dimensions that you don't care about. In the car example, we could stop caring about maintenance. Maintenance plans are straight forward. Almost every major manufacturer has a good policy. Let's get rid of that completely.

  2. Create "threshold" dimensions that you care about up to a certain point, but not beyond. For example, safety matters to my family, with our small children. But beyond a specific safety rating, any car is sufficiently safe, and we don't need to optimize any further.

  3. Establish trade budgets. This is not dimension reduction per se, but helpful in understanding the relationships between different things. For example, if we care more about efficiency than style, and getting a high gas mileage is worth twice as much as having a sexier car. This approach gives us a rough calculator to prioritize the dimensions we genuinely care about.

The beauty of this framework is that you can quickly sort through the dimensions that matter and devalue or completely discard the ones that don't. Moreover, when we end up with a few real choices at the end of the process, we are assured that all of them satisfy our constraints and would make us happy. Beyond this point, all decisions are good decisions.

Framework 2: Mediating Assessments Protocol (MAP)

This approach is from an excellent article by Daniel Kahneman, Dan Lovallo, and Olivier Sibony. If this summary piques your interest, I encourage you to read the article in full. It is clear, easily understandable, and practical. If you're at a tech company, such as Google or Facebook, and are using a structured interviewing process, you're already using MAP without knowing about it.

Remember the "giant spreadsheet" approach to making decisions? The problem with that approach was that it threw out all human intuition. What if we kept an element of intuition in the mix, but had a way to neutralize a variety of cognitive biases? This is the central idea behind the MAP framework proposed by Kahneman and team.

Let's revisit the feature prioritization problem. We need to make a decision on which feature to build next. There is a trap here — we can easily mislead ourselves into believing that we are following a structured process by sitting through presentations about each option, evaluated in its entirety, with pros and cons, followed by a decision making or voting meeting. This method is subject to precisely the same biases - confirmation bias for things you like, and recency bias for the last option presented. It is essentially the equivalent of a holistic gut call.

Here is the MAP alternative:

  1. Agree upfront on what the goals are. To continue with our example, let's say the objectives are 1) increase the number of daily users, 2) improve the performance of the app, and 3) reduce our operational costs.

  2. One presentation per goal. This method allows us to compare all proposals, on a particular dimension, instead of looking at all the aspects of one proposal. If we have a scoring rubric, we can score proposals per goal at this stage. These assessments are called mediating assessments.

  3. A final evaluation of all proposals, while looking at the mediating assessments. Note: we are not merely taking a weighted average of the intermediate scores. Instead, we are using our judgment at this juncture while keeping all the data in front of our eyes.

The changes seem subtle, but the impact can be profound. The best proof of this approach is in the use of a structured interviewing process to evaluate candidates. If you have interviewed at modern tech companies, like Facebook, Google, or most modern startups, you have experienced this. Instead of having each interviewer simply provide an overall score, the interview process involves a series of mediating assessments.

In structured interviewing processes, each person interviews and makes a judgment about one area of competency - coding, system design, communication, people management, etc. Interviewers score candidates per dimension. The hiring committee looks at all the intermediate scores and then determines an overall rating. This approach is different from each interviewer judging the candidate in all of the different areas and giving one overall score. Structured interviewing is the norm in almost all tech companies. Studies on personnel selection have conclusively shown that using such approaches to interviewing leads to more accurate long term outcomes.

Framework 3: WRAP

This framework is a summary of the WRAP process outlined in the Heath Brothers' fabulous book Decisive. I strongly encourage you to read the book, as well as use the summaryresources on their website (free registration required.)

The WRAP framework focuses on avoiding or overcoming cognitive biases that creep into all human thinking. It is easy to understand and practical to apply. Each maxim can be used independently toward decision making; apply a few or all.

1. Widen the frame

Let's go back to the car-buying example. We are trying to choose between a minivan or a Prius. This problem statement implies a particular frame: we have to decide between A and B.

However, the car is a means to an end - commuting to work, transporting children to school, picking up groceries, or traveling for leisure. In our choice, did we consider solving the more significant problem using some other means? Do we need a car at all? Could we use an electric bike to commute? Or Instacart for all groceries? How would that change our set of options?

A narrow frame is a common decision-making trap. It focuses our thinking on available options, instead of opening our minds to all possibilities, some of which may solve the problem in unique or non-traditional ways.

A classic sign of this trap is the "whether or not" question. When you hear your friend ask you "whether or not they should quit their job" or "whether or not they should build a feature" or "whether or not they should buy an iPad," you should smell a trap. One way out of this trap is removing the option you are leaning toward and making that a non-option. What if you absolutely could not quit your job or buy the iPad? What would you do then?

2. Reality Test Your Assumptions

When we survey the set of available options, we build models in our head of how those options are going to work out. These models get tested when they meet reality, and usually don't survive. We try to improve our models by finding evidence that supports or disproves the model. However, because of confirmation bias, we are much more likely to seek validating proof, rather than the contrary, or disconfirming evidence.

One way to get around this pernicious problem is to look for opposing or disconfirming evidence. Imagine we are in love with a particular feature. Instead of looking for reasons to support our instinct, look for the holes in our reasoning. Why could this feature fail or underperform?

How do other similar features perform? This line of questioning helps us determine the base rate. If most similar features underperform (low base rate), it is unlikely that this particular one is going to be the breakout.

Looking for disconfirming evidence can be difficult, especially when we're already heavily biased toward pursuing a particular path. One trick is to do a joint "premortem" exercise. Get together in a room, and imagine that you're six months into the future. The feature has been built and launched and isn't doing well. What went wrong?

Another approach to reality testing assumptions is to dip a toe in without diving in all the way. In the car buying example, we could rent a minivan for a week, followed by renting another car for a week, to test out what it would feel like living with that car. The cost of a mistake (perhaps you hate the way the minivan drives or turns out that the sedan is entirely too small for your family) is tiny compared to buying the car and discovering you made a mistake.

3. Attain Distance

One of the most striking passages in Andy Grove's book "Only The Paranoid Survive" is about Intel's decision to pivot from making computer memory to making microprocessors. Intel started as a memory company - and they were the world leader in manufacturing memory chips in the late 70s through the early 80s. The microprocessor business was niche, dwarfed by the massive memory business. However, the memory business was seeing enormous pressure from Japanese manufacturers and steadily losing margin. Pivoting the company from their roots, to go all-in on microprocessors was an incredibly difficult decision. Grove described how they finally did it:

— via Google Book Search

For complicated psychological reasons, we seem to make clearer decisions when we are deciding for others instead of ourselves. One of the most effective techniques for attaining distance is to ask:

"What would you tell your best friend to do in the same situation?" — Personal Context

"If you were let go and we hired someone else, what would they do in the same situation?" — Professional Context

4. Prepare To Be Wrong

We typically overestimate the impact of any particular decision. In reality, most decisions are reversible, or at least have escape hatches that are less catastrophic than we initially believe them to be. Jeff Bezos summarizes the concept of reversibility:

Some decisions are consequential and irreversible or nearly irreversible – one-way doors – and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. If you walk through and don't like what you see on the other side, you can't get back to where you were before. We can call these Type 1 decisions. But most decisions aren't like that – they are changeable, reversible – they're two-way doors. If you've made a suboptimal Type 2 decision, you don't have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high judgment individuals or small groups.

— Jeff Bezos, Amazon Annual Shareholder Letter

What if we made a wrong car-buying decision? We own a car that we don't like, which we need to sell subsequently, and buy another car. There is a quantifiable dollar cost and some hassle in selling and buying cars and filing paperwork. But that's it. With that understanding, we no longer fear making a decision, knowing that the cost of reversing that decision is not life-altering.

In Conclusion

Life is full of decisions. In the majority of cases, our instinct is a great decision-maker. However, when faced with highly complex decisions, the evolutionary processes that helped us survive the lions on the savannah can mislead us into making poor, often irrational choices. Using frameworks to make such complex decisions allows us to counter some of those cognitive biases and make good long term choices.


4 Interesting Reads

Featuring works by Clayton Christensen, Steven Sinofsky, Simon Wardley, and Erik Bernhardsson.

A different post this week. Instead of the usual dose of problems and frameworks, here are four articles I came across in the past few weeks, along with summaries of why they're worth reading.

How Will You Measure Your Life by Clayton Christensen

https://hbr.org/2010/07/how-will-you-measure-your-life

Prof. Christensen passed away last week. He was a giant in the business world, with his theory of how low-end disruption can topple the biggest, most well-funded companies, explained in The Innovator's Dilemma. He also gave us the Jobs To Be Done framework, in his more recent book, Competing Against Luck.

The above article is a break from business and explains his ideas on how to measure life. It is a beautiful and sobering read.

  1. Have a purpose and follow a strategy to achieve that purpose

  2. Allocate time wisely, knowing that intimate and loving relationships with family and friends are the most potent and enduring sources of happiness.

  3. Create a culture, at work, and home, of doing things that are hard and learning what works.

  4. Never compromise your principles. "It's easier to hold to your principles 100% of the time than it is to hold to them 98% of the time."

  5. Remembering to stay humble, even if you're the smartest person in the room.

Functional vs. Unit Organizations by Steven Sinofsky

https://medium.learningbyshipping.com/functional-versus-unit-organizations-6b82bfbaa57

Sinofsky's post is an essential read for any leader planning an organization's structure. This long post includes the history of Sinofsky's organization of Office, followed by Windows. He goes into a lot of detail on the pros and cons of functional and unit organizations. Functional organizations are organized by discipline (eg. engineering, product, design, research), with functional leaders all reporting to the CEO. Unit organizations are led by general managers, with smaller functional teams reporting to them. Lots of unit organizations constitute the company. Neither are correct per se, and the pendulum swings between the two. Sinofsky also points out his three golden rules:

  1. Don't ship the org chart (or accept that you will always ship the org chart, and design the org to match what you'd like to ship)

  2. Know the problem the re-org is solving.

  3. Ship schedule is everything. (bonus: ship dates are dates. "January" or "Q3" are not ship dates. January 15th is.)

Pioneers, Settlers, and Town Planners by Simon Wardley

https://blog.gardeviance.org/2015/03/on-pioneers-settlers-town-planners-and.html

Wardley argues that different phases of innovation need different types of people and thinking.

  1. Pioneers. They can explore never-before discovered concepts, the uncharted land. They show you wonder, but they fail a lot. Half the time, the thing doesn't work correctly. You wouldn't trust what they build. They create 'crazy' ideas.

  2. Settlers. They can turn the half baked thing into something useful for a broader audience. They build trust. They build understanding. They make the possible future happen. They turn the prototype into a product, make it manufacturable, listen to customers, and turn it profitable.

  3. Town Planners. They can take something and industrialize it taking advantage of economies of scale. You trust what they build. They find ways to make things faster, better, smaller, more efficient, more economical, and good enough.

The article was an interesting way to think about the different mindsets, people, and teams, that work well on products in various phases of development.

How To Hire Smarter Than The Market by Erik Bernhardsson

https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the-market-a-toy-model.html

With the help of a toy statistical model, the author argues that there is an arbitrage opportunity in hiring engineers. When companies collectively place a hefty premium on some trait (e.g., graduate of a top tier engineering school), they ignore a large pool of candidates that another company can go after. He lists several such traditionally undervalued traits: candidates who didn't get a CS degree, or who have gaps on their resume, or who are not confident in interviews, who don't have experience with your exact tech stack, the list goes on.

This article made me think, rather than being immediately actionable. I don't know how to efficiently screen and interview candidates without the help of some traditional markers. I do agree with the hypothesis that we are collectively ignoring a lot of great talented people out there.

Selfish Note

If you liked this sort of post, or if you hated it, I'd love to hear from you. I spend a lot of time reading articles and books about leadership, engineering, product management, and technology. I would love to share more summaries and pointers if they are of interest. Thank you for being a reader.

On Execution

Thanks to Ben Liebald, Lily Zhang, and Maher Saba for ideas and feedback.

"Our customers love our product and are hungry for more. We have a solid strategy. But we seem to be moving very slowly. Our team is great, but we can't seem to execute."

If this struck a chord, I have good news. First, you're not alone. Most leaders I talk to ask for "execution" help. Second, execution can be improved through hard work and persistent, directed effort. However, execution is a discipline, and there are no silver bullets, despite what the internet will tell you. Improvement will take time and effort, but if you're committed, read on.

Building A Race Car

Racing cars is a fascinating sport: it requires skilled drivers, but the world's best driver cannot win with a poorly designed car. Take Formula One racing as an example. Formula One, also called F1, is a series of races around the world, with the world's best drivers in custom-built cars. F1 cars are built for extremely high cornering speeds: hitting peak forces of 6.5g, while maxing out speeds of 350km/h (215 mph). F1 is highly regulated - accidents can be hazardous, sometimes claiming lives, such as the tragic death of Ayrton Senna at the Imola Grand Prix in Brazil in 1994. The regulatory authority, F1A, changes rules often, sometimes minor changes, but the occasional major change that requires a substantial redesign of the car. Car design and development is such an essential part of F1 that there are actually two prizes: the Driver's prize and the Constructor's prize. Building F1 cars is as much an artistic endeavor, as it is a feat of ultimate engineering. It requires concerted hard work, year over year, improving cars, fixing existing problems, adjusting to rules changes, adapting to different drivers, stretching for varying road and weather conditions.

I think of high performing teams as a custom-built race car. The car comes together as a whole — engine design matters, but there is more to it. Similarly, any process a team employs matters, but there is much more to execution than process. The internet discussions on execution often point to a change in process, from waterfall to scrum, or having daily standup meetings, or weekly retrospectives. These things matter, but like the car's engine, do not capture the complete picture. I believe that you exact processes must be tuned for the team; there are broad rules of what everyone must do (all cars have 4 wheels, are under a certain weight, and have an engine), but within the general framework, there are a lot of choices to be made.

This post is divided into three broad sections:

  1. Defining execution and making the case to treat it as a discipline, rather than a process.

  2. The Five pillars of execution.

  3. A toolbox of things that have worked: practical ideas for you to copy-paste and edit.

What Is Execution?

The words "strategy" and "execution" are insufficiently understood by product development teams. My post, "Strategy and Tactics," attempts to explain the difference. Here's the short version:

  • Vision is what the future looks like if you're successful.

  • Strategies are possible paths that allow you to attain your vision. A strategic choice or decision is the act of choosing which particular way to follow.

  • Execution is walking that path.

Success depends on all three; which one matters more is highly dependent on the nature of the work, the company, the team, and a variety of other factors.

At a company level, the vision is typically to change the world in some meaningful way by making some problems go away. Google seeks to understand the world's information and make it available. Facebook aims to connect the world.

Strategy is how you get there. Exceptional strategic thinking is hard to come by because it involves so many leaps of faith. Strategies are forward-looking and often hard to test on any meaningfully short timescale. Amazon's strategy is three-pronged: decrease prices, increase product selection and increase customer convenience. Amazon could have chosen a variety of other approaches — build bricks and mortar stores, or purely match buyers to sellers, or designing and selling only their own products, or making a white-label e-commerce site. Notably, each of these strategies has a company built around it: Barnes and Noble, eBay, Everlane, and Shopify.

Execution is about the nitty-gritty. Designing experiences. Shipping hardware or code. Nimbly iterating to get through or around new roadblocks. Similar to other disciplines, excellent execution takes sustained effort over a long time. It is not a Thing to be achieved and checked off. The principles of Deliberate Practice apply here at the team level. Having a clear goal, focus and hard work, feedback, accountability, and deadlines — these are the pillars of a high achieving team.

Pillars of Execution (OSPAD)

  1. One Clear Goal

  2. Sustained Focus

  3. Progress Indicators

  4. Accountability

  5. Deadlines

1. One Clear Goal 🏁

To execute well, teams must know what they are executing toward. It all begins with a clear and concrete goal, one that everyone can understand and rally behind.

One of the biggest challenges that organizations face is not the lack of goals, but having too many. When you have multiple goals, priorities get muddled. Given a choice, teams will follow the path of least resistance and focus on goals that are achievable over goals that are important. It is paramount, for a leader, at any level, to identify and set a small number of concrete goals for their organization, ideally one.

The second challenge is communication. Ask 5 people at different levels in the organization what their goals are. If you hear different goals depending on who you ask, or worse, "I'm not sure what our goals are," you have a problem. After picking a goal, you must communicate it as clearly and as often as you can. The test comes at goal evaluation time, at the end of the quarter. If the team hits a secondary, unimportant goal, but misses the primary one, and they believe they did well and deserve commendation, then you have failed at clear goal-setting and communication.

A note on measurability: we in tech are obsessed with measurement. "What we cannot measure, we cannot improve." Sometimes things are just not measurable, or worse, the only thing you can measure is a misleading proxy of the actual thing. Don't let the lack of easy measures stop you from setting goals. For example, shipping is a perfectly good goal — especially when a team is building a new product with no priors.

Your job as a leader is to do the following:

  1. Pick a clear goal. This is the goal that you consider critical to the success of the team or organization. It is the goal that everyone should rally behind. If you have a dozen goals, prioritized into big and small goals, whittle the list down to one or two. This exercise is challenging, but worthwhile.

  2. Talk about your goal as often as possible. If your priorities are clear to you, and perhaps your executive team, but not to every engineer, PM, designer, and analyst - you haven't talked about them loudly or often enough.

2. Sustained Focus

YouTube in 2011 was primarily known as the site for cat videos that people forwarded and shared on Facebook. There was a blossoming creator ecosystem, but creators were incentivized to make small, bite-sized content. The reason was simple: YouTube algorithms valued views as the primary metric and overwhelmingly recommended short videos. Sometime in 2012, leadership made the decision to switch to watch-time as the primary metric. The central idea was that as long as people were watching content on YouTube, we didn't care whether it was short or long, just that they were entertained (or informed) by the things that they watched. The analytics team came up with a chart that showed a growth curve to 1 billion hours of watch-time. We all thought that a billion hours was impossible, but the chart served as strong motivation to hit a clear goal. When did YouTube hit the billion-hour goal? 2017.

Large bets take time to build. We, in Silicon Valley, are obsessed with instant feedback, numbers moving up and to the right instantly, trying to get to product-market fit as quickly as possible. If it doesn't work in 3 months, it must be a terrible idea and should be abandoned because of opportunity cost.

I believe many organizations are afflicted with organizational attention deficit disorder. We do lots of annual and quarterly roadmap planning, and the minute we see something shiny, internally or externally, we collectively go "Squirrel!" and chase after it.

Execution requires focus. Goals are the setup. Focused effort is the follow-through. As a leader, you should keep your team focused on the goals, for months to years, based on your conviction about your strategy.

3. Progress Indicators

A game without a scoreboard does not inspire players to do their best. The mere presence of a score is not enough, the score must be visibly present, so everyone sees it, every day.

Picking a good progress indicator is an art. These are operational metrics, not business metrics. As a result, the number you select to keep tabs on progress can be different from the overall goal chosen in section one. Rules of thumb for operational metrics:

  1. Indicators should be fast. The point of a visible indicator is to motivate the team. A rapidly moving number helps build momentum.

  2. Use leading indicators instead of lagging ones. When I was trying to lose weight (a perpetual goal), I made the mistake of using my weight as the indicator. It is a lagging indicator, and will move down after strenuous effort along other fronts, is subject to a lot of intra-week variation, and is mostly demotivating. Better markers like activity-minutes, or calorie counts, or standing or walking time possess the dual properties of being on the leading edge and moving quickly.

  3. Use indicators connected to the eventual goal. Measuring the wrong thing is akin to going really fast around the wrong race track. Things will feel good, but nobody will make progress.

Examples of good indicators:

  1. Kanban boards. I love these for projects where the primary goal is to ship something. They publicly and visibly capture the set of things that are in flight, and who is responsible for what. They get updated in real-time. Kanban boards are supported by most task management software; putting up a large TV with a display of your team's kanban board is a great way to check off the Visible Indicators pillar.

  2. Real-time dashboards. When working on projects to improve performance, improve efficiency, reduce bugs, I like dashboards that reflect the current state of the world (latency measurements, bug counts, etc.) I get inspired by seeing steady progress toward the horizontal goal line.

  3. To-do list. Low tech solutions are on occasion, even better than the high tech ones. A whiteboard chart showing deals closed and progress toward the ARR goal can be more inspiring than a fancy screen with a soulless number.

4. Accountability

In 2018, the Facebook app was slow, and bugs were out of control. Most of Facebook's mobile users are on Android; device and OS fragmentation make Android development particularly painful and error-prone. We knew we weren't doing a good job serving our users but were stuck in the constant pull between new feature work and stability. Leadership eventually decided that enough was enough, and something had to be done. We had clear goals: fixing bugs and faster startup. The progress indicators were natural (bug count and startup latency). Leadership all the way through Mark was committed. Things still didn't move.

The solution was a weekly meeting with all the engineering leaders and the head of the Facebook app. When some team wasn't doing well (I was a frequent offender), people would ask questions about why and what they could do to help. The social pressure, my word as a leader, made me feel very accountable to deliver on my commitments.

As humans working cooperatively toward a higher goal, our word stands for something. I try my best to deliver on public promises. Holding a mirror up to my face when I fail is enough of a stick to keep me motivated.

The word "accountability" has frequent connotations of negative performance management, or of letting people go. I believe this is far too extreme; accountability is simply about a feeling of responsibility. Every parent knows they can't control their young children but have a sense of responsibility for their actions.

I have two simple techniques for driving accountability:

  1. Write detailed weekly status updates, publicly visible to the whole company. The leader must write these updates themself, rather than delegating them to an assistant. The process of writing forces the leader to read the constituent material and stay on top of execution issues in their team. Much more on this in the Execution Toolbox section below.

  2. Weekly check-ins for the most critical projects. There is nothing that beats face to face (or video) interaction for driving accountability. These meetings are large and can get expensive really fast, so preparation and a concerted effort toward efficiency pays dividends quickly. Balance is essential — a status update meeting for every project in the organization is an egregious waste of time and unnecessary.

5. Deadlines

Confession: I am a serial procrastinator. Why do something today that I could push off until tomorrow? The harder something is, the more likely I am to procrastinate. I know I am not alone. And teams procrastinate just as much as individual humans do.

Deadlines are an effective deterrent for a few reasons:

  1. They create pressure to get things done. Creative work is not steady; it happens in bursts. Fixed periods provide a way to channel that bursty energy in a coordinated way.

  2. Deadlines serve as a focusing function. When people have a deadline to get something done, other priorities get downgraded.

  3. They provide opportunities to change directions. We all make incorrect decisions at times, bite off more than we can chew, or underestimate the complexity of a project. Changing decisions midway kills productivity and focus. However, changing direction at a particular deadline is a reasonable way of re-evaluating decisions and correcting course.

Deadlines go hand in hand with the curse of estimation. No sufficiently complex software project I have been on has been estimated correctly at the start. Are estimates even worth the work? I don't believe in long term estimates — these are mostly garbage and serve to create artificial pressure by committing teams to death marches that are months long. Short term estimates - in the order of 2-6 weeks - are reasonably accurate and should be done. Call it a sprint, call it a milestone, call it whatever you want. Software engineering teams should be able to predict, with a small margin of error, what they can accomplish in 2-6 weeks and then reliably deliver on this estimate.

Execution Toolbox

The toolbox is a set of things that have worked. I've included two ideas from leaders that I have worked with and respect greatly: Ben Liebald, VP Engineering at Branch, and Lily Zhang, Director of Engineering at Instacart.

  1. Weekly updates: perfect communication.

  2. Daily office hour blocks: faster decision making.

  3. Who-What-When: world's most straightforward project tracking.

  4. Recognition: positive reinforcement.

  5. Retrospectives: feedback about how we work.

1. Weekly Updates

Teams and leaders often write weekly updates, but without care and effort, these can feel like a lot of make-work. Weekly updates are one of the most powerful tools a leader has. They can keep tabs everything, as well as tell the company what's on their mind.

Tricks to writing regular weekly updates:

  1. Build a process that works for your team. In my case, PMs wrote an update for every major work-stream, with a deadline of Friday evening. I spent 2-3 hours over the weekend, understanding and pulling updates together. I would tune it on Monday and send it out Monday evening.

  2. Avoid unanchored metrics — "Participation moved up by 0.5% last week because of 3D photos." Is this good or bad? What is the baseline? What is the goal? Vague numbers engender more questions. Instead, try: "We rolled out 3D photos to 100%. As expected, participation rate increased to 3% (incremental 0.5%), bringing us close to our goal of 3.2%. We expect some follow on increases, as the rollouts continue." Longer, but provides way more context and understanding than the former.

  3. Remember the audience: exec leadership and the rest of the company. Stick to what is essential. It is okay for a week to be quiet and not much to report. If there are many weeks of quiet, leaders should check-in.

Getting to meaningful weekly updates took months. My most effective technique was rewriting individual PM updates as feedback. It helped everyone on the team write in one voice and significantly reduced the burden on me when writing the full update.

2. Daily Office Hour Blocks

Most teams schedule leadership or exec reviews at a regular cadence of weeks. These reviews fall into two broad categories: decision making and information dissemination. When a team is deep in execution mode, slow decision making can break a team's flow. Rebuilding that flow could take days.

In mid-2017, my team at Facebook was charged with building an AR-enabled camera feature for the Facebook family of apps. Our teams were running as hard as they could, but would get stuck when decisions needed to be made. Groups disagreed with one another and, on occasion, disagreed with themselves. At times, they just needed advice or someone to say, "go ahead."

We would typically have reviews once a week. Due to packed schedules, any given team could get review time perhaps once a month. This was too long to wait on a decision, especially in the middle of an intense sprint.

Our solution was to devise a 2 hours block of office hours, every day, broken into 30-minute slots. The entire leadership team committed to being present. We decreased the level of homework needed to get into office hours — pre-reads were still ideal, but a two-paragraph email was a sufficient pre-read, instead of a ten slide deck.

The net result of this was that teams rarely sat on decisions for more than 48 hours. They could come in, talk about their problem, get help, advice, or a decision. Everyone on the leadership team would be aware and aligned. Teams would still have to communicate the decision out to the broader organization. However, thanks to liberal audience management, key team members were instantly in sync.

I found this incredibly useful but extremely taxing on time. There is some risk that you train teams to come to you for every small decision that they should be making or to break every minor disagreement that they had. This did happen, and we did ask teams to go back and sort it out on their own. However, in most cases, the issues were legitimate, and teams were able to move significantly faster.

3. Who-What-When

Thanks to Maher Saba, Facebook Engineering VP, and leader extraordinaire.

Modern startups are drowning in project management tools. Asana, Jira, Monday, Notion, Coda, Google Sheets. These tools are flexible and sophisticated because they try to model an inherently messy, complex system. They can also be overkill — imposing concepts of owners or timelines or points or whatever ideas were built into the system.

Saba's technique was to have the most straightforward tool possible: a doc tracking 3 things:

  1. Who is accountable/responsible?

  2. What is the item being delivered or shipped?

  3. When will it be shipped by?

Below the table, there is a log to keep track of everything that is happening every week. This sounds ridiculously simple compared to all the powerful tools out there. Because it is. But it works well to run projects of small to medium size and complexity.

The goal of the process is to have an ongoing, credible plan where the dependencies and workload are well understood and balanced.

Here is a sample coda doc to duplicate if you'd like to implement this system.

4. Recognition

Thanks to Ben Liebald, Head of Engineering at Branch.co

Execution grinds can take a toll on team health and morale. Good leaders find ways to keep spirits up. Celebrating wins along the way, or even milestones with significant progress, can do wonders. Here are some ideas to create opportunities for people (not just the leader) to recognize good work:

  1. Friday Demos and Beer. At the end of the day on Friday, the team would gather, and anyone could present things they had been working on. The bar for demos was low — this was a safe space for showing off work in progress, not a polished presentation for the CEO.

  2. Recognition rituals. One of my teams at Facebook would award a giant stuffed pusheen cat to the person who went above and beyond their job to help the team. This person would then pass the award on to the next person at the team's weekly meeting. It was a great way to recognize work that would otherwise go unacknowledged publicly.

  3. Gratitude emails. I love receiving emails that say something like, "I appreciate your hard work on feature X. I know this was challenging, and we were under a lot of pressure. You really stepped up and delivered. I'm so glad to have you on the team." Sending these emails is effortless for the sender, but carries a lot of meaning for the recipient.

5. Retrospectives

Thanks to Lily Zhang, Director of Engineering at Instacart.

Teams, like people, need time to reflect on what they have done, and where they could have done better. It is hard to find time for this reflection during regular business since everyone is chugging away at their task and moving as quickly as possible.

To improve, we need feedback loops. Immediate product feedback loops are rare, or the signals are often weak. Leaders can ease people into a habit of seeking feedback and improving, by providing a safe space to collectively reflect, give each other feedback, and figure out how to work better. There are two natural points for retrospectives:

  1. Incident level. When there is a significant outage ("SEVs" in Facebook parlance), the first order of business is to deal with the outage. Facebook has this down to a science: the tooling and coordination needed for crisis management has been tuned to the point of near perfection. Once the team is past the crisis, there is a meeting called the "SEV Review," where the person causing the SEV presents an analysis of what happened, why, what could have been done to prevent it, and the next steps to minimize chances of a similar future incident. Senior engineering leaders are in attendance — giving feedback by pattern matching against other SEVs. This is not a blame meeting; it is strictly a learning meeting where everyone in the company learns as a result.

  2. Project level. One benefit of having deadlines (pillar #5) is that the end of a particular milestone presents an opportunity for reflection and improvement. I am a fan of using sticky notes to individually write what is working and what isn't, before opening up the discussion to the entire room. We are human and biased, writing before presenting helps reduce availability bias.

#executionishard

This post is not meant to be read once, understood, and put on the side. I hope you scanned it once, and return to it, when you have specific problems that these techniques can help with.

Remember, execution is really hard. It takes a long time to build execution muscle. If you're a leader: be patient and keep improving with your teams. If you're an individual contributor, and you see some key pillar missing, ask your leadership to fill in the gap.

Onward!

Coaching

The Feedback Series 3/3

The final note in the series brings theory and practice together into two ideas: first, in order to improve at anything, you need fast feedback loops. Second, coaches are a great way to create these feedback loops in complex environments. I believe coaching is an essential part of management and part of a manager’s job. I also suggest two approaches to getting coaching when your manager is unable to coach.

-- Photo by Moises Alex, Unsplash

Atul Gawande is a master surgeon. Specializing in endocrine surgery, he has performed thousands of operations over a long career. For years, Gawande had been steadily improving - his rates of complications were dropping steadily, and he consistently beat the averages, when compared to national data. Then he hit a wall. It seemed like he had hit peak performance, and the only place he could go from there was down.

On an out of town trip, Gawande was trying to play a game of tennis, for fun. He had been an aspiring tennis player growing up but had long since given up that dream. He couldn't find a game, and the ball machine was reserved for members. He paid the local pro to play with him for some time. After running Gawande around, the pro couldn't help get into coaching mode. He pointed out that Gawande could get more power into his serve by paying attention to his feet. Gawande gave it a go and within a short period, the pro had Gawande serving at least 10 mph faster than before: a new personal record.

Everyone knows that athletes have coaches. So do musicians. Why not doctors? 

Gawande decided to try an experiment and got another expert doctor to coach him. The coach sat in Gawande's surgery, took extensive notes, and gave him lots of feedback about things that he could improve. Bear in mind: this is one of the world's foremost surgeons, an expert in their field, at their performance peak, getting feedback and improving!

-- excerpts from Personal Best, Atul Gawande, New Yorker

Building Fast Feedback Loops

By this point, I have hopefully convinced you of the importance of feedback. Implementing feedback loops, however, is not easy.

Sometimes, tasks have in-built fast feedback loops. Remember Ericsson's experiment in Deliberate Practice? If the student was able to accomplish the job, it got harder. Else it got easier.

We see this characteristic in many software problems, as well. Imagine an engineer working on making their service faster. The performance of the service either improves or doesn't (or worse, regresses): an in-built fast feedback loop.

Some feedback loops can be very long. One can learn management by building a team, leading them into disaster, learning, starting again, creating the next team, etc. The feedback loop exists, but just takes years to run its course, and includes a large number of confounding factors.

To get good at anything, we need fast feedback loops - ideally realtime, but failing that, as close to realtime as possible. 

Feedback Loops For Complex Domains

Coaches for athletics, music, and even medicine, are effective because they are taking in a vast amount of complex information, pattern matching against what they know to be optimal, and efficiently communicating the result.

This process currently requires a human. I imagine that over the next decade, we will build better AI-powered coaches (using computer vision to analyze your tennis serve), and the human coaches will move on to increasingly complex domains.

People management in any organization is a very complex domain. Everyone working in a modern company is seeking to improve. To improve, they need fast feedback.

Managers as Coaches

Managers can provide fast and accurate feedback. They observe most of what their team is doing and the team's struggles. They may have done the same things before, or they may have done similar things and can pattern match, or they may have a wider lens. Regardless of the reason, they can provide helpful feedback and dramatically improve their reports' performance. 

I believe that coaching is a crucial part of a manager's job. If you're a manager and aren't providing great coaching to your team, you're failing at an essential part of your job. 

Coaching is not the only thing that managers do - they have to build teams, deliver outcomes, and in general, lead. Managers are human - and neglecting the important in favor of the urgent is a very human fault. To be great coaches, managers must put in time and effort toward this goal.

Other Coaches

Sometimes your manager cannot provide you all the coaching you need. They may not be great coaches or are working on their coaching skills. They may not have the time - this is a frequent occurrence in executive situations. They may not believe in its value or have the inclination. In any case, if your manager cannot provide you adequate coaching, here are some other ideas to explore.

  1. Peer coaching. Your peers have skills that you may be lacking and vice-versa. Put enough people in a group together, and there can be a mutual sharing of skills that helps the whole group improve. Running coaching circles takes time and effort: there has to be a set of people dedicated to running them, bringing people together, setting the agenda, and helping moderate the discussion. When done well, coaching circles can be effective and help form close bonds among colleagues as a side effect.

  2. Hire a coach. Many large tech companies, such as Facebook and Google, offer employees at a certain level, the ability to get external coaches. Executive coaches are expensive, and the company will likely partner with another organization or agency to pair coaches with people. One can also directly hire a life coach. Many companies will help you find the right one. However, I have no experience with any of these companies or their services, and cannot recommend one platform over another.

Important note when hiring a coach: a great coach can improve your game; a bad coach can destroy it. Be careful from who you seek counsel. Check referrals, talk to them, and do your homework.

Recap

This is the third of three notes on Feedback. This note made the case for building rapid feedback loops, getting a coach, and why managers should treat coaching as a critical part of their job.

  1. Delivering Critical Feedback

    1. Be timely. The effectiveness of feedback decays rapidly over time.

    2. Be firm and kind. Don’t beat around the bush, don’t sugar coat, and don’t be an asshole.

    3. Show the way. Point out flaws, but illustrate solutions with examples.

  2. Deliberate Practice

    1. Have a specific goal.

    2. Focus on achieving that goal.

    3. Get rapid feedback.

    4. Persevere through the pain.

  3. Coaching 👈🏽this note.

    1. Feedback is critical.

    2. Build fast feedback loops, automated or human.

    3. Coaching is a manager’s job.

About

I’m your host, Rushabh Doshi. After spending nearly a decade managing some of the brightest engineering teams at Facebook and YouTube, I’m taking a break and doing some writing, spending time with my family and figuring out my next thing.

If you like this post, please feel free to forward and share it. If you’re not signed up for my newsletter, I write on engineering, product, and management, about once a week. You can sign up below.

You can find me on Twitter @radoshi and my writings at rushabhdoshi.com or medium.com/@radoshi.

Loading more posts…