I’m not a big believer in post-mortem analysis. It’s very hard to do it without infecting analysis with what you know now. For example, when it comes to “misses” in startup investing, one often looks at where “things went wrong.” But maybe nothing went wrong. Maybe you created a systematic plan and process, executed the process based on the facts that you had at the time and made a decision based on the probabilities. Your decision wasn’t wrong or a “miss”; the probabilities just played themselves out.
Put another way:
[D]ecisions are about probabilities is that decisions should not be judged by outcomes but by the quality of the decision-making, though outcomes are certainly one useful input in that evaluation. Any individual decisions can be badly thought through, and yet be successful, or exceedingly well thought through, but be unsuccessful, because the recognized possibility of failure in fact occurs. But over time, more thoughtful decision-making will lead to better overall results, and more thoughtful decision-making can be encouraged by evaluating decisions on how well they were made rather than on outcome. In managing trading rooms, I always focused on evaluating and promoting traders not on their results alone, but also and very importantly, on the thinking that underlay their decisions.
This may be a tall tale, but I once heard that a successful hedge fund manager started his career by identifying every investment decision that Warren Buffett ever made. He then analyzed each decision with the information that Buffett had at that time. That process was probably as painstaking as it sounds. Because a lot of Buffett’s investment decisions were made before the Internet even existed. So he had to access all the newspaper and publicly available information at the time. It seems like that’s what’s needed if you’re going to do a real post-mortem analysis. When we consider our misses (which are plenty and painful) at SV Angel, we try to do this and think about our process and not the outcome. With each miss, we may tweak our process. Even that could be a mistake – confusing procedure with substance. The challenge is to decide whether the process ever needs to be tweaked in light of the outcomes.