Decision Economics: designing an effective feedback loop

The annual investment performance review session is one of the most critical interactions for both the team and the client. For the team, it provides the ideal opportunity to review results objectively and make systematic improvements to both process and skills; for the client, it’s an ideal forum to both educate and deepen the relationship.

In our experience, however, the performance review is often run as a score-carding exercise: "The benchmark did X and we did Y.” When results are positive, few (if any) changes are made. When results are disappointing, allocations and managers are often modified to eliminate underperformers. But when outcomes are primarily viewed through a binary lens—good or bad—without consideration for the underlying decision process, it’s difficult, if not impossible, to determine if performance was largely driven by luck or skill.

The key to process improvement is building an effective feedback loop, including capturing both the environment in which decisions were made and the rationale. Why is this important? As an industry, we’re conditioned to evaluate results over very short time periods despite having investment time horizons that often exceed five, ten, twenty years, or more. Markets can easily make well-reasoned decisions conceived at the beginning of the year appear ill informed by the end of the year, leading to second-guessing or worse—a cycle of trial and error decision-making. Understanding the investment landscape at the point when a decision was made provides critical context to properly evaluating process and results versus relying on memory alone.

Elizabeth Loftus, Ph.D., a professor of cognitive science at Stanford University, is considered one of the foremost authorities on memory and recall. She's been involved in high-profile criminal trials to demonstrate the fallibility of eyewitness testimony. Her conclusion: Our ability to accurately recall information is poor at best. Further, she finds that our memories are subject to distortion, including contamination from others through the power of suggestion. Amplifying this are some common behavioral tendencies, including hindsight bias and attribution bias. The former is our tendency to look back on events that weren’t predictable or obvious at the time and conclude that "we must have known.” The latter is our tendency to attribute positive outcomes to our own skill but negative ones to exogenous factors that were out of our control. Combined, our inability to accurately recall conversations and events and our behavioral tendencies can impede our ability to learn from past mistakes and increase the odds that we’ll repeat them.

As mentioned, the key to an effective feedback loop is the ability to re-create the environment in which decisions were made as well as capture the rationale for our decisions. Without both, it’s virtually impossible to link outcomes to process and make systematic improvements. Previously in this series, we discussed the use of a market dashboard to focus our attention on the data with empirical value to drive markets and the economy. By understanding concepts such as the direction of credit spreads and inflation expectations, we put ourselves in a position to make more informed decisions about how we want to allocate portfolios. By maintaining a signals dashboard, the work of re-creating the environment in which decisions were made is largely complete. We’ve also discussed the use of base rating as a method to avoid performance chasing. By better understanding our entry points into asset classes and managers, we’re in a better position to set expectations for future returns on our capital allocations as well as reality check the forecast returns of others. Combined with documenting the bull, bear, and base case for major policy decisions in the Investment Policy Committee meeting, we’ve captured the rationale for our decisions and formed the backbone of an effective feedback loop.

1   Be grounded—Create an investment philosophy statement.

2   Be focused—Build a market dashboard to separate signals from noise.

3   Be disciplined—Apply base rates to promote the capture of mean reversion.

4   Be skeptical—Employ counterintuition to address biases and groupthink.

5   Be thorough—Design a feedback loop to promote learning and process improvement.

The investment consulting group at John Hancock Investment Management is here to help, offering a range of services from formal model reviews to manager selection to the investment decision process. For more information on structuring your team’s investment policy meeting, contact us today.