What is the best way of evaluating portfolio performance allocation strategies? Should we just compare risk, return or risk adjusted return? Are complex investment allocation models and tools more effective than simpler ones? Is one framework better than other? When we look at fund managers or fund performance what data should we seek and collate? How do we know if we one investment strategy is better than the other? What benchmarks can we use to assess how well we have done relative to the market?

All great questions. We answered these and more in the latest iteration of the portfolio management and optimization models course last October in Dubai. Let’s take a look at what we found out.

## Experiment design

We used a simple experiment design with a universe of preselected investment securities and 5 investment allocation models. The securities data set looked at prices from 2008 to 2018. 2008-2016 data was used to train allocation models and allocate capital to specific positions. 2016-2018 data was used for evaluating performance of each allocation.

The model increased in their sophistication moving from a simple risk adjusted return approach to more complex designs. Models focused on minimizing probability of loss as well models that focused on increasing the probability of upside by optimizing the distribution of returns.

For performance evaluation we used three benchmarks. Two linked to index performances (NYSE and NASDAQ) and one linked to a security that outperformed all other securities within the portfolio during the observation period (AMZN).

Using historical securities price data, we allocated capital to investment securities. The allocation model had access to prices from 2008 till 2016. Performances were evaluated separately for two periods. 2016-2017 and 2016-2018. The performance metric was holding period return (HPR) which measured total return over the evaluation period. The 2016-2018 evaluation served as a post allocation performance evaluation. We then evaluated their performance by comparing the results against performance of the three benchmarks.

All models used historical return series to allocate capital. This implies that future performance can be projected using historical returns. We understand that this is not true but this is the basis of the original allocation model. Once the optimal allocation strategy is identified it can be tweaked to take into consideration cyclical performance and the impact of product, user, demographic and economic cycles.

Any guesses on which models out performed others. Before you read further take a second to think about the following questions.

- Given two allocation models, one more sophisticated than the other, which one is likely to outperform the other? Simpler over more complex or more complex over simpler?
- Is it possible to maximize upside and limit downside at the same time? Is there a model that would allow us to trade between the upside potential and downside risk?

## The distribution of returns

We have spent a great deal of time on understanding how returns distribution are generated in an earlier part of our course on Portfolio Optimization models. If you are not familiar with the concept of returns distribution, how to calculate them or how to plot them using Excel please see building portfolio management worksheets in excel.

## Strategies and evaluation benchmarks

Let us take a quick look at the five investment allocation models used in the experiment. The first two allocation models are metric driven. The last three are returns distribution driven. Of the metric driven, the first focuses on return, the second focuses on risk. Of the distribution driven, the first two focus on the down side, the last one focuses on the upside.

**Strategy I – base case – Simpler is better**. We
calculate return per unit of risk and then optimize that as our internal
metric. Return is defined as holding
period return over a historical observation period. Risk is defined by observed
annual standard deviation of daily return over the same period.

**Strategy one** is essentially a simplified version of
the good old Sharpe ratio, one of the oldest portfolio allocation designs that
works of two dimensions at the same time – **risk and return**.

**Strategy II – Minimize downside**. Represented by worst
case single day loss (WCSDL) using historical returns. WCSDL is defined as the
minimum value observed in daily price returns for a given security in the
observation period. At a portfolio level it is defined as the minimum value in
the portfolio return series during the observation period. The optimization model
minimizes the portfolio worst case single day loss.

**Strategy two** is a Value at risk (VaR) based design
that sets a simple threshold using historical data set on the worst case possible
loss the portfolio return distribution has experienced during the observation
period.

**Strategy III – Minimize probability of shortfall**.
Minimize the probability that returns will fall below some threshold. Probability
is measured by the distribution of portfolio returns. In the first shortfall
model it the loss threshold is set at 5%. The optimization model minimizes the
probability that this threshold will be hit by portfolio returns. In simpler words this means that the portfolio
allocation model will focus on reducing the probability of losing more than 5%.

**Strategy three** come from the shortfall school. The
shortfall school is related to the value at risk school but rather than
focusing on a single day loss, it focuses on reducing the probability that a
loss threshold would be hit.

**Strategy IV – Minimize probability of shortfall**. Same
model as **Strategy III**. The loss threshold is set 1%. The optimization
model minimizes the probability that we will hit a loss of 1%.

**Strategy four** is the same as strategy three with a
lower loss threshold.

**Strategy V – Optimize skewness of returns**. Skewness
refers to attribute of return distributions that shifts it in a certain
direction. Will it help performance if we shift the historical distribution of
returns to emphasize positive returns more than negative returns? We test this
assumption by maximizing positive skewness of the portfolio return
distribution.

**Strategy five** is the most complex of all five
strategies. While strategy two, three and four are also driven by the return
distribution, five actually tries to shift the portfolio distribution towards
the positive end of return spectrum. The hope is that perhaps doing so we will
improve the risk return trade off.

**Benchmarks used**. NYSE, NASD and AMZN return series over the
same period.

Once again before you read ahead, which strategy are you likely to choose as a portfolio manager? Which model do you think out performed all others?

## One ring to rule them all?

How well did your chosen strategy perform? Did you expect these results? Were you surprised? Can a deeper dive into performance metrics explain what happened?

The simplest strategy risk adjusted return, outperformed the more complex one and came very close to beating all three benchmarks in the primary evaluation period. It still dominated all other strategies and 2 of the three performance benchmarks in the post allocation evaluation period that included 2018.

Can you come up with a justification or rationale for these results? Let’s take a look at additional metrics to see if we can find any hints?

## The results

The primary performance metric was holding period return or HPR. HPR describes the total realized return over the observation period. It is a better performance metric than expected return or average expected return which is the reason why we used it for performance evaluation.

Our second performance metric was downside as indicated and measured by worst case single day loss. Our third measure was annualized volatility as a measure of risk.

In addition to these other metrics of notes that were tracked across strategies were Beta and Alpha with respect to NYSE (Dow Jones) index, percentile returns at the 1% threshold, maximum single day gain, skewness and kurtosis of the portfolio return distribution.

Take a minute to compare the metrics shared above with post allocation performance for each strategy. Is there anything that stands out?

Here is a hint. In your opinion which attributes highlighted above are the strongest predictors of future portfolio performance?

## Risk adjusted returns

The answer once you take a deeper look at the figures is **risk
adjusted return**. Changes in expected
return, volatility, beta, worst case single day loss or max single day gain are
not sufficient enough to attribute changes in expected performance as well as
risk adjusted return does. The same also holds true for percentile returns,
skewness and kurtosis.

Alpha is a special animal. While it appears that it may have the same predictive powers as risk adjusted return, we have to be a little careful with this assumption. Take a look at our discussion around alpha cyclicality and optimal portfolio alpha allocation before you commit to alphas as your primary performance metric.

## Implication for portfolio managers?

**One**, if you don’t have a fancy performance monitoring
dashboard that is fine. You just need risk adjusted return or what is commonly
known as the good old Sharpe ratio.

**Two**, as you move to more sophisticated approaches or
attempt to limit downside, you also limit your upside. Theoretically speaking
maximizing positive skewness has a great deal of technical appeal.

It maximizes upside by shifting the returns distribution in the positive direction. If you take a look at the maximum single day gain row, you will notice that maximizing skewness has the highest score for that metric. But what impact does that have on expected return and realized holding period return? As you limit your downside, you will by definition limit your upside.

**Three**, it appears that there is a clear trade off and
no arbitrage possible between the two extremes. At least within this data set.
You could change the securities universe and try again but results would remain
similar. Sounds counter intuitive but it is true.

Whatever you save in terms of downside you will end up giving up in upside. A distribution of returns comes with a certain amount of risk. The two are linked, you can’t have more of one without having more of the other.

For instance, when we push positive skewness higher, even though we increase the maximum single day gain, we also increase returns volatility and we end up reducing holding period return by a fairly significant amount.

Then why bother with all the metrics?

The metrics are useful when it comes to exploring performance and to answer question posed above. If you want to compare capital allocation strategies, you want to compare them across multiple dimensions not just one. When it comes to designing performance evaluation systems, you want to focus on the one metric that the organization needs to optimize. One that is simple, effective and relevant. While you may understand that simpler is better, you still need good data to convince the world that your baseline model does outperform the more sophisticated editions.

## Conclusions and takeaways

Remember the questions we asked above right at the start. Let’s try and answer them one by one

**Should we just compare risk, return or risk adjusted
return? **

Risk adjusted return leads to better performance than just optimizing risk or return. It beats all other benchmarks because they focus on one dimension – risk or return. Risk adjusted return work with two – risk and return.

**Are complex investment allocation models more effective
than simpler ones? **

They can certainly do a better job of limiting downside but there is a cost. In terms of actual performance measured in terms of realized returns, they don’t perform as well as simpler metrics. That is because as you reduce risk beyond a certain threshold, you also reduce the potential for higher returns. Nothing illustrates this more powerfully than the positive skewness strategy in the example above.

**Is one framework better than other? **

That depends on what you want to measure and achieve and what metrics and benchmarks your performance is measured against. In the end you will best be served with tools that are aligned well with your own performance management benchmarks.