7 mins readHow to effectively assess portfolio performanceWhat is the best way of evaluating portfolio performance allocation strategies? Should we just compare risk, return or risk adjusted return? Are complex investment allocation models and tools more effective than simpler ones? Is one framework better than other? When we look at fund managers or fund performance what data should we seek and collate? How do we know if we one investment strategy is better than the other? What benchmarks can we use to assess how well we have done relative to the market?Evaluating portfolio performance allocation strategiesAll great questions. We answered these and more in thelatest iteration of the portfolio management and optimization models courselast October in Dubai. Let\u2019s take a look at what we found out. Experiment designWe used a simple experiment design with a universe ofpreselected investment securities and 5 investment allocation models. Thesecurities data set looked at prices from 2008 to 2018. 2008-2016 data was usedto train allocation models and allocate capital to specific positions.2016-2018 data was used for evaluating performance of each allocation. The model increased in their sophistication moving from asimple risk adjusted return approach to more complex designs. Models focused onminimizing probability of loss as well models that focused on increasing theprobability of upside by optimizing the distribution of returns. For performance evaluation we used three benchmarks. Twolinked to index performances (NYSE and NASDAQ) and one linked to a security thatoutperformed all other securities within the portfolio during the observationperiod (AMZN). Using historical securities price data, we allocated capitalto investment securities. The allocation model had access to prices from 2008 till2016. Performances were evaluated separately for two periods. 2016-2017 and2016-2018. The performance metric was holding period return (HPR) whichmeasured total return over the evaluation period. The 2016-2018 evaluation servedas a post allocation performance evaluation. We then evaluated their performance bycomparing the results against performance of the three benchmarks. All models used historical return series to allocatecapital. This implies that future performance can be projected using historicalreturns. We understand that this is not true but this is the basis of theoriginal allocation model. Once the optimal allocation strategy is identifiedit can be tweaked to take into consideration cyclical performance and theimpact of product, user, demographic and economic cycles. Any guesses on which models out performed others. Before youread further take a second to think about the following questions. Given two allocation models,one more sophisticated than the other, which one is likely to outperform theother? Simpler over more complex or more complex over simpler?Is it possible to maximizeupside and limit downside at the same time? Is there a model that would allowus to trade between the upside potential and downside risk?The distribution of returnsUnderstanding portfolio returns distributionWe have spent a great deal of time on understanding how returns distribution are generated in an earlier part of our course on Portfolio Optimization models. If you are not familiar with the concept of returns distribution, how to calculate them or how to plot them using Excel please see building portfolio management worksheets in excel. Strategies and evaluation benchmarksLet us take a quick look at the five investment allocationmodels used in the experiment. The firsttwo allocation models are metric driven. The last three are returns distributiondriven. Of the metric driven, the first focuses on return, the second focuseson risk. Of the distribution driven, the first two focus on the down side, thelast one focuses on the upside. Strategy I \u2013 base case – Simpler is better. Wecalculate return per unit of risk and then optimize that as our internalmetric. Return is defined as holdingperiod return over a historical observation period. Risk is defined by observedannual standard deviation of daily return over the same period. Strategy one is essentially a simplified version ofthe good old Sharpe ratio, one of the oldest portfolio allocation designs thatworks of two dimensions at the same time \u2013 risk and return. Strategy II – Minimize downside. Represented by worstcase single day loss (WCSDL) using historical returns. WCSDL is defined as theminimum value observed in daily price returns for a given security in theobservation period. At a portfolio level it is defined as the minimum value inthe portfolio return series during the observation period. The optimization modelminimizes the portfolio worst case single day loss. Strategy two is a Value at risk (VaR) based designthat sets a simple threshold using historical data set on the worst case possibleloss the portfolio return distribution has experienced during the observationperiod. Strategy III \u2013 Minimize probability of shortfall.Minimize the probability that returns will fall below some threshold. Probabilityis measured by the distribution of portfolio returns. In the first shortfallmodel it the loss threshold is set at 5%. The optimization model minimizes theprobability that this threshold will be hit by portfolio returns. In simpler words this means that the portfolioallocation model will focus on reducing the probability of losing more than 5%.Strategy three come from the shortfall school. Theshortfall school is related to the value at risk school but rather thanfocusing on a single day loss, it focuses on reducing the probability that aloss threshold would be hit. Strategy IV \u2013 Minimize probability of shortfall. Samemodel as Strategy III. The loss threshold is set 1%. The optimizationmodel minimizes the probability that we will hit a loss of 1%.Strategy four is the same as strategy three with alower loss threshold. Strategy V \u2013 Optimize skewness of returns. Skewnessrefers to attribute of return distributions that shifts it in a certaindirection. Will it help performance if we shift the historical distribution ofreturns to emphasize positive returns more than negative returns? We test thisassumption by maximizing positive skewness of the portfolio returndistribution. Strategy five is the most complex of all fivestrategies. While strategy two, three and four are also driven by the returndistribution, five actually tries to shift the portfolio distribution towardsthe positive end of return spectrum. The hope is that perhaps doing so we willimprove the risk return trade off. Benchmarks used. NYSE, NASD and AMZN return series over thesame period. Once again before you read ahead, which strategy are you likely to choose as a portfolio manager? Which model do you think out performed all others? One ring to rule them all?Evaluating portfolio performance allocation strategies \u2013 score cardHow well did your chosen strategy perform? Did you expectthese results? Were you surprised? Can a deeper dive into performance metrics explainwhat happened?The simplest strategy risk adjusted return, outperformed themore complex one and came very close to beating all three benchmarks in theprimary evaluation period. It still dominated all other strategies and 2 of thethree performance benchmarks in the post allocation evaluation period thatincluded 2018. Evaluating portfolio performance allocation strategies- summarized score cardCan you come up with a justification or rationale for theseresults? Let\u2019s take a look at additional metrics to see if we can find anyhints?The resultsEvaluating portfolio performance allocation strategies \u2013 detailed score cardThe primary performance metric was holdingperiod return or HPR. HPR describes the total realized return over the observationperiod. It is a better performance metric than expected return or averageexpected return which is the reason why we used it for performance evaluation. Our second performance metric was downside as indicated andmeasured by worst case single day loss. Our third measure was annualizedvolatility as a measure of risk. In addition to these other metrics of notes that weretracked across strategies were Beta and Alpha with respect to NYSE (Dow Jones)index, percentile returns at the 1% threshold, maximum single day gain,skewness and kurtosis of the portfolio return distribution. Take a minute to compare the metrics shared above with postallocation performance for each strategy. Is there anything that stands out?Here is a hint. In your opinion which attributes highlightedabove are the strongest predictors of future portfolio performance?Risk adjusted returnsThe answer once you take a deeper look at the figures is riskadjusted return. Changes in expectedreturn, volatility, beta, worst case single day loss or max single day gain arenot sufficient enough to attribute changes in expected performance as well asrisk adjusted return does. The same also holds true for percentile returns,skewness and kurtosis. Alpha is a special animal. While it appears that it may havethe same predictive powers as risk adjusted return, we have to be a littlecareful with this assumption. Take a look at our discussion around alphacyclicality and optimal portfolio alpha allocation before you commit toalphas as your primary performance metric. Implication for portfolio managers? One, if you don\u2019t have a fancy performance monitoringdashboard that is fine. You just need risk adjusted return or what is commonlyknown as the good old Sharpe ratio. Two, as you move to more sophisticated approaches orattempt to limit downside, you also limit your upside. Theoretically speakingmaximizing positive skewness has a great deal of technical appeal.It maximizes upside by shifting the returns distribution in thepositive direction. If you take a look at the maximum single day gain row, youwill notice that maximizing skewness has the highest score for that metric. Butwhat impact does that have on expected return and realized holding periodreturn? As you limit your downside, you will by definition limit your upside. Three, it appears that there is a clear trade off andno arbitrage possible between the two extremes. At least within this data set.You could change the securities universe and try again but results would remainsimilar. Sounds counter intuitive but it is true. Whatever you save in terms of downside you will end upgiving up in upside. A distribution of returns comes with a certain amount ofrisk. The two are linked, you can\u2019t have more of one without having more of theother. For instance, when we push positive skewness higher, eventhough we increase the maximum single day gain, we also increase returns volatilityand we end up reducing holding period return by a fairly significant amount. Then why bother with all the metrics?The metrics are useful when it comes to exploring performanceand to answer question posed above. If you want to compare capital allocation strategies,you want to compare them across multiple dimensions not just one. When it comesto designing performance evaluation systems, you want to focus on the onemetric that the organization needs to optimize. One that is simple, effectiveand relevant. While you may understand that simpler is better, you still needgood data to convince the world that your baseline model does outperform themore sophisticated editions. Conclusions and takeawaysRemember the questions we asked above right at the start. Let\u2019stry and answer them one by oneShould we just compare risk, return or risk adjustedreturn? Risk adjusted return leads to better performance than justoptimizing risk or return. It beats allother benchmarks because they focus on one dimension \u2013 risk or return. Riskadjusted return work with two \u2013 risk and return. Are complex investment allocation models more effectivethan simpler ones? They can certainly do a better job of limiting downside butthere is a cost. In terms of actual performance measured in terms of realizedreturns, they don\u2019t perform as well as simpler metrics. That is because as youreduce risk beyond a certain threshold, you also reduce the potential forhigher returns. Nothing illustrates this more powerfully than the positiveskewness strategy in the example above. Is one framework better than other? That depends on what you want to measure and achieve andwhat metrics and benchmarks your performance is measured against. In the endyou will best be served with tools that are aligned well with your ownperformance management benchmarks.