Tracking error as a measure to evaluate the tracking quality of an ETF
The first wrong assumption he stressed was that investors believe tracking error is a good measure to evaluate the quality of an ETF; they think a low tracking error is good, while a high tracking error is bad. He pointed out that this assumption is wrong, since tracking error is just a measure of uncertainty. A low tracking error mistakenly gives an investor confidence, since the tracking error value does not say anything about the direction of the out- or underperformance. That is, a low tracking error indicates only that the ETF had a low difference in the standard deviation of its returns compared to the returns of its index. I totally agree with this point and always say that an investor can’t use tracking error as a single measure to evaluate the tracking quality of an ETF.
Is correlation a better measure?
Some investors use correlation to evaluate the tracking quality of an ETF, but this statistic doesn’t provide them the full picture either. Correlation only tells the investor whether the ETF is moving in the same direction as the underlying index. It contains no information about how widely this movement differs from the returns of the index. Correlation is therefore also not a good standalone measure to evaluate the tracking quality of an ETF.
From my point of view, a combination of these two measures would do the trick. A high correlation (somewhere around +1) tells the investor that the ETF and its index move in the same direction, and a low tracking error adds the information that daily returns are close to those of the index. But this combination has a weak spot: it doesn’t tell the investor whether the ETF is out- or underperforming its index and how big the performance gap between the ETF and the index is in terms of net returns. Even a low tracking error on a daily basis can add up to large performance gap over a mid- to long-term investment period. As Jean-René said during his presentation, at the end of day the number that matters most for the investor is the net return of the ETF compared to its index–the so-called tracking difference.
Is tracking difference the ultima ratio?
Tracking difference, aka relative performance, compares the return of the ETF with the return of its index and shows the investor exactly by how many percentage points the ETF has out- or underperformed its index. Tracking difference might therefore be the ultima ratio for investors, especially if they are monitoring the evolution of the tracking difference over time. I would like to add a big caveat here: it works only if investors don’t care about how the result was achieved; the tracking difference does not provide any information on how closely the ETF followed the index during the journey. So, it seems to me there is no single solution to provide an investor all the information needed to evaluate and select an ETF.
Cheap is better
That said, there is another assumption made by a number of investors that can be sorted out with the tracking difference: if one wants to invest in a standardized broad market cap-weighted index such as the S&P 500 or the EuroStoxx 50, one should simply buy the cheapest available ETF. Costs are a burden that holds down an ETF’s performance; therefore, cheap funds are better than their more expensive peers. This assumption sounds logical at first, but since all ETFs are able to use some performance-enhancing management techniques, one has to think twice about the assumption. Jean-René provided a scatter chart during his presentation that showed the overall performance (after costs) in relation to the management fees. This chart showed no evidence that cheap funds perform any better than their expensive peers. He stated that he even found some low-cost funds showing a big underperformance, meaning an ETF with low fees might become an expensive one if one looks at the overall performance realized in the investor’s portfolio.
From my point of view, investors can’t answer with a single ratio all the questions raised to become successful in investing with ETFs. Investors have to accept some complexity in their analysis, even when they look at ETFs that track standardized market cap-weighted indices. I am fairly sure it will be worthwhile to the overall performance of a portfolio to make some extra effort in the ETF analysis. It is also clear that research efforts shouldn’t stop at the performance level, since good execution on the right trading venue in either direction–into and out of the market–is also a key driver for investors’ success.