Keeping It Simple – A Lesson From Backtesting

I’ve gotten many questions about slicing up the equity allocation of the Permanent Portfolio to try to capture the “value premium” by biasing the portfolio towards small cap value stocks both domestically and internationally.

Here are my thoughts:

Backtesting investing performance is dangerous. With backtesting, there is this temptation to tweak a portfolio to obtain the best possible historical returns. The one problem with this is: We aren’t going to re-live the past. We are going to live a brand new future that will be different. Even worse, you could introduce risks to the asset allocation that you may overlook as you stare mesmerized by the high annual returns you’ve discovered.

One of the changes you hear people talking about with the Permanent Portfolio (or most index investing strategies) is that some asset or another should be changed because this other asset did best in the past. After all, look at the results. If you just substituted Sub-Saharan Africa Micro Cap Value Stocks for the US Total Stock Market fund, the historical performance went up by X% a year and over Y number of years it creates a dramatic difference.

Too bad we didn’t know that information years ago when we could have actually made use of it. Now would you mind telling us what’s going to do best going forward? That’s something we can actually use.

Also, there are costs involved for taxable and non-taxable investors and many of these specialty funds have higher expenses. Further, much of the data used to sell you on the latest hot asset class performance is usually simulated because those indices never even existed over the time period covered. They were built with the benefit of hindsight.

For a taxable investor we need to incorporate not only the annual expense ratio and advisor fees, but also the tax cost ratio that Morningstar reports on fund efficiencies. The tax cost ratio reflects the amount of returns you can expect to lose in the fund due to tax impacts. When looking at the tax cost ratio, you need to look at funds with at least 10 years of data so you can see how it did through a bull and bear market. Not doing so can skew the numbers downward. Many funds in fact don’t have 10  years worth of data so their tax cost ratio can be deceptively low.

With that criteria in place, let’s look at some commonly used funds for value tilting vs. the Total Stock Market index from Vanguard:

Vanguard Total Stock Market (TSM)

Expense Ratio: 0.16% (Even lower with the ETF = 0.07%)
Tax Cost Ratio: 0.39%
Advisor Fee: 0.00%

DFA Intl. Small Value (ISV)

Expense Ratio: 0.69%
Tax Cost Ratio: 1.25%
Advisor Fee: 1% (Can range lower or higher)

DFA Small Cap Value (SV)

Expense Ratio: 0.52%
Tax Cost Ratio: 1.80% (Holy smokes that’s high!)
Advisor Fee: 1% (Can range lower or higher)

Total Overhead Costs

Vanguard TSM: -0.55% a year (or lower with the ETFs or Admiral status shares)
DFA Intl Small Value: -2.94% a year
DFA Small Value: -3.32% a year

If you are non-taxable you can eliminate the tax cost ratios. If you use a non-DFA fund you can get rid of the advisor fee. But the higher expenses are still there because small cap funds have higher turnover which means higher fees. International small funds also have these fees plus the higher trading costs associated with dealing in small markets internationally. That’s at least 0.50% a year reduction in returns under best conditions and likely larger under typical conditions.

Also, don’t forget that if you are a taxable investor and you need to rebalance each year because you own lots of funds.  On those gains you need to pay Uncle Sam his cut which is currently 15%, but has historically averaged 25% and has even been much higher.

Now the extra performance is not so clear cut. In this case you are losing perhaps 3% of the returns per year you are seeing in the spreadsheet results. At best you are probably losing close to 1% a year even for non-taxable investors simply because of higher fees.

Here’s the best part: You are losing that money whether or not the fund beats the Total Stock Market (TSM).

If your specialty funds beat TSM going forward you need to deduct the expenses. If your specialty funds lag TSM going forward you still need to deduct the expense.

So it comes back to Harry. Not Harry Browne, but Dirty Harry: “Do you feel lucky?”

You can buy the TSM and get a guaranteed low expense investing vehicle or you can go for the specialty asset classes and pay much more. If your bet pays off that’s great. But if it doesn’t (and with a 1-3% overhead it very well may not), then you’ll lose to TSM. It works out in these simple examples:

Year 1:

TSM Returns: 10%

Small Cap Value Returns: 12%

Looks like you made a good bet! Oh not so fast…

TSM Returns: 10% – 0.16% (expenses) – 0.39% (taxes)  = 9.45% to you

Specialty Fund Returns: 12% – 0.69% (expenses) – 1.25% (taxes) – 1% (advisor fee) = 9.06% to you

Winner: TSM

Year 2:

TSM Returns: 10%

Small Cap Value Returns: 9% (no guarantee you’ll win each year is there?)

After expenses:

TSM Returns: 10% – 0.16% (expenses) – 0.39% (taxes)  = 9.45% to you

Specialty Fund Returns: 9% – 0.69% (expenses) – 1.25% (taxes) – 1% (advisor fee) = 6.06% to you

Winner: TSM

Etc.

You may think I’m being unfair to prove a point. Yes, I’m aware there are tax-managed versions of these funds that could decrease tax costs. Yes, I’m also aware that there are even some cheaper funds recently introduced. But these things will never make them as cheap and tax efficient as the Total Stock Market fund. Also, there is the chance that your value funds may beat the market, but there have been very long stretches of times where they didn’t. For instance: From 1979-1999 Large Cap stocks returned 17.2% CAGR vs. Small Cap Value 15.9% CAGR (before costs).

This period covers the greatest bull market in US History for stocks. Yet, it was the big companies that walked away the winner. That’s 20 years lagging before the small value bet paid off starting in 2000.

I’m aware that you could rebalance between funds to capture excess returns on down years (of course keeping in mind the expenses of doing this). The problem is the value indexing phenomena is just not reliable enough that I’m willing to wager on it. In fact, as more people become aware of value indexing it is less likely any significant outperformance will continue. The markets are very efficient and have a way or making these types of advantages disappear. Heck, a few years ago I remember a broker from a major firm pitching to me the advantages of value investing and how that’s the strategy the brokerage was recommending for all their clients. Now with that many people in on the secret what are the odds it’s going to pay off as it did in the past? Certainly a lot less.

Here’s the most important thing you need to remember in all of this:

With the Permanent Portfolio your diversification comes from your Stock, Bond, Cash and Gold split and not from owning different stock asset classes.

As 2008 showed, owning multiple stock asset classes provides negligible diversification benefits in a bad market. They all did poorly together. It’s much more important to focus on keeping your stock, bond, cash and gold allocation and don’t worry about eeking out every last possible ounce of performance from your stocks based on backtests.

But, if you just have to value tilt I offer this advice: Buy the cheapest value index fund you can and is not actively managed. Vanguard and IShares offer funds that do this (S&P 600 Value and Vanguard Small Cap Value are good choices). Avoid funds wrapping the index up in some new gimmick like “Fundamental Indexing” which simply runs the costs of the fund up and may not produce tangible benefits. Lastly, be aware of all the costs involved if you are holding the funds in a taxable account and be aware that history does not repeat and you could be waiting a long time for the value premium to show up.

Or, just include these funds in your variable portfolio as a speculative play.

For my money, I own a Total Stock Market fund and a very small allocation to a Total International-style index fund which is the most I really gamble with stock asset class selection. Other than that, I don’t worry about my stocks. With the Total Stock Market, I own large stocks, small stocks, value stocks and growth stocks so I don’t care which one does best because I own them all. Even better is I own them for the cheapest possible price and that’s guaranteed money in the bank each year in investment cost savings. My advice: Keep it simple.

————————–

* The spreadsheet I use for backtesting I call “The Backtesting Spreadsheet from Hell.” It has not only historical data, but also includes typical expense ratios of funds, Morningstar tax cost ratio for funds (with at least 10 years worth of data to capture bull and bear markets), and variable long term capital gains rates to deduct when you rebalance each year (so you can simulate a 15% rate vs. 28% for instance). Also the capital gains rates can use historical rates so you can actually see what the tax losses were for each year when you rebalanced (US avg long term capital gains rate is around 25%). It is designed not to provide the best case rosy scenario of a portfolio, but the worse case cost and tax burden on a strategy. I don’t care when things are going great as much as I care about when things are going poorly.

What I found with my spreadsheet is that more complicated portfolios with specialty asset classes largely do no better than simpler portfolios. Investors who load up on risky funds hoping to get higher returns generally give those extra returns to Uncle Sam, an advisor or the mutual fund company. In essence, you are taking on the risk, but others are getting the rewards.

I am not releasing the spreadsheet because I don’t want to support it. However I think anyone doing backtests should incorporate these expenses at a minimum to get an idea how their strategy performs under real world conditions. The conclusions I drew from it were “Keep It Simple” and “Costs matter.”

** “The Backtesting Spreadsheet from Hell” – Name based on William Bernstein’s “Retirement Calculator from Hell“. My spreadsheet has similar goals of looking at what happens when the investment returns touted in marketing brochures meet reality.

Craig Rowland

I own the place.

  • chris leow

    Don’t know about share investing but Malaysia is booming thanks to China, so if we need money we just job hop to a higher salary ! We in Malaysia are enjoying high growth and high inflation, I do not understand all this complaining. We have to thank China for our strong growth as our economy was going down until March 2009 and China rescued us by buying our commodities. Currently there is strong job market, 2 jobs for every worker, we have to import in foreign labour to do jobs that locals do not want to do ! We have high inflation, an example is a local dessert called “cendol” selling for $1.20 in local currency a month ago, is now selling for $ 1.80 in local currency. Thats a hefty increase, so don’t complain, enjoy the boom !

  • Max

    The oldest DFA fund is the micro-cap fund. From 12/23/81 (inception) to 12/31/08 it returned 10.79%. For comparison, from 1/1/82 to 12/31/08 the Vanguard 500 index fund returned 10.41%.

  • http://nedsferatu.com Nedsferatu

    thanks for the post, i really enjoyed it :-)

  • Matthew

    A few points to consider:

    1) All slices of stocks tend to be rather correlated so substituting one for another will still maintain the proper operation of the PP. Consider that Browne suggested actively managed small growth funds for a while and later suggested just a S&P500 index because it was simpler and less prone to manager error.

    2) A TSM fund may technically contain small stocks but, they have no impact on performance.

    3) Expenses are very important, but you selected a strawman for the small-value fund example. Conside VBR, a vanguard small-value ETF with 0.11% expense ratio and low turn-over. Or IWN with higher yield and 0.25% expense. ETFs are quite tax efficient as I am sure you know.

    4) This conversation wouldn’t be worth having except for the little intsy wintsy fact that small-value stocks UTTERLY CRUSH the TSM over the long term. In fact they outperformed in 97% of the rolling 20 year periods since 1928 (see Figure 9-9 http://www.ifa.com/12steps/step9/step9page3.asp). $1 dollar invested in the TSM in 1928 would have turned into about $1,265 after 81 years but $1 dollar invested in small cap stocks would have turned into $17,230 (http://www.ifa.com/images/12steps/step9/GrowthofDollarover81Year.jpg).

    5) As for the size and value premiums disappearing, I don’t have enough time/space to discuss the competing theories for why this may or may not take place. Just consider that it has not happened yet even though the secret has been out. For this too take place investors in aggregate would have to start getting as excited about $5mil cap concrete manufacturers as AAPL and CROX ;-)

    Thank for the great blog – keep it up!!

  • http://www.crawlingroad.com craigr

    Hi Matthew,

    Let me respond first by saying I’m not opposed to changes in the allocation that can improve performance. After all, I sometimes recommend people use ST treasuries for the cash allocation once they get beyond a year or so in living expenses covered in the Treasury MMF. But I do this knowing what the risks are. If someone is aware of the value tilting risks then I’m not going to tell them not to do it. I just think that there are some questions that need to be better answered and that keeping things simple has its own advantages.

    Also I should add that many years in the investment markets have taught me to be extremely skeptical of all claims of a better mousetrap. If someone is offering returns that beat the market my red flags go up simply because history has shown that these methods do not hold up to scrutiny.

    Now, to your points:

    “1) All slices of stocks tend to be rather correlated so substituting one for another will still maintain the proper operation of the PP. Consider that Browne suggested actively managed small growth funds for a while and later suggested just a S&P500 index because it was simpler and less prone to manager error.”

    This is true and not true. Yes, stocks tend to move together, but it is also true that some asset classes can move ahead or behind of others over stretches of time. After all, the principle of the value tilt argument is they tend to move higher than a broad based index fund. But this means they can also lag the index, too.

    It is also true that the early portfolio Browne tried to pick funds that would add Beta to the stock allocation. When I asked his longtime partner, John Chandler, about why they stopped this advice he said that the problem is the markets tend to erase these advantages as the strategies of outperformance gain popularity. Eventually they found it just wasn’t worth the hassle.

    “2) A TSM fund may technically contain small stocks but, they have no impact on performance.”

    It is true that small stocks make up a relative tiny part of the index fund. The primary advantage the index has is in overall tax efficiency and costs as companies are not being constantly added and removed during index reformulations. The only reason a company typically leaves the index is when they go bankrupt. It is also, I believe, the most efficient way to capture market returns.

    “3) Expenses are very important, but you selected a strawman for the small-value fund example. Conside VBR, a vanguard small-value ETF with 0.11% expense ratio and low turn-over. Or IWN with higher yield and 0.25% expense. ETFs are quite tax efficient as I am sure you know.”

    This is not a Strawman because these funds were only recently made available at those fee price points. Before that you were paying much higher fees to access these sectors and some funds still charge those higher fees.

    ETFs can be tax efficient but it depends on the index. The Russell 2000 for instance has very high turnover and is not tax efficient for small stocks. The S&P 600 (and value versions) with the iShares ETF (ticker: IJR and IJS) are better constructed and executed to reduce taxes. If I had to choose between the two for small value tilting I’d use the S&P 600 Value version.

    Even then, these funds have only around five years of data for tax efficiency. It is quite likely their tax costs will continue to go up if we hit a sustained bull market in stocks.

    “4) This conversation wouldn’t be worth having except for the little intsy wintsy fact that small-value stocks UTTERLY CRUSH the TSM over the long term. In fact they outperformed in 97% of the rolling 20 year periods since 1928 (see Figure 9-9 http://www.ifa.com/12steps/step9/step9page3.asp). $1 dollar invested in the TSM in 1928 would have turned into about $1,265 after 81 years but $1 dollar invested in small cap stocks would have turned into $17,230 ”

    I’m skeptical of data from IFA. They created many of their indices for comparison from piecing together index funds results from unrelated funds. I posted about this at the Diehards forum, but point you to their webpages where they say, among other things:

    http://www.ifa.com/btp/
    http://www.ifa.com/library/support/data/IFAindexdata.asp

    “””Backtested performance also differs from actual performance because it is achieved through the retroactive application of model portfolios (in this case, IFA’s twenty index portfolios) designed with the benefit of hindsight. As a result, the models theoretically may be changed from time to time to obtain more favorable performance results. “””

    Also it was simply impossible to have invested a dollar back in 1928 in this asset class anyway. The people pushing these funds took a bunch of data and found what worked best in hindsight. Plus there could be survivorship bias involved and they are definitely using simulated data for parts because the records of small publicly traded companies are not as complete as the larger entities. Not only that, but there are no fees deducted from these findings and there is a conflict of interest in the research. IMO.

    The conflict comes from the fact French and Fama both sit on the board of DFA who make their money by promoting funds that use their ideas. I’m not saying they are being untruthful, but I do think there MAY be an incentive to look for evidence that only supports their findings.

    “5) As for the size and value premiums disappearing, I don’t have enough time/space to discuss the competing theories for why this may or may not take place. Just consider that it has not happened yet even though the secret has been out. For this too take place investors in aggregate would have to start getting as excited about $5mil cap concrete manufacturers as AAPL and CROX ”

    Well I remember now that the broker bragging to me about using value index investing for their clients was Merrill Lynch. I suspect they aren’t the only ones letting their clients in on this secret. Also, there are myriad of cheap value index options available today that didn’t exist even 10 years ago. So it’s quite easy for anyone to adopt this strategy of value tilting.

    I’m not so confident that any historical edge that did exist is going to remain. It’s far too easy and cheap today for any Joe on the street to value invest and I think this is definitely going to have an impact on the approach. We’ll just have to see I suppose.

    Thanks for your comments. Heck, if it turns out in a few years that I’m wrong I’ll happily concede. I’m all for higher performance with the same or less risk. But my natural skepticism is preventing me from getting back onto the value tilting bandwagon (I was on it before).

    But I don’t think you’re going to do that much potential damage to the portfolio if you decide to value tilt. Just be aware of the costs and the possibility that it may not work out as planned.

  • Max

    “2) A TSM fund may technically contain small stocks but, they have no impact on performance”

    Over the last 10 years, a very good period for small-caps, TSM has outperformed the S&P500 by about 1%/year. Roughly equivalent to 80% S&P 500 / 20% small-cap.

  • Heather

    Hi Craig,
    Are you concerned about the PP’s performance if/when long term bonds under perform over long periods? It appears this has not happened since 1972 and I am concerned about the PP under performing if long bonds decline over a long period. Please advise. thxs

  • http://www.crawlingroad.com craigr

    Anything could happen. There could be some combination of events that could hurt it in a way not foreseen.

    But even assuming that this concern is valid (I think LT bonds were paying around 6% in early 1970s??? and were up to the low teens by early 1980s), how do we know if/when it is going to happen or how it is going to unfold? During the 70s LT bonds severely underperformed for an entire decade because inflation was so bad. But the gold managed to do OK and eliminate the losses.

    So we’ve had a period of bad LT bond performance and the portfolio did OK. The past does not predict the future, but we at least have one extended period of time where things worked out.

    Then again, those 4% LT bond rates we have today could look pretty good if we get a Japan style slump where their LT bonds have languished in the 2% range for 20 years now.

    Harry Browne talked about the bond questions in his radio shows where he used to get from people and how they could never buy them because they were so expensive only to watch them get more expensive.

    In the end, you just can’t predict the future.

  • Pingback: Complicated costs. Simple saves. | Crawling Road