Why It’s So Hard for Even the Best Investors to Beat the Market

This illustration shows four people trying to maintain profits by picking a money tree to represent investing.
Getty Images

ScoreCard Research

In investing, there’s a competition that’s a lot like “The Tortoise and the Hare.”

Remember the story? The tortoise, exasperated with the hare’s incessant antagonizing, challenges him to a race. 

Our industrious tortoise doggedly pursues his objective. But the hare, confident of his victory, takes several detours. Hearing the commotion in the distance as the tortoise approaches the finish line, the hare makes a furious dash — only to lose to the tortoise by mere inches. 

The moral: Slow and steady wins the race. 

There’s a parallel in the debate between active management vs. indexing.

Active management is viewed as the faster, sleeker, more sophisticated approach to investing. 

Indexing, on the other hand, with its low fees and academic theorizing, is seen as a strategy for ivory tower academics and unsophisticated investors. 

But which one is the better investing strategy?

What Is Indexing?

First off, let’s distinguish between an index — a noun — and index-ing which is an approach to investing. 

An index is simply a list of securities — usually stocks or bonds — grouped together according to some predetermined criteria, such as:

  • Price
  • Percentage of overall market value
  • Location (domestic vs. international)
  • Revenue growth
  • Credit quality

Some examples of larger indexes you may have heard of include the Dow Jones and the S&P 500 Index. But there are literally thousands of indexes measuring just about every kind of investment or investment strategy imaginable. 

Indexing is the act of investing in a particular type of investment vehicle, such as an exchange-traded fund, that tracks an underlying index.

Frequently, indexing is described as “passive” management, though this is somewhat of a misnomer for two reasons.

First of all, passive investing can include investment strategies beyond indexing. Buying and holding onto a handful of stocks, for example, can also be considered a passive approach to investing. Indexing, on the other hand, is a specific approach to investment management that seeks to replicate and track the performance of a particular market index. 

Secondly, replicating an index by trading the individual stocks or other securities is an incredibly intense and proactive endeavor.

What Is Active Management?

Active investment management is any investment decision that rests on the assumption that an investor will be able to earn better returns than the market average, as reported by one or more indexes.

The index is used as a benchmark to measure an investment manager or strategy against.

Intuitively, active investment management makes all the sense in the world.

Shouldn’t anyone with a little business savvy should be able to discern a superior investment opportunity from an inferior one? And shouldn’t professionals who spend most of their waking hours analyzing investments and the economy be that much more likely to improve upon the performance of the collective masses constituting the “average” investor? 

The answer is frequently “no.” 

Indexing vs. Active Management: Which Is Better?

Hands down, the primary advantage of indexing vs. active management is the cost. 

Without an army of analysts, office space and other overhead, index funds and ETFs can be managed for very low costs. 

Consider the difference in the expense ratios, which is the fee for managing an investment and is calculated as a percentage of its total value.

Most actively managed investments (usually mutual funds) charge around 0.5% to 1.5% in management fees. But an index fund may cost as little as 0.1%, or $10 for every $10,000 you have invested.

Some brokerage firms have even begun to offer their own proprietary funds linked to various indexes for ZERO management fees. (Don’t worry — they still make money in other ways.)

Most active investment managers fail to outperform their indexed counterparts. 

Even the best active investment managers have, historically, only managed to outperform their respective benchmarks by around 0.25% to 0.5% after fees. Most do not even break even. 

Considering the likelihood of identifying these successful managers in advance in the first place, this is hardly enough to alter the fundamental principles of sound financial management.

Does that mean that there is no role for active investment management at all? Not necessarily. 

Proponents of active management argue that as the number of active managers and the fees they charge decline, the opportunity for active investment management could improve. 

The Moral of the Story: Just Keep Investing

The choice between active management vs. indexing — even after taking fees into account — is FAR less important than deciding how much to save (or spend) in the first place. 

Investing in a well-diversified portfolio, whether indexed or actively managed, eliminates your risk of losing all your savings because a single company goes into bankruptcy or default. 

While you can eliminate this type of risk by not investing in a single company, you can’t eliminate the risks associated with the market, no matter how diverse your investments are. Such risks include:

  • Recession
  • Runaway inflation
  • Political turmoil
  • Anything impacting investor sentiment 

But indexing is hardly a low-risk alternative to active investment management.

As anyone who was invested during the dot.com bubble or 2008 financial crisis will tell you, although (relatively) rare, stock index funds can and do lose 50% or more of their value — depending on the type of stocks owned by the fund — during a severe market downturn or financial crisis. 

It’s important to consider your risk tolerance relative to your objectives.

From there, you can set realistic expectations for the returns you can expect from investing, choose an asset allocation that’s appropriate for your risk tolerance and decide on an appropriate amount to save each pay period to have a reasonable chance of achieving your goals.

Whatever investment strategy you decide is a fit for you, be sure to go into it with a firm understanding of what you should expect. 

At that point, whether you’re a tortoise or a hare, you should be well on your way to successfully running your race — no matter who comes in “first.” 

David Metzger is a fee-only wealth manager in Chicago. He is a certified financial planner (CFP) and a chartered financial analyst (CFA). He has taught courses on personal financial planning and investing at DePaul University in Chicago and Christian Brothers University in Memphis, Tennessee.