How Does Statistical Arbitrage Work?

 

Statistical arbitrage is the term usually involving both long and short hedged positions or paired trades that tend to revert to the mean historically. You are using anything with a statistical edge and usually using mean reversion strategies. This works well when there is a volatile trading environment and lower correlation among assets. THe period from 1997 to 1999 tended to trend more and mean revert less as everyone was selling everything to buy dot com stocks and capital kept flowing into internet stocks. As the crowd began to participate the smart money then began to sell early and buy stuff that had been down so from 1999 to 2000 and beyond you saw statistical arbitrage begin to outperform, despite the rotation and sell offs being sharp and continuing as the market entered bear market.

There are several narratives for why statistical arbitrage occurs, but one example I like also displays the potential for a variation of statistical arbitrage involving cash vs a long or short position provided you are able to totally mitigate margin risks. As that is not practical without using very small position size a hedge is needed and then you end up with more of a standard statistical arbitrage. So instead let's only use a long only position vs cash to illustrate.

Let's assume a totally random market. How would you gain? Some may say it's impossible, but they must not have read or heard about Claude Shannon in detail because he was a brilliant man accomplished for a lot of reasons that also constructed a model for beating a random market. Modern day game theory would come to the same conclusion which is identifying what is called "nash equilibrium". Positioning yourself in "nash equilibrium" in some conditions (this is one of them) allows gains to the degree which opponent makes mistakes. In other situations like a game of rock paper scissors it only mitigates chances for opponent to exploit patterns by going equal parts rock, paper and scissors and using some random generation method.

In this case, with no directional bias and only two assets, equilibrium would be 50% cash and 50% stock. As cash becomes more valuable vs the stock position (stock goes down, you add enough stock to create a new equilibrium. As stock rises you sell enough or buy cash position using stock to maintain this equilibrium. Given enough volatility and enough time for prices to normalize your net worth would rise over time. If you wanted to simply recover to new highs faster but with less gain and less volatility to total net worth, you could trade more capital to the less volatile position (cash) and maintain that allocation. The allocation of say 25% stock, 75% cash would still work, it would just produce less gains.

Nash equilibrium isn't necessarily the best possible strategy for maximizing gains vs all other strategies as it assumes oppponents are all of equal and perfect skill and the moment participants begin to switch strategies, a new strategy becomes dominant and players would adjust until no one have an advantage. If for example market participants were dramatically under positioned or over positioned and you had reason to believe it had reached an extreme, you could go 100% long or more. If they were over positioned and you could go 100% cash or even go short. In between the extremes, it's possible you can identify trends towards extreme and not adjust your strategy until the extremes are reached. The point is not that you should seek this strategy out if it isn't right for you, but to explain that even though equilibrium may not make money as fast as another strategy, it can still profit regardless of what others do over enough time and enough volatility

While mentioning this it's important to keep in mind that statistical arbitrage is NOT necessarily an equilibrium strategy itself as calculating that would be too difficult but it in general benefits for similar reasons in that capital flows as people move away from equilibrium and it eventually finds its way back and away again in whatever direction it does for whatever reasons that are only obvious in hindsight.

It's possible because of how many variables influence strategy of actual participants that there is no actual equilibrium and instead it's something that always changes. Equilibrium for a more complicated market is difficult to identify, but you can create something that functions in a similar way.

This sort of illustrates why statistical arbitrage works. Anyone who is aware of the possibility would generally be able to buy a basket of stocks vs an equal amount cash position, and so if market was out of equilibrium after a sell off in any given stock, the risk is to the upside. RSI measures buying and selling pressure so you are buying positions that have become underowned relative to the time frame at a fast enough rate for long enough time.

When the selling pressure cannot continue as people have already sold that wanted to, the least amount of buying pressure can produce the fastest amount of gain.

Statistical arbitrage is a little bit of contrarian strategy only the RSI 5 only refers to 5 day period so this is only contrarian on the short term time horizon and some stocks could be in an uptrend for years while others could be trading near their lows so it isn't a contrarian strategy on any other time frame but the given time period of the strategy.

Another explanation is emotion. People tend to emotionally overreact and sell now and ask questions later. when stops get triggered it can trigger more stops and soon people find themselves out of the position. Once emotions normalize, they realize they didn't need to sell everything and these previous holders may become buyers at higher prices

 

Above you will see a statistical arbitrage strategy that goes long when the RSI 5 is below 20 and takes profit and sells the full position when it is above 50 and sells short any stock above an RSI of 90 and takes profit when it is below 50. It uses any stock in the Russel 3000 that meets the condition and doesn't adjust the number of positions to maintain some kind of balance and instead relies on what the market offers. While this is a little different than typical statistical arbitrage that usually does create balance, it's close enough for you to understand the effectiveness.

Shorting stocks is a little more dangerous because stocks can go up more than 100% requiring you to come up with more than double the capital you started with. If you are only short you are at significant risk of this happening. Nevertheless, you can see it does well in bear markets and even was able to find some trades that worked

We can isolate how it did from May 2007 to June 2010 to get an idea for how it performs just before and just after a bear market.

Above is a long only strategy which produces a greater compounded rate of return but does so at greater volatility and has a lower sharp ratio as a result. Theoretically leveraging up a hedged strategy or reducing down the position sizes of the long only portfolio so that they have equal levels of volatility would make the long AND short portfolio superior for return and it produces a greater return on risk.

However, market timers may instead select a strategy right for market environment, or at least
position 60/40 long or 40/60 short depending on predicted market conditions.

While not everyone will be a statistical arbitrage trader nor should they be, we can learn some important lessons from it. If you are trading a particular strategy that does NOT work in almost all market environments, you should be sure to have some means to measure what environment your trading strategy works in. If you trade momentum breakouts, look for growth periods and anticipate trending markets. If you trade price patterns, you may wish to consider buying dips in anticipation of breakouts or post breakout moves to retest if that strategy works for you. And if you decide to trade a particular strategy, understand that it may not work in all market environments and be willing to do some work identifying the reasons why or the conditions in which the strategy works.

Another important understanding is that you may wish to do a periodic review to determine if what was working is still working. Below in green shows buying falling wedge breakouts and holding for 3 months. It worked well from 2011 to 2014 but since then at least until July 2017 it no longer has worked. That may be because the conditions for buying or selling have changed, (buying the breach of the low and selling into the breakout may still work) or the pattern itself has become less effective vs alternatives. You would have to backtest a number of variables to decide and even then it only works in hindsight, so be mindful of the conditions we are in, what made that strategy work in the past, and what made it not work and what you expect moving forward if you are going to use that strategy.

 

Historically growth has not spent a ton of time outperforming value, but when it does it can make some really strong moves near market highs like in 1999-2000. Lately Growth has outperformed around the time period shortly before falling wedge breakouts with 3 month holding period began failing.

Some things in market are related and others are not, and conditions can quickly change so in order to remain a leg up on the competition you have to determine what role volatility has in the strategy and what the volatility is (measured by the VIX for example).
Notice whether we are in trending or mean reverting markets and bull or bear conditions and if there are any anticipatory signals for conditions changing. Notice how long trends are lasting and what correlations are.

Just a bit of information can give you a great overview if you track them, or you can track market internals looking at individual stocks, industries, sectors, etc. which perhaps is more  complicated but when the market doesn't always move with stocks (correlation is low) then using the broad market as your only indicator may miss opportunities to identify sectors and industries that work according to your system or what you are most comfortable with and what fits your psychological make up.

Even a winning system doesn't mean much if you have poor execution so you had better find out what works for you personally and always manage risk. Statistical arbitrage can work for some if they align the time frame and market conditions and risk management appropriately but it may not be best for everyone nor may any other strategy mentioned, but hopefully this sort of detailed analysis as to what works and why will give you a thin slice of things you can look at to determine what works best for you.

Even passive investing can look at changes that lasts from months to years and have 3 types of changes they can make and look for longer term extremes to shift allocations from a "green light" to "yellow light" or "red light" type of strategy going from conservative allocation to stocks and preferring slower moving stocks and bond markets can be looked at separately and aggressive small cap or growth stocks and greater allocations from extreme lows or a monthly signal that momentum has shifted from down to up and another signal when things get extreme and markets may stop trending may shift them to neutral. There are different ways to every approach that can work, but regardless of what works for you, be aware that there are conditions in which changes to that strategy may be preferable. Additionally there are a few strategies that may work in any conditions, but even so volatility and account drawdown may increase or decrease requiring adjustments in position size or potential hedging in certain time periods if you want to reduce the potential drawdowns to a range you are comfortable with.  Even day traders can be away of both the long and short term conditions in which their edge may change. If they wait for movement to scalp a trade or look for some kind of daily breakout then higher volatility markets may require adjustments CHanges to margin rates and volatility may impact speed of moves and changes to algorithmic high frequency trading may also change your edge. There are always things changing and even legislation may pass that change the rules of business and taxes and investment and cause reactions and behavior and paradigm shifts. The market isn't like blackjack or poker where the odds are always the same, it's like being blindfolded and spun around in circles and sitting down at a different table with a different game and different rules. Fortunately everyone else is too and you have a lot of time to monitor the conditions and determine the rules that fit the conditions best.

Modeling expectations of a portfolio Part 1

Some people like to use a combination of options, long term investments and short term trades in a more complex portfolio. This can be tricky, particularly if the amount of capital used for each changes with the conditions of the market.

However, you can still model an entire portfolio over a time period with given assumptions. Just be sure you also test the extreme end of assumptions that would be an unprecedented result. For instance, although we've had a 20% loss in a day in 1987 crash, what about a 60% decline in a month? This sort of decline would be very very very rare, but if an event like this wipes you out erasing decades of gains, your system may not even be profitable at all in the long run due to the very rare event.

Let's start with a hypothetical coinflip to keep things simple. Heads you net double what you risk. Risk 1, win 3, profit 2. Tails you lose 1.

If we flipped a coin and then adjusted our bet based upon the new position size we can calculate our return over several flips very simply. Assume 50 heads and 50 tails.
Let's calculate the win result for 50 heads. For a position size of 1%.
It's 1.02^50  or 1 plus the win ratio times the position size. This equals your multiplier factor. The result is 2.691588. in other words, over 50 wins your wealth will have multiplied by ~2.7 or an increase of ~170%.
Now we also have 50 losses. Since each loss we will adjust our position size, the wealth will decrease by a factor of .99 per loss and size will be adjusted downwards. .99^50=0.60500607 or about 60% of what we started with if we lose 50 times in a row or a loss of about 40%.
So now we can combine them ~2.7*~.60=~1.62 or a 62% gain over 100 flips.
But hold on just a second, it's very possible that you only get 40 heads or perhaps only 30. We need to test the outliers in terms of number of flips as well as the magnitude of the results (how big each individual win or loss can be). Since this trade is fixed (the upside does not change and the downside does not change) we don't have to adjust the magnitude of the win. But we should test for what sort of drawdown we are looking at at 1% position size if we lose 70/100. The results are (1.02^30)*(.99^70=~.896 or a loss of ~10.4%. What about 20/80? THe loss would be closer to 33.5% in a year. what about 10/90? Results are a loss of about 51%. You can see how even modest positioning with limited upide and a clear edge can still produce a lot of volatility over time if you go through a rough streak. We can test the upside as well if you'd like. 90 heads to 10 tails would be 437.5%. 80/20=299% 70/30=195.85%.

This is similar to a hypothetical system, but it is not equal. Before we go looking at more complex outcomes with multiple outcomes, let's look at smaller as well as larger position sizes. For now let's summarize 1% position size.

Expected results (50/50)=62.8% gain per 100 flips
outside range results 40/60 to 60/40= 20.8% to 119.5%
extreme range (30/70 to 70/30) = -10.37% to 195.8%
very extreme range 20/80 to 80/20 = -33.5% to 298.8%
unprecedented extreme 10/90 to 90/10 = -50.67 to 437.48%

I'll go ahead and do the work to summarize 5% position sizes and 0.25% position sizes, but for those looking to make sure they do it right, it's position size times expectation plus starting portfolio for one trade. For instance starting portfolio in this case will always be 100% or 1. expectations in this system will always be either minus a full position or plus 2 full positions. So if position size is 5% a 5% loss from the 100% will leave us with 95% of what we started with or .95. A win will leave us with 10% plus what we started with of 1.10. So now we can build out the results table for 5%

Expected results (50/50)=803% gain per 100 flips
outside range results 40/60 to 60/40= 8.5% to 3812%
extreme range (30/70 to 70/30) = -51.86% to 16851%
very extreme range 20/80 to 80/20 = -88.89% to 733.32%
unprecedented extreme 10/90 to 90/10 = -97.43% to 318010%
The biggest problem with this strategy in real life is there is very little ability to separate a broken system from an extreme outcome and by the time you figure it out, the damage can already be done.

The other extreme of 0.25% positions will see .9975 of what we started with per trade or 1.0050

Expected results (50/50)=13.27% gain per 100 flips
outside range results 40/60 to 60/40= 5.1% to 22%
extreme range (30/70 to 70/30) = -2.5% to 31.5%
very extreme range 20/80 to 80/20 = -9.50% to 41.75%
unprecedented extreme 10/90 to 90/10 = -16.1% to 52.78%

We can see the dramatic difference in outcomes. Just for fun, let's see what happens if we position size way too large at 60%.
expected gain at 50/50=~-100%*. Even a winning strategy with less than 100% going as expected results in ruin if we bet too large. *The exact loss is 99.83%

While we could simulate more realistic trades with options and more than 2 possible results to more accurately model the outcomes there is another problem we run into comparing reality to the theory as currently modeled. In reality you won't wait until one trade is done to start another. So we will stick to the coinflip argument until we resolve it.
This simultaneous trading provides both benefits (in that you can put more capital to work while less is at risk since the odds of a lot of small trades simultaneously not working out at once is less than if all that capital were tied up in one trade) as well as a hindrance (In that while losing 40 1% positions over 40 trades will only lose you 1-(.99^40)=.33 or 33%, 40 losses simultaneously will lose you 40. In addition while 40 1% coinflips that win one at a time will compound and win you 1.02^40=~2.2 or 120%, when placed all at once they will only win you 40%.

Still, it's far less riskier with far better results to bet 1% on 40 coinflips than 40% of the capital on a single coinflip. Even as we will transition into a model that models actual trades, no two trades are exactly correlated, so there is still a bennefit for diversification (before fees) that can act to reduce risk if used properly.

Nevertheless, we have to change the calculation entirely to how many simultaneous trades over a period to where it adds up the sum of all results as opposed to multiplying. For instance, if we have 1% position with 30 wins and 20 losses simultaneously, rather than (30^1.02) * (20^.99)=1.48 or 48% return, the calculation is (1.02*30)-(.01*20)=.40 or 40% return if we have multiple overlapping periods like this we can add the separate periods. For instance if we had 2 periods of 30 wins and 20 losses, we can still adjust position sizes after a period so the result is 1-(1.4^2)=.96 or 96% instead of just 40%*2 or  40%+40%=80%.

In reality, a very important variable determining success or failure when using overlapping trades is how correlated each trade is to another. For instance, if all you have are calls in each individual S&P stock expiring at the same time, you are going to be very exposed to a broad market sell off during that time, and you might as well have a single call in the S&P. There is no real diversification of risk if you are exposed to the same correction in the broad market or rotation out of it. Having 40 1% of various S&P stocks is about the same as having a 40% position on for the S&P over that period. If instead you have a variety  of strike prices across a variety of time periods and each are in a variety of underlying stocks in different sectors of the market as well as focusing on the underowned stocks, then a rotation from the overowned into the underowned or the big cap into the small or the index correlated to the uncorrelated can actually be your gain as the market declines. However, 40 1% positions bought over approximately the same way (within the same price range or time range) with approximately the same expiration date (within a month of each other) even if you are seeking out underowned names is still vulnerable to the type of everything down correlated sell off common during major corrections (down 15-20%) and bear markets. Although there will be numerous sell offs where you are not impacted by uncorrelated sell offs, the selloffs that are correlated can do enough damage where you better plan for a strategy that can weather multiple extreme moves over a few years and still be profitable over a time period, otherwise you will need to position differently or adjust your strategy. One such adjustment may be buying insurance in the form of index puts over a long period of time to pay the minimum and then rolling that time premium as needed. This doesn't protect against a long period where your individual trades fail to gain traction or where the market doesn't ever rotate into underowned names and instead is lead by the names leveraged to the index.

Consider that when modeling against all extremes which I will show you how to do briefly. Remember when I showed you the table of expected return, outside range, outside extreme etc? One thing you may wish to do is run a calculation for a 10 year period that contains a couple outside extremes to the downside, one to the upside and the rest either in the normal or outside range or results. You can test different position sizes over this time period. Keep in mind that it will be very easy and possibly appropriate to adjust following a significant decline as you don't know whether or not that was just an extreme or representative. You don't know if your expectations moving forward will equal the past, be better than or worse than the past. Emotionally, you may also determine you've had enough. For this reason, your effective bankroll may actually be smaller than you calculated and your proportional risk to that bankroll may be higher than you realize.


We are ultimately going to seek out a balance between multiple strategies across multiple portfolios to try to mitigate risk of our overall wealth and obtain our goals.
For instance one such overall wealth building strategy may look something like this.
1)401k aims to capture the very long term wealth effect over multiple credit cycles (usually 7-11 years) as well as overall movement from and to stocks and other assets like bonds, and may reposition to try to capitalize off of moves lasting months to years.
2)Roth IRA aims to bennefit from swings in stocks lasting weeks to months.
3)Individual option account combined with individual stock accounts aim to insure against the longer term moves when insurance is relatively cheap and in appropriate times, while primarily seeking the non-correlated moves and counter moves and spotting individual bullish and bearish setups when appropriate. The taxible individual trading accounts will dynamically adjust position size by having some exposure to the index and some exposure to cash and hedges and increasing or decreasing that exposure as the short term and long term trades swell up in size to maintain a proportional bias consistent with expectations and avoid overexposure to any one period of time or expiry or overallocation to risk.

This is just one such possible idea, but another involves complex rotation. Another variable is market condition. certain strategies do better during sideways markets, others during quiet markets, others during choppy markets, others during bull and others during bear market and combination of the 2 variables of volatility and trend/direction (or no direction if it's a sideways correction through time).

Here is one such strategy I sort of tentatively mapped out. The idea is you get some market correlated names and perhaps include owning companies run by money managers like Warren Buffett like birkshire hathaway shares. At the time I made this I mistakenly thought WCC (Westco Financial) was still run by Charlie Munger, but there may be some sort of publicly held company that has a large float of investible securities that is run by a quality asset manager. The idea is to own market correlated names that you believe in over the long run to equal perform or outperform. Your 401k might also be allocated into long term outperform but in a more permenant sort of way so you may view that separately. Then you may consider a basket of commodities that start as a very small position and you might buy for the long term at historically significant prices when possible. You might have a currency trading account or just trade calls/puts  in currency ETFs to place trades that move differently from the market itself. You also may want to own some proportion of long term corporate bond and treasury bonds, however, we are at a historical period where rates are probably rising so keeping this percentage very small or finding some way to hedge against rising rates on the extreme end (perhaps long term puts in TLT) may be appropriate.

Then you have your stock timing portion of your portfolio that tries to maintain a mixture across varying strike prices. And finally you have some hedging capital and cash, and since you won't need all of your cash at even extremes, some percentage can be held in income.

The design of this portfolio is very important to understand. The idea is that overall some percentage of your portfolio will be allocated to "risk on" and some percentage to "risk off" That should be consistent with your expectations of winners vs losers plus a buffer for your personal tolerance to varience over time. However, individual option trades that post huge wins can expand and contract. This is where the balancing act comes in in the form of rebalancing your allocation strategy appropriately. For instance, if your stock and option trades are working well, they will expand in size and overall you will be more exposed to violent corrections. As such, you simply reduce your market correlated calls or shares in S&P or leveraged ETFs like SPXL or TNA to maintain the desired balance. Beyond that if you are still more risk on than you want, you can purchase a hedge.

The goal of holding commodities and bonds is to capture rotations in capital into and out of the stock market as well as provide the buffer for your individual stock timing/outperforming portion of your account(s). The goal of having large cash and income amount is both for future opportunity of better prices as well as some degree of protection from deflation as well as reducing account volatility to desirable size, particularly if you are seeking maximum volatility. The goal of having market correlated names is to both have something to balance out growth in stocks and to generally add lower and reduce higher while options are by design going to increase exposure higher and decrease lower and some trading strategies involving stocks or options may stop out lower and add higher.

In order to optimize the portfolio according to risk tolerance and our goals we will need to run several calculations like the calculations that we already did stress testing the portfolio against hypothetical extremes and determining the blend that we desire. That will be for another time.
The goal will be to accomplish a goal of return (or better) while keeping max drawdown limited to a particular amount (or less)... or else over a time period to maximize the probability of an event with a secondary time period where an amount must be reached. This may put less focus on the drawdown in a given month or year, but ultimately the drawdown must be limited such that you are able to recover and obtain your goal.