How Does Statistical Arbitrage Work?

 

Statistical arbitrage is the term usually involving both long and short hedged positions or paired trades that tend to revert to the mean historically. You are using anything with a statistical edge and usually using mean reversion strategies. This works well when there is a volatile trading environment and lower correlation among assets. THe period from 1997 to 1999 tended to trend more and mean revert less as everyone was selling everything to buy dot com stocks and capital kept flowing into internet stocks. As the crowd began to participate the smart money then began to sell early and buy stuff that had been down so from 1999 to 2000 and beyond you saw statistical arbitrage begin to outperform, despite the rotation and sell offs being sharp and continuing as the market entered bear market.

There are several narratives for why statistical arbitrage occurs, but one example I like also displays the potential for a variation of statistical arbitrage involving cash vs a long or short position provided you are able to totally mitigate margin risks. As that is not practical without using very small position size a hedge is needed and then you end up with more of a standard statistical arbitrage. So instead let's only use a long only position vs cash to illustrate.

Let's assume a totally random market. How would you gain? Some may say it's impossible, but they must not have read or heard about Claude Shannon in detail because he was a brilliant man accomplished for a lot of reasons that also constructed a model for beating a random market. Modern day game theory would come to the same conclusion which is identifying what is called "nash equilibrium". Positioning yourself in "nash equilibrium" in some conditions (this is one of them) allows gains to the degree which opponent makes mistakes. In other situations like a game of rock paper scissors it only mitigates chances for opponent to exploit patterns by going equal parts rock, paper and scissors and using some random generation method.

In this case, with no directional bias and only two assets, equilibrium would be 50% cash and 50% stock. As cash becomes more valuable vs the stock position (stock goes down, you add enough stock to create a new equilibrium. As stock rises you sell enough or buy cash position using stock to maintain this equilibrium. Given enough volatility and enough time for prices to normalize your net worth would rise over time. If you wanted to simply recover to new highs faster but with less gain and less volatility to total net worth, you could trade more capital to the less volatile position (cash) and maintain that allocation. The allocation of say 25% stock, 75% cash would still work, it would just produce less gains.

Nash equilibrium isn't necessarily the best possible strategy for maximizing gains vs all other strategies as it assumes oppponents are all of equal and perfect skill and the moment participants begin to switch strategies, a new strategy becomes dominant and players would adjust until no one have an advantage. If for example market participants were dramatically under positioned or over positioned and you had reason to believe it had reached an extreme, you could go 100% long or more. If they were over positioned and you could go 100% cash or even go short. In between the extremes, it's possible you can identify trends towards extreme and not adjust your strategy until the extremes are reached. The point is not that you should seek this strategy out if it isn't right for you, but to explain that even though equilibrium may not make money as fast as another strategy, it can still profit regardless of what others do over enough time and enough volatility

While mentioning this it's important to keep in mind that statistical arbitrage is NOT necessarily an equilibrium strategy itself as calculating that would be too difficult but it in general benefits for similar reasons in that capital flows as people move away from equilibrium and it eventually finds its way back and away again in whatever direction it does for whatever reasons that are only obvious in hindsight.

It's possible because of how many variables influence strategy of actual participants that there is no actual equilibrium and instead it's something that always changes. Equilibrium for a more complicated market is difficult to identify, but you can create something that functions in a similar way.

This sort of illustrates why statistical arbitrage works. Anyone who is aware of the possibility would generally be able to buy a basket of stocks vs an equal amount cash position, and so if market was out of equilibrium after a sell off in any given stock, the risk is to the upside. RSI measures buying and selling pressure so you are buying positions that have become underowned relative to the time frame at a fast enough rate for long enough time.

When the selling pressure cannot continue as people have already sold that wanted to, the least amount of buying pressure can produce the fastest amount of gain.

Statistical arbitrage is a little bit of contrarian strategy only the RSI 5 only refers to 5 day period so this is only contrarian on the short term time horizon and some stocks could be in an uptrend for years while others could be trading near their lows so it isn't a contrarian strategy on any other time frame but the given time period of the strategy.

Another explanation is emotion. People tend to emotionally overreact and sell now and ask questions later. when stops get triggered it can trigger more stops and soon people find themselves out of the position. Once emotions normalize, they realize they didn't need to sell everything and these previous holders may become buyers at higher prices

 

Above you will see a statistical arbitrage strategy that goes long when the RSI 5 is below 20 and takes profit and sells the full position when it is above 50 and sells short any stock above an RSI of 90 and takes profit when it is below 50. It uses any stock in the Russel 3000 that meets the condition and doesn't adjust the number of positions to maintain some kind of balance and instead relies on what the market offers. While this is a little different than typical statistical arbitrage that usually does create balance, it's close enough for you to understand the effectiveness.

Shorting stocks is a little more dangerous because stocks can go up more than 100% requiring you to come up with more than double the capital you started with. If you are only short you are at significant risk of this happening. Nevertheless, you can see it does well in bear markets and even was able to find some trades that worked

We can isolate how it did from May 2007 to June 2010 to get an idea for how it performs just before and just after a bear market.

Above is a long only strategy which produces a greater compounded rate of return but does so at greater volatility and has a lower sharp ratio as a result. Theoretically leveraging up a hedged strategy or reducing down the position sizes of the long only portfolio so that they have equal levels of volatility would make the long AND short portfolio superior for return and it produces a greater return on risk.

However, market timers may instead select a strategy right for market environment, or at least
position 60/40 long or 40/60 short depending on predicted market conditions.

While not everyone will be a statistical arbitrage trader nor should they be, we can learn some important lessons from it. If you are trading a particular strategy that does NOT work in almost all market environments, you should be sure to have some means to measure what environment your trading strategy works in. If you trade momentum breakouts, look for growth periods and anticipate trending markets. If you trade price patterns, you may wish to consider buying dips in anticipation of breakouts or post breakout moves to retest if that strategy works for you. And if you decide to trade a particular strategy, understand that it may not work in all market environments and be willing to do some work identifying the reasons why or the conditions in which the strategy works.

Another important understanding is that you may wish to do a periodic review to determine if what was working is still working. Below in green shows buying falling wedge breakouts and holding for 3 months. It worked well from 2011 to 2014 but since then at least until July 2017 it no longer has worked. That may be because the conditions for buying or selling have changed, (buying the breach of the low and selling into the breakout may still work) or the pattern itself has become less effective vs alternatives. You would have to backtest a number of variables to decide and even then it only works in hindsight, so be mindful of the conditions we are in, what made that strategy work in the past, and what made it not work and what you expect moving forward if you are going to use that strategy.

 

Historically growth has not spent a ton of time outperforming value, but when it does it can make some really strong moves near market highs like in 1999-2000. Lately Growth has outperformed around the time period shortly before falling wedge breakouts with 3 month holding period began failing.

Some things in market are related and others are not, and conditions can quickly change so in order to remain a leg up on the competition you have to determine what role volatility has in the strategy and what the volatility is (measured by the VIX for example).
Notice whether we are in trending or mean reverting markets and bull or bear conditions and if there are any anticipatory signals for conditions changing. Notice how long trends are lasting and what correlations are.

Just a bit of information can give you a great overview if you track them, or you can track market internals looking at individual stocks, industries, sectors, etc. which perhaps is more  complicated but when the market doesn't always move with stocks (correlation is low) then using the broad market as your only indicator may miss opportunities to identify sectors and industries that work according to your system or what you are most comfortable with and what fits your psychological make up.

Even a winning system doesn't mean much if you have poor execution so you had better find out what works for you personally and always manage risk. Statistical arbitrage can work for some if they align the time frame and market conditions and risk management appropriately but it may not be best for everyone nor may any other strategy mentioned, but hopefully this sort of detailed analysis as to what works and why will give you a thin slice of things you can look at to determine what works best for you.

Even passive investing can look at changes that lasts from months to years and have 3 types of changes they can make and look for longer term extremes to shift allocations from a "green light" to "yellow light" or "red light" type of strategy going from conservative allocation to stocks and preferring slower moving stocks and bond markets can be looked at separately and aggressive small cap or growth stocks and greater allocations from extreme lows or a monthly signal that momentum has shifted from down to up and another signal when things get extreme and markets may stop trending may shift them to neutral. There are different ways to every approach that can work, but regardless of what works for you, be aware that there are conditions in which changes to that strategy may be preferable. Additionally there are a few strategies that may work in any conditions, but even so volatility and account drawdown may increase or decrease requiring adjustments in position size or potential hedging in certain time periods if you want to reduce the potential drawdowns to a range you are comfortable with.  Even day traders can be away of both the long and short term conditions in which their edge may change. If they wait for movement to scalp a trade or look for some kind of daily breakout then higher volatility markets may require adjustments CHanges to margin rates and volatility may impact speed of moves and changes to algorithmic high frequency trading may also change your edge. There are always things changing and even legislation may pass that change the rules of business and taxes and investment and cause reactions and behavior and paradigm shifts. The market isn't like blackjack or poker where the odds are always the same, it's like being blindfolded and spun around in circles and sitting down at a different table with a different game and different rules. Fortunately everyone else is too and you have a lot of time to monitor the conditions and determine the rules that fit the conditions best.

Modeling expectations of a portfolio Part 1

Some people like to use a combination of options, long term investments and short term trades in a more complex portfolio. This can be tricky, particularly if the amount of capital used for each changes with the conditions of the market.

However, you can still model an entire portfolio over a time period with given assumptions. Just be sure you also test the extreme end of assumptions that would be an unprecedented result. For instance, although we've had a 20% loss in a day in 1987 crash, what about a 60% decline in a month? This sort of decline would be very very very rare, but if an event like this wipes you out erasing decades of gains, your system may not even be profitable at all in the long run due to the very rare event.

Let's start with a hypothetical coinflip to keep things simple. Heads you net double what you risk. Risk 1, win 3, profit 2. Tails you lose 1.

If we flipped a coin and then adjusted our bet based upon the new position size we can calculate our return over several flips very simply. Assume 50 heads and 50 tails.
Let's calculate the win result for 50 heads. For a position size of 1%.
It's 1.02^50  or 1 plus the win ratio times the position size. This equals your multiplier factor. The result is 2.691588. in other words, over 50 wins your wealth will have multiplied by ~2.7 or an increase of ~170%.
Now we also have 50 losses. Since each loss we will adjust our position size, the wealth will decrease by a factor of .99 per loss and size will be adjusted downwards. .99^50=0.60500607 or about 60% of what we started with if we lose 50 times in a row or a loss of about 40%.
So now we can combine them ~2.7*~.60=~1.62 or a 62% gain over 100 flips.
But hold on just a second, it's very possible that you only get 40 heads or perhaps only 30. We need to test the outliers in terms of number of flips as well as the magnitude of the results (how big each individual win or loss can be). Since this trade is fixed (the upside does not change and the downside does not change) we don't have to adjust the magnitude of the win. But we should test for what sort of drawdown we are looking at at 1% position size if we lose 70/100. The results are (1.02^30)*(.99^70=~.896 or a loss of ~10.4%. What about 20/80? THe loss would be closer to 33.5% in a year. what about 10/90? Results are a loss of about 51%. You can see how even modest positioning with limited upide and a clear edge can still produce a lot of volatility over time if you go through a rough streak. We can test the upside as well if you'd like. 90 heads to 10 tails would be 437.5%. 80/20=299% 70/30=195.85%.

This is similar to a hypothetical system, but it is not equal. Before we go looking at more complex outcomes with multiple outcomes, let's look at smaller as well as larger position sizes. For now let's summarize 1% position size.

Expected results (50/50)=62.8% gain per 100 flips
outside range results 40/60 to 60/40= 20.8% to 119.5%
extreme range (30/70 to 70/30) = -10.37% to 195.8%
very extreme range 20/80 to 80/20 = -33.5% to 298.8%
unprecedented extreme 10/90 to 90/10 = -50.67 to 437.48%

I'll go ahead and do the work to summarize 5% position sizes and 0.25% position sizes, but for those looking to make sure they do it right, it's position size times expectation plus starting portfolio for one trade. For instance starting portfolio in this case will always be 100% or 1. expectations in this system will always be either minus a full position or plus 2 full positions. So if position size is 5% a 5% loss from the 100% will leave us with 95% of what we started with or .95. A win will leave us with 10% plus what we started with of 1.10. So now we can build out the results table for 5%

Expected results (50/50)=803% gain per 100 flips
outside range results 40/60 to 60/40= 8.5% to 3812%
extreme range (30/70 to 70/30) = -51.86% to 16851%
very extreme range 20/80 to 80/20 = -88.89% to 733.32%
unprecedented extreme 10/90 to 90/10 = -97.43% to 318010%
The biggest problem with this strategy in real life is there is very little ability to separate a broken system from an extreme outcome and by the time you figure it out, the damage can already be done.

The other extreme of 0.25% positions will see .9975 of what we started with per trade or 1.0050

Expected results (50/50)=13.27% gain per 100 flips
outside range results 40/60 to 60/40= 5.1% to 22%
extreme range (30/70 to 70/30) = -2.5% to 31.5%
very extreme range 20/80 to 80/20 = -9.50% to 41.75%
unprecedented extreme 10/90 to 90/10 = -16.1% to 52.78%

We can see the dramatic difference in outcomes. Just for fun, let's see what happens if we position size way too large at 60%.
expected gain at 50/50=~-100%*. Even a winning strategy with less than 100% going as expected results in ruin if we bet too large. *The exact loss is 99.83%

While we could simulate more realistic trades with options and more than 2 possible results to more accurately model the outcomes there is another problem we run into comparing reality to the theory as currently modeled. In reality you won't wait until one trade is done to start another. So we will stick to the coinflip argument until we resolve it.
This simultaneous trading provides both benefits (in that you can put more capital to work while less is at risk since the odds of a lot of small trades simultaneously not working out at once is less than if all that capital were tied up in one trade) as well as a hindrance (In that while losing 40 1% positions over 40 trades will only lose you 1-(.99^40)=.33 or 33%, 40 losses simultaneously will lose you 40. In addition while 40 1% coinflips that win one at a time will compound and win you 1.02^40=~2.2 or 120%, when placed all at once they will only win you 40%.

Still, it's far less riskier with far better results to bet 1% on 40 coinflips than 40% of the capital on a single coinflip. Even as we will transition into a model that models actual trades, no two trades are exactly correlated, so there is still a bennefit for diversification (before fees) that can act to reduce risk if used properly.

Nevertheless, we have to change the calculation entirely to how many simultaneous trades over a period to where it adds up the sum of all results as opposed to multiplying. For instance, if we have 1% position with 30 wins and 20 losses simultaneously, rather than (30^1.02) * (20^.99)=1.48 or 48% return, the calculation is (1.02*30)-(.01*20)=.40 or 40% return if we have multiple overlapping periods like this we can add the separate periods. For instance if we had 2 periods of 30 wins and 20 losses, we can still adjust position sizes after a period so the result is 1-(1.4^2)=.96 or 96% instead of just 40%*2 or  40%+40%=80%.

In reality, a very important variable determining success or failure when using overlapping trades is how correlated each trade is to another. For instance, if all you have are calls in each individual S&P stock expiring at the same time, you are going to be very exposed to a broad market sell off during that time, and you might as well have a single call in the S&P. There is no real diversification of risk if you are exposed to the same correction in the broad market or rotation out of it. Having 40 1% of various S&P stocks is about the same as having a 40% position on for the S&P over that period. If instead you have a variety  of strike prices across a variety of time periods and each are in a variety of underlying stocks in different sectors of the market as well as focusing on the underowned stocks, then a rotation from the overowned into the underowned or the big cap into the small or the index correlated to the uncorrelated can actually be your gain as the market declines. However, 40 1% positions bought over approximately the same way (within the same price range or time range) with approximately the same expiration date (within a month of each other) even if you are seeking out underowned names is still vulnerable to the type of everything down correlated sell off common during major corrections (down 15-20%) and bear markets. Although there will be numerous sell offs where you are not impacted by uncorrelated sell offs, the selloffs that are correlated can do enough damage where you better plan for a strategy that can weather multiple extreme moves over a few years and still be profitable over a time period, otherwise you will need to position differently or adjust your strategy. One such adjustment may be buying insurance in the form of index puts over a long period of time to pay the minimum and then rolling that time premium as needed. This doesn't protect against a long period where your individual trades fail to gain traction or where the market doesn't ever rotate into underowned names and instead is lead by the names leveraged to the index.

Consider that when modeling against all extremes which I will show you how to do briefly. Remember when I showed you the table of expected return, outside range, outside extreme etc? One thing you may wish to do is run a calculation for a 10 year period that contains a couple outside extremes to the downside, one to the upside and the rest either in the normal or outside range or results. You can test different position sizes over this time period. Keep in mind that it will be very easy and possibly appropriate to adjust following a significant decline as you don't know whether or not that was just an extreme or representative. You don't know if your expectations moving forward will equal the past, be better than or worse than the past. Emotionally, you may also determine you've had enough. For this reason, your effective bankroll may actually be smaller than you calculated and your proportional risk to that bankroll may be higher than you realize.


We are ultimately going to seek out a balance between multiple strategies across multiple portfolios to try to mitigate risk of our overall wealth and obtain our goals.
For instance one such overall wealth building strategy may look something like this.
1)401k aims to capture the very long term wealth effect over multiple credit cycles (usually 7-11 years) as well as overall movement from and to stocks and other assets like bonds, and may reposition to try to capitalize off of moves lasting months to years.
2)Roth IRA aims to bennefit from swings in stocks lasting weeks to months.
3)Individual option account combined with individual stock accounts aim to insure against the longer term moves when insurance is relatively cheap and in appropriate times, while primarily seeking the non-correlated moves and counter moves and spotting individual bullish and bearish setups when appropriate. The taxible individual trading accounts will dynamically adjust position size by having some exposure to the index and some exposure to cash and hedges and increasing or decreasing that exposure as the short term and long term trades swell up in size to maintain a proportional bias consistent with expectations and avoid overexposure to any one period of time or expiry or overallocation to risk.

This is just one such possible idea, but another involves complex rotation. Another variable is market condition. certain strategies do better during sideways markets, others during quiet markets, others during choppy markets, others during bull and others during bear market and combination of the 2 variables of volatility and trend/direction (or no direction if it's a sideways correction through time).

Here is one such strategy I sort of tentatively mapped out. The idea is you get some market correlated names and perhaps include owning companies run by money managers like Warren Buffett like birkshire hathaway shares. At the time I made this I mistakenly thought WCC (Westco Financial) was still run by Charlie Munger, but there may be some sort of publicly held company that has a large float of investible securities that is run by a quality asset manager. The idea is to own market correlated names that you believe in over the long run to equal perform or outperform. Your 401k might also be allocated into long term outperform but in a more permenant sort of way so you may view that separately. Then you may consider a basket of commodities that start as a very small position and you might buy for the long term at historically significant prices when possible. You might have a currency trading account or just trade calls/puts  in currency ETFs to place trades that move differently from the market itself. You also may want to own some proportion of long term corporate bond and treasury bonds, however, we are at a historical period where rates are probably rising so keeping this percentage very small or finding some way to hedge against rising rates on the extreme end (perhaps long term puts in TLT) may be appropriate.

Then you have your stock timing portion of your portfolio that tries to maintain a mixture across varying strike prices. And finally you have some hedging capital and cash, and since you won't need all of your cash at even extremes, some percentage can be held in income.

The design of this portfolio is very important to understand. The idea is that overall some percentage of your portfolio will be allocated to "risk on" and some percentage to "risk off" That should be consistent with your expectations of winners vs losers plus a buffer for your personal tolerance to varience over time. However, individual option trades that post huge wins can expand and contract. This is where the balancing act comes in in the form of rebalancing your allocation strategy appropriately. For instance, if your stock and option trades are working well, they will expand in size and overall you will be more exposed to violent corrections. As such, you simply reduce your market correlated calls or shares in S&P or leveraged ETFs like SPXL or TNA to maintain the desired balance. Beyond that if you are still more risk on than you want, you can purchase a hedge.

The goal of holding commodities and bonds is to capture rotations in capital into and out of the stock market as well as provide the buffer for your individual stock timing/outperforming portion of your account(s). The goal of having large cash and income amount is both for future opportunity of better prices as well as some degree of protection from deflation as well as reducing account volatility to desirable size, particularly if you are seeking maximum volatility. The goal of having market correlated names is to both have something to balance out growth in stocks and to generally add lower and reduce higher while options are by design going to increase exposure higher and decrease lower and some trading strategies involving stocks or options may stop out lower and add higher.

In order to optimize the portfolio according to risk tolerance and our goals we will need to run several calculations like the calculations that we already did stress testing the portfolio against hypothetical extremes and determining the blend that we desire. That will be for another time.
The goal will be to accomplish a goal of return (or better) while keeping max drawdown limited to a particular amount (or less)... or else over a time period to maximize the probability of an event with a secondary time period where an amount must be reached. This may put less focus on the drawdown in a given month or year, but ultimately the drawdown must be limited such that you are able to recover and obtain your goal.

How FindingThe Best Stocks is A Needle in The Haystack Problem

Finding the best stocks in certain ways is like finding a needle in the haystack. I don't mean with regards to difficulty necessarily but procedure. Awhile ago I was watching a show called Mythbusters: The Search in which they had contestants compete to become a part of the next generation's "Mythbuster's group". One of the problems that they had to solve for was how to find a needle in the haystack. Contestants invented ways of burning the hay that would not burn the needle, using magnates that would work on the needle but not the hay, using water in which the hay would float and the needle would sink and other ideas en route to attempting to filter out the noise.

One of the more eccentric contestants named Hackett suggested that this was a "signal and noise problem". He explained that you had a lot of noise (hay) and had to filter it out without also filtering out the signal (the needle).

I realized that stock picking methods involved mostly the same process. A lot of people like to look at only just the winners and seeing what they have in common. The problem with that is you might be very effective at identifying traits that are also true with losing stocks. Even if 75% of winning stocks have a certain characteristic, if 90% of all stocks share that trait then the trait in itself is not useful. Another problem is that even if you can compare it to the baseline and say that 90% of big winners paid no dividend and 75% of stocks that made big moves were under $10 per share and 60% of stocks that made big moves were under a $1B market cap--and even if you can also say that these big winners percentages are higher than the baseline rate for each stat---it is still possible in combination that this isn't the best match of filters. For example, perhaps although a small percentage of all stocks pay no dividend, 92% of $10 stocks with a small market cap pay no dividend. You'd have to decide which filter is more important.

Nevertheless, as long as you set up a way to objectively find out how to increase your hit rate of capturing big winners while reducing your chance of getting a non winner and still providing enough opportunities you are being productive in your methodology.

I want to focus on methods not stocks and not markets here. You could look at quarterly performance or yearly performance... but for an example I"m going to assume daily performance as you can run this every day and build a large sample size of what works and also plot some sort of measure of market condition in terms of trend, breadth and volatility just in case you notice certain scans work conclusively better in certain conditions.

So here is the method:
1)Develop a scan from a universe of stocks (say 5,500 stocks as an example)
2)Develop scan A to seek stocks up more than 1% and scan B to seek stocks up more than 4% in a single day (or 3% or 5% or whatever you prefer).
3)Either subtract the results or come up with a scan to seek stocks NOT up more than 1% and 4%.
4)Determine the baseline "hit rate" or percentage of all stocks that are up 1% vs up 4%.
This establishes a baseline. Let's imagine that out of 5,500 stocks 400 of them are up 1% or more and 150 of them are up 4%.

Eliminating The Noise And Burning The Hay

Now you are going to come up with a variety of scans you may like to use of what happened prior to today and not including today such as a stock was oversold 3 days ago or a stock was down 3 days in a row until today. Or perhaps you come up with a bollinger band squeeze or a scan that a stock over last several days prior to the breakout closed lower but not too much lower and never made a move more than 1%. Come up with at least a few methods between consolidating volatility, oversold, breakouts, trends, etc.


1)Develop these scans.
2)Scan them from all stocks to see the total number of stocks that pass the filter (say 500 stocks for a particular scan).
3)Add into a version of one of these scans that they pass this scan AND are up more than 1% and 4% or sort by today's change and count them (say 60 stocks in the scan are up 1% or more and 30 of them are up 4%)
4)Determine if randomly selecting a stock from this scan is better than randomly selecting from all stocks in terms of hit percentage and if so how much?
Example: In the example 400/5500 stocks or 7.27% are up 1%+. 150/5500 or 2.72% are up 4%+. In the example scan 60/500 are up 1% or 12%. And 30/500 or 6% are up 4% or more. This is a clear increase in your success rate over picking stocks randomly, so the filter added value today.

Repeat this process
I would run multiple scans and filters and even multiple "universes" from which you run the scan. Is scanning just the russel 3,000 going to produce a better hit rate than all stocks? What if you create a list of stocks that IPO'd in the last 5 years or less vs more than 5 years? What about stocks with positive earnings vs negative? What about stocks with accelerating earnings growth (growth % greater than last year) vs decelerating? What if you create a list of stocks that have been out of favor at one point or another and peaked in 2015 or earlier and are still down from their peak vs those within 15% of a prior peak in any year before 2015 (possible multiyear breakouts from ranges) or a list of stocks with positive uptrend or oversold on a longer term basis? This would be separate from the scan and it needs to have a pretty large number of names on that list to be statistically valid.  You can remove the results that would have stopped out if you want as well.

You can improve the results by identifying the samples most representative of the current market and sticking to methods that worked in the past under those EXACT same conditions.

This may seem like a lot of work but once it's done you can have confidence in the process and selection method's. Once you have collected samples over various periods and begin to determine the best scans to trade from, you may even develop the habit of reviewing the lists and developing your own personal skill or eye for identifying unique features that are hard to quantify in a scan or algorithm and begin to only focus on the best names from the list according to your experience.

If you'd like to shorten the time you have to wait to do this process you can set up the 4% movers for yesterday and just set up more scans and just keep running it for each day before today of how the scan would have came out. It's more work but you can do this all at once if you'd like. Or perhaps you'd like to do all of this over the weekend for each day of the week rather than once after close every day. Or perhaps you'd like to play for weekly moves or monthly moves of larger amounts rather than 1 day moves.

More Accurate Modeling

A more accurate way to model how your portfolio will result over a period assuming certain conditions is to actually look at 5 different distributions of outcome based upon the scan rather than just the success rate. You can sell at the stop, sell at the target, sell break even, sell below the stop or sell above the target. Using spreadsheets and getting a little app that runs monte carlo simulations on random numbers, you can set it up so that a range of random numbers between zero and one represents an outcome consistent with the actual results. For example if 40% of stocks stop out at the stop rather than gaping down or gaping down you'd set it up so that a number less than or equal to 0.40 results in a stop out and set up 5 different cells that determine 1 for the outcome and zero for no outcome and then translates that to a result for a single trade. Trying to plot simultaneous trades is more difficult particularly if your outcomes have any degree of correlation (they will) so modeling your entire portfolio of multiple trades with simultaneous holding periods and different entry and exit times that overlap is really hard to program for me. But you can do a simplified version of perhaps ALWAYS holding 10 stocks so that way your results over a period of 4 trade periods doesn't allow you to compound more than 4 times of the sum of all 10 results for each period.

This can be set up with rules such as if the result of a particular trading year or series of 100 or 1000 trades is over a certain amount to trigger a 1 otherwise a zero. Then a montecarlo simulation determining the average will give you the probability that that outcome is reached. SO you can determine a particular allocation and risk % how many series of trade results in obtaining your goal of minimum (what percentage of these periods are not down more than 20%) and maximum thresholds (what percentage of these periods are up more than 50%). Or you can look at the total distribution of all simulated outcomes at the end of a period.

I'm planning on trying to do a similar process at some point but I'm working on setting it up. This post serves as me getting the logic down so I have an idea of how to run it but now I actually have to set up the scans and process. I plan to track it on an excel spreadsheet which I have not yet build yet. I've attempted such a feat for quarterly results and some fundamentals, but those results were based upon recent data rather than the data BEFORE the stock made their move so it may not be accurate.

SNAP it up

Took these screenshots earlier today BABA and TWTR went through a similar process after IPO. It consolidated downward but fails to break down and soon the short sellers and bulls waiting for lower prices have to chase to get on and the short sellers get squeezed out.

THe psychology is also interesting. Conviction in the short run can be a weakness, and in a long run it may break and you may not be able to have it. The short sellers say "but I'm right" and add on or are forced out only to try again higher. Collectively it is the short sellers conviction that creats more selling than can be sustained in the short term and lower prices than can be sustained in the short term that allows for prices to be driven higher. And it is the short covering rally that creates buying that pushes stocks higher and forces shorts collectively to buy higher.

There are people underneath signalling to be patient and wait for lower prices to buy. That creates a potential bid should prices go lower and also a potential chase effect should they run out of patience if/when prices begin going green and shooting higher.

Pay attention to price action. If it breaks down and doesn't respond by reversing then perhaps the trade is over, but if it begins going higher and you see green shoots appearing, then beware a short squeeze ahead. Have a stop and manage risk, but remember that if more people short and more people sell that's more ammo for a greater short squeeze should it occur. Of course it may not so you have to be flexible, but what I see is a lot of conviction from bears and they have no other move left but to post on twitter and stocktwits about how obviously over priced it is and how the longs are "bagholders". IN order to be a bagholder you actually have to be willing to hold the bag as opposed to having a tight stop. But every stock in the late 90s was overpriced that doubled in price and so was TWTR when it went from $38 to $70. NFLX had nothing proprietary about it and Amazon has chronically been overpriced while it rallied from $50 to $850.

Again, I may be wrong, and it's also possible I'm right and the bears will eventually be right after it goes much higher. But to me this represents opportunity if you can manage the downside and keep a modest position size or modest risk overall.

If the idea begins to gain traction I may add to it, but be warned... the best performing stocks don't offer too many dips when they start moving while everyone waits on the sidelines wishing they were in as the green shoots push the stock higher.


edit:Using a time machine I was able to look at the intraday patterns

Full Blown Bull Market Not Yet Here?

One thing that dow theorists used to propose is that it isn't a bull market until all major indices are making new highs. However, due to the US coming off the gold standard in the 70s and the way currency exchanges and the increased global investing that is being done that may not be enough. One thing Martin Armstrong once said that always stuck with me is it's not a full blown bull market until its leading relative to all currencies. That works for different asset classes and in this case the Russel is only just about to approach new highs relative to the dollar index.

The last time that we were in a "full blown bull market" by this metric was in mid 2013 to 2014.
We can also look and see emerging markets aren't ready and that the all world index is not ready to take out its highs. This suggests a concentration into US stocks until proven otherwise.

The Russel is and all world index is pushing against resistance so it is possible we will stall here, but if/when we make it through to highs it is an all clear buying signal until it retraces the candle that takes out the high and fails its breakout (failed move and 2b sell signal) or until it creates another topping pattern after completing its rally.

For the record, Martin Armstrong has also suggested that the public average Joe investor won't as a group begin to pour into the market until if/when we get above 23000 and if that happens lookout we are headed towards a parabolic move or what he calls a "phase transition".

Risk Arbitrage And Options

Risk Arbitrage is identifying a deal of merger and aquisition that has been announced but has not gone through officially yet and in an all cash offer, buying the company to be acquired, and in an all stock deal buying the to be acquired company and selling the company doing the acquiring.

Risk Arbitrage with options is instead using calls and/or puts to accomplish the same thing, but with unique leveraged instruments.

In the early days, Warren Buffett used to participate in these until he met Charlie Munger and Munger convinced him of sticking with other methods.

Long Call Options and Risk Arbitrage

A call option gives the call owner the right to buy 100 shares of a stock (per contract) at a particular agreed upon price (the "strike price" is the term to define the agreed upon price). It forces the seller of the call option to sell the stock itself at that price should the call owner decide to "execute" your option rather than just sell it to someone else.

For example, if a stock is priced at $90 and there is a deal announced for $100 per share, you may decide to buy call options with a strike price of $95 and make the difference between the price at the close of the deal and this strike price (in this case $5 per share), and the cost of the option. If you can buy the option for $1.70 per share (or $170 per contract) and the deal goes through, you can sell the option for $5 and make $3.30 per share (or $330 per contract) if the deal of the stock closes at $100. However, if the deal fails, you'll likely lose 100% of what you risk. In fact, even if the deal is delayed and the stock goes up from $90 to $96.69 or less you will lose money, and if it doesn't go at least to $95.01 you will still lose all of it. With options you have to be right on timing, price, AND for out of the money options you need to be right on the magnitude of the move. This example allows a return of almost 3 times the risk, which means you can lose 3 times and nearly break even on the 4th. So to make money the deal has to go through more than 25.4% of the time. We actually should only plan to make this trade if the odds are much higher (like 35%) because of volatility, risk, and to pay for missed opportunity.

One variable I haven't mentioned yet is time. Eventually options expire and when they expire you either sell them, exercise them, or they expire worthless. Unless they are trading above the strike price for call options, (and sometimes if  you sell them before the end of the day on the final trading minute of the final trading day before options expiry) they will almost always expire worthless because no one else will want to buy the, knowing they will soon be worthless. The tricky thing about arbitrage deals is they can take longer than you initially expect.

One of the tricks of options is position sizing. Normally when you buy a stock you might put 10% in a stock and if it drops 10% you might sell. This is a 1% loss to your portfolio. But if you risk 10% on an option you have a 10% loss to your portfolio. Lose multiple times and you will have a hard time making it back. A 20% loss requires a 25% gain, a 50% loss requires a 100% gain. While options do allow leverage, making back those gains eventually It doesn't take too many losses for the position size to hurt you even with an edge. However, if you risk 1% per option, you only incur a 1% loss if you are wrong and you can lose 10 of those and be down 10% and only need an 11% portfolio gain to get back to even. The thing to realize about options is the problem is time PLUS price.

Options can work well for risk arbitrage plays because there usually is a clearly defined time frame and a clearly defined buy out price, so you can calculate your exact upside if the deal goes through, and your downside can be 100% as opposed to worrying about a stock that could easily gap down substantially before you could sell the stock itself.

So with this basic understanding we can look at other strategies for merger arbitrage plays.

Covered Calls And Risk Arbitrage

Rather than just buying call options on an all cash deal for risk arbitrage, you may attempt covered calls. Covered calls is where you buy the underlying stock and sell a call at a strike price against it.
For example, awhile ago RAD was selling at $5.60 with a deal that would either be $6.50 or $7.00. The FTC denied the initial deal at $9 for the entire company but after revising the deal downward and offering to divest from a certain number of stores to satisfy the FTC it could go through again at either price depending on how many stores required to divest from. The deal was expected to go through at the end of July.
The call options with expiration date of the 3rd week in July with a strike price of $6 were trading at $0.45 (per share or $45 per contract of 100 shares). That means that if you buy the option AND the deal goes through for $6.50, you'll only make 5cents and you're risking 45 cents per share if it doesn't. That means the deal has to go through 90% of the time to break even as a call buyer. Even if it goes through for $7. That's still $1 made on $.45 risked which is a little higher than I'd prefer. Also, I'd usually want to buy a couple months past when the deal is said to go through because it may get delayed again, this actually doesn't even finish out the month of July which means there's a good chance we can lose even if the deal goes through. If it's a bad situation for the call buyer, you should think about being the call seller.

Selling calls without owning the stock is dangerous. Although this would be the best possible situation a call seller who doesn't own the stock could be in with a likely capped upside, there is still a rare possibility of another bidder coming in (RAD would have probably already looked at all possible bidders and someone else would have probably come in by now), or something weird like the deal failing but the shares going up beyond $7. Plus, I think the deal still probably goes through more than 50% of the time so we want to own some of the exposure.

This is a perfect situation in my view for a covered call.

In other words, we buy the stock and for every 100 shares that we buy, we sell a July option with a $6 strike price. In the most likely situation given a buyout goes through before expiration, we keep $0.45 per share that we collected from the premium, we keep $1 that we make from the stock from $5 to $6, and we give up our stock at $6 per share and forfeit any additional gains, should they occur. In other words, when we are right, we make $1.45 per share. If the deal fails, we still collect $0.45, but we have to hang onto the stock until the option expires, unless we want to close on the option first and then sell the stock afterwards (but there's usually extra transaction fees for this).

The options are like a $0.45 insurance policy in exchange for giving up the gains beyond $6. It insures $0.45 of damages should the stock decline, but we'll still have to take the loss of the difference.
If we expect this deal to go through 2/3rds of the time, we'll make $1.45 each time or $2.90 for the two times, and on the 3rd we'll make $.45 more for the options or $3.35 for all 3 times including the one that failed. So when it fails the stock will have to drop less than $3.35 to make money (not including fees, transactions, and weird situations like a revision of the takeover price or early execution that can mess with the expectations) Since the price is $5.66, that means it would have to fall to $2.31 or lower when we're wrong to lose money (or we'd have to overestimated the chances that it goes through).

I don't know exactly what will happen if the deal fails, but it seems reasonable enough that this trade is profitable. Some people will look at the price a stock was trading at before it announced the deal and use that as the expected loss. I think the actual probability might be higher. Also, I think I'll probably collect the premium in July and that will possibly allow me to collect an extra $.50 to $1 when the deal goes through, or possibly sell August options, or if it's trading close to $6 I may even exit early rather than risk the possible deal failure.

Long Stock And Risk Arbitrage

There are plenty of situations which you would not wish to buy the option but would wish to own the stock without selling covered calls. One example is when the calls have such a wide spread between the bid and ask that you can't buy or sell them at any reasonable price. This happens more often than you might think. But if this happens, you still may wish to consider risk arbitrage if the deal seems good enough. You also may wish to do this if the stock is trading at $1 and the deal will go through at $2.50 or less and the only options available are $2.50 and they are only available for buying at $0.05 and sellers are unlikely to collect. In other words, if the strike price of the option doesn't make sense for either side. Not all stocks trade options so that's another reason you may not wish to play options.

The way options are priced usually make deals that have higher net profit better than shorter time frame. However, if you can get a deal that is expected to clsoe next week for only a 3% gain, and the loss is less than say a 9% loss if it fails, and you can make lots of these trades over a year, that can add up. This is a good reason to make long stock risk arbitrage. 

When selecting risk arbitrage, you have to realize that the highest percentage profit probably have the lowest chance of going through or will be delayed

The Baseline Odds of Risk Arbitrage Deals

According to insidearbitrage,  last year saw 225 deals closed. It also saw 20 deals that failed. It also saw 88 pending deals, some of which were probably delayed and may not close yet, and some of which may have been carry overs from 2015 that still haven't closed. We also don't know how many deals closed at a lower total than initially expected that could have resulted in a loss in options price, a much lower gain in the stock, and possibly even a loss overall in the deal (depending on where you bought). So let's just assume all pending deals are bad deals. We'll say 225/333 closed which is just over 2/3rds. I think this is a good number to use when planning a deal but with higher percentage deals you should expect lower probability of it going through.

I think using the pre-announcement price is a good price to plan on the stock going to if the deal fails.

How Much To Risk?

How much you should risk on these deals and on merger and arbitrage depends on your edge and instrument in general, but when Warren Buffett used arbitrage, he only used it as part of his portfolio. He would take a controlling interest in some and he would just find undervalued situations in others. 

One of the benefits of risk arbitrage is the low correlation to the market. The expectation of gain is not very high, but it also will tend to outperform in bear markets because of the low correlation to the market. Buffett would at times use margin to buy risk arbitrage names due to the high probability nature, and his measurable downside and measurable edge, but he would carefully weight the downside, and this was a time when risk arbitrage was not popular, and thus the opportunities were much better. He also was convinced that these were not the best way to play the market over time.

I like it due to the ability to outperform on the way down which can allow capital to buy if things go wrong. Buffett now almost always has some (usually large like 20%) percentage of capital in cash, buys insurance companies which have access to their float and now has access to the federal funds rate to borrow at lower interest than most, plus he has such a good reputation that he is unlikely to see a lot of capital leave his company just because the market sells off and he can also raise money by selling bonds. He also collects earnings directly from companies he owns privately that are relatively immune to the economy. Because of this, he doesn't have a shortage of means to buy when everyone else is selling and raise more capital should stocks sell off. 

However, for the average Joe experienced investor, this may be a good option if you do so intelligently. There's a book by Mary Buffett explaining how Warren did these deals if you want to learn more.

Trading System - Order Within Chaos

Stockbee has a good post out recently on how to organize your trading plan.
For some people, operating under chaos is how they're used to operating. These individuals may be able to manage the simultaneous chaos of the market and find setups. But even this type of individual would benefit from organization. There are too many tools available that can improve your efficiency and too many options out there for individuals to be able to optimize them all. This is where a trading plan can be structured to be more efficient.

Even having software that helps you with one particular process such as watchlist development or entrypoint isn't necessarily enough to tell you position size, asset management, decisions on how many to enter and which ones and how to prioritize entries, how aggressive to be given the conditions and so on.

A full trading plan has to quickly sort through the mess of multiple variables and manage the chaos. Many make the mistake of confusing chaos with randomness. The market is chaotic which means many properties such as distribution of outcomes and day to day predictability of moves may seem random, while operating with an order under the surface.

But chaos is sensitive to small changes and relationships between multiple variables may not be equal to the sum of its parts. So how do we operate in a multi-variable world?

Mohnish Pabrai has been using checklists to accomplish this task. It is the same process that airlines have implemented to reduce error and many medical arenas are now adapting to avoid mistakes.
Checklists should at least be designed with the most fatal mistakes in mind and make sure there aren't too many decisions on the checklist.

Develop the habit of reading through a checklist.

Make the checklist when you are out of the market or at least on a weekend when the market is closed so you can maintain some objectivity.
Backtest and forward test and/or model certain conditions and decisions to see if your assumptions about your decision make sense prior to implementing it into a working checklist. A checklist should have no more than 11 items and probably closer to 5-7.

Example checklist:
1)Check to make sure portfolio tracking data is updated
2)Check to see if any thresholds have been met that require action such as reducing size or limiting buying.
3)Check to see if your portfolio rules allow for the addition of stock and/or option purchases
4)If so, use breadth tracking and/or other methods like oscilators and indicators to determine if market is in a condition where taking the action is acceptable.
5)Check watchlist and risk/reward of potential entries and ensure they follow your purchase rules before purchasing.
6)Look for entry trigger.
7)If portfolio rules and market conditions and entry triggers permit, buy.
8)Track portfolio end of day just before close (5-15m before close) and sell anything below stop or beyond target that meets exit criteria.
9)Update watchlist after close consistently (perhaps once a week, twice a week or once per day).
10)Input watchlist data of possible stops and entries.
11)Update alerts for prices below stop or above target and also for any prices that may trigger a trade.

Optional:Update portfolio tracking data after close or after next open?

Condense this to:
1)Update and check portfolio tracking.
2)Check breadth and portfolio tracker to determine if you are possibly (or definitely) buying or selling today.
3)Check watchlist for possible entry, confirm with trigger.
4)Review positions before close.
5)Update watchlist, watchlist data and alerts

You may wish to set an alert or alarm to inform you when it's time for this. You might have a phone alarm and where it says "location" or "title" you can put the details for what that time of day signals. Example:
Alarm 9:35: portfolio tracking - determine today's possible actions. Update portfolio data
Alarm 10:35: check breadth and determine if buying or selling today.
Alarm 11:35, 12:35, 1:35: breadth + portfolio + watchlist
Alarm 2:35  Final period to consider buying
Alarm 3:50 Check portfolio tracking and look for trade exits.
Alarm 3:59 Market close, update watchlist.
Alarm 4:15 Update watchlist stop and targets in spreadsheet
Alarm 4:45 Update Portfolio data
Alarm 5:00 Update email alerts for trade triggers

Another way to organize it is perhaps that within each checklist you might have more details available as a reference if you need them, or you might try a checklist of checklists. Your first checklist tells you which order to view the checklists and how to use them to come to a decision. Your second goes through the steps of what specifically to do rather than generally and provides more details to avoid any mistakes within that particular process itself and gets you used to not trying to make any decision that isn't allowed by the process. This is a more in depth way to go about it but it keeps you organized to a particular procedure.

Checklist of Checklist example:
1)Portfolio tracking checklist
2)Breadth tracking checklist
3)Watchlist development checklist
4)Monitoring Watchlists for entries checklist
5)Entry checklist + Position size checklist
6)Management/exit checklist
7)Trade review checklist

-------------------------------------
1)I like updating portfolio and determining the plan for the day at the open (adding any trades from yesterday and updating the prices to set the tone for the rest of the day. I have no interest in trading the first hour or two after the open anyways so any time in the first two hours to do this is fine)
2)I like managing trades/exiting right before market close on an exit signal.
3)I like setting up a watchlist after the close and setting the targets and the stops if I were to take the trade after the close... or on the weekend
4)I like setting alerts just after that. Alerts let me know when trades I really want hit points where I'd consider an entry or when certain candlestick trades are triggered that I use when opportunity is scarce or when I'm trying to place a bearish trade.
5)After the first hour or longer of trading I like checking breadth throughout the day and I check that just before I look at my watchlist.


Setting a phone alarm with a note to go off Monday through Friday is a good system even as just a reminder. You may want to treat one or two days of the week differently and have different alarms.
Set it to go off once sometime during the open.
Have it go off once 5 or 10 or 15 minutes before the close depending on what you need
Set it to off after the first hour or two to go off once every hour and once every 15 or 30 minutes on Fridays (options expiry).

As you spot nuances you'd like to include, try to adapt your checklist to reflect those changes but make sure you don't get carried away. For example, perhaps you want to check the RSI(5) on the sector SPDR ETFs and check the breadth by sector and by some of the largest industry groups to determine where to focus your buy priorities or watchlist priority names. As you're building a watchlist rather than just being about risk/reward you probably want to highlight a few of the most quality names or those within themes and place emphasis on those