Blog Charts FAQ




ALGO TRADING


August 22, 2022


The purpose of algo trading is to select or develop a trading system that appears to have good prospects of achieving net profits in future.  It also includes executing its buy and sell signals in a disciplined manner while mostly ignoring one's own opinions of the market direction. It is harder than it may seem at first glance, but it also offers various advantages over other types of trading.

 

Why Algo Trading?

To be able to capitalize on market moves it is necessary to either have a lot of experience as a discretionary trader that translates into consistent profits, or you have to follow some statistically sound rules that improve your chances of achieving more gains than losses.

Among novice traders algo trading is usually not as popular as discretionary trading. One reason might be ego. A winning trade feels better if you can tell anyone that it was based on your own correct judgment rather than some rigid rules. But this attitude does not work very well in the longer run and can lead to big mistakes that give back most past profits.

"Many discretionary traders avoid the systematic route because of laziness or because they lack the quantitative skills to really understand system development. This is a mistake. Discretionary trading is probably the hardest trading there is; it takes more work, more time, more analysis, more self-reflection, and more self-control to achieve success in this arena than in any other type of trading." Adam Grímes

Discretionary trading is much harder to do than systematic trading with strict rules or algorithms. Beginners are quickly lost in the maelstrom of choppy price action and emotional chaos. Algo trading provides a clear framework for trades, which helps reduce uncertainty and confusion. An algo could be seen as a digital mentor that holds your hand and helps to avoid overtrading.

"Trading systems limit the scope of market behavior, and therefore make this activity a little easier for our minds to manage. They also give us direction and suggestions about what to do in a given market situation. Without them traders could easily feel as if they are floating aimlessly in an endless sea of possibilities and opportunities with no land in sight." Mark Douglas

There is a broad range between discretionary and systematic trading, but it is usually best to start out systematically and only experiment with an increasing degree of discretion as your experience grows. Depending on the trader's personality, one or the other route is more viable.

"A lot of the excellent traders I talk to tend to start with quite strict rules and gradually give themselves more discretion as their experience grows. This is a sensible approach and one I followed myself." CryptoCred

Algo trading even has lesser risk than trading with fixed rules written down on a piece of paper. Manually checking if the rules apply is prone to perception errors, interpretation problems and other biases and mistakes. However, in both cases there is still the risk that a trader does not execute a signal even if it is very clear and eye-catching.


Types of Entry/Exit Algos

Most trading algorithms that are offered are based on averages: moving averages, RSI, MACD or even more exotic types of averages. These type of algos are rather simple and easy to develop, which might be the main reason for their dominance. But because they need a longer range of price data to make decisions, they are usually comparably slow. This is usually not a problem during bull markets when trends last longer, but can cause many losses during bear markets when price quickly changes direction. They avoid overtrading by not entering and exiting too often, which is an advantage.

Algos normally use price data, but it is also possible to use volume for generating trade signals. Volume represents the force behind price moves, but is more noisy than price. While volume worked much better in the 20th century, electronic trading introduced more noise because trades are split up much more because of low fees, more intraday trading and big players trying to hide their commitment. But volume may still give clues that price is not able to give. This usually requires more noise suppression and sophisticated calculation, but can provide valid signals.


Interpreting price action is usually done by discretionary traders while algorithmic traders tend to ignore it. It is much more complex to code, but price action signals potentially allow very quick entries and exits. Too many trades and potential faster accumulation of losses might be the downside of this advantage. But if a system is well balanced, it can result in much bigger net profits than possible with averages.

Patterns are another field dominated by non-systematic traders. It requires sophisticated detection algos, but it might be worth the effort. Often a too naive and simplistic approach is used when patterns are involved. Relying on patterns alone without context information and a good strategy does not work. These algos can also produce fast entries and exits, which potentially makes them more profitable during bear markets.

We offer algorithms based on all the four mentioned approaches: averages (BECKI), volume (BSVI), price action (BEAST, MCS) and patterns (BEYOND). However, it is important to keep the weak spots as well as strength of each type of algorithm in mind.

 

Adaptability

Even if you found a great pattern, setup or formula for entering & exiting trades, it will likely only work e.g. 75% of the time. There will always be periods when it won't work at all. If it produces too many or too big losses during that time, it will sabotages the whole strategy.

"Livermore would always stay out of market trends that were indecisive or sideways. He would always make sure the market led first, and then he would trade accordingly... Loeb and Darvas learned to stay out of declining or bear markets." John Boik

For example, a swing trading strategy may work great during bull markets but give back most profits during bear markets. Or a day trading strategy may have too big drawdowns from time to time, which makes it unprofitable.

"Going back for the last year and a half it's been an environment that, if you were a short-term trader and you were buying breakouts and selling them into the first pop in the first few days, you could have made a fortune." Mark Minervini

To avoid this several measures could be used. It might be possible to detect these periods and stop trading, reduce the number of trades or switch to different strategy, e.g. only shorting or aiming at many small profits. If not possible, an algo could try to reduce losses by avoiding certain trades or by aborting struggling trades prematurely.

"As is always the case, your fund management could have done better [in 1957], but this year we missed an opportunity to do considerably worse. To do worse, all we had to do was maintain a fully invested position." Jack Dreyfus

Compounding also reduces losses because position size goes down with lower capital. Other methods of varying the position size can help even more.

 

Position Sizing

The most simple way to trade with an algo is to use the maximum 100% of your standard position size on each signal. Various famous traders recommended to not use more than 20-25% of your total trading capital on one trade. This is a good idea when trading rather volatile assets.

"If you have a significant edge, diversification does not protect you; it dilutes you. I want to concentrate as much of my money as I can in a position up to 25% of my portfolio. I may not start out at 25%, but that’s where I would like to be for my best positions." Mark Minervini

When trading higher liquidity assets, e.g. a stock index, forex or Bitcoin, it can make sense to use 100% of the trading capital if the trader and the algo are up to the task. But this should only be done during bull markets. Bear markets with much lower win rates are better traded with a lower percentage.

"I strongly believe the only way to make long-term returns in our business that are superior is by being a pig. I like putting all my eggs in one basket and then watching the basket very carefully." Stan Druckenmiller

To reduce risk a trader or algo can also scale in/out, which means starting to buy or sell only a part of the position once a setup occurs or a signal triggers, and then increase or decrease the position based on price action. Thís may reduce losses if price action is erratic and bad signals cannot be avoided, e.g. in bear markets. When trends take off quickly it can reduce profits. A lower time frame can also be used to do it.

"If you are on the buying side, you can begin with just a little capital and buy ... in small quantities, and when you start making profits, you can increase the volume of your purchase until you buy the volume you had targeted originally." Munehisa Honma (1755)

Using a maximum of 20-100% of the trading capital on any single position might be called macro position sizing as it imposes an upper limit. Scaling in/out would be micro position sizing then, because it is applied on the level of individual trades. It is also possible to size a position based on the risk of a trade defined by its entry and stop loss price, which is mainly used by discretionary traders. Additionally, it is smart to adapt the position size over multiple trades and periods of a market cycle.

Usually the odds are skewed in your favor during bull markets and heavily against you in bear markets. Would it not be foolish to use the same amount of money when your chances of winning are reduced? Additionally, the dominant trend in a bull market is up and in a bear market it is down. So it makes perfectly sense to use bigger position sizes for going long than for shorting. Vice versa, for bear markets it is wise to use more money for shorting downtrends.

Here are position size suggestions for swing trading stocks or crypto currencies:

Bull Market:
Long: 100-200% of standard position size
Short: 0-50%

Bear Market:
Long: 0-50%
Short: 50-100%

Longs cause smaller drawdowns during bull markets. Thus it can make sense to use a bit of leverage, especially if the leverage is scaled up as a trade moves in the right direction. Shorts cause larger drawdowns during bear markets. Why it is better to not leverage them.

The bigger the pullbacks an asset exhibited during a bull market in the past, the bigger the position size for shorts can be. But it can make totally sense to not short at all. Similarly, with bigger bear market rallies in the past the position size for long-side trades in a bear market can be increased. The least risk would be to not trade the long side during bear markets. If you or your algo does not have a good short selling strategy, it is better to remain totally in cash during a bear market.

Some trading styles, like day trading, don't know bull and bear markets (or only have miniature versions of them within a day), because they focus on much smaller trading periods. In such cases the position sizing could concentrate on the win rate or drawdown of the trading results. If the trading works great, the position size could be increased. If losses mount during a certain period, the position size could be reduced. E.g. the Kelly Criterion formula with an appropriate lookback period can be used for doing this. Of course, this approach only works for trading systems that perform many fast trades and not mid to long term algos with only a few trades per year.

The reason behind increased losses or profits often has to do with a change in the market cycle. But it could also be caused by a change in the performance of a trading algo or discretionary trader. The algo may use a strategy that slowly becomes obsolete or the trader may have personal issues that suddenly reduce his performance. Intelligent position sizing can soften the blow that such problems cause.

Discretionary swing traders also use positive or negative streaks to implicitly detect the start of bull and bear markets. If trades work out mostly, it usually implies that a bull market or bigger rally started, so increasing position sizes often means increased profits. If most trades are stopped out, it can be the first sign of a approach bear market or larger correction. Thus reducing position size or stopping trading reduces losses.

Profits of entry/exit parameters increase as better position sizes are used while max drawdown is kept more or less constant. The lowest profits use no position sizing (long signals only). Highest profits use 200% longs in bull markets and 100% shorts in bear markets. Both are not recommendable because of a too high 30%+ drawdown. Optimal with 20% max drawdown seem to be 150% long-only during bull & 50-75% short-only during bear market. This position size is located on a nice plateau in the landscape above as indicated by the red circle. Additionally, shorts in bull markets & longs in bear markets have a minimal effect on the gains of this algorithm and increase max drawdown unnecessarily.


Just like entry & exit rules the position sizing rules of a trading system have to be reliable and robust. This means the rule that differentiates bull from bear markets or detects a slump or surge in trading performance has to work sufficiently. Increasing position size, especially with leverage, at the wrong time can have a devastating effect on the trading results. When done correctly, it had the potential to great increase profits and effectively reduce losses.

"If a sound position sizing strategy (or algorithm) is not employed, the effective rate at which trading equity is compounded will remain suboptimal... Professional trading strategists believe that the sizing principle is more important than the trading strategy itself... This suggests that perhaps it will be more productive to find the most effective sizing principle than it is to spend a great deal of time attempting to improve the entries and exits of the strategy." Robert Prado

Position sizing might be the only way to sufficiently reduce losses during certain trading periods. Entry and entry suppression rules might not work sufficiently during bear markets, which contain a lot of noisy price action. Then the only option is to trade with reduced position sizes (or not at all) to avoid that a trading system suffers the "death by a thousand cuts". As a new bear market might last longer than previous ones, it is better to err on the side of caution.

Our algos offer a position sizing feature that roughly detects bull and bear markets and suggests different position sizes for longs and shorts during these periods.

 

Dealing With Randomness

The market is a game of probabilities. No one can predict where price is going in the mid to long run. You never know what will happen with your trade, because the market can go any way it likes. Especially if you are quite sure it will move your way, the final outcome might be even more surprising.

"Ask yourself what can't happen? What can't the market do? When you find yourself rationalizing the market's behavior to support your position, you are operating in the realm of illusion and setting yourself up for a painful forced awareness. Remember the market can do anything, even take your profits away if you allow it." Mark Douglas

Depending on the time frame that one trades there is only a small window that allows "predictions". Therefore, the outlook on the market needs to be reviewed often, e.g. every day when trading the daily time frame, and a trade managed accordingly by reacting to significant price moves.

Why is the market unpredictable? The answer is simple. It consists of thousands or even millions of participants that can change their opinion any second. Thus future price moves are unknowable and fluid. We only know how price behaved in the past and can only hope it will "stay in character" in future.

"When we fully accept the idea that the market can do anything, and we don't know what it will do, our trading drastically improves. We limit our risk and build strategies that give us a multiple of our risk on winning trades. We trust the process, because we accept that we don't know." Cory Mitchell

Even if you got the best trade setup or signal, which worked a dozen times in the past, it could nevertheless fail this time. And it might work the next dozen times or never again. This is why you can never risk too much on any single trade.

"Markets are extremely competitive. They are usually close to efficient & most observed price movements are random... This is the trading environment - high-risk decisions, made under pressure with insufficient information." Adam Grimes

You or your algo might be right 40 to 60% of the time over the long term, but wrong almost as often. Once a trade is executed you can only wait and hope for the best. Regardless if a trade is working or not, the only option you have is to exit a full position partially or completely.

"Once an automatic trading system has been decided upon, it should be followed rigidly. The automatic trend trader must be prepared to accept the whipsaws that will inevitably occur from time to time, as well as the substantial profits that will usually result from the occasional extended price moves." Arthur Sklarew

You should always know where you will exit a trade if it goes in the opposite direction after entry. The trading system should provide this exit. If it calculates the exit as the trade unfolds, you should at least know at how much % loss it will exit usually.That is the risk you take. The best way to deal with it is to mentally write off the risked money as if you already lost it. It is the risk fee of the trade.

"After many years and many mistakes fighting [hope & fear], I found a solution that works for me. It is deceptively simple, but it is difficult to do consistently. Here it is: for every trade you put on, immediately assume you are wrong. This is your baseline assumption, and, if you find evidence to the contrary (that you are right), be pleasantly surprised. This works because it takes all pressure off you and all hope out of the trade." Adam Grimes

Another good attitude is to never see one trade in isolation. A strategy only works over many trades not over one, two or three trades. Why it is about probability and not certainty. Many loss trades in a row happen (usually during bear markets) as well as many win trades in a row. If you want a system to work, you have to trade it consistently for a longer time. Each individual trade in unimportant.

"Good trading is not about being right, it's about trading right. If you want to be successful, you need to think of the long run and ignore the outcomes of individual trades." Curtis Faith

The best traders are like the market. They don't expect to be right and make profits each time, they don't worry about losses, they stay focused in the present and execute trades without hesitation.

 

The Confidence Problem

You may have the best possible trading system, but you can still use it wrongly or give up too early if you do not have enough confidence in it. As a result you may make losses or few gains whereas people who used the very same system correctly may have made great returns. This has nothing to do with the effectiveness of the trading system, but the trader's own confidence, attitude or psychological condition.

"Loss aversion affects one's ability to follow mechanical trading systems because the losses incurred in following a system are felt more strongly than are the potential winnings... People feel the pain of losing much more strongly when they follow rules than... from a missed opportunity or by ignoring the rules of the system." Curtis Faith

The problem is that most people expect to make a profit with every trade. But even systems that achieve a 90% win rate in a backtest can still make at least 2 losing trades in a row. For many trading systems it is normal to do 3 or 4 bad trades in a row. Most often it happens during bear markets. This can be shattering for many novice algo traders if they don't have realistic expectations.

"Most people who have little confidence in a system (such as one they have purchased or one they get through a subscription to someone else's advice) will abandon it after three or four consecutive losses. However, suppose you have a system that you have tested. You've had a negative run of perhaps five or six straight losses. What do you do? You start to doubt the system. Perhaps the markets have changed. Perhaps my testing had some flaws in it. Streaks generally cause people to doubt their systems. As a result, we are most likely not to trade it, but when we do, we trade it with little confidence and thus trade with a minimum position size. We vastly under trade the system and when the good trade comes along, the system does not perform up to expectations." Van Tharp

Trading systems are based on probabilistic calculations. There is no absolute certainty that they will work perfectly each time. Instead there is a likelihood that they will work e.g. 6 out of 10 times (for a 60% profit rate) if used on a larger number of trades. But this also means that they could have e.g. 5 bad trades in a row and a maximum drawdown of the trading capital that is higher than in back tests. This is usually bad luck or indicative of a bad market period and not a flaw in the system. It could be due to a fundamental change in the price action of an asset. But if it is not possible to find grave reasons for this suspicion, it is better to stay with the back luck assumption.

"Don’t get frustrated & quit trading it during drawdowns. All systems have losing streaks, the key is to manage risk & stick to your plan until the system has time to work out... as the market becomes conducive to your system’s method." Steve Burns

Here is a table (based on research by Van Tharp) that shows how many consecutive loss trades are normal or not normal based on the percentage of profitable trades of a system:

  Number of Consecutive Loss Trades
System Win Rate Normal Bad Luck You got a problem!
90% 1 2 3
80% 2 3 4
75% 3 4 5
70% 3 4 5-6
65% 3 4 6-7
60% 4 5 7
55% 4 5 8
50% 5 6 9


However, it also depends on the amount of loss. Five -2% loss trades are not as problematic (but indicate a chop trade problem) as two -15% trades. You should stop trading and start investigating once your consecutive losses exceed 20% to rule out that there is a fundamental problem with the trading system.

"Everyone who trades must take a loss. Those who expect to do so & accept this fact gracefully will be successful. Those who do not... will prejudice their judgment & allow emotions rather than reason to guide them... Those who brood over their losses always miss the next opportunity, which more than likely will be profitable." Roy Longstreet

In essence, you will very likely sabotage your own success as a trader if you are too focused on winning every trade and attribute a loss to a flaw in the trading system. Losses are a normal property of trading algos and rarely indicate a problem. The trades of the algo needs to be rather erratic to substantiate such a suspicion.

"I think that successful traders have a personality, that they're not afraid to have 19 losing trades out of 20, because the twentieth can be a trade that's much greater than the other 19 put together. They're not hung up on losing money. They want to protect what they've got and wait for the opportunity to make a lot of money." George Segal

So don't expect a profit on each trade. Only expect to make net gains over a series of trades. You never know if the next trade will be successful or not based on past performance. If possible, you should execute every signal that is presented to you. Expecting too much might be so frustrating that you give up right before a winning streak starts. Every new trade is a new opportunity that you should not miss. The best attitude would be to forget the results of past trades and act as if you had just started using the system.

"The key is consistency and discipline. Almost anybody can make up a list of rules ... What they couldn't do is give them the confidence to stick to those rules even when things are going bad." Richard Dennis

You cannot succeed with a trading system if you do not have confidence or even conviction in it. You have to be certain that it has an edge, an advantage over other market participants. Otherwise you might stop using it too early. So you should do everything possible to convince yourself of the validity of a trading system. Compare it to other systems. Check each of its past trades and see if they made sense. Better investigate thoroughly if you are uncertain about a property of the trading system.

"At the trade-by-trade level... the micro level... we should believe that the market is uncertain and because the market can and probably will do anything... At the level of a large sample size, or the macro level, we should hold the second belief, that markets are predictable, and to the extent that, our trading plan has an edge, the market will be predictable and certain." Ray Barros

Another possibility is to wait and see how the system works with the next few signals in real-time while betting little to no money on it, or even paper trade it. With every successful trade the position could then be increased slightly. This is also called out-of-sample testing. It should reduce anxiety and increase confidence.

"What could be threatening about the market? Nothing, if you had the confidence and completely trusted yourself to act appropriately under any given set of market conditions. Essentially, what you fear is not the markets but rather your inability to do what you need to do, when you need to do it, without hesitation." Mark Douglas

If you are not able to consistently execute the trades of an algorithm - although you know that it is performing much better than your own discretionary trading - there could be some emotional baggage that sabotages your execution. Every trader did some really bad trades with big losses in the past. To be successful at trading these feelings of helplessness and shame have to be consciously processed and overcome in such a way that they don't interfere with future trade execution. This take quite some time and requires patience.

"If you can't execute your trades properly, even when you perceive the most perfect opportunity, it is because you have not released yourself from the pain contained in the memories of past trading experiences and because you still don't trust yourself to act appropriately in any given set of conditions. If you did, there would be no fear or immobility." Mark Douglas

One potential method to side-step these problems is to execute the algo trades automatically. This requires some technical effort, but you still have to monitor them to make sure that the trading system works as it should. As you basically cannot restrict yourself from interfering, the above mentioned confidence problems still applies. It is only one additional barrier that hinders you from not letting the system do its work.

So you have to solve the confidence problem one way or another to be able to trade successfully. As everyone is different no one can get around working on his own individual solution that fits his personality best.

 

Common Sense Rules

Here are some rules that might be helpful when it comes to algorithmic trading:

1. Don't get out of a trade because you are afraid to give back profits. Instead wait patiently for a sell signal even if your profits go down again. A trading system needs some leeway to stay in during minor pullback in order to make potentially more gains in future. If you do not give the trading system this freedom, you may get out early in a bull run and miss out on future gains. If you cannot resist the urge to take profits, it would be better to exit only 50% of your position and let the rest ride until a sell signal occurs.

"Sometimes the system will give you signals to trade in ways that are completely contrary to your logic and reasoning... and be right, and sometimes you will agree with the system and it will be wrong. You need to understand that technical trading systems are not designed to be outguessed." Mark Douglas

2. Don't ignore a sell signal because you think the market is destined to go higher. The trading system always signals an exit if it detects high probabilities of a down trend. If the trend should nevertheless not change, it should tell you to get in again. This might involve missing some gains, but you also avoid the risk getting caught in a quickly descending price spiral.

"Once you have settled on a plan, be consistent. You may have multiple rules and qualifying conditions, but, within those rules, you must be absolutely, perfectly consistent. The market is random enough; don’t make your results more volatile by adding random decisions to your process." Adam Grimes

3. Don't change your trading strategy continuously. Especially if you are in a trade. This could turn a profitable trade using the old rules into a loss trade with the new rules. If you found another promising strategy, wait for the next signal. Switching trading rules or strategies too often increases your risk of locking in losses.

"The advantage of the mechanical trading approach is that we learn to execute consistently. We take every trade, without hindering our execution with fear or hope." Ray Barros

4. Don't change your position size based on one or two previous loss or win trades. Don't double your position size because you made a huge gain with a previous trade. If your next trade is a losing one, you might give back more profits than necessary. Also, don't reduce your position size by an large percentage because losing trades. You may lock in your losses because a smaller position size does not allow you to make back lost money. It is better to increase or decrease the position size slowly. That is also what the popular Kelly Criterion does based on previous loss or win trades. Only if you are rather certain that a bear market started or if you have the strong suspicion that your trading system has a problem, you should reduce your position size significantly.

"Mostly I follow the rules. As I keep studying the markets, I sometimes find a new rule which breaks and then replaces a previous rule. Sometimes I get to a personal breakpoint. When that happens, I just get out of the markets altogether and take a vacation until I feel that I am ready to follow the rules again. Perhaps some day, I will have a more explicit rule for breaking rules." Ed Seykota

5. Unless you need a break, don't ignore a valid buy signal. At times it is necessary to exit the market and take a break to relax from the stress of trading in order to be able to calmly execute trades in future. But if it is just fear of losing that keeps you getting into a trade, it could very well backfire, because you might miss a trade that would provide the necessary gains to increase your confidence and trading capital. You could make a compromise to reduce your fear by starting with a smaller position size or placing a stop loss. As bull markets follow bear markets there is always a chance that the market will turn around and place the odds in favor of your trading system. Consistency is key to trading success, not giving in to fear.

 

Rules For Extraordinary Situations

Uncommon situations may occur when algo trading, which might leave traders clueless or confused, because of events that lie outside the competence of the algorithm or question the validity of a signal. Why it is good to have some pre-defined strategies to cope with such cases.

When trading daily candles, a signal usually becomes valid only at the daily candle close. Huge green as well as red candles are annoying, because you would miss or give back gains if a signal is already shown near the open or middle of a candle and you would have to wait for the candle to close. This can easily result in missing or giving back 5% or more profits. You might also question if it is still a good idea enter so high and chase price.

"Japanese traders circumvent this problem..... by initiating a light position if the reversal amount is met or exceeded on an intra-session basis. If the market then closes by the reversal amount, they can either add more at the close or wait for a correction to add more... If the market fails to confirm the reversal by the close, the traders would offset [i.e. liquidate] the light position they had earlier added." Steve Nison

If a buy signal preview occurs and price takes off strongly, a solution might be to start a smaller position intraday, e.g. up to 50% of the standard size. If the signal is still there on the daily close, the rest of the position would be added. If the signal vanishes again, the smaller position would have to be sold again. Same can be done for a sell signal: Selling a part of the position intraday and the rest on close. If it should not work out, the sold part of the position needs to be bought back higher.

As pullbacks frequently occur after such big moves, one could alternatively wait for a small correction, e.g. on the next day, to add to the incomplete position. But if a strong uptrend kicks in, waiting could turn out to be a bad decision.

This strategy is, of course, discretionary and could produce losses. A large move may occur within a day and break down again, which results in a big wick up or down. So applying this method requires trading experience.

Another potential problem occurs when a fundamental change questions the validity of a signal. For example, a big price gap may occur due to fundamental developments, trading might be suspended for a longer period or altered trading fees might suddenly produce different trading volume. Such events can bring an algorithm out of balance and often requires multiple days or even weeks for the algo to readjust to the changed data.

If a signal occurs during such a readjustment period, you may reasonably question its validity. But what to do with such an ambiguous signal? If there is a chance that it could be valid, you can put on a smaller position size and only increase it on further confirmation.

Remaining confused and not reacting at all might not be the best solution in such situations. But it is wise to act with caution by applying a multi-step strategy as discussed above.

 

Trading Multiple Algos

If you have more than one trading algo or signal type for an asset at your disposal, you can also diversify the capital over two or more of them. This reduces the drawdown if one of the algos does not work very well. There are different ways to accomplish this.

With an averaging strategy you would simply trade each algo with the same position size. This is the same as trading separate algos. By doing that you are scaling in and out of trades based based on signals instead of price action or time.

  Position Size
Number of Algos Averaging Strategy Median Strategy Inbetween Strategy Fastest Strategy
2 50% on each signal 100% between signals
(or on 2nd signal)
100% between signals
(or on 2nd signal)
100% on whichever signal triggers first
3 33% on each signal 100% on 2nd signal 50% before & after 2nd signal 100% on whichever signal triggers first
4 25% on each signal 100% between 2nd & 3rd signal
33% between signals
100% on whichever signal triggers first
5 20% on each signal 100% on 3rd signal 25% between signals 100% on whichever signal triggers first


The median strategy would select the mid signal to put on or liquidate a full position. If there is an even number of signals, you would simply buy or sell between the two mid signals or on the later of both if you missed the entry.

An inbetween strategy would always enter or exit between 2 signals. With only 2 signals it would enter or exit sometime after the first signal, but not later than the second. With 3 signals, each half of the position would be built or liquidated after the first and second signal. Not later than the third signal. You would simply try to find an entry or exit that is as favorable as possible in each case.

Another option is to always use the fastest signal. This will only work if the algos are working in sync and if you are able to reliably judge if an algo signal might be bad or not. Imagine one of the algos produces most losing trades and usually triggers faster than the others. This would increase losses and a median strategy would work better. Or imagine that one algo has the tendency of exiting trades too soon. Again profits would be reduced and using a median signal would be more successful.

So, the averaging and median strategies are recommended if the algo trader does not have good trading knowledge or does not fully understands how the algos work. If one of the algos is less efficient or much better than the others, trading the median signal will likely produce the best result. For an averaging strategy to excel all algos have to be similarly efficient.

 

To Compound Or To Not Compound

Compounding means that you keep accumulating gains in your trading account and increase future positions based on them. If you use 20% of the trading capital for a trade and this would be $2000, you would still use 20% on new trades, but it would translate to $2400 per trade if your account grew by 20%.

Compounding is sometimes called the 8th wonder of the world, because past gains have the power to increase future gains dramatically. As your standard position size increases, so do your net profits. This fact alone makes it a worthwhile consideration.


However, compounding strategies have a higher risk of total loss, which is easily underestimated. So it might generally be a good idea to move money out of your trading account after huge gains. This is money that the markets cannot take again from you. The money can also be used as new trading capital if the worst case of blowing up the account should occur.

If you choose to compound, it may happen that your first or second trade incurs a bigger loss. This can have a big effect on future gains with a compounding strategy. So one way to avoid it would be to stick to the same position size, which would be the same dollar amount. If you use 100% of the trading account for trades, you would have to get the money from another strategy that ran more successfully or use money from non-trading funds. At a later point in time after successful trades you could extract the compensation sum again. This strategy makes only sense at the very beginning and not later after many successful trades.

 

Backtest vs Reality

Backtests almost always look rather good. This has to do with the fact that algorithms are often optimized on the data that they are backtested on. You always have to expect that real-time results will be worse.

One thing to remember about ... indicators is that the new ones always seem to work far better than the old ones. This is because all ... indicators are tailored carefully to the idiosyncrasies of past booms and slumps. Little exceptions are built in that would have kept the followers they didn't have - because the indicators weren't yet invented - from losing the shirts they may or may not have had. This gives the new indicators almost perfect records. The old ones, suffering from the fact that no two market swings are exactly alike, look a little battered." Burton Crane (1959)

For example, if an algorithm has a 20% maximum drawdown, it could produce a drawdown of 30% under real trading conditions. If an algo exhibits 33% drawdown in a backtest, it might even produce 50% in real time, which is unacceptable. You also have to expect that gains will be lower in reality, because less trades will work perfectly, which drags down profits. This is perfectly normal, to be expected and nothing to worry about if the algo does not deviate extremely from past results.

"Drawdown can reasonably be expected to be larger in real time because of increased volatility... if, for example, a period of congestion occurs that is substantially longer than any encountered during testing... yet the strategy can still be functioning normally." Robert Prado

These 3D landscape visualize the profits of a combination of 2 parameters. Mountains represent high profits .Valleys mean less profit and water means loss. Left Image: The peaks in the red areas indicate outliers while the green zone contains more robust parameters. But overall this strategy is not very robust, because of the many spikes. Right Image: This strategy is quite robust, because it has smooth hills and not a single parameter produces a loss.


When slightly varying the parameters of an algo, you can get a better impression of the real results that should be expected. It is usually more important to make sure that an algo works rather stable within certain parameter ranges. Within these ranges profits can easily drop by 50%. Sometimes merely 25-50% of what could be achieved under optimal conditions is the result of real trading if a strategy is fit too strongly to past data. You should always keep that in the back of your mind and not get too excited about fantastic backtest results.

"Whichever way you review a strategy there will be a balancing between robustness and performance measures. It will be one-part science and one-part art. ... If I review a strategy with little or no out-of-sample performance but it shows ... robustness, through its versatility and good design, I’d allow a few of the performance measures to be less than desirable. If I feel a strategy, through its complexity, has been overly curve fitted I’d put it aside for no further consideration, despite ... how irresistible it looks." Brent Penfold

Unfortunately some algo developers don't even provide a backtest feature. This should be taken as a warning, because the algo might have a serious weakness that the developer may want to disguise. It is also possible to use future data in an algo, in which case a signal may appear too late to be tradable, but it still looks as if the algo did exceptionally well in the past. To check if this is the case you can use the replay feature of a charting platform to display past signals in "real-time". In any case, it is best to carefully inspect each trade an algo did in the past and see if they make sense to you.

 

Overfitting vs Underfitting

Trading systems and algos are always based on past price data. They incorporate a model about how price behaved in the past and provide rules for successfully dealing with future price action, which may or may not be rather different.

There is ... no way to predict what the model will do in the succeeding periods [after optimization]... Since we are merely fitting a model to past data, we are not capturing all the fundamental and psychological forces driving the market. Our poor ability to predict the future based only on past price data is not surprising." Tushar S Chande


An algo may overfit past data, which means that it does not generalize price behavior enough and detects too specific properties of price moves, which do not repeat this way in future. It is also possible to underfit data. Then the algo generalizes too much and overlooks characteristic properties of the price action of an asset.

"A forecasting model can make valid predictions only to the extent that there is nonrandom, hence predictable, behavior in the data. A model... is fit properly when adapted to that portion of price movement that is nonrandom. A model is improperly or overfit when excessively adapted to the random portion of price movement. The proportion of the random to nonrandom components of price action sets a natural limit to the predictability of a market." Robert Prado

It is easier to underfit with an average-based algo and easier to overfit with a price action or pattern based algo. The maximum version of overfitting is entering or exiting on every price move. The maximum version of underfitting is buy & hold. The first reduces your trading capital very quickly and the second may lead to regret in future and a lot of wasted time if you are not a pro.

"In these kinds of systems, the use of more than one or two parameters reduces the efficiency of the system. The different parameters are, so to speak, on different wavelengths and block each other, undermining the system. This is why the best systems that use mathematical indicators employ very few parameters, which is the only way to get them working in harmony." Felipe Tudela

It can be quite hard to determine if a system is overfitted, but there are some hints. Algos that use averages should not use more than 2 averages (best only 1) and the used curve should not be influenced by more than 2 parameters (best only 1). Additionally, it is often necessary to use noise reducing filter mechanisms, but they should not act too strongly to avoid missed signals. Algos that detect price action or patterns usually need a lot of parameters, which is unavoidable. It is then important to make sure that they deal appropriately with the price moves they are meant to detect and don't have a weak spot that could unexpectedly produce high losses.

"[If] parameters belong to the market core behavior itself... They are not merely statistical entities, but real entities that belong to ... the market. The result is that this multiplicity of parameters do not block each other. Instead, as wheels within wheels, they ensure a smooth working of the entire ... machinery." Felipe Tudela

Underfitting can be noticed when valid trades are not taken, too low profits occur, too much profit is given back or too slow entries exist. Overfitting usually results in too many chop or erratic trades, too many losses and too fast exits. However, making various losses after starting to trade a strategy, which worked great in backtests, does not necessary mean that the it is flawed. If you start trading it during a bear market or choppy bottom, you should not be surprised that it does not show its best side.

"Real-time performance for a sound trading strategy can, in fact, be less than in-sample performance because of worse market conditions or having less market opportunity." Robert Prado

So-called out-of-sample testing can help fighting overfitting. It means that the algo is adjusted to only a portion of the available data and tested on the remaining data. There are various ways to do this. The most reliable way is to do multiple forwards tests, e.g. adjust the algo according to strict rules to a one year period, then test on the next one year period and so forth. This has to be continued after real trading started, because the algo has to be constantly readjusted to work as intended.

However, this is not always a viable option. There might not be enough data or too much outdated data to independently adjust and test the algo. Especially when trading higher time frames or when markets change their character rather quickly this may well be a problem. Artificially creating realistic price data is very difficult and may only help finding general weak spots of an algo. In such cases it is better to rely on rules of thumb, apply some common sense and not overdo it.

"All markets have their own unique personalities. A trading strategy may perform well in one market with one set of values and poorly in another with those same values... I tend to prefer different values for a trading model for different markets, in that it offers an additional dimension of portfolio diversification... I have seen profitable real-time trading strategies that do use different parameter values for different markets and do benefit from periodic reoptimization." Robert Prado

Some people think that algos can only be sound if they work on multiple assets and markets at the same time. They are indeed more robust and less overfit if they do, but they may produce lower profits than an algo that trades a very characteristic behavior of one market. Highly specialized trading strategies as well as rather general ones can both produce net profits.

 

Algo Optimization Traps

When backtesting trading algos and trying to optimize them, there are several traps that can easily seduce you to overdo it. As a result the algo will work too unstable in future and produce too many losses, although backtest results look great.

"No matter how much traders and investors learn about the dangers of overoptimization, they still want to optimize... Understand the concept you are using so well that you will not even feel that you need to optimize." Van Tharp


Trap 1: Maximum Profit

Optimizing the backtest results for maximum profits can easily backfire. It is motivated by greed and a false sense of pride about one's trading system. More robust parameters are often achieved by sacrificing 10-20% of total backtest profits and focusing more on avoiding losses and bringing down the maximum drawdown value. Less losses mean higher net profits, so by reducing losses, higher profits are indirectly achieved in the long run.

"Research has shown that highest net profit alone is not an adequate measure of the overall trading performance and robustness of a trading model. For example, 60 percent of the highest net profit could be generated by a strategy with one large and likely unrepeatable trade... Or, the bulk of this highest net profit could have been produced in the first half of the test period and could be masking a devastating loss in the second and most current half... The strategy with the highest net profit could also have a very large and unacceptable drawdown... The most robust and stable trading strategy may or may not be the most profitable." Robert Prado


Trap 2: Avoiding Losses At All Cost

Avoiding losses is a good idea as long as it does not become an obsession. Introducing a new rule or exception just to erase a single loss is too risky, because it may cause strange trading behavior in future. Removing multiple loss trades while sacrificing a valid trade is also a bad idea. This may easily result in missing a trade with a big profit that more than compensates for past losses.

"Most of the profits [of a trend following strategy] come from a relatively small number of trades... if you get out too soon, you are likely to miss one of the mega-trades [4% of all trades] that make trend-following worth the aggravation." Tushar S Chande

It is better to allow loss trades to occur if the algorithm works more consistently in future. Otherwise, you could overfit the algo to noise in the price data. Price contains more noise during bear markets, so more caution is advised for adjusting to these periods. You don't want your algo to work great during bear markets, but suboptimal during bull markets.


Trap 3: Nasty Profitable Trades

"When you design a system, your goal should be to design a system that produces low-risk ideas... In any worthwhile trading or investing methodology, you must have all kinds of backups to protect you when you’re in a trade that’s going against you." Van Tharp

The trades of an algo should always be logical, make sense and don't expose the trader to high risk. Trades that endure too large pullbacks in order to produce big profits are too risky. In future, a similar trade might be triggered that does not recover from a pullback and instead causes a huge loss. Another example are trades that enter very late, but nevertheless produce a profit. The algo might not be so lucky one day and multiple such trades may cause a too large drawdown. If feasible, these late-entry trades should be avoided by the algo.


Trap 4: Optimizing Individual Trades

When choosing parameters or adding new rules, you should always make sure that they affect as many trades as possible. They should never be tailored to improve one or two individual trades. If a parameter or rule only benefits very few trades, it's best to adjust or remove it again. You don't want to have an algo that only works optimally over a few trades, but instead performs consistently over many months or even years.


Trap 5: Complexity

Adding new features or improvements to an algo is desirable as long as the algo does not get too complex. Too much complexity may produce side-effects that reduce trading performance.

It is not always easy to see if an algo is still simple enough or already too complex. You can try to replace multiple rules or parameters with a single one that does the same job. Or think of another solution that may make them obsolete. Instead of stuffing too many things into an algo, consider creating a new one. Also, if you need to patch up an algo too much, it might also be better to start from scratch. The new algo should use core principles that achieve similar performance with less complexity.


"Are you beginning to understand why the task of trading system development is so full of psychological biases? My experience is that most people will not be able to deal with the issues that come up in trading system design until they've solved some of their personal psychological issues dealing with fear or anger." Van Tharp

There is often a fine line between optimally adjusting a trading algo and overdoing it. Keeping out emotions like pride or greed during this process increases the likelihood that the algorithm will trade successfully in future.

"The unfortunate fact is that parameters that work best on past data rarely provide similar performance in the future. But the real issue is not whether a particular [parameter] set is the best. It is whether you believe sufficiently in the system to trade it without deviations. The primary benefit of optimization may be that you improve your comfort level with a particular system." Tushar S Chande

Ultimately, we only know which parameters worked in the past, but we don't know which ones will work optimally in future. This means that optimizing a trading system is guesswork at best and we can only use a few statistically sound principles that avoid taking too much risk and make a system more robust. Why it often makes more sense to optimally trade a system than to optimize its parameters too much.

 

Readjusting Algos Or Stopping Trading

Regardless of how well an algo works, at some point in the future it should be necessary to readjust it. Normally this should only be the case every few years for swing trading algos if the system is well designed and the traded asset behaves rather consistently. This is more the case for very liquid and popular assets.

"To achieve optimal trading performance over long periods of time with a trading strategy, it will need to be updated, at least occasionally... The main reason a trading system must be reoptimized is that markets change with some frequency. They do not change, however, with great regularity." Robert Prado

Here are some signs that might make it necessary to readjust parameters or add additional features to an algorithm:

1. Too many consecutive losing trades (see table above)
2. Trades with higher losses than usual
3. Trade entries or exits seem "irrational" and do not fit price action
4. Too many too fast exit signals during a bull run that reduce profits

Another option is to use a totally different or new algo that models price behavior better and produces more robust results. But that is the more time-consuming alternative and only necessary if the market behavior changed fundamentally.

If you a longer period of time to adjust a strategy, the strategy will work longer in real-time trading, because there is a higher probability that all types of price action (including a bear market) were considered. However, it also depends on the responsiveness of the trading system how earlier or late it needs to be readjusted. For example, you could create a trading system based on 4 years of data that uses a slow 200 day moving average and it might work for 10 years. Or you could create a fast pattern based system based on 4 years of data and it might produce too many losses after one year already.

"If the performance of the strategy is in free fall, it may prove to be prudent to stop trading the strategy even before it hits the level of the strategy stop-loss... It is best if the actual strategy stop-loss uses some multiple of maximum drawdown." Robert Prado

It is necessary to stop trading a system if it produced a too big drawdown in order to prevent further losses and retain a level of capital that allows restarting trading with a new system. A 33% drawdown should be the ultimate stop loss. But this already requires a 50% gain to return to break even. Stopping at -20% (with only +25% to break even) would be much better, unless the strategy has to endure such a drawdown to work sufficiently. Personally, I would never trade a system that produces more than a 20% max drawdown in backtests.

 

Conclusion

Algo trading provides some challenges, but has many advantages. It is easier than many other types of trading. It requires less experience in order to achieve good results. But trader and algo are more or less in a relationship with each other. The trader has to invest some trust, otherwise it will not work out.

Algos can also teach you a lot of things about trading if you reflect about why it might have or not have entered or exited at certain points. Especially eye-opening is the experience of doing what an algo recommends, although you are very skeptical, and have your opinion proven wrong in the end. Algorithms selectively focus on small amounts of data and make decisions without the tiniest bit of emotion or bias. This is an attitude worth emulating as a trader.

"The bottom line is you must have an edge. If you are not trading with a statistical advantage over the market, everything else is futile. Discipline, money management, execution skills & positive thinking add value in support of an actual edge, but they are not edges themselves." Adam Grimes

Most importantly, an algo can provide you with an edge that may otherwise take years and a lot of losses to develop. Your main challenge is to put it into practice and execute the trades consistently. This can only be done with enough experience. You are only able to gather the necessary experience if you don't give up, which in turn is only possible if you don't risk too much to kill your primary motivation to trade.