LSTM Top Mistake In Price Movement Predictions For Trading

Good luck for your trading and mostly for your algorithmic trading/learning!

🍓 Follow structured courses with more details and practice exercises check my "About" page for Discount Coupons on my Udemy courses covering:
- Python basics Kickstarter,
- Object Oriented Programming
- Data Analysis with NumPy and Pandas including financial analysis exercises,
... more courses are on the way drop me a message if you have a particular interesting topic!
Good luck!

#forexanalysis #neuralnetworks #deeplearning #trading #tradingbot #forex #stockmarket #stocktrading #stocktradingstrategies #algotrading #python

💲 Discount Coupon for My Udemy course on Algorithmic Trading:

The python code link:

[MasamuneX]

ive got a 6 gb csv file with the last 5000 days of price info for around 4800 companies merged with quarterly financial data like assets revenue etc merged with macro indicators like interest rates and all of that interpolated and cleaned up before passing to an lstm with many many layers with tuned hyper params. instead of using a standard loss value you should use mean absolute percentage error because if a stock is trading at 1$ and you have a loss of 1$ you would be off by 100% but if its trading at 1000$ and yoru loss is 10 then you would be off by 1%.

also try to use time distributed lstm layers at the input and to use leaky relu not just relu so that parts of the model that arnt use often dont just become dead weight . Also use a deeper model to capture the complex relationship between the columns in the training data and MOST Kill any over fitting with lots of dropout layers and batch norm

[Mammel]

The biggest pitfall of anyone starting with time series predictions (of any kind)! Predicting actual values instead of predicting the difference between values. Great video explaining why!

[ShaneZarechian]

Hi, just want to say I've watched many videos about this topic and that this is the best one because you let me see the mistake in other's results that look good superficially

[jakub]

Dude it was the same problem that made mu furious 2years ago, and I couldn't understand it.

THANKS YOU! This class was a blessing!

[mjever]

Great video, excellent observation about the good predictions phenomena, almost 90% of articles and papers i came across online show high accuracy, that is not applicable for the future only for test data, which is useless!

[noimnotnice]

This is a valuable lesson about uncertainty and how much 'magic' one can squeeze from AI.

I think you are asking too much from the model. Using the highest w/l strategies, not even the best traders can predict what the price will do at the *next* candle, for *every* candle. They are content with making predictions about the near, unspecified future, at times when many concepts converge on a bias. In essence, *nobody* can predict exact price movements at all times.
Apart from making this demand, the worst 'crime' you ended up committing was throwing away volume information. Your model is tasked with price-action prediction, and you took away *its most critical* input.

I don't know how one would train a model with the fuzziness of "tell me when prices will rise/fall, by approximately how much", which is what traders do.

Probably best, however, would be to skip this approach entirely and task the machine to predict order signals, SL, and TP prices, then measure strategy performance, instead. And please, don't throw away volume.

[AndyCreedx]

I tried to do exactly the same 2 years ago and I gave up. Congrats for the achievement!!

[kadourkadouri]

Nice ! But I sincerely think that data scaling is also a source of problems as most of the features are continuous (ATL, +inf). Therefore, usual scaling methods assume that the previous ATH will never be reached in the future in which case the model, no matter how sophisticated it is, will never predict a breakthrough (new ATHs).

[cahitkarahan]

1- If you calculate power of predicted results few times looks like you will get barely enough results for price change.
2- Instead of predicting how many pips price gonna change, predict how much percent price gonna change rational to the last candle. In other words don't confuse our little brained neural net with exact numbers or pips, instead teach it to predict relatively change of price as daily percent.

[VadimChes]

why to limit LSTM. It can use any size of memory. I think no need to set any number of candles to limit... If you use that limit then just use usual ANN instead of LSTM

[Rudaosong]

You are certainly right. I have traded cryptocurrency in practice for 3 months. I do the swing trading and clearly realize that it is hardly to calculate or predict the oscillate range of any coin. But I think trend prediction would be a better direction. Transferring the problem from regression to classification may be more efficient🧐

[arashsadeghibablan]

You just saved me a LOT of time, thank you 🙏🏼❤️

[pink_cloudsky]

You mean using the actual values instead of the difference it's a mistake because for price prediction we are looking for accurate results. But if we simply want a forecast to understand the direction of the time series, wouldn't we be happy with the results ? In some use cases we might not need an exact number. Do you think in this cases, the model with actual value prediction is still okay ?

[vyacheslavfiodorov]

The problem in shifted values is in fact that regression model predicts value of Close very similar to close of current bar, not future, even if you have targeted NextClose column with future close values. That’s why you have shifted values. So, they are very similar to previous closes … So I can’t understand why it works like this. Why people invented regression ML models if they work like this

[bosypuspus]

For the bad predictions graph, I think it can be used as a golden crossover strategy between test and predicted prices

[qwertasd]

hm based upon avaraged indicators predict..>> Something slow moving, stears something wildly moving.. this might not be ideal engineering for LSTM use.
LSTM's need patterns, not noice. I'll analyse it deeper tomorow, as it helps me thinking about my tensorflow projects too.

[julianbonomini]

Sorry, I'm quite dumb. I'm trying to understand why for a simple classification model using the next closing price wouldn't work

If I'm trying to catch a trend (and really don't care about how much will go up or down), looking at the predicted close isn't enough? So for example if I do:

prev_actual_close < actual_close && prev_actual_close < predicted_close

and gather all results for the testing data, with (just for the example) 90% success rate, doesn't that mean I *could* use it as a "going go go up, not sure how much" type of flag?

I know there are other (maybe) better classification models, but I'm just getting started and have much to learn still, your videos are helping a lot so thank you!

[ucthanhchau]

Does that mean that predicting price fluctuations on a line that is not too attractive like that is not necessarily wrong? Because I was also studying and working on the same problem, and the results were similar!

[pedrerorey]

Isn't this the principle of using stationary series for predictions instead of wrongly using non-stationary series ?

[truthforeman]

Great video! how would we modify this to predict multiple targets instead of one?