Will Transformers Revolutionize Time-Series Forecasting? - Advanced Insights, Part 2

The 2nd part analyzes the ultimate dilemma in detail.

The Transformer is a general-purpose, trainable computation engine.

That’s what Andrej Karpathy mentioned on Lex Fridman’s podcast. When asked about the most beautiful idea in AI, he mentioned the Transformer.

Since the rise of Deep Learning in 2012, various neural network structures have been developed for different tasks, like CNNs for images.

After the release of "Attention is All You Need", the Transformer is now used for almost everything—video, text, speech, and images.

Transformers are known for excelling in understanding unstructured data due to their general-purpose architecture. But can Transformers be successful in forecasting?

And if so, how can we use them most effectively?

Let's find out in this article.

Recap of Part 1

In Part 1, we explored the evolution of Deep Learning and Transformers, focusing on their use in time-series forecasting.

Specifically, we explored:

• What was the secret sauce of the successful Deep Learning forecasting models? Not all early DL models performed well for time series.

• Why is the application of DL in time series lagging behind other domains?

Here’s Part 1, in case you missed it:

Will Transformers Revolutionize Time-Series Forecasting? Advanced Insights, Part 1

Transformers Under Fire

Every revolutionary breakthrough that aims to change the status quo is met with skepticism and criticism. This is expected.