AI Horizon Forecast

AI Horizon Forecast

Share this post

AI Horizon Forecast
AI Horizon Forecast
Will Transformers Revolutionize Time-Series Forecasting? - Advanced Insights, Part 2

Will Transformers Revolutionize Time-Series Forecasting? - Advanced Insights, Part 2

The 2nd part analyzes the ultimate dilemma in detail.

Nikos Kafritsas's avatar
Nikos Kafritsas
Jul 27, 2024
∙ Paid
11

Share this post

AI Horizon Forecast
AI Horizon Forecast
Will Transformers Revolutionize Time-Series Forecasting? - Advanced Insights, Part 2
3
Share

The Transformer is a general-purpose, trainable computation engine.

That’s what Andrej Karpathy mentioned on Lex Fridman’s podcast. When asked about the most beautiful idea in AI, he mentioned the Transformer.

Since the rise of Deep Learning in 2012, various neural network structures have been developed for different tasks, like CNNs for images.

After the release of "Attention is All You Need", the Transformer is now used for almost everything—video, text, speech, and images.

Transformers are known for excelling in understanding unstructured data due to their general-purpose architecture. But can Transformers be successful in forecasting?

And if so, how can we use them most effectively?

Let's find out in this article.

Recap of Part 1

In Part 1, we explored the evolution of Deep Learning and Transformers, focusing on their use in time-series forecasting.

Specifically, we explored:

  • What was the secret sauce of the successful Deep Learning forecasting models? Not all early DL models performed well for time series.

  • Why is the application of DL in time series lagging behind other domains?

Here’s Part 1, in case you missed it:

Will Transformers Revolutionize Time-Series Forecasting? Advanced Insights, Part 1

Will Transformers Revolutionize Time-Series Forecasting? Advanced Insights, Part 1

Nikos Kafritsas
·
June 25, 2024
Read full story

Transformers Under Fire

Every revolutionary breakthrough that aims to change the status quo is met with skepticism and criticism. This is expected.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Nikos Kafritsas
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share