Tabular Foundation Models Are Not Plug-and-Play - Part 1

The pretrained models beating gradient boosting come with hidden settings. A guide to TabPFN-3, TabICL-v2, and TabFM

Image by author

Tabular foundation models arrived with a simple promise. No training, no tuning, and we get predictions that beat a tuned gradient boosting model.

The first half of that promise is real. We do not train these models. The second half hides a catch.

Under the fit and predict calls there’s a layer of settings and hidden defaults that decide whether we get good or mediocre results.

Pass the same table as a NumPy array instead of a dataframe, and things can change in ways you might not notice. One model may completely stop handling categorical features, while another silently drops categories that appear only once in the training data. These are not rare edge cases, they can easily turn a fair benchmark into a misleading one.

In this article, we’ll look at the 3 leading tabular foundation models right now: TabPFN-3, TabICL-v2, and TabFM. For each model, we’ll look at who created it, how to use it, and which settings you should pay attention to.

Accuracy is not the whole story either. Once we try to use one of these for real work, speed and calibration matter just as much. The most accurate model here is also the slowest.

I will say something that might sound strong. I think the progress in tabular foundation models this year is a bigger deal than the latest round of models from OpenAI or Anthropic. Most business data lives in tables, and for the first time a pretrained model beats the tools that owned this space for a decade.

Let’s get started!

✅ Find the tabular foundation model benchmark and every model's hidden configurations in the AI Projects Folder (Project 34). Plus more cool projects inside!

Preliminaries

This all started with TabPFN, and the idea behind it comes straight from large language models. We can hand an LLM a few examples inside the prompt and it works out the task without any training. TabPFN does the same for tables.

The fit() call does not actually train the model. It simply stores the training rows. When you call predict(), those rows are used as context for a frozen network, which then generates the predictions.

For years, boosted trees dominated the tabular world. Benchmark after benchmark put models like CatBoost and XGBoost on top, with neural networks a step behind. The best-known result is [1], a 2022 paper that found that tree-based models outperform Deep Learning on tabular data.

The research community relied heavily on trees and did not take early neural models seriously. That skepticism made sense at the time. What it missed was how quickly these models improved. Early versions also had clear limits. TabPFN-v1 only worked on small tables, with a few thousand rows and 100 features at most.

The jump from TabPFN-v1 to v2 was the biggest single leap. Then Google released TabFM this year, and the narrative changed. Now the hype runs the other way.

Table 1: Comparing the top 3 models at the time of writing this article (Image by author - Source)

TabPFN-3 and TabFM achieve the top spots on accuracy. TabICL-v2 gives up a little raw accuracy for a big win in speed and openness, and we will see why in Part 2. For each one, here is a short description and the settings that matter most, because it is not a simple forward pass.

TabPFN-3

TabPFN comes from Prior Labs, a team that grew out of Frank Hutter’s research group. The first version was released in 2022. Version 2 was published in Nature in early 2025[2], and version 3 was released in May 2026[4].

The weights are free to download, but the license is non-commercial. We can use TabPFN-3 for research, evaluation, and internal testing, which covers a benchmark like ours.

This is a good place to explain the trick behind all 3 models. They are never trained on real data. They are trained on millions of synthetic tables, each one generated from a random cause-and-effect structure like the graph above. This generator is called the prior.

Figure 1: A structural causal model (SCM) graph. Synthetic training data is generated from a random cause-and-effect structure like this one. [Source 2]

The prior is to a tabular foundation model what the pretraining corpus is to an LLM (the secret sauce). A better prior means better results on real tables the model has never seen. Most teams keep their prior private, and I come back to this in the TabICL section, because that team is doing something unusual with theirs.