RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
arXiv:2610.17867v3 Announce Type: replace-cross Abstract: Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models to unseen datasets requires tens of thousands of examples. Here we introduce RT-J, an 86M-parameter relational foundation model (RFM) that makes accurate predictions even with hundreds of in-context examples or fewer. To achieve this, we first assemble the Joi…