Tabular data drive most real-world machine learning applications, yet building general-purpose models for them remains difficult. Mixed numeric and categorical fields, weak feature structure, and limited labeled data make scaling and generalization challenging. To this end, we introduce Orion-Bix, a tabular foundation model that combines biaxial attention with meta-learned in-context reasoning for few-shot tabular learning. Its encoder alternates standard, grouped, hierarchical, and relational attention, fusing their outputs through multi-CLS summarization to capture both local and global dependencies efficiently. A label-aware ICL head adapts on the fly and scales to large label spaces via hierarchical decision routing. Meta-trained on synthetically generated, structurally diverse tables with causal priors, Orion-Bix learns transferable inductive biases across heterogeneous data. Delivered as a scikit-learn compatible foundation model, it outperforms gradient-boosting baselines and remains competitive with state-of-the-art tabular foundation models on public benchmarks, showing that biaxial attention with episodic meta-training enables robust, few-shot-ready tabular learning. The model is publicly available at https://github.com/Lexsi-Labs/Orion-BiX .

Tabular Foundation Models (TFMs) have recently shown strong in-context learning capabilities on structured data, achieving zero-shot performance comparable to traditional machine learning methods. We find that zero-shot TFMs already achieve strong performance, while the benefits of fine-tuning are highly model- and data-dependent. Meta-learning and PEFT provide moderate gains under specific conditions, whereas full supervised fine-tuning often reduces accuracy or calibration quality. This work presents the first comprehensive study of fine-tuning in TFMs across benchmarks including TALENT, OpenML-CC18, and TabZilla. We compare zero-shot, meta-learning, supervised (SFT), and parameter-efficient (PEFT) approaches, analyzing how dataset factors such as imbalance, size, and dimensionality affect outcomes. Our findings cover performance, calibration, and fairness, offering practical guidelines on when fine-tuning is most beneficial and its limitations.

Feature engineering is a critical yet often manual step in building effective machine learning models. While automated machine learning (AutoML) has streamlined many aspects of model development, the generation of high-quality, interpretable features remains a key bottleneck, requiring case-by-case domain knowledge and significant effort. This challenge highlights the importance of automated feature engineering (AutoFE) as a critical component within AutoML pipeline. To address this, we recently proposed SMART, a novel AutoFE approach that combines knowledge graph reasoning and deep reinforcement learning to guide the generation of interpretable features. In this demonstration, we introduce KnowFE, a web-based AutoFE platform powered by SMART. KnowFE enables users to generate high-quality, human-understandable features without writing any code, striking a balance between explainability and predictive performance. With a user-friendly interface, it empowers data practitioners to efficiently enhance machine learning workflows across diverse domains. A video demonstration is available at https://www.KnowFE.com .

Feature engineering stands as a pivotal step in enhancing the performance of machine learning (ML) models, particularly with tabular data. However, traditional feature engineering methods are often time-consuming and requires case-by-case domain knowledge. In addition, as ML systems become more common, interpretability becomes increasingly important, especially among domain experts. To this end, we propose ReaGen, an automated feature engineering (AutoFE) approach that combines knowledge graphs (KGs) with large language models (LLMs) to generate interpretable features. ReaGen begins by symbolic REAsoning over the KG to extract relevant information based on datasets description. Then, it uses an LLM to iteratively GENerate meaningful features. Finally, to overcome challenges such as hallucinations and handling long contexts typical in LLMs, our model performs logical reasoning on the KG to ensure that the generated features maintain interpretability. ReaGen provides Python code for automatic feature generation and detailed explanations of feature utility. It leverages both LLM's internal knowledge and retrieved information from KGs. Experiments on public datasets demonstrate that ReaGen significantly improves prediction accuracy while ensuring high interpretability through human-like explanations for each feature. This work highlights the potential of integrating LLMs and KGs in feature engineering, paving the way for interpretable ML models.