Function calling, also called tool use, refers to an LLM’s ability to invoke external functions, APIs, or user-defined tools in response to user queries—an essential capability for agentic LLM applications. Despite its prominence, there did not exist a standard benchmark to evaluate function calling abilities, due to two reasons – the challenging nature of evaluating when a function call is valid, and the challenge of acquiring diverse, real-world functions. We present the Berkeley Function Calling Leaderboard (BFCL), a comprehensive benchmark designed to evaluate function calling capabilities in a wide range of real-world settings. The BFCL benchmark evaluates serial and parallel function calls, across various programming languages using a novel Abstract Syntax Tree (AST) evaluation method that can easily scale to thousands of functions. We construct the benchmark using a combination of expert curated, and user-contributed functions and associated prompts. Finally, BFCL benchmark evaluates the ability of models to abstain and reason in stateful multi-step agentic setting. Evaluating a wide range of models, we observe that while state-of-the-art LLMs excel at singleturn calls, memory, dynamic decision-making, and long-horizon reasoning remain open challenges. Since its preview, BFCL has become the defacto standard for evaluating function-calls, and can be accessed at gorilla.cs.berkeley.edu/leaderboard.html.

Accurate load forecasting is essential for power system stability and operational planning. While deterministic forecasts provide point estimates, they fail to capture uncertainty, which can lead to grid imbalances or reserve misallocations. This study evaluates six non-parametric probabilistic forecasting approaches which are Adaptive Conformal Inference (ACI), Deep Quantile Regression (QR), Bayesian LSTM (BLSTM), CatBoost Quantile Regression, Mamba ACI and Mamba QR applied to short-term load forecasting. The stacked LSTM and Mamba models are first optimized via grid search and then used to generate point forecasts over which prediction intervals are constructed. Using 3.4 years of hourly load data from Tamil Nadu, India, the models are evaluated on an unseen dataset of 8.21 months based on coverage, mean interval width (MIW), Continuous Ranked Probability Score (CRPS), and Winkler score. Mamba ACI achieved the highest coverage (92.47%) with MIW of 8.5% of peak load, while LSTM ACI followed with 90.24% coverage and 5.9% MIW. CatBoost yielded the sharpest intervals (3.8% of peak) but with lower coverage (83.81%). DQR showed moderate performance, and BLSTM achieved 89.60% coverage with an MIW of 8.0% of peak load, striking a balance between reliability and sharpness. Mamba QR, though covering 90.29%, produced excessively wide intervals (26.9% of peak). The results highlight that ACI delivers the best balance between sharpness and reliability across architectures. Given the importance of coverage in avoiding forecasting-related operational risks, Mamba ACI emerges as the most practical and robust choice for uncertainty quantification in very short-term load forecasting.