Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks range from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human developers, the majority of their evaluations are limited to short and self-contained algorithmic tasks or standalone function calls. Solving challenging and practical tasks requires the capability of utilizing **diverse function calls as tools** to efficiently implement functionalities like data analysis and web development. In addition, using multiple tools to solve a task needs compositional reasoning by accurately understanding **complex instructions**. Fulfilling both of these characteristics can pose a great challenge for LLMs. To assess how well LLMs can solve challenging and practical tasks via programs, we introduce BigCodeBench, a benchmark that challenges LLMs to invoke multiple function calls as tools from 139 libraries and 7 domains for 1,140 fine-grained tasks. To evaluate LLMs rigorously, each task encompasses 5.6 test cases with an average branch coverage of 99%. In addition, we propose a natural-language-oriented variant of BigCodeBench, BigCodeBench-Instruct, that automatically transforms the original docstrings into short instructions containing only essential information. Our extensive evaluation of 60 LLMs shows that **LLMs are not yet capable of following complex instructions to use function calls precisely, with scores up to 60%, significantly lower than the human performance of 97%**. The results underscore the need for further advancements in this area.

Across industries, there is an ever-increasing rate of data sharing for collaboration and innovation between organizations and their customers, partners, suppliers, and internal teams. However, many enterprises are restricted from freely sharing data due to regulatory restrictions across different regions, performance issues in moving large volume data, or requirements to maintain autonomy. In such situations, the enterprise can benefit from the concept of federated learning, in which machine learning models are constructed at various geographic sites. In this paper, we introduce a general framework, namely BDSP, to share data among enterprises based on Blockchain and federated learning techniques. Specifically, we propose a transparency contribution accounting mechanism to estimate the valuation of data and implement a proof-of-concept for further evaluation. The extensive experimental results show that the proposed BDSP has a competitive performance with higher training accuracy, an increase of over 5%, and lower communication overhead, reducing 3 times, compared to baseline approaches.