The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform? To enable greater study of this question, we present LegalBench: a collaboratively constructed legal reasoning benchmark consisting of 162 tasks covering six different types of legal reasoning. LegalBench was built through an interdisciplinary process, in which we collected tasks designed and hand-crafted by legal professionals. Because these subject matter experts took a leading role in construction, tasks either measure legal reasoning capabilities that are practically useful, or measure reasoning skills that lawyers find interesting. To enable cross-disciplinary conversations about LLMs in the law, we additionally show how popular legal frameworks for describing legal reasoning—which distinguish between its many forms—correspond to LegalBench tasks, thus giving lawyers and LLM developers a common vocabulary. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.

Augmented Reality (AR) as a new field regarding Human Computing Interaction (HCI) has been gaining momentum in the last few years. Being able to project interactive graphics into real-life environments can be applied in various fields, research and commercial goals. In the field of education, textbooks are still considered to be the primary tool used by students to learn about new topics. Since AR requires interaction and exploration, it brings a ludic component that is hard to replicate using regular textbooks. The application we developed allows elementary school students to interact with a fully three-dimensional human skeleton model, using specialized virtual buttons. Students can understand this complex structure and learn the names of important bones just by using a tablet, a picture and their hands. Results show that the majority of students consider that our AR application helped them visualize and learn more about the human skeletal system. Additionally, the data we gathered shows that there was a 16% increase in correct responses regarding bone names after using our AR application. Our AR application successfully helped the students learn about the human skeletal system by introducing them to AR technologies.