Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Carbune, Jason Lin, Maria Wang, Yun Zhu, Jindong Chen. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Improving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users. To stimulate research in this direction, we release a human-annotated dataset with approximately 500k unique annotations aimed at increasing the understanding of the functionality of UI elements. This dataset augments images and view hierarchies from RICO, a large dataset of mobile UIs, with annotations for icons based on their shapes and semantics, and associations between different elements and their corresponding text labels, resulting in a significant increase in the number of UI elements and the categories assigned to them. We also release models using image-only and multimodal inputs; we experiment with various architectures and study the benefits of using multimodal inputs on the new dataset. Our models demonstrate strong performance on an evaluation set of unseen apps, indicating their generalizability to newer screens. These models, combined with the new dataset, can enable innovative functionalities like referring to UI elements by their labels, improved coverage and better semantics for icons etc., which would go a long way in making UIs more usable for everyone.

Parallelization of traditional accelerated techniques for integral equation solvers has been shown to be inefficient and to scale poorly with the number of parallel computing nodes. This is because traditional methods typically represent the solution using piecewise constant (PWC) basis functions, resulting in gigantic systems of linear equations to solve. In this paper, we propose instantiable basis functions, which generate smaller systems than PWC basis functions for the same accuracy. Furthermore, they redistribute computation from the system solving step to the embarrassingly parallelizable system setup step, hence enabling highly scalable and efficient parallelization. In the examples, we tested, our new solver is six to ten times faster than FASTCAP in serial execution and achieves 90% parallel efficiency in a ten-core distributed-memory system. We developed a toolkit that automates a complete extraction flow from GDSII layout files to capacitance matrices. Our code has been released in the public domain.

Traditional parallel boundary element methods suffer from low parallel efficiency and poor scalability due to the long system solving time bottleneck. In this paper, we demonstrate how to avoid such a bottleneck by using an instantiable basis function approach. In our demonstrated examples, we achieve 90% parallel efficiency and scalability both in shared memory and distributed memory parallel systems.