Benjamin T. Jones

dblp:228/6104 · DBLP profile ↗
← Back
9ranked-venue papers
7as first author
7since 2021 · last 2025
—ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 7 · 5 first-author · 7 since 2021Human-computer interaction and ubiquitous computing · 3 · 2 first-author · 1 since 2021Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2025 A Solver-Aided Hierarchical Language for LLM-Driven CAD Design
abstract
Abstract Parametric CAD systems use domain‐specific languages (DSLs) to represent geometry as programs, enabling both flexible modeling and structured editing. With the rise of large language models (LLMs), there is growing interest in generating such programs from natural language. This raises a key question: what kind of DSL best supports both CAD generation and editing, whether performed by a human or an AI? In this work, we introduce AIDL, a hierarchical, solver‐aided DSL designed to align with the strengths of LLMs while remaining interpretable and editable by humans. AIDL enables high‐level reasoning by breaking problems into abstract components and structural relationships, while offloading low‐level geometric reasoning to a constraint solver. We evaluate AIDL in a 2D text‐to‐CAD setting using a zero‐shot prompt‐based interface and compare it to OpenSCAD, a widely used CAD DSL that appears in LLM training data. AIDL produces results that are visually competitive and significantly easier to edit. Our findings suggest that language design is a powerful complement to model training and prompt engineering for building collaborative AI–human tools in CAD. Code is available at https://github.com/deGravity/aidl .
Benjamin T. Jones, Felix Hähnlein, Wojciech Matusik, Maaz Bin Safeer Ahmad, Vladimir G. Kim, Adriana Schulz
Comput. Graph. Forum1
2024 Computational Illusion Knitting
abstract
Illusion-knit fabrics reveal distinct patterns or images depending on the viewing angle. Artists have manually achieved this effect by exploiting "microgeometry," i.e., small differences in stitch heights. However, past work in computational 3D knitting does not model or exploit designs based on stitch height variation. This paper establishes a foundation for exploring illusion knitting in the context of computational design and fabrication. We observe that the design space is highly constrained, elucidate these constraints, and derive strategies for developing effective, machine-knittable illusion patterns. We partially automate these strategies in a new interactive design tool that reduces difficult patterning tasks to familiar image editing tasks. Illusion patterns also uncover new fabrication challenges regarding mixed colorwork and texture; we describe new algorithms for mitigating fabrication failures and ensuring high-quality knit results.
Amy Zhu, Yuxuan Mei, Benjamin T. Jones, Zachary Tatlock, Adriana Schulz
ACM Trans. Graph.3
2023 Self-Supervised Representation Learning for CAD
abstract
Virtually every object in the modern world was created, modified, analyzed and optimized using computer aided design (CAD) tools. An active CAD research area is the use of data-driven machine learning methods to learn from the massive repositories of geometric and program representations. However, the lack of labeled data in CAD's native format, i.e., the parametric boundary representation (B-Rep), poses an obstacle at present difficult to overcome. Several datasets of mechanical parts in B-Rep format have recently been released for machine learning research. However, large-scale databases are mostly unlabeled, and labeled datasets are small. Additionally, task-specific label sets are rare and costly to annotate. This work proposes to leverage unlabeled CAD geometry on supervised learning tasks. We learn a novel, hybrid implicit/explicit surface representation for B-Rep geometry. Further, we show that this pre-training both significantly improves few-shot learning performance and achieves state-of-the-art performance on several current B-Rep benchmarks.
Benjamin T. Jones, Michael Hu, Milin Kodnongbua, Vladimir G. Kim, Adriana Schulz
CVPR1
2023 ReparamCAD: Zero-shot CAD Re-Parameterization for Interactive Manipulation
abstract
Parametric CAD models encode entire families of shapes that should, in principle, be easy for designers to explore. However, in practice, parametric CAD models can be difficult to manipulate due to implicit semantic constraints among parameter values. Finding and enforcing these semantic constraints solely from geometry or programmatic shape representations is not possible because these constraints ultimately reflect design intent. They are informed by the designer’s experience and semantics in the real world. To address this challenge, we introduce ReparamCAD, a zero-shot pipeline that leverages pre-trained large language and image model to infer meaningful space of variations for a shape We then re-parameterize a new constrained parametric CAD program that captures these variations, enabling effortless exploration of the design space along meaningful design axes. We evaluated our approach through five examples and a user study. The result showed that the inferred spaces are meaningful and comparable to those defined by experts. Code and data are at: https://github.com/milmillin/ReparamCAD.
Milin Kodnongbua, Benjamin T. Jones, Maaz Bin Safeer Ahmad, Vladimir G. Kim, Adriana Schulz
SIGGRAPH Asia2
2023 B-rep Matching for Collaborating Across CAD Systems
abstract
Large Computer-Aided Design (CAD) projects usually require collaboration across many different CAD systems as well as applications that interoperate with them for manufacturing, visualization, or simulation. A fundamental barrier to such collaborations is the ability to refer to parts of the geometry (such as a specific face) robustly under geometric and/or topological changes to the model. Persistent referencing schemes are a fundamental aspect of most CAD tools, but models that are shared across systems cannot generally make use of these internal referencing mechanisms, creating a challenge for collaboration. In this work, we address this issue by developing a novel learning-based algorithm that can automatically find correspondences between two CAD models using the standard representation used for sharing models across CAD systems: the Boundary-Representation (B-rep). Because our method works directly on B-reps it can be generalized across different CAD applications enabling collaboration.
Benjamin T. Jones, James Noeckel, Milin Kodnongbua, Ilya Baran, Adriana Schulz
ACM Trans. Graph.1
2022 Computational Design of Knit Templates
abstract
We present an interactive design system for knitting that allows users to create template patterns that can be fabricated using an industrial knitting machine. Our interactive design tool is novel in that it allows direct control of key knitting design axes we have identified in our formative study and does so consistently across the variations of an input parametric template geometry. This is achieved with two key technical advances. First, we present an interactive meshing tool that lets users build a coarse quadrilateral mesh that adheres to their knit design guidelines. This solution ensures consistency across the parameter space for further customization over shape variations and avoids helices, promoting knittability. Second, we lift and formalize low-level machine knitting constraints to the level of this coarse quad mesh. This enables us to not only guarantee hand- and machine-knittability, but also provides automatic design assistance through auto-completion and suggestions. We show the capabilities through a set of fabricated examples that illustrate the effectiveness of our approach in creating a wide variety of objects and interactively exploring the space of design variations.
Benjamin T. Jones, Yuxuan Mei, Haisen Zhao, Taylor Gotfrid, Jennifer Mankoff, Adriana Schulz
ACM Trans. Graph.1
2021 AutoMate: a dataset and learning approach for automatic mating of CAD assemblies
abstract
Assembly modeling is a core task of computer aided design (CAD), comprising around one third of the work in a CAD workflow. Optimizing this process therefore represents a huge opportunity in the design of a CAD system, but current research of assembly based modeling is not directly applicable to modern CAD systems because it eschews the dominant data structure of modern CAD: parametric boundary representations (BREPs). CAD assembly modeling defines assemblies as a system of pairwise constraints, called mates , between parts, which are defined relative to BREP topology rather than in world coordinates common to existing work. We propose SB-GCN, a representation learning scheme on BREPs that retains the topological structure of parts, and use these learned representations to predict CAD type mates. To train our system, we compiled the first large scale dataset of BREP CAD assemblies, which we are releasing along with benchmark mate prediction tasks. Finally, we demonstrate the compatibility of our model with an existing commercial CAD system by building a tool that assists users in mate creation by suggesting mate completions, with 72.2% accuracy.
Benjamin T. Jones, Dalton Hildreth, Duowen Chen 0003, Ilya Baran, Vladimir G. Kim, Adriana Schulz
ACM Trans. Graph.1
2018 Human-AI Interaction in Symbolic Problem Solving
abstract
Despite the increasing need for computer assistance in solving problems involving complex systems and large amount of data, professional mathematicians, scientists, and engineers currently avoid the use of computer algebra systems during creative problem-solving phases of their work due to problems with transparency, familiarity, and inflexibility in input. I have designed and prototyped a new approach to interaction with computer algebra systems that is compatible with current working styles, flexible in its input and output. I propose a user study to validate this tool, and tool extensions to allow creative problem solvers to interactively define their own notation as they work.
Benjamin T. Jones
VL/HCC1
2018 Searching Over Search Trees for Human-AI Collaboration in Exploratory Problem Solving: A Case Study in Algebra
abstract
Artificial intelligence and machine learning work very well for solving problems in domains where the optimal solution can be characterized precisely or in terms of adequate training data. However, when humans perform problem solving, they do not necessarily know how to characterize an optimal solution. We propose a framework for human-AI collaboration that gives humans ultimate control of the results of a problem solving task while playing to the strengths of the AI by persisting an agent's search trees and allowing humans to explore and search this search tree. This allows the use of AI in exploratory problem solving contexts. We demonstrate this framework applied to algebraic problem solving, and show that it enables a unique mode of interaction with symbolic computer algebra through the automatic completion and correction of traditional derivations, both in digital ink and textual keyboard input.
Benjamin T. Jones, Steven L. Tanimoto
VL/HCC1