Recent advances in text-to-image diffusion models have demonstrated remarkable capabilities in generating high-quality visual content with style and feature controlled. A fundamental challenge remains in simultaneously maintaining three critical properties of generated image sequences: (1) fine-grained style control, (2) strict image-prompt alignment, and (3) cross-image content coherence. To overcome the challenge, we leverage AnyStyleDiffusion to overcome the challenge. Specifically, we interpret any artistic style required by users on generated image as a feature in models' weight space. Interpolation between weight space obtains models expressing middle styles with linear transition. Hyper-receptive Motion Layers is proposed to align outputs of diverse weight spaces, operating as adaptive style modulators. These HRMLs are separated from interpolated diffusion models, leveraging zero-shot compatibility with existing model checkpoints. By employing Homogeneous Stable Diffusion, direct interpolation on weight space is avoided to improve synthesis efficiency. Comprehensive evaluations across personalized models demonstrate our method's superiority in generating content-coherent sequences with dynamic style transformations. Code will be released at https://github.com/shermandozer/AnyStyleDiffusion.git.

We present an incremental sampling-based task and motion planner for retrieving near-cylindrical objects, like bottle, in cluttered scenes, which computes a plan for removing obstacles to generate a collision-free motion of a robot to retrieve the target object. Our proposed planner uses a two-level hierarchy, including the first-level roadmap for the target object motion and the second-level retrieval graph for the entire robot motion, to aid in deciding the order and trajectory of object removal. We use an incremental expansion strategy to update the roadmap and retrieval graph from the collisions between the target object, the robot, and the obstacles, in order to optimize the object removal sequence. The performance of our method is highlighted in several benchmark scenes, including a fixed robotic arm in a cluttered scene with known obstacle locations and a scene, where locations of some objects or even the target object are unknown due to occlusions. Our method can also efficiently solve the high-dimensional planning problem of object retrieval using a mobile manipulator and be combined with the symbolic planner to plan complex multistep tasks. We deploy our method to a physical robot and integrate it with nonprehensile actions to improve operational efficiency. Compared to the state-of-the-art approaches, our method reduces task and motion planning time up to 24.6$\%$with a higher success rate, and still provides a near-optimal plan.

We present a fast collision checking/avoidance algorithm for collaborative robot arms that work in close proximity. We formulate forward kinematics and separating distance function using DH convention and Taylor models (the tight enclosure of a function), and then compute their tight bounds for determining interference between robot arms. Our algorithm allows the collaborative robot arms to perform fine and dexterous tasks in close spatial proximity.

Visual servoing of robot manipulators can be viewed as an optimization problem and recurrent neural network is widely accepted as a powerful tool for solving optimization problems. Inspired by this, an adaptive neural network method is proposed for image-based visual servoing (IBVS) with uncertain parameters. It is the first work focused on IBVS simultaneously considering the uncertain camera configuration parameters and uncertain kinematics of robot manipulators in the framework of recurrent neural networks. Theoretical analysis including convergence and stability of the proposed method is presented. In order to verify the effectiveness of the theoretical results and the portability of the proposed method, simulations are conducted on different robot manipulators for different tracking tasks with excellent performance. In addition, comparisons with control schemes employing traditional gradient neural network (GNN), Kalman filter (KF) and model-based recurrent neural network highlight the great advantages of the proposed control system.

Abstract We present an improved image caption generation model that incorporating multimodal attention mechanism. We use ResNet‐101 to extract image features while incorporating channel attention mechanism and spatial attention mechanism. We use Faster R‐CNN for object detection and use a multi‐head attention structure consisting of spatial attention and self‐attention. This allows our algorithm to improve the model's capability to learn and use the internal grammatical features of natural sentences. Moreover, we use GPU parallel computing to accelerate the entire model training. We apply our model and algorithm to early education scenarios: show and tell for kids. We compare our algorithm with the state‐of‐the‐art deep learning algorithms. Our experimental results show that our model improves the captioning accuracy in terms of standard automatic evaluation metrics.

For large-scale tasks, coverage path planning (CPP) can benefit greatly from multiple robots. In this paper, we present an efficient algorithm MSTC∗for multi-robot coverage path planning (mCPP) based on spiral spanning tree coverage (Spiral-STC). Our algorithm incorporates strict physical constraints like terrain traversability and material load capacity. We compare our algorithm against the state-of-the-art in mCPP for regular grid maps and real field terrains in simulation environments. The experimental results show that our method significantly outperforms existing spiral-STC based mCPP methods. Our algorithm can find a set of well-balanced workload distributions for all robots and therefore, achieve the overall minimum time to complete the coverage.

We present a new approach to transfer grasp configurations from prior example objects to novel objects. We assume the novel and example objects have the same topology and similar shapes. We perform 3D segmentation on these objects using geometric and semantic shape characteristics. We compute a grasp space for each part of the example object using active learning. We build bijective contact mapping between these model parts and compute the corresponding grasps for novel objects. Finally, we assemble the individual parts and use local replanning to adjust grasp configurations while maintaining its stability and physical constraints. Our approach is general, can handle all kind of objects represented using mesh or point cloud and a variety of robotic hands.

We present a realtime virtual grasping algorithm to model interactions with virtual objects. Our approach is designed for multi-fingered hands and makes no assumptions about the motion of the user's hand or the virtual objects. Given a model of the virtual hand, we use machine learning and particle swarm optimization to automatically pre-compute stable grasp configurations for that object. The learning pre-computation step is accelerated using GPU parallelization. At runtime, we rely on the pre-computed stable grasp configurations, and dynamics/non-penetration constraints along with motion planning techniques to compute plausible looking grasps. In practice, our realtime algorithm can perform virtual grasping operations in less than 20ms for complex virtual objects, including high genus objects with holes. We have integrated our grasping algorithm with Oculus Rift HMD and Leap Motion controller and evaluated its performance for different tasks corresponding to grabbing virtual objects and placing them at arbitrary locations. Our user evaluation suggests that our virtual grasping algorithm can increase the user's realism and participation in these tasks and offers considerable benefits over prior interaction algorithms, such as pinch grasping and raycast picking.

We present a new parallel algorithm for collision detection using many-core computing platforms of CPUs or GPUs. Based on the notion of a $(p)$-partition front, our algorithm is able to evenly partition and distribute the workload of BVH traversal among multiple processing cores without the need for dynamic balancing, while minimizing the memory overhead inherent to the state-of-the-art parallel collision detection algorithms. We demonstrate the scalability of our algorithm on different benchmarking scenarios with and without using temporal coherence, including dynamic simulation of rigid bodies, cloth simulation, and random collision courses. In these experiments, we observe nearly linear performance improvement in terms of the number of processing cores on the CPUs and GPUs.

We present a new method for efficiently approximating the global penetration depth between two rigid objects using machine learning techniques. Our approach consists of two phases: offline learning and performing run-time queries. In the learning phase, we precompute an approximation of the contact space of a pair of intersecting objects from a set of samples in the configuration space. We use active and incremental learning algorithms to accelerate the precomputation and improve the accuracy. During the run-time phase, our algorithm performs a nearest-neighbor query based on translational or rotational distance metrics. The run-time query has a small overhead and computes an approximation to global penetration depth in a few milliseconds. We use our algorithm for collision response computations in Box2D or Bullet game physics engines and complex 3D models and observe more than an order of magnitude improvement over prior PD computation techniques.

We present a new bounding volume structure, k-IOS that is an intersection of k spheres, for accelerating proximity query including collision detection and Euclidean distance computation between arbitrary polygon-soup models that undergo rigid motion. Our new bounding volume is easy to implement and highly efficient both for its construction and runtime query. In our experiments, we have observed up to 4.0 times performance improvement of proximity query compared to an existing well-known algorithm based on swept sphere volume (SSV) [1]. Moreover, k-IOS is strictly convex that can guarantee a continuous gradient of distance function with respect to object's configuration parameter.

We present a simple and efficient approach for continuous collision detection of deforming triangles based on conservative advancement. The efficiency of our approach is due to a sequence of simple collision-free conditions for deforming triangles. In our experiment, we show that our CCD algorithm achieves 2-30 times performance improvement over existing algorithms for triangle primitives.

We present a simple algorithm to generate a topology-preserving, error-bounded approximation of the outer boundary of the volume swept by a polyhedron along a parametric trajectory. Our approach uses a volumetric method that generates an adaptive volumetric grid, computes signed distance on the grid points, and extracts an isosurface from the distance field. In order to guarantee geometric and topological bounds, we present a novel sampling and front propagation algorithm for adaptive grid generation. We highlight the performance of our algorithm on many complex benchmarks that arise in geometric and solid modeling, motion planning and CNC milling applications. To the best of our knowledge, this is the first practical algorithm that can generate swept volume approximations with geometric and topological guarantees on complex polyhedral models swept along any parametric trajectory.

We present a fast continuous collision detection (CCD) algorithm for articulated models using Taylor models and temporal culling. Our algorithm is a generalization of conservative advancement (CA) from convex models [Mirtich 1996] to articulated models with non-convex links. Given the initial and final configurations of a moving articulated model, our algorithm creates a continuous motion with constant translational and rotational velocities for each link, and checks for interferences between the articulated model under continuous motion and other models in the environment and for self-collisions. If collisions occur, our algorithm reports the first time of contact (TOC) as well as collision witness features . We have implemented our CCD algorithm and applied it to several challenging scenarios including locomotion generation, articulated-body dynamics and character motion planning. Our algorithm can perform CCDs including self-collision detection for articulated models consisting of many links and tens of thousands of triangles in 1.22 ms on average running on a 3.6 GHz Pentium 4 PC. This is an improvement on the performance of prior algorithms of more than an order of magnitude.

We present an interactive and accurate collision detection algorithm for deformable, polygonal objects based on the streaming computational model. Our algorithm can detect all possible pairwise primitive-level intersections between two severely deforming models at highly interactive rates. In our streaming computational model, we consider a set of axis aligned bounding boxes (AABBs) that bound each of the given deformable objects as an input stream and perform massively-parallel pairwise, overlapping tests onto the incoming streams. As a result, we are able to prevent performance stalls in the streaming pipeline that can be caused by expensive indexing mechanism required by bounding volume hierarchy-based streaming algorithms. At runtime, as the underlying models deform over time, we employ a novel, streaming algorithm to update the geometric changes in the AABB streams. Moreover, in order to get only the computed result (i.e., collision results between AABBs) without reading back the entire output streams, we propose a streaming en/decoding strategy that can be performed in a hierarchical fashion. After determining overlapped AABBs, we perform a primitive-level (e.g., triangle) intersection checking on a serial computational model such as CPUs. We implemented the entire pipeline of our algorithm using off-the-shelf graphics processors (GPUs), such as nVIDIA GeForce 7800 GTX, for streaming computations, and Intel Dual Core 3.4G processors for serial computations. We benchmarked our algorithm with different models of varying complexities, ranging from 15K up to 50K triangles, under various deformation motions, and the timings were obtained as 30 approximately 100 FPS depending on the complexity of models and their relative configurations. Finally, we made comparisons with a well-known GPU-based collision detection algorithm, CULLIDE [4] and observed about three times performance improvement over the earlier approach. We also made comparisons with a SW-based AABB culling algorithm [2] and observed about two times improvement.

This paper presents an effective approach to generate a set of small textures from an input texture that can be tiled together to synthesize large textures. Such a small set can be useful in texturing any large area realistically and efficiently while consuming only a small amount of texture memory. Our approach is advantageous in its ability to generate a smaller number of tiles that can embed much more texture patterns and with less conspicuous seams within each tile than earlier approaches. As a result, our approach can generate large textures that look as if each were from a continuous part of the input texture while avoiding highly repetitive patterns. In general, our approach performs very well and shows a particular strength, compared to earlier approaches, for input textures of elaborate or relatively large features, or with distinctive colors.