Automating surgical systems enhances precision and safety while reducing human involvement in high-risk environments. A major challenge in automating surgical procedures like suturing is accurately modeling the suture thread, a highly flexible and compliant component. Existing models either lack the accuracy needed for safety-critical procedures or are too computationally intensive for real-time execution. In this work, we introduce a novel approach for modeling suture thread dynamics using control barrier functions (CBFs), achieving both realism and computational efficiency. Thread-like behavior, collision avoidance, stiffness, and damping are all modeled within a unified CBF and control Lyapunov function (CLFs) framework. Our approach eliminates the need to calculate complex forces or solve differential equations, significantly reducing computational overhead while maintaining a realistic model suitable for both automation and virtual reality surgical training systems. The framework also allows visual cues to be provided based on the thread's interaction with the environment, enhancing user experience when performing suture or ligation tasks. The proposed model is tested on the MagnetoSuture system, a minimally invasive robotic surgical platform that uses magnetic fields to manipulate suture needles, offering a less invasive solution for surgical procedures.

The deployment and coordination of mobile sensor networks for coverage control applications can present several practical challenges, including how to efficiently share limited communication resources and how to reduce the use of localization devices (e.g., radars and lidars). One potential solution to these challenges is to reduce the frequency at which agents communicate or sample each other's position. In this article, we present a distributed asynchronous self-triggered control policy for centroidal Voronoi coverage control that is shown to decrease the sampling or communication instants among agents without degrading the performance of the mobile sensor network. Each agent independently decides when to sample the position of nearby agents and uses outdated information of its neighbors until new information is required. We prove that the locational cost function describing the distribution of agents monotonically decreases everywhere outside of a bounded neighborhood around the group's optimal configuration and that the agents asymptotically converge to their Voronoi centroids if the data-sampled centroid errors approach zero. In addition, we show that the sampling intervals are always positive and lower bounded and, as illustrated by simulations and experiments, they tend to stabilize at a large value as the mobile sensor network comes to a steady state. Simulations and experiments with ground vehicles validate the control strategy and show that the proposed policy can achieve similar level of performance as a continuous or fast periodic implementation.

Human-Swarm Interaction (HSI) is a fast-growing research area in swarm robotics. One challenging aspect of HSI is facilitating effective handling of the many degrees-of-freedom present in robot swarms by humans. One emergent option is the use of Augmented Reality (AR) systems to encode information. AR based interfaces can help provide human operators with visual cues about the swarm's states and control to facilitate decision-making. In research settings, AR systems can address issues such as limited availability of lab spaces, limited access to robotics resources, and the need for the ability to simulate dynamic environments with which robots and humans can interact. Further, to make swarm robotics more accessible and ubiquitous, HSI systems that support remote interaction would allow humans to interact with robot swarms and multi-robot systems regardless of the geographical distance between humans and swarms. Taking these into consideration, we aim to investigate the effectiveness of AR based interfaces as tools for remote interaction in HSI systems. We developed a simple AR based interface and evaluated its effectiveness against an unaugmented interface, by means of remote human user studies where a human operator would control a team of robots remotely through a video call. Our finding suggests that augmentation can improve control accuracy and reduce collision safety violations when performing navigation tasks. Through experimental surveys, it is shown that operators with varying levels of robotics and technology experience overwhelmingly prefer the augmented interface to facilitate swarm control. These results suggest that AR-based interfaces are effective in improving the control experience in remote HSI.

This paper presents a distributed interactive framework to provide high-level position instructions for multi-robot aerial cinematography based on coverage over a hemisphere. The control strategy based on optimization of the coverage functional and geometric relationships over a hemisphere is presented. It enables multiple Unmanned Aerial Vehicles (UAVs) to coordinate their motion while tracking a dynamic (real or virtual) target, and can accommodate high-level human inputs to influence UAV concentration. In this framework, each UAV uses local information combined with exogenous inputs to determine its motion. The two inputs to the system, i.e., the predicted trajectory of the target and user-defined aesthetic preferences, are agnostic to the size of the multi-robot system (MRS). The proposed framework is validated using the PX4 SITL Autopilot simulator in Gazebo, and the scalability of the framework is verified via simulations.

Exerting forces and torques instantaneously on rigid magnetic bodies with no physical connection is an attractive feature of magnetic robotics. This demonstrates great potential for manipulating tools that are externally controlled through the use of magnetic fields in minimally invasive surgeries. The magnetic field can be controlled by the application of currents to electromagnets positioned around the surgical site, and the necessary currents for a specific desired manipulation can be derived from magnetic field models. However, the magnetic field generated by electromagnetic coils are highly nonlinear, especially in the vicinity of the magnetic field sources, which complicates the modeling process. While simple dipole models provide a good approximation for these fields far away from the electromagnets, these models tend to be highly inaccurate near the sources. Magnetic surgical applications benefit from models which accurately describe fields and gradients both near and far from the field source. Particularly, since forces and torques decay inversely proportionally with the cube of the distance to the coil, inaccurate modeling near the coil makes large regions near the coil unfit for applications requiring precisely predicted motion. Estimation errors near coils generate inaccuracies in field models that significantly reduce control performance for rigid magnetic bodies. In order to tackle this problem, we utilize Zernike basis functions to analytically represent the nonlinear magnetic field distribution more accurately. The accuracy of the controller is tested experimentally by driving a magnetic surgical suture needle with a length of 22 mm in the MagnetoSuture™ system along a lemniscate trajectory. The magnetic needle's tip position and the needle orientation, autonomously controlled by the proposed controller, shows RMS tracking error of 2.35 mm using typical dipole models and 1.71 mm for the Zernike fitting approach, a 27% improvement in tracking error. This suggests that the use of Zernike basis functions to capture the nonlinearities of the magnetic field may assist in implementing fast and precise autonomous control strategies for magnetic suture needles.

Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effects. However, the nonlinear magnetic fields produce unintuitive forces and demand delicate position-based control that exceeds the capabilities of direct human manipulation. This makes automatic needle localization a necessity. Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0.73 mm RMS error in clean environments and 2.72 mm RMS error in challenging environments with blood and occlusion. The average localization RMS error is 2.16 mm for all environments we used in the experiments. We combine this localization method with our closed-loop feedback control system to demonstrate the further applicability of localization to autonomous control. Our needle is able to follow a running suture path in (1) no blood, no tissue; (2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood, with tissue environments. The tip position tracking error ranges from 2.6 mm to 3.7 mm RMS, opening the door towards autonomous suturing tasks.

This paper addresses the problem of a domain becoming non-convex while using coverage control of a multirobot system over time-varying domains. When the domain moves around in the workspace, its motion and the presence of obstacles might cause it to deform into some non-convex shape, and the robot team should act in a coordinating manner to maintain coverage. The proposed solution is based on a framework for constructing a diffeomorphism to transform a non-convex coverage problem into a convex one. A control law is developed to capture the effects of time variations (e.g., from a time-varying density, time-varying convex hull of the domain and time-varying diffeomorphism) in the system. Analytic expressions of each term in the control law are found for uniform density case. A simulation and robotic implementation are used to validate the proposed algorithm.

This paper proposes a magnetic needle steering controller to manipulate mesoscale magnetic suture needles for executing planned suturing motion. This is an initial step towards our research objective: enabling autonomous control of magnetic suture needles for suturing tasks in minimally invasive surgery. To demonstrate the feasibility of accurate motion control, we employ a cardinally-arranged four-coil electromagnetic system setup and control magnetic suture needles in a 2-dimensional environment, i.e., a Petri dish filled with viscous liquid. Different from only using magnetic field gradients to control small magnetic agents under high damping conditions, the dynamics of a magnetic suture needle are investigated and encoded in the controller. Based on mathematical formulations of magnetic force and torque applied on the needle, we develop a kinematically constrained dynamic model that controls the needle to rotate and only translate along its central axis for mimicking the behavior of surgical sutures. A current controller of the electromagnetic system combining with closed-loop control schemes is designed for commanding the magnetic suture needles to achieve desired linear and angular velocities. To evaluate control performance of magnetic suture needles, we conduct experiments including needle rotation control, needle position control by using discretized trajectories, and velocity control by using a time-varying circular trajectory. The experiment results demonstrate our proposed needle steering controller can perform accurate motion control of mesoscale magnetic suture needles.

This paper presents a framework for multirobot motion planning that characterizes pairwise interactions between agents, e.g., crossing paths while en route to a destination. Mixing is identified as the number of pairwise crossings exhibited by the robot motion. Mixing patterns specified through elements of the braid group provide sufficient level of abstraction to describe interactions without concern for the geometry of the motion. Controllers are constructed explicitly reasoning about the spatial collocation of robots to execute mixing patterns, achieving rich motion in a shared space, e.g., to exchange inter-robot information. We do not focus on achieving a particular pattern, but rather on the problem of being able to execute a whole class of them (e.g., all patterns with at most $M$ pairwise interactions). The result is a hybrid system driven by symbolic inputs that are mapped onto paths, realizing desired mixing levels. Controllers derived from optimal control provide theoretical bounds on the achievable amount of mixing, satisfaction of spatio-temporal constraints, and collision-free trajectories. Designs are carried to implementation on real robot platforms.

An approach is presented for influencing teams of robots by means of time-varying density functions, representing rough references for where the robots should be located. A continuous-time coverage algorithm is proposed and distributed approximations are given whereby the robots only need to access information from adjacent robots. Robotic experiments show that the proposed algorithms work in practice, as well as in theory.

This paper investigates the problem of finding shortest paths through 3-dimensional cluttered environments. In particular, an algorithm is presented that determines the shortest path between two points in an environment with obstacles which can be implemented on robots with capabilities of detecting obstacles in the environment. As knowledge of the environment is increasing while the vehicle moves around, the algorithm provides not only the global minimizer - or shortest path - with increasing probability as time goes by, but also provides a series of local minimizers. The feasibility of the algorithm is demonstrated on a quadrotor robot flying in an environment with obstacles.