Few-shot knowledge distillation recently emerged as a viable approach to harness the knowledge of large-scale pre-trained models, using limited data and computational resources. In this paper, we propose a novel few-shot feature distillation approach for vision transformers. Our approach is based on two key steps. Leveraging the fact that vision transformers have a consistent depth-wise structure, we first copy the weights from intermittent layers of existing pre-trained vision transformers (teachers) into shallower architectures (students), where the intermittence factor controls the complexity of the student transformer with respect to its teacher. Next, we employ an enhanced version of Low-Rank Adaptation (LoRA) to distill knowledge into the student in a few-shot scenario, aiming to recover the information processing carried out by the skipped teacher layers. We present comprehensive experiments with supervised and self-supervised transformers as teachers, on six data sets from various domains (natural, medical and satellite images) and tasks (classification and segmentation). The empirical results confirm the superiority of our approach over state-of-the-art competitors. Moreover, the ablation results demonstrate the usefulness of each component of the proposed pipeline. We release our code at https://github.com/dianagrigore/WeCoLoRA.

This article develops and experimentally tests a supervisory risk controller used to increase the safety of drone operations. Its task is to monitor the state of the drone and environment and to use this information to automatically change safety-critical parameters in real-time during operation. The main contribution of this article is to demonstrate how methods from the risk sciences can be combined with methods from the field of artificial intelligence to design risk awareness into automated systems. More precisely a system theoretic process analysis (STPA) is performed to identify how the system can fail. The results of the STPA are used to build a Dynamic Decision Network (DDN), which is used as an online risk model. An optimization algorithm then uses the online risk model to find the optimal parameter configuration that ensures an acceptable risk level. A case study of a tethered industrial inspection drone is considered. Through experimental trials, it is demonstrated how the supervisory risk controller is able to identify the state of the drone and the environment by combining information from multiple measurements over time and how it uses this information to modify maximum speed, safety distance, and maximum vertical acceleration such that the risk level remains acceptable. When no parameter set can ensure an acceptable risk level then a recommendation of aborting the mission is sent to the human operator.

Traditional large Earth observation satellites (EOSs) are designed with simple imaging strategies. However, small satellites can satisfy immediate user needs through agile imaging modes. In this article, we model several agile imaging mode concepts and test them experimentally on the push-broom hyperspectral imaging cubesat HYPerspectral Smallsat for Ocean observation satellite 1 (HYPSO-1). Six imaging modes are tested: slewing, multitarget, dual-angle, wide swath, on-board processing, and dynamic pointing. The results show that the slewing and multiangle mode increased estimated signal-to-noise ratio (SNR) by$1.4\times $to$1.7\times $, the on-board processing mode decreased data latency, and the multitarget, wide swath, and dynamic pointing modes enhanced spatial coverage.

Attitude control of fixed-wing unmanned aerial vehicles (UAVs) is a difficult control problem in part due to uncertain nonlinear dynamics, actuator constraints, and coupled longitudinal and lateral motions. Current state-of-the-art autopilots are based on linear control and are thus limited in their effectiveness and performance. Gls drl is a machine learning method to automatically discover optimal control laws through interaction with the controlled system that can handle complex nonlinear dynamics. We show in this article that deep reinforcement learning (DRL) can successfully learn to perform attitude control of a fixed-wing UAV operating directly on the original nonlinear dynamics, requiring as little as 3 min of flight data. We initially train our model in a simulation environment and then deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller with no further online learning required. Learning with significant actuation delay and diversified simulated dynamics were found to be crucial for successful transfer to control of the real UAV. In addition to a qualitative comparison with the ArduPlane autopilot, we present a quantitative assessment based on linear analysis to better understand the learning controller's behavior.

Spectroscopic sensors provide online information about the composition and concentration of species in a sample by analyzing the interaction of light and matter. At the industrial scale, external variables such as temperature, pressure, and particle size distribution affect spectroscopic measurements. Thus, conventional quantitative analytical methods that do not consider these external factors provide poor estimates. Their effects have to be compensated through proper modeling and processing to improve the concentration estimation. This work presents an integrated discrete-time model considering the process dynamic and a physics-based sensor model. Then, we suggest a novel application of an adaptive Kalman filter to provide concentration estimates by correcting external factor effects. The convergence of the Kalman filter requires the fulfillment of uniform observability (persistent excitation) conditions for both inputs and external signals. Simulation results illustrate the modeling methodology and the main characteristics of the proposed Kalman filter approach for performing online correction of the spectroscopic sensor signals. The results show that the proposed adaptive Kalman filter can estimate concentrations with small error under temperature variations and measurement noise.

This paper presents a low-cost lightweight UAV payload for remote sensing purposes using a small imaging spectrometer made from COTS components. The novel design with an additional upward-facing spectrometer allows for reflectance computation by correcting the push-broom imaging spectrometer data of the ocean surface for different lighting- and atmospheric conditions.

The along track spatial resolution in push broom imaging systems depends on platform parameters and imaging settings. This paper present a method for achieving a consistent spatial resolution for varying exposure times. We consider the geometry of a simplified push broom imaging system to derive relationships between spatial resolution, platform parameters (altitude, speed, focal length, slit width) and recording settings (exposure time, frame rate). The method is tested and verified using data from the HYPerspectral Smallsat for Ocean observation-1 (HYPSO-1). The spatial resolution is consistent over the exposure time range of 4.41ms to 49.3ms for low off-nadir angles when choosing frame rates according to the method.

Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, MPC has some significant challenges for such systems, such as its high computational complexity. Further, the MPC parameters must be tuned, which is largely a trial-and-error process that affects the control performance, the robustness, and the computational complexity of the controller to a high degree. This paper presents a multivariate optimization method based on reinforcement learning (RL) that automatically tunes the control algorithm’s parameters from data to achieve optimal closed-loop performance. The main contribution of our method is the inclusion of state-dependent optimization of the meta-parameters of MPC, i.e. parameters that are non-differentiable wrt. the MPC solution. Our control algorithm is based on an event-triggered MPC, where we learn when the MPC should be re-computed, and a dual-mode MPC and linear state feedback control law applied in between MPC computations. We formulate a novel mixture-distribution RL policy determining the meta-parameters of our control algorithm and show that with joint optimization we achieve improvements that do not present themselves with univariate optimization of the same parameters. We demonstrate our framework on the inverted pendulum control task, reducing the total computation time of the control system by 36% while also improving the control performance by 18.4%.

The method described in this paper proposes a supervised Deep Neural Network (DNN) approach for the prediction of anomalies in camera-based navigation. The method is inspired by the unsolved issues of Integrity Monitors (IMs) when some of the sensor measurement covariances are unknown or inconsistent. Especially, the focus is on predicting when the estimation error distribution would require fatter tails to include outliers. The developed method takes into account single-frame image features as well as transient changes in the error. In the best of our knowledge, this is the first work that predicts anomalies in the error covariance of Simultaneous Navigation and Mapping (SLAM) estimates and associates them with low-level image features. Finally, the prediction method can be used with other sensors as well, allowing the future development of navigation algorithm- and sensor-agnostic safety monitoring frameworks.

HYPSO-1 is a Small Satellite with a hyperspectral camera payload inside a six unit cubesat platform, launched january 2022. This paper describes how the operation of the same hyperspectral camera is validated by deployment on an experimental airplane, and how field trials are performed together with other remote sensing and in-situ agents. The payload is presented, in addition to how the operation was planned. The raw and radiance data are presented, and various practical aspects regarding implementation are discussed. Notwith-standing some minor issues, the procedures described were successfully used to gather valid data that can be used to infer properties of the imaged regions.

The assignment problem is a fundamental optimization problem and a crucial part of many systems. For example, in multiple object tracking, the assignment problem is used to associate object detections with hypothetical target tracks and solving the assignment problem is one of the most compute-intensive tasks. To enable low-latency real-time implementations, efficient solutions to the assignment problem is required. In this work, we present Sparse and Speculative (SaS) Auction, a novel implementation of the popular Auction algorithm for FPGAs. Two novel optimizations are proposed. First, the pipeline width and depth are reduced by exploiting sparsity in the input problems. Second, dependency speculation is employed to enable a fully pipelined design and increase the throughput. Speedups as high as 50 × are achieved relative to the state-of-the-art implementation for some input distributions. We evaluate the implementation both on randomly generated datasets and realistic datasets from multiple object tracking.

Sporadic ocean color events with characteristic spectra, in particular algal blooms, call for quick delivery of high-resolution remote sensing data for further analysis. Motivated by this, we present the mission design for HYPerspectral Smallsat for Ocean observation (HYPSO-1), a 6U CubeSat at 500 km orbital altitude hosting a custom-built pushbroom hyperspectral imager with wavelengths 387–801 nm at 3.33 nm bandpass and a swath width of 70 km. The imager’s expected signal-to-noise ratio is characterized for typical open ocean water-leaving radiance which can be flexibly increased by binning pixels. Using geometric principles, the satellite shall execute a slew maneuver during a scan to induce greater overlap in the pixels with a goal to enable better than 100 m spatial resolution. Since high-dimensional hyperspectral data need to be transmitted over limited space-to-ground communications, we have designed a modular FPGA-based onboard image processing architecture that significantly reduces the data size without losing important spatial-spectral information. We justify the concept with a simulated scenario where HYPSO-1 first collects numerous hyperspectral images of a 40 km by 40 km coastal area in Norway and aims to immediately transfer these to nearby ground stations. Using CCSDS123 lossless compression, it takes about one orbital revolution to obtain the complete data product when considering overhead in satellite bus communications and less than 10 min without the overhead. It is shown that even better latency can be achieved with more advanced onboard processing algorithms.

In this article, a new approach for ship-ship collision probability estimation based on the Cross-Entropy (CE) method is introduced, which can be treated as an adaptive importance sampler. It has the advantage of attaining low variance estimates of small collision probabilities, which will most often be the case in realistic scenarios. Furthermore, a risk-based Collision Avoidance (COLAV) system being able to take obstacle kinematic uncertainty and intention uncertainty into account is presented, namely the Probabilistic Scenario-Based Model Predictive Control (PSB-MPC). The collision probability estimator (CPE) is used in the risk assessment of the PSB-MPC, and tested in a simulation study, where the total system is validated. Simulation results show that the MPC is able to utilize the CPE for better risk assessment than the original version, in order to make safer decisions in close quarter situations and cases where nearby obstacles make unexpected maneuvers. It is also shown that when all vessels involved use the PSB-MPC, situations are resolved according to the traffic rules in a safe manner.

A robust and efficient Collision Avoidance (COLAV) system for autonomous ships is dependent on a high degree of situational awareness. This includes inference of the intent of nearby obstacles, including compliance with traffic rules such as COLREGS, in order to enable more intelligent decision making for the autonomous agent. Here, a generalized framework for obstacle intent inference is introduced. Different obstacle intentions are then considered in the Probabilistic Scenario-Based Model Predictive Control (PSB-MPC) COLAV algorithm using an examplatory intent model, when statistics about traffic rules compliance and the next waypoint for an obstacle are assumed known. Simulation results show that the resulting COLAV system is able to make safer decisions when utilizing the extra intent information.

Harmful algal blooms occur frequently and deteriorate water quality. A reliable method is proposed in this paper to track algal blooms using a set of autonomous surface robots. A satellite image indicates the existence and initial location of the algal bloom for the deployment of the robot system. The algal bloom area is approximated by a circle with time varying location and size. This circle is estimated and circumnavigated by the robots which are able to locally sense its boundary. A multi-agent control algorithm is proposed for the continuous monitoring of the dynamic evolution of the algal bloom. Such algorithm comprises of a decentralised least squares estimation of the target and a controller for circumnavigation. We prove the convergence of the robots to the circle and in equally spaced positions around it. Simulation results with data provided by the SINMOD ocean model are used to illustrate the theoretical results.

This paper presents a viable approach for incorporating collision avoidance strategies into existing guidance and control systems on marine vessels. We propose a method that facilitates the use of simulation-based Model Predictive Control (MPC) for collision avoidance (COLAV) on marine vessels. Any COLAV strategy to be applied in real traffic must adhere to the international regulations for preventing collisions at sea (COLREGS). The proposed MPC COLAV method does not rely on an accurate model of the guidance system to achieve vessel behaviors that are compliant with the COLREGS. Rather, it depends on transitional costs in the MPC objective for collision avoidance maneuvers that are being executed by the marine vessel. Hence, it is straightforward to implement the MPC COLAV on different vessels without specific knowledge of the vessel's guidance strategy. Moreover, it offers the possibility to switch between different (possibly application specific) guidance strategies on the same vessel while running the same MPC COLAV algorithm. We present results from full scale experiments that show the viability of our method in different collision avoidance scenarios.

In recent years, the use of Unmanned Aerial Systems (UAS) has become commonplace in a wide variety of tasks due to their relatively low cost and ease of operation. In this paper, we explore the use of UAS in maritime Search And Rescue (SAR) missions by using experimental data to detect and classify objects at the sea surface. The objects are chosen as common objects present in maritime SAR missions: a boat, a pallet, a human, and a buoy. The data consists of thermal images and a Gaussian Mixture Model (GMM) is used to discriminate foreground objects from the background. Then, bounding boxes containing the object are defined and used to train a Convolutional Neural Network (CNN). The CNN achieves the average accuracy of 92.5% when evaluating a testing dataset.

This paper addresses the source localization problem of an acoustic fish-tag using the Time-of-Arrival measurement of an acoustic signal, transmitted by the fish-tag. The Time-of-Arrival measurements denote the pseudo-range information between the acoustic receiver and the fish-tag, except that the Time-of-Transmission of the acoustic signal is unknown. Starting with the pseudo-range measurement equation, a globally valid quasi-linear time-varying measurement model is presented that is independent of the Time-of-Transmission of the acoustic signal. Using this measurement model, an Uniformly Globally Asymptotically Stable (UGAS), three stage estimation strategy (eXogenous Kalman Filter) is designed to estimate the position of an acoustic fish-tag and evaluated against a benchmark Extended Kalman Filter based estimator. The efficacy of the developed estimation method is demonstrated experimentally, in presence of intermittent observations using an array of receivers mounted on three Unmanned Surface Vessels (USVs).

We propose a collision avoidance method that incorporates the interactive behavior of agents and is proactive in dealing with the uncertainty of the future behavior of obstacles. The proposed method considers interactions that will be experienced by an autonomous surface vessel (ASV) in an environment governed by the international regulations for preventing collisions at sea (COLREGs). Our approach aims at encouraging dynamic obstacles to cooperate according to COLREGs. Therefore, we propose a strategy for assessing the cooperative behavior of obstacles, and the result of the assessment is used to adapt collision avoidance decisions within the Reciprocal Velocity Obstacles (RVO) framework. Moreover, we propose a predictive approach to solving known limitations of the RVO framework, and we present computationally feasible extensions that enable the use of complex dynamic models and objectives suitable for ASVs. We demonstrate the performance and potentials of our method through a simulation study, and the results show that the proposed method leads to proactive and more predictable ASV behavior compared with both Velocity Obstacles (VO) and RVO, especially when obstacles cooperate by following COLREGs.

The Simultaneous Localization And Mapping (SLAM) estimation problem is a nonlinear problem, due to the nature of the range and bearing measurements. In latter years it has been demonstrated that if the nonlinearities from the attitude are handled by a separate nonlinear observer, the SLAM dynamics can be represented as a linear time varying (LTV) system, by introducing these nonlinearities and nonlinear measurements as time varying vectors and matrices. This makes the SLAM estimation problem globally solvable with a Kalman filter, however, the noise structure is no longer trivial. In this paper, a new bearings only SLAM estimation algorithm is presented, including a novel design of the noise covariance matrices. Simulations of the SLAM estimator are presented, and show the performance of the state and uncertainty estimates, as well as the stability of the proposed estimator.

Multi-vehicle operations using various types of unmanned vehicles (UVs) can increase efficiency of marine data acquisition, reduce the crew risk and lower mission costs. These types of missions are very complex and often involve systems that are not interoperable. From an operational perspective however, some level of integration is necessary. Typically, a common network system architecture and Situation Awareness (SA) platform are required. The architecture allows operators to transfer data between vehicles and their operators, while the SA platform allows to monitor mission progress and react to changes. This paper presents a network system architecture used during an experiment realized in Spring 2016 in Norway. 8 departments from 5 institutions worked together to combine operation of 4 UVs (aerial, surface, underwater), a support vessel and on-shore team. The description is followed by a backbone network performance analysis. Several cases are presented, with focus on a transmission between manned vessel and Unmanned Surface Vehicle (USV), including direct connection, and data-relay mechanism via Unmanned Aerial Vehicle (UAV).

This paper describes a concept for a collision avoidance system for ships, which is based on model predictive control. A finite set of alternative control behaviors are generated by varying two parameters: offsets to the guidance course angle commanded to the autopilot and changes to the propulsion command ranging from nominal speed to full reverse. Using simulated predictions of the trajectories of the obstacles and ship, compliance with the Convention on the International Regulations for Preventing Collisions at Sea and collision hazards associated with each of the alternative control behaviors are evaluated on a finite prediction horizon, and the optimal control behavior is selected. Robustness to sensing error, predicted obstacle behavior, and environmental conditions can be ensured by evaluating multiple scenarios for each control behavior. The method is conceptually and computationally simple and yet quite versatile as it can account for the dynamics of the ship, the dynamics of the steering and propulsion system, forces due to wind and ocean current, and any number of obstacles. Simulations show that the method is effective and can manage complex scenarios with multiple dynamic obstacles and uncertainty associated with sensors and predictions.

Harmonic voltages and currents in electrical systems, when present to a certain degree, represent not only a\npower quality problem but they are also strongly associated with the electrical system overall losses and they\nare arguably a source of instability and a safety concern. Mitigating harmonics distortion across the entire\nsystem by actively reducing harmonic currents propagation is an effective way of coping with these issues and\ncan be dealt with the injection of a compensating current waveform with an active filter installed at a given\nbus. This paper shows how, by shaping the compensating current waveform in an optimal way, the overall\nelectrical system harmonic distortion can be optimally reduced in a cost effective manner with a minimum\nsize of the compensating device. The process of shaping this optimal compensating current is shown by how\nits components are defined by the optimization algorithm using the phase and amplitude of each harmonic as\ndegrees of freedom in the process of finding the optimal waveform. A marine vessel distribution grid is used\nas representative example to prove the concept.

Examines the development of marine vehicle power systems from an historical perspective. from the earliest forms of power systems to the current development of all electric ship technologies.

Obtaining synoptic observations of dynamic ocean phenomena such as fronts, eddies, oxygen minimum zones and blooms has been challenging primarily due to the large spatial scales involved. Traditional methods of observation with manned ships are expensive and, unless the vessel can survey at high-speed, unrealistic. Autonomous underwater vehicles (AUVs) are robotic platforms that have been making steady gains in sampling capabilities and impacting oceanographic observations especially in coastal areas. However, their reach is still limited by operating constraints related to their energy sources. Unmanned aerial vehicles (UAVs) recently introduced in coastal and polar oceanographic experiments have added to the mix in observation strategy and methods. They offer a tantalizing opportunity to bridge such scales in operational oceanography by coordinating with AUVs in the water-column to get in-situ measurements. In this paper, we articulate the principal challenges in operating UAVs with AUVs making synoptic observations for such targeted water-column sampling. We do so in the context of autonomous control and operation for networked robotics and describe novel experiments while articulating the key challenges and lessons learned.

A thrust allocation method with a functionality to assist power management systems by using the hull of the ship as a store of potential energy in the field of environmental forces has been recently proposed and demonstrated to work in simulation. This functionality allows the thrust allocation algorithm to decrease the power consumption in the thrusters when a sharp increase in power consumption is demanded elsewhere on the ship. This way, the high-frequency part of the load variations on the power plant can be reduced, at the expense of minor (typically less than 1 meter) variations in the position of the vessel. The advantages from reduced variations in load include reduced wear-and-tear of the power plant, more stable frequency on the electric grid, reduced risk of blackout due to underfrequency, and more reliable synchronization when connecting additional generators or connecting bus segments. In the present work, this functionality is improved further by continuously monitoring the environmental forces and modifying the setpoint of the dynamic positioning algorithm to place the vessel a short distance (e.g. 20 cm) in the direction of steepest increase of the environmental force potential, thus maximizing the available potential energy. The increased potential energy creates additional capacity for assisting the power plant, which is shown in simulation to be significant.

Adaptive pattern-based control strategies adapt their parameters from an analysis of response patterns exhibited by the system. This work presents an analysis of a class of artificial neural network (ANN) pattern-based adaptive control. It provides conditions under which the adaptive algorithm will converge, and it also characterizes the closed-loop stability properties. In addition, a method for monitoring the adaptation is also proposed. Several simulation examples illustrate our findings.

A preliminary design of a remote palpation instrument for minimally invasive surgery (MIS) is given. The lack of the tactile sense in MIS limits the surgeon's abilities to examine and palpate internal organs. Based on this problem, the aim of this paper is to describe a surgical instrument which can serve as an extension of the surgeon's fingers. A piezoelectric sensor array attached to the instrument's end effector provide tactile information, which is sent to the surgeon's fingers via a tactile display to provide a feeling of the shape and hardness of the tissue. The sensor array is 24 mm /spl times/ 8 mm and consists of 30 piezoelectric sensors, while the tactile display constitutes of 30 micro motors adding up to a total size of 32 mm /spl times/ 18 mm /spl times/ 45 mm.

The problem of identifying the parameters of the constituent local linear models of Takagi-Sugeno fuzzy models is considered. In order to address the tradeoff between global model accuracy and interpretability of the local models as linearizations of a nonlinear system, two multiobjective identification algorithms are studied. Particular attention is paid to the analysis of conflicts between objectives, and we show that such information can be easily computed from the solution of the multiobjective optimization. This information is useful to diagnose the model and tune the weighting/priorities of the multiobjective optimization. Moreover, the result of the conflict analysis can be used as a constructive tool to modify the fuzzy model structure (including membership functions) in order to meet the multiple objectives. Simple illustrative examples as well as experimental results show the usefulness of the method.

The stability conditions for Takagi-Sugeno fuzzy systems suggested in the above mentioned article by Lo-Chen (ibid. vol.7 (1999)) are shown by a counterexample not to be sufficient. In reply, Lo-Chen point out that the example also indicates the fuzzy stability problem is related to the premise part of fuzzy rules.

Dynamic Takagi-Sugeno fuzzy models are not always easy to interpret, in particular when they are identified from experimental data. It is shown that there exists a close relationship between dynamic Takagi-Sugeno fuzzy models and dynamic linearization when using affine local model structures, which suggests that a solution to the multiobjective identification problem exists. However, it is also shown that the affine local model structure is a highly sensitive parametrization when applied in transient operating regimes. Due to the multiobjective nature of the identification problem studied here, special considerations must be made during model structure selection, experiment design, and identification in order to meet both objectives. Some guidelines for experiment design are suggested and some robust nonlinear identification algorithms are studied. These include constrained and regularized identification and locally weighted identification. Their usefulness in the present context is illustrated by examples.

A nonlinear controller based on a fuzzy model of MIMO dynamical systems is described and analyzed. The fuzzy model is based on a set of ARX models that are combined using a fuzzy inference mechanism. The controller is a discrete-time nonlinear decoupler, which is analyzed both for the adaptive and the fixed parameter cases. A detailed stability analysis is carried out, and the main result is that the closed loop is globally stable and robust with respect to unstructured uncertainty, which may include modeling error and disturbances. In addition, bounds on the asymptotic and transient performance are given. The main assumptions on the system and model are that they must not have strong nonminimum-phase effects, except time-delay, and the unstructured uncertainty must not be too large. A simulation example illustrates some of the properties of the modeling method and model based control structure.>