Model binarization has made significant progress in enabling real-time and energy-efficient computation for con-volutional neural networks (CNN), offering a potential solution to the deployment challenges faced by Vision Transformers (ViTs) on edge devices. However, due to the structural differences between CNN and Transformer architectures, simply applying binary CNN strategies to the ViT models will lead to a significant performance drop. To tackle this challenge, we propose BHViT, a binarization-friendly hybrid ViT architecture and its full binarization model with the guidance of three important observations. Initially, BHViT utilizes the local information interaction and hierarchical feature aggregation technique from coarse to fine levels to address redundant computations stemming from excessive tokens. Then, a novel module based on shift operations is proposed to enhance the performance of the binary Multi-Layer Perceptron (MLP) module without significantly increasing computational overhead. In addition, an innovative attention matrix binarization method based on quantization decomposition is proposed to evaluate the token’s importance in the binarized attention matrix. Finally, we propose a regularization loss to address the inadequate optimization caused by the incompatibility between the weight oscillation in the binary layers and the Adam Optimizer. Extensive experimental results demonstrate that our proposed algorithm achieves SOTA performance among binary ViT methods. The source code is released at: https://github.com/IMRL/BHViT.

Track-track association (TTA) in complex environment for multi-sensor fusion is a challenging topic due to the uncertainty of measurements, biased data, mismatch caused by different resolution etc. In this work, we proposed an end-to-end deep learning model, named the spatial-temporal attention network (STAN) for TTA tasks in complex scenarios. Three modules in the backbone of STAN for intra-track and inter-track feature representation are based on self-attention mechanism, e.g., the motion mode encoder (MME) module to encode the motion pattern of single moving targets, the spatial structure extraction (SSE) module for capturing the inter-track spatial interaction relation of an individual sensor, and the spatialtemporal fusion (STF) module for intra-track modeling on temporal dimensions, respectively. A relation reasoning head (RRH) is built for track-track relation reasoning based on the encoded track features. Experimental results on different tasks show that our proposed method achieved superior performance for track-track association compared with previous methods.

Segmentation of roads in remote sensing images is a challenging task due to the inhomogeneous intensity, non-consistent contrast, and very cluttered background in remote sensing images. Recent approaches, mostly relying on convolutions or self-attention, make it difficult to extract weak and continuous road objects. Fourier neural operators provide another novel mechanism for capturing long-range and fine-grained features beyond self-attention. Based on it, we propose an adaptive Fourier convolution network (AFCNet) on the spatial-spectral domain for road segmentation in this paper. The AFCNet is built on the pipeline of the classical U-Net model and its core is the proposed Fourier neural encoder (FNE), which is built on a feed-forward layer and a flexible Fourier convolutional structure composed of Fourier-domain pooling layers, asymmetric convolutions, squeeze-excitation inspired self-attention and adaptive multiscale fusion layers. Furthermore, we combine the FNE and bottleneck in ResNet to form a hybrid global-local feature representation scheme to capture the long and weak road objects in remote sensing images. The experiments on two public datasets, the Massachusetts Roads and DeepGlobe Road Datasets, have shown that AFCNet worked with fewer parameters and outperformed most previous methods in terms of accuracy, precision, recall, and mean intersection over union (mIoU), etc.

Road segmentation from remote sensing images is a challenging task in capturing weak, long, and irregular road features due to the limited connectivity-preserving modeling capability. In this work, we proposed a U-shaped Fourier-deformable convolution network (FDNet) for road segmentation, which integrates the merits of deformable convolutions (DCs) and Fourier convolutions compactly. Specifically, a saliency-aware DC (SD-Conv) layer is proposed for tracing salient road features based on an iterative dynamic offset learning mechanism to grasp extremely tender and weak road objects. Meanwhile, a lightweight global feature extracting module based on spectral convolutions, namely, the adaptive Fourier convolution (AF-Conv) layer, is adopted to learn long-range dependency to extract long and continuous road structures. The proposed SD-Conv layer worked in parallel with the AF-Conv layer to construct a basic and compact block to build the U-shaped FDNet model for road segmentation. Furthermore, to maintain the continuity of road objects in complex road conditions, we introduced a topology-oriented loss function based on the Hausdorff distance (HD) on the persistence diagram (PD) of segmented results, and further combined with softDice loss components for fully supervised training. Our FDNet has been trained and evaluated on two benchmarks, and experimental results show that FDNet achieved state-of-the-art (SOTA) performance. Specifically, it achieved 80.34% on accuracy, 88.42% on precision, and 84.70% on mean intersection over union (mIoU), respectively, on the Massachusetts dataset, and achieved 99.05% on accuracy, 89.21% on precision, 88.61% on recall, and 81.37% on mIoU, respectively, on the DeepGlobe dataset, outperforming most previous methods on both datasets. Codes are available at:https://github.com/zhoucharming/FDNet.

Piezoelectric ultrasound transducers have been commercialized by industry. However, the trade-off between acoustic pressure output and the sensitivity of resonant frequencies to the non-uniformity of residual stresses in the piezoelectric layer (known as stress sensitivity) has not been fully addressed. We present a novel combination of our patented slitted membrane design with our Sodium Niobate piezoelectric material. Our combination not only reduces the stress sensitivity by 4.9×, but also increases the acoustic pressure output by 35×. This brings our figure-of-merit to ($4.9\times 35$) 172× improvement over fully clamped Aluminum Nitride membranes.

Extraction road from remote sensing (RS) images is a challenging topic because of the inhomogeneous intensity, nonconsistent contrast, and very cluttered background of satellite images. Most previous approaches, relying on convolutions or self-attention, are built on the local operation or global modeling on the spatial domain but are difficult to capture weak and continuous road objects. The spectral representation of road image features and modulation learning on it provides a novel long-range-dependent and fine-grained feature representation mechanism. Based on it, we propose a novel road extraction network on RS images, called an adaptive Fourier filtered U-shaped network (AFU-Net) in this letter, which relies on modulation learning on the Fourier domain. The AFU-Net is composed of modulation learner (MoL) basic blocks and follows the pipeline of the classical U-Net model. The basic MoL block includes a global MoL (GML) block for global spectral modulation learning and an attentive MoL (AML) block which contains two parallel layers, i.e., phase-modulated filter (PMF) and magnitude-modulated filter (MMF), for fine-grained spectral modulation on the Fourier spectrum. The experiments on two public datasets, such as Massachusetts roads and DeepGlobe road datasets have shown the outstanding performance of AFU-Net on the metrics of accuracy, precision, recall, and mean intersection over union (mIoU).

In this paper, we rethink our earlier work on self-attention based crack segmentation, and propose an upgraded CrackFormer network (CrackFormer-II) for pavement crack segmentation, instead of only for fine-grained crack-detection tasks. This work embeds novel Transformer encoder modules into a SegNet-like encoder-decoder structure, where the basic module is composed of novel Transformer encoder blocks with effective relative positional embedding and long range interactions to extract efficient contextual information from feature-channels. Further, fusion modules of scaling-attention are proposed to integrate the results of each respective encoder and decoder block to highlight semantic features and suppress non-semantic ones. Moreover, we update the Transformer encoder blocks enhanced by the local feed-forward layer and skip-connections, and optimize the channel configurations to compress the model parameters. Compared with the original CrackFormer, the CrackFormer-II is trained and evaluated on more general crack datasets. It achieves higher accuracy than the original CrackFormer, and the state-of-the-art (SOTA) method with$6.7 \times $fewer FLOPs and$6.2 \times $fewer parameters, and its practical inference speed is comparable to most classical CNN models. The experimental results show that it achieves the F-measures on Optimal Dataset Scale (ODS) of 0.912, 0.908, 0.914 and 0.869, respectively, on the four benchmarks. Codes are available athttps://github.com/LouisNUST/CrackFormer-II.

A robust and efficient pyramid signal Transformer model, called SigFormer for automatic modulation recognition was proposed in this paper. In SigFormer, a pyramid Transformer architecture is introduced to encode the relationship between the internal features of modulated signals. Specifically, a dual-attention block composed of self-attention layer and scaling-attention layer is proposed for simultaneous global feature repre-sentation and noise resistance learning for modulated signals, and small-kernel convolution layers embedded to dual-attention block and feed-forward block is proposed for fine-grained modulation recognition as well. Experiments on RML2018.01a, RML2016.10a and RML2016.10b show that the SigFormer outperformed most other deep learning models on recognition accuracy, and it is more parameter-efficient than most other models and more robust on low signal-to-noise ratio (SNR) signals.

Recent years have witnessed significant advancements made in the field of unsupervised domain adaptation for semantic segmentation. Depth information has been proved to be effective in building a bridge between synthetic datasets and real-world datasets. However, the existing methods may not pay enough attention to depth distribution in different categories, which makes it possible to use them for further improvement. Besides the existing methods that only use depth regression as an auxiliary task, we propose to use depth distribution density to support semantic segmentation. Therefore, considering the relationship among depth distribution density, depth and semantic segmentation, we also put forward a branch balance loss for these three subtasks in multi-task learning schemes. In addition, we also propose a spatial aggregation priors of pixels in different categories, which is used to refine the pseudo-labels for self-training, thus further improving the performance of the prediction model. Experiments on SYNTHIA-to-Cityscapes and SYNTHIA-to-Mapillary benchmarks show the effectiveness of our proposed method.

Object detection, as one of the most important environment perception tasks for traffic safety in intelligent transportation systems, has been widely investigated recently. However, most of the researches focus on the fully supervised scenario, and inevitably lead to model failure. With the continuous development of Zero-Shot Learning (ZSL) models, Generalized Zero-Shot Detection (GZSD) has attracted great attention due to its ability of detecting unseen objects. Many researchers tend to map the detected visual features to semantic attributes and then separate seen and unseen domains during inference. But they have ignore that the generative methods generally have higher performance than these visual-semantic mapping methods, and they have been confirmed from previous GZSL methods. In order to make up for the vacancy of GZSD in the generative methods, we propose an idea of using curriculum learning to generate more precise unseen visual features. And with the excellent performance of WGAN-based method in sample synthesis, we realize the function of using semantics to generate visual features for unseen domains. In addition, we also adopt part of the idea of meta-learning to progressively correct the capability of the generator for better mitigating domain shift problem during the generation process. Through the above ideas, we can detect both seen and unseen bounding boxes and classify them accurately, by combining with the excellent detection ability of Faster-RCNN. Extensive experimental results on two popular datasets, i.e., MSCOCO and KITTI, show that our proposed method can outperform the state-of-the-art methods.

Cracks are irregular line structures that are of interest in many computer vision applications. Crack detection (e.g., from pavement images) is a challenging task due to intensity in-homogeneity, topology complexity, low contrast and noisy background. The overall crack detection accuracy can be significantly affected by the detection performance on fine-grained cracks. In this work, we propose a Crack Transformer network (CrackFormer) for fine-grained crack detection. The CrackFormer is composed of novel attention modules in a SegNet-like encoder-decoder architecture. Specifically, it consists of novel self-attention modules with 1x1 convolutional kernels for efficient contextual information extraction across feature-channels, and efficient positional embedding to capture large receptive field contextual information for long range interactions. It also introduces new scaling-attention modules to combine outputs from the corresponding encoder and decoder blocks to suppress non-semantic features and sharpen semantic ones. The CrackFormer is trained and evaluated on three classical crack datasets. The experimental results show that the CrackFormer achieves the Optimal Dataset Scale (ODS) values of 0.871, 0.877 and 0.881, respectively, on the three datasets and outperforms the state-of-the-art methods.

Abstract A major challenge in feature matching is the lack of objective criteria to determine corresponding points. Recent methods find match candidates first by exploring the proximity in descriptor space, and then rely on a ratio‐test strategy to determine final correspondences. However, these measurements are heuristic and subjectively excludes massive true positive correspondences that should be matched. In this paper, we propose a novel feature matching algorithm for image collections, which is capable of providing quantitative depiction to the plausibility of feature matches. We achieve this by exploring the epipolar consistency between feature points and their potential correspondences, and reformulate feature matching as an optimization problem in which the overall geometric inconsistency across the entire image set ought to be minimized. We derive the solution of the optimization problem in a simple linear iterative manner, where a k‐means‐type approach is designed to automatically generate consistent feature clusters. Experiments show that our method produces precise correspondences on a variety of image sets and retrieves many matches that are subjectively rejected by recent methods. We also demonstrate the usefulness of the framework in structure from motion task for denser point cloud reconstruction.

This paper introduces an approach to performance animation that uses window-based principal component analysis (WLPCA). Our key idea is to construct a series of local models from a prerecorded motion database and utilize them to construct full-body human motion in a maximum a posteriori frame work. We have demonstrated the effectiveness of our approach by synthesizing a variety of human actions. Given an appropriate motion capture database, the results are comparable in quality to the ground truth data. We have also evaluated the performance of our approach by leave-one-out experiments and by comparing to two baseline algorithms.

This paper introduces an approach to performance animation that employs a small number of motion sensors to create an easy-to-use system for an interactive control of a full-body human character. Our key idea is to construct a series of online local dynamic models from a prerecorded motion database and utilize them to construct full-body human motion in a maximum a posteriori framework (MAP). We have demonstrated the effectiveness of our system by controlling a variety of human actions, such as boxing, golf swinging, and table tennis, in real time. Given an appropriate motion capture database, the results are comparable in quality to those obtained from a commercial motion capture system with a full set of motion sensors (e.g., XSens [2009]); however, our performance animation system is far less intrusive and expensive because it requires a small of motion sensors for full body control. We have also evaluated the performance of our system by leave-one-out-experiments and by comparing with two baseline algorithms.

One difference between real-time CSCW systems and traditional distributed systems is that the former one needs to provide a natural, free and fast interface for multi-user interaction. However, typical multi-user interaction methods in 3D CAD systems apply strict consistency maintenance, such as lock mechanism and floor control which result in a stagnant and unnatural interface. This article proposes a semantic-based operational transformation (OT), which is similar to the traditional OT form, to support less constraint multi-user interaction and to achieve consistency in collaborative CAD editing (co-CAD) systems. Major technical contributions of this article include: a 3D semantic priority to select operations from waiting list, an OT strategy to decide OT direction, OT rules and a kernel OT function for co-CAD systems.

This paper presents a generative human motion model for synthesis, retargeting, and editing of personalized human motion styles. We first record a human motion database from multiple actors performing a wide variety of motion styles for particular actions. We then apply multilinear analysis techniques to construct a generative motion model of the form x = g(a, e) for particular human actions, where the parameters a and e control "identity" and "style" variations of the motion x respectively. The new modular representation naturally supports motion generalization to new actors and/or styles. We demonstrate the power and flexibility of the multilinear motion models by synthesizing personalized stylistic human motion and transferring the stylistic motions from one actor to another. We also show the effectiveness of our model by editing stylistic motion in style and/or identity space.

Data exchange is one of key issues in collaborative design. The purpose of feature-based data exchange is that the target model is editable after being exchanged. This is the reason why nowadays feature-based data exchanges become the research focus beyond the traditional geometry-based data exchange. This paper presents a two-stage mechanism to recover a complete modeling process of a parametric CAD mode in source systems. Therefore, what we exchange is the procedure of modeling steps. In target systems, we use the exchanged procedure to simulate a real human to reconstruct the parametric CAD mode in any heterogeneous CAD systems. The proposed method has been tested with case studies among typical CAD systems, such as SolidWorks, UG, Pro/E and Catia.

From CSCW view, the name issues in replicated collaborative solid modeling involve several fundamental challenges, such as consistency maintenance, multi-user interaction and undo/redo mechanism. Therefore, a consistency and awareness naming approach will contribute to all of above three challenges, either directly or potentially. The paper begins with the multi-user interaction framework in replicated collaborative solid modeling. And then a collaborative name structure is constructed to consistently name merged faces. Finally an awareness method is presented to distinguish which type of operation to create the merged faces. The proposed methods are illustrated with case studies in replicated collaborative solid modeling.

At first, graphics media are abstracted into high level and low level representation. The high level representation includes the design history and feature description. The low level includes topology, geometry and attribution. Secondly, the consistency conditions for different abstracted levels are established. The consistency conditions can be described as strong consistency or weak consistency. They also can be described as strong no-consistency or weak no-consistency. Thirdly, according to layered consistency conditions, different abstracted levels of graphics media are associated with suitable concurrent control methods respectively, which can be strict or relaxed. Fourthly, the strategy how to apply hierarchical consistency model into flexible collaboration transparent infrastructure are analyzed. Finally, we applied the model and methods in our test-bed for collaborative design of engineering graphics.

In cross-country navigation, autonomous land vehicles (ALVs) must traverse harsh natural terrains, which are uneven, rough, and sloping. One of challenges is to evaluate the terrain's characteristics quantitatively so as to prepare for smooth and stable trajectory planning subsequently. In this paper, we proposed a separate-and-integrate model to analysis rugged terrains, and developed a more universal and robust untraversable regions detection method on elevation maps. When separate, we extract the necessary and sufficient terrain characteristics such as slope, roll variance and roughness from elevation maps respectively and when integrate, the fuzzy inference is applied to combine the above terrain features in order to obtain its traversability assessment and local quantitative evaluations. Experimental results show the method can accurately evaluate terrains' characters and properly classify rugged terrains, and the classification results are robust to the uncertainty and imprecision of the terrain information. And because it's based on terrains' geometry clues, the method provides a more generic framework for rugged terrain analysis.