Multimodal Large Language Models (MLLMs) employing the Mixture-of-Experts (MoE) structure exhibit encouraging results in visual language tasks. However, they struggle with catastrophic forgetting due to a lack of effective collaboration among experts and negative transfer across tasks. This happens because the router typically employed in MoE for managing expert assignments is inadequate when there are significant shifts in data distribution across various tasks. A drop in the effectiveness of earlier tasks is caused by negative transfer, which occurs due to conflicts in shared knowledge between tasks, disturbing the knowledge already acquired. To address these issues, we propose the Knowledge Space Synergy Framework in Mixture of Experts (KSS-MoE) for Continual Visual Instruction Tuning (CVIT). It dynamically combines the knowledge subspaces of experts to improve the integration of fine-grained complementary knowledge and collaborative abilities of experts, thus addressing the limitations of the basic router. Furthermore, we introduce a general expert that maintains orthogonal subspaces for shared knowledge, enabling effective cross-task knowledge utilization while reducing negative transfer. Extensive experiments conducted on eight CVIT tasks confirm the excellence of KSS-MoE, showcasing its top-tier performance.

Considerable attention has been paid to predicting student performance on exercises. The performance of prior studies is determined by the quality of the trait features of students and exercises. Nevertheless, most of the prior study primarily examines simple pairwise interactions in learning trait features, like those between students and exercises or exercises and concepts, while disregarding the complex higher-order interactions that typically exist among these components, which in turn hinders the prediction results. In this paper, we using an innovative Multi-Channel Graph Contrastive Learning (MCGCL) framework that integrates various high-order interactions for predicting student performance. MCGCL characterizes graph structures reflecting various high-order relationships among students, exercises, and concepts through multiple channels, thereby enhancing the trait features of both students and exercises. Moreover, graph contrastive learning is employed to enhance the representation of trait features acquired from high-order graph structures in diverse views. Extensive experiments on real-world datasets show that MCGCL achieves state-of-the-art results on the task of predicting student performance. The code is available at https://github.com/sunlitsong/MCGCL.

Concolic testing combines concrete execution with symbolic execution to automatically generate test inputs that exercise different program paths and deliver high code coverage. This approach has been extended to multithreaded programs for exposing data races. Multithreaded programs frequently rely upon concurrent dynamic data structures whose implementations may contain data races that manifest only when certain dynamic data structure shapes, program paths, and thread interleavings are exercised. The lack of support for exploring different data structure shapes compromises the detection of races. This paper presents a summarization-guided approach for concolic testing capable of efficiently exploring different dynamic data structure shapes to expose data races. Via unit testing of key functions, function summaries are generated that capture data structure shapes that cause various function paths to be exercised. The shapes are captured in the form of pointer-pointee relations among symbolic pointers. By reusing function summaries during concolic testing, much of the overhead of handling symbolic pointers and dynamic objects in summarized functions is avoided. The summary also contains symbolic memory accesses and synchronization events that guide application-level concolic testing first to identify and then confirm potential data races. We demonstrate the efficiency and efficacy of our approach via experiments with multithreaded programs performing concurrent operations on four widely used dynamic data structures - Skip List, Unrolled Linked List, Priority Queue, and AVL Tree. It increases the number of races detected from 34 to 74 in total in comparison to Cloud9, and reduces both constraints solving time and number of constraints needed to be solved via summarization.

Airborne multichannel circular stripmap synthetic aperture radar (CSSAR)-ground moving target indication (GMTI) has drawn increasing attention in wide-area surveillance, reconnaissance, and traffic monitoring due to its short revisit times. In this article, a novel scheme for CSSAR-GMTI systems is proposed to offer high-resolution focusing of moving targets and enable efficient detection and accurate parameter estimation, which are ensured by integrating moving target imaging with a space–time adaptive processing (STAP) clutter suppression step. The presented scheme develops a target imaging algorithm that forms SAR images focused on a particular 2-D motion parameter to maximize target signal energy and recover its high resolution. Moreover, benefiting from the suitable phase compensation designed in the 2-D frequency domain, moving targets can finally be properly focused in the SAR image without displacement, and their parameters can be uniquely determined. Experiments on an emulated radar dataset are conducted to validate the effectiveness of the proposed scheme.

In the era of big data, data mining algorithms usually handle plenty of data and the data may involve sensitive information about people in some scenarios, so privacy protection becomes an important task. Support Vector Machine (SVM) is a popular data mining algorithm widely used in various fields. In this paper, we mainly study the problem of privacy protection during SVM classification. Nowadays, lots of privacy-preserving SVM approaches have been proposed, but many of them have some flaws. Negative database (NDB) is a new type of data representation, which stores the complementary information of data. It is considered as a promising technique for privacy-preserving data mining, since reversing NDB has been proven as an NP-hard problem. Therefore, we propose a privacy-preserving SVM method based on NDBs. We also design a model for estimating the results of the dot product from NDBs, which is the basis of SVM kernel functions. Private data are converted into NDBs before being transmitted to other parties, and thus privacy is protected. Then SVM classification can be effectively performed using the proposed estimating model. We conduct several experiments on five public datasets, and the results demonstrate the effectiveness of the proposed method.

This letter proposes a machine learning inversion scheme for P-band polarimetric interferometric synthetic aperture radar (Pol-InSAR), which can achieve the single-baseline random volume over ground (RVoG) model inversion without the assumption of the null ground-to-volume ratio. First, the potential variables—including the incidence angle and the PDHigh coherence acquired with phase diversity optimization—that are related to the forest vertical structure are analyzed for their correlations with the extinction coefficient in the RVoG model. Then, the machine learning approach is applied to forecast the extinction coefficient characterized by those potential variables. Ultimately, in the case of fixing the extinction coefficient, the forest height is estimated by a geometric process on the complex plane. The actual Pol-InSAR data verification illustrates that the inversion performance of the proposed scheme overmatches that of the traditional schemes.

The polarimetric interferometric synthetic aperture radar (Pol-InSAR) model under P-band observations exhibits vertical structure diversity. Compared with the exponential-based random volume over ground (RVoG) model, the Gaussian vertical backscatter volume over ground (GVBVoG) model expresses a more complex forest vertical structure via introducing more parameters. On account of the influence of topographic fluctuation on the model, this article establishes the sloped Gaussian vertical backscatter volume over ground (SGVBVoG) model by drawing into the terrain slope. Based upon the SGVBVoG model, this article develops the 2-D SGVBVoG (2-D-SGVBVoG) model by defining the structure factor, which effectively reduces the model complexity from three to two dimensions. In the 2-D-SGVBVoG model inversion, in view of the diversity of forest species, age, shape, density, etc., in the natural scene and the variation of specific radar systems, a structure factor prediction scheme relying on machine learning is proposed. In the machine-learning model training, the radar incidence angle and the PDHigh coherence acquired by coherence optimization with terrain phase removal are utilized as the variables for characterizing the structure factor. Ultimately, in the case of fixing structure factor, a geometric inversion process on the complex plane is put forward to extract the forest height. The BIOSAR 2008 P-band Pol-InSAR data validation shows that the proposed method achieves an RMSE of 3.07 m, which is 24.0% better than the three-baseline SRVoG inversion.

Finite-state machine (FSM) is a fundamental computation model used by many applications. However, FSM execution is known to be “embarrassingly sequential” due to the state dependences among transitions. Existing solutions leverage enumerative or speculative parallelization to break the dependences. However, the efficiency of both parallelization schemes highly depends on the properties of the FSM and its inputs. For those exhibiting unfavorable properties, the former suffers from the overhead of maintaining multiple execution paths, while the latter is bottlenecked by the serial reprocessing among the misspeculation cases. Either way, the FSM parallelization scalability is seriously compromised.

Concolic testing combines concrete execution with symbolic execution along the executed path to automatically generate new test inputs that exercise program paths and deliver high code coverage during testing. The GKLEE tool uses this approach to expose data races in CUDA programs written for execution of GPGPUs. In programs employing concurrent dynamic data structures, automatic generation of data structures with appropriate shapes that cause threads to follow selected, possibly divergent, paths is a challenge. Moreover, a single non-conflicting data structure must be generated for multiple threads, that is, a single shape must be found that simultaneously causes all threads to follow their respective chosen paths. When an execution exposes a bug (e.g., a data race), the generated data structure shape helps the programmer understand the cause of the bug. Because GKLEE does not permit pointers that construct dynamic data structures to be made symbolic, it cannot automatically generate data structures of different shapes and must rely on the user to write code that constructs them to exercise desired paths. We have developed DSGEN for automatically generating non-conflicting dynamic data structures with different shapes and integrated it with GKLEE to uncover and facilitate understanding of data races in programs that employ complex concurrent dynamic data structures. In comparison to GKLEE, DSGEN increases the number of races detected from 10 to 25 by automatically generating a total of 1,897 shapes in implementations of four complex concurrent dynamic data structures -- B-Tree, Hash-Array Mapped Trie, RRB-Tree, and Skip List.

JSON (JavaScript Object Notation) and its derivatives are essential in the modern computing infrastructure. However, existing software often fails to process such types of data in a scalable way, mainly for two reasons: (i) the processing often requires to build a memory-consuming parse tree; (ii) there exist inherent dependences in processing the data stream, preventing any data-level parallelization. Facing the challenges, developers often have to construct ad-hoc pre-parsers to split the data stream in order to reduce the memory consumption and increase the data parallelism. However, this strategy requires more programming efforts. Moreover, the pre-parsing itself is non-trivial to parallelize, thus introducing a new serial bottleneck. To solve the dilemma, this work introduces a scalable yet fully automatic solution - a compilation system, namely JPStream, that compiles standard JSONPath queries into parallel executables with bounded memory footprints. First, JPStream adopts a stream processing design that combines the querying and parsing into one pass, without generating any in-memory parse tree. To achieve this, JPStream uses a novel joint compilation technique that compiles the queries and the JSON syntax together into a single automaton. Furthermore, JPStream leverages the "enumerability'' of automaton to break the dependences and reason about the transition rules to prune infeasible states. It also features a runtime that learns structural constraints from the input to enhance the pruning. Evaluation on real-world JSON datasets with standard JSONPath queries shows that JPStream can reduce the memory consumption significantly, by up to 95%, meanwhile achieving near-linear speedup on multicore and manycore processors.

A linear frequency modulation continuous-wave mini-synthetic aperture radar (SAR) system mounted on small aircrafts promises a high-flexibility and cost-effective microwave remote sensing technology. However, the mini-SAR system suffers from considerable trajectory deviations due to aircraft's lightweight, low flight height, and limited capacity for a high-accuracy inertial measurement unit (IMU). With the rapid increasing requirements for resolution, the residual range cell migration (RCM) exceeds a single range cell. Under such circumstances, traditional autofocus algorithms fail to guarantee well-focused mini-SAR images and further accurate interferometric SAR (InSAR) applications. To solve these problems, this paper proposed a novel residual RCM correction scheme for a mini-SAR system mounted on small aircrafts without high-accuracy IMU. The core idea is to estimate the misalignments of adjacent range profiles based on fast-time split-band signal interferometry at each azimuth time and integrate them with time to obtain an estimation of residual RCM. The proposed method promises a high-accuracy misalignment estimation result without resampling operation in the traditional cross-correlation methods. Simulation and experimental results show the improvement of mini-SAR image focusing quality and refinement of coherence map between the master and slave images for InSAR applications, which demonstrated the effectiveness and reliability of our proposed residual RCM correction scheme for the mini-SAR system.

In this paper, to address problems in the registration of synthetic aperture radar (SAR) and optical images due to large gray differences, the scale-invariant feature transform (SIFT) approach based on phase congruency (PC-SIFT) is proposed. This approach is used to address the gradient inversion in multi-source images, and it is based on optimizing the dominant direction interval of the descriptors. We construct a new descriptor by combining phase consistency and the gradient amplitude, which is referred to as PCG-SIFT descriptor. The proposed algorithm is suitable for multi-sensor images with large gray differences and significant edge features The results of experiments show that compared to the traditional gradient-based SIFT descriptor, the PC-SIFT descriptor and PCG-SIFT descriptor improve the robustness and matching probability of the registration algorithm for multi-source images.

This work proposes a forest model based on Gamma distribution to better describe the forest vertical structure for height inversion using P-band polarimetric synthetic aperture radar interferometry (Pol-InSAR) data. The proposed model takes into account the forest vertical heterogeneity and asymmetry, to which volume interferometric coherence is sensitive. The interferometric coherence associated with a volume where the vertical backscattered power varies following a Gamma distribution is derived. The effect of scattering center height standard deviation and mean elevation to the volume interferometric coherence is investigated. Finally, the strategy of multi-baseline on the proposed model for forest height inversion using P-band Pol-InSAR data is proposed.

For the mini-interferometric synthetic aperture radar system mounted on small aircraft or unmanned aerial vehicles, yaw and pitch angle deviations can be considerably high due to their small size and atmospheric turbulence. Moreover, we cannot install a large-volume, heavy-weight, and high-cost inertial navigation system limited by the aircraft's carrying capacity and system cost. In view of the problem, this letter proposes a robust yaw and pitch angle estimation method based on the relationship between range-variant Doppler centroid and attitude angles. For each azimuth moment, estimate the range-variant Doppler centroid for each range gate and solve the range-variant Doppler centroid model using a total least squares method to obtain a robust yaw and pitch angle estimation result. The comparison of the estimated and recorded yaw and pitch angles by a high-accuracy position and orientation system validated the effectiveness and reliability of our proposed yaw and pitch angle estimation method.

During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.