The surging volume of big data underscores the imperative of integrating heterogeneous datasets into a unified, semantically consistent format. We introduce Pyramid, a comprehensive framework for heterogeneous data integration, addressing schema transformation, feature encoding, entity matching, deduplication, and mapping retrieval. At its core, a hierarchical graph captures relationships across databases, bridging diverse data sources. We employ a bottom-up encoding strategy, factoring in data context, and a top-down matching mechanism, curbing attribute misalignment across entity types. Enhanced by the transformer model and contrastive learning, our approach realizes unsupervised feature synthesis, bolstering integration. Extensive experiments and evaluations validate the broad applicability and superior performance of our method across a variety of heterogeneous datasets.

Recently, cross-modal hashing has attracted much attention in large-scale image retrieval scenarios. However, most existing methods ignore the potential higher-order relationships and label semantic information between heterogeneous modality data. Besides, the imbalanced training samples could bias the learning process in most classes and affect the retrieval performance. To solve the above problems, we proposed a Joint-semantics Multi-Similarity Hashing method for cross-modal retrieval (JMSH). We first construct a joint semantic similarity matrix, which supervises hash learning by integrating multi-modal features and semantic labels. This method generates higher-order semantic features that maintain semantic correlation effectively. Then, we propose a multi-similarity loss based on adaptive margin, which can collect and weight informative pairs efficiently and accurately, thus producing more discriminative hashing code and improving retrieval performance. Extensive experiments on two benchmark datasets show the superiority of JMSH in cross-modal retrieval tasks.

Outlier detection can identify anomalies in large-scale data. To provide reliability and security for Internet of Things (IoT)-enabled maritime transportation systems (MTSs), in this paper we propose an outlier detection method based on the neighborhood rough residual network (NRRN). We calculate the neighborhood approximation accuracy and neighborhood conditional entropy to obtain the neighborhood combined entropy describing the discrimination ability of the condition attribute subset to the information system. We then delete the redundant attributes according to the attribute combination importance derived from the neighborhood combined entropy. The data after attribute reduction are used to train the convolutional neural network, and the residual network (ResNet50) is used to avoid the degradation of model performance caused by the increase in the number of network layers. The proposed method is compared with mainstream outlier detection algorithms on a fishing vessel operation dataset. Experiments show that the proposed method can greatly improve the accuracy of outlier detection while taking into account interpretability and computational efficiency, thereby ensuring the data integrity of IoT-enabled MTSs.

Cameras onboard autonomous, as a critial component of the sensor system of automatic driving, plays a vital role in perception of driving and road environment. However, in some bad weather or unpredictable situations, the image quality obtained by the in-vehicle sensing camera is not ideal, which will become an extremely unsafe factor for autonomous driving. In order to improve the safety of self-driving vehicles, we proposed a novel high-quality image of invehicle cameras generation approach CE-GAN, a conditional generative adversarial network that attempt to leverage the point cloud data from on-board lidar to compensate the defect of visible image to improve the image quality of on-board cameras. Inspired by the generative adversarial networks, our method establishes an adversarial game between the generator and the discriminator We designed specifically loss function for different reasons for image quality impairment including partially obscured and fogged. Consequently, extensive experiments show that CE-GAN renders better performance in detail texture, compared with conventional Cycle-GAN, pix2pix methods without assistance of LiDAR data.

Realtime foreground/background segmentation based on sequence video was of great significance for autonomous vehicles perception, edge device application and higher level data analysis. A new fast background subtraction method for dynamic objects detection was proposed by using the digital features of the whole block of pixels. The algorithm considered that the change of the current pixel was closely related to the surrounding pixels, took the current pixel and its eight neighboring pixels as a whole block, and used the digital features - average and variance to reflect the pixel level of the block and establish the background model. At the same time, a local remodeling method was proposed, which made the algorithm can process and eliminate ghost quickly. The results based on CDnet2014 dataset showed that our algorithm could adapt to various dynamic objects detection scenarios and initialize fast under the condition of low hardware cost, and provided a good overall performance.