Recently, we have witnessed impressive achievements in cancer survival analysis by integrating multimodal data, e.g., pathology images and genomic profiles. However, the heterogeneity and high dimensionality of these modalities pose significant challenges in extracting discriminative representations while maintaining good generalization. In this paper, we propose a Cohort-individual Cooperative Learning (CCL) framework to advance cancer survival analysis by collaborating knowledge decomposition and cohort guidance. Specifically, first, we propose a Multimodal Knowledge Decomposition (MKD) module to explicitly decompose multimodal knowledge into four distinct components: redundancy, synergy, and uniqueness of the two modalities. Such a comprehensive decomposition can enlighten the models to perceive easily overlooked yet important information, facilitating an effective multimodal fusion. Second, we propose a Cohort Guidance Modeling (CGM) to mitigate the risk of overfitting task-irrelevant information. It can promote a more comprehensive and robust understanding of the underlying multimodal data while avoiding the pitfalls of overfitting and enhancing the generalization ability of the model. By cooperating with the knowledge decomposition and cohort guidance methods, we develop a robust multimodal survival analysis model with enhanced discrimination and generalization abilities. Extensive experimental results on five cancer datasets demonstrate the effectiveness of our model in integrating multimodal data for survival analysis. Our code is available at https://github.com/moothes/CCL-survival.

Ultrasound imaging (US) is one of the most commonly used techniques for detecting breast lesions. However, due to the inherent properties of low contrast, speckle noise, and blurred boundaries in B-mode ultrasound images, the performance of breast lesion segmentation was significantly limited. In contrast, original radiofrequency (RF) data contains more detailed information. To enhance the performance of breast lesion segmentation and compensate for the information missing in ultrasound images, this study proposed a radiofrequency and ultrasound image feature attention and fusion network (RUIFAF Net) to comprehensively fuse the complementary information from both modalities. Specifically, we proposed a Feature Attention Module (FAM) and a Multi-level Fusion (MF) method to obtain multi-scale feature information and integrate contextual semantic information, respectively. Experimental results on OASBUD dataset demonstrate that our proposed RUIFAF_Net achieved higher Dice (0.8264 vs. 0.6468-0.8184) and IoU (0.7161 vs. 0.5243-0.7025) compared with the other six advanced methods.

Deep Learning-based Unsupervised Salient Object Detection (USOD) mainly relies on the noisy saliency pseudo labels that have been generated from traditional handcraft methods or pre-trained networks. To cope with the noisy labels problem, a class of methods focus on only easy samples with reliable labels but ignore valuable knowledge in hard samples. In this paper, we propose a novel USOD method to mine rich and accurate saliency knowledge from both easy and hard samples. First, we propose a Confidence-aware Saliency Distilling (CSD) strategy that scores samples conditioned on samples' confidences, which guides the model to distill saliency knowledge from easy samples to hard samples progressively. Second, we propose a Boundary-aware Texture Matching (BTM) strategy to refine the boundaries of noisy labels by matching the textures around the predicted boundaries. Extensive experiments on RGB, RGB-D, RGB-T, and video SOD benchmarks prove that our method achieves state-of-the-art USOD performance. Code is available at www.github.com/moothes/A2S-v2.

In unsupervised 3D face reconstruction, existing methods modeling the canonical face typically exclude the skip connections between encoder-decoder pairs. Consequently, they have difficulty capturing appearance details necessary for the task. However, directly applying original skip connections only induces these methods to degrade to a trivial 2D texture reconstruction algorithm. In this paper, we propose novel Reprogramming Skip Connections (RSCs), which escape from bringing about degradation and improve the 3D face reconstruction quality. Specifically, the proposed method filters out inappropriate information causing degradation by aggregating the features from the encoder in spatial dimensions into several prototypes. These prototypes preserving beneficial information are subsequently combined with the corresponding decoder features with the help of expansion masks. Further, we design the masks reconstruction consistency loss to improve the quality of the expansion masks. Our experiments verify the superiority of our method compared to other competitors.

Few shot object detection aims to generalize the model to previously unseen classes with only a few training samples, which has been attached great attention due to its practicability in real scenes. Many existing methods hold the missed detection issue, mainly due to the problem of incompletely annotated samples. Specifically, only partial objects in a sample are labeled, resulting in unlabeled objects being used as the background for training. This problem is especially serious for few-shot learning. To solve this noisy label problem, we first propose a label calibration method based on confidence to correct potential incorrect labels, and then introduce the class center library to eliminate the negative impact of unlabeled objects. We conduct extensive experiments on PASCAL VOC and MS-COCO benchmarks, which proves the effectiveness of our approach and achieves the state-of-the-art results.

This paper focuses on the Unsupervised Salient Object Detection (USOD) issue. We come up with a two-stage Activation-to-Saliency (A2S) framework that effectively excavates saliency cues to train a robust saliency detector. It is worth noting that our method does not require any manual annotation in the whole process. In the first stage, we transform an unsupervisedly pre-trained network to aggregate multi-level features into a single activation map, where an Adaptive Decision Boundary (ADB) is proposed to assist the training of the transformed network. Moreover, a new loss function is proposed to facilitate the generation of high-quality pseudo labels. In the second stage, a self-rectification learning strategy is developed to train a saliency detector and refine the pseudo labels online. In addition, we construct a lightweight saliency detector using two Residual Attention Modules (RAMs) to learn robust saliency information. Extensive experiments on several SOD benchmarks prove that our framework reports significant performance compared with existing USOD methods. Moreover, training our framework on 3,000 images consumes about 1 hour, which is over 10 times faster than previous state-of-the-art methods. Code has been published athttps://github.com/moothes/A2S-USOD.

In ship detection task, target objects with extreme aspect ratios are common in practical applications. However, existing ship detection methods seldom make efforts to tackle this issue. In this paper, we present a novel Cross-level Attention and Ratio Consistency (CARC) Network for ship detection. First, we propose a Cross-Level Attention (CLA) module to generate attention signals by integrating information from both higher and lower level features. Specifically, for each feature, we calculate its similarity with features from adjacent levels. These similarities are utilized as weights to enhance the channels that consist of different-level information. By fusing multi-level information, a channel-wise attention vector is generated to enhance the learned representations in the base feature. Second, we propose a Ratio Consistency loss that promotes the networks to localize the ships with more accurate aspect ratios. Existing ship detection methods have different sensitiveness to width and height predictions, significantly increasing the learning difficulty for localizing target ships. We append an auxiliary supervision signal to the detection head in our method. This supervision signal measures the error between the predicted and the ground truth aspect ratios of target ships. Experiment results show that our model achieves significant performance gains compared to existing methods.

One essential goal for Highly Automated Vehicles (HAVs) safety test is to assess their risk rate in naturalistic driving environment, and to compare their performance with human drivers. The probability of exposure to risk events is generally low, making the test process extremely time-consuming. To address this, we proposed a surrogate-based method in scenario-based simulation test to expediate the assessment of the risk rate of HAVs. HighD data were used to fit the naturalistic distribution and to estimate the probability of each concrete scenario. Machine learning model-based surrogates were proposed to quickly approximate the test result of each concrete scenario. Considering the different capabilities and domains of various surrogate models, we applied six surrogate models to search for two types of targeted scenarios with different risk levels and rarity levels. We proved that the performances of different surrogate models greatly distinguish from each other when the target scenarios are extremely rare. Inverse Distance Weighted (IDW) was the most efficient surrogate model, which could achieve risk rate assessment with only 2.5% test resources. The required CPU runtime of IDW was 2% of that required by Kriging. The proposed method has great potential in accelerating the risk assessment of HAVs.

Personalized Fixation-based Object Segmentation (PFOS) aims at segmenting the gazed objects in images conditioned on personalized fixations. However, the performances of existing PFOS methods are degraded when facing anomalous fixation maps (some fixations fall in the background) or enormous objects because of their poor localization ability. In this paper, we propose a novel Selective Intra-image Similarity Network (SISNet) that achieves significant performance by precisely localizing the gazed objects. First, we propose a Response Purifying Module (RPM) to eliminate the false response regions caused by anomalous fixations in the background. By suppressing these false responses, we can significantly reduce the negative impacts caused by anomalous fixations. Second, we propose an intra-image similarity module (ISM) to better localize large objects by integrating more long-range information. In addition, we propose a new Discriminative Intersection-over-Union metric that evaluates whether PFOS methods can produce distinctive predictions for varying fixations. Experiments on the PFOS and our proposed OSIE-CFPS-UN datasets prove that our network achieves remarkable improvements and outperforms existing state-of-the-art methods. Code has been published athttps://www.github.com/moothes/SISNet.

Automated Vehicles (AVs) need to be thoroughly evaluated in order to ensure their driving capabilities. However, comprehensive evaluations are intractable due to both time and monetary costs. To address this problem, we propose an Adaptive Design of Experiments (ADOE) method to evaluate the safety of AVs. Using this method, a Surrogate Model (SM) is established and updated iteratively. SM in ADOE is used to approximate the results of AV testing and help to select the next concrete scenario to be tested in each iteration. Two different ADOE approaches are proposed in this study for different testing purposes. Since the choice of the surrogate model has a profound impact on the performance of the ADOE method, 6 surrogate models were compared with two logical scenarios at different scales – a car following logical scenario and a cut-in logical scenario. Results show that Extreme Gradient Boosting (XGB) is suitable for both ADOE approaches. And both proposed ADOE approaches achieved desired performance. Scenario-oriented ADOE made full use of each concrete scenario, capturing one collision case for every 1.12 test runs in the car following logical scenario, while SM-oriented ADOE successfully depicted the boundary between safety and danger. Using 0.46% test resources compared to enumeration, the SM-oriented ADOE found 93.9% dangerous scenarios with 90.9% precision. ADOE approaches have great potential in accelerating the evaluation of AV safety.

Highly Automated Vehicles (HAVs) must undergo strict safety testing before being released to the public. Mileage-based on-road testing suffers from unaffordable time costs and high safety risks. Simulated scenario-based testing has been found to be a trustworthy alternative for testing HAVs’ built-in algorithms and functionalities. Test automation is typically used to generate target scenarios. This approach facilitates customized testing and avoids wasting time on simple and redundant scenarios. This study aims to review test automation methods and discuss how to accentuate their strengths rather than be trapped in their weaknesses under certain applicable conditions. According to their main purposes, we classify test automation methods into coverage-oriented, unsafe-scenario-oriented, and naturalistic-assessment-oriented categories. To further demonstrate the differences of these methods, we then design numerical experiment to compare the capabilities of seven test automation methods. Finally, we compile our observations to form a comprehensive guide for selecting test automation methods with different test requirements in mind.

Video salient object detection (VSOD) aims to locate and segment the most attractive object by exploiting both spatial cues and temporal cues hidden in video sequences. However, spatial and temporal cues are often unreliable in real-world scenarios, such as low-contrast foreground, fast motion, and multiple moving objects. To address these problems, we propose a new framework to adaptively capture available information from spatial and temporal cues, which contains Confidence-guided Adaptive Gate (CAG) modules and Dual Differential Enhancement (DDE) modules. For both RGB features and optical flow features, CAG estimates confidence scores supervised by the IoU between predictions and the ground truths to re-calibrate the information with a gate mechanism. DDE captures the differential feature representation to enrich the spatial and temporal information and generate the fused features. Experimental results on four widely used datasets demonstrate the effectiveness of the proposed method against thirteen state-of-the-art methods.

We present a learning model that makes full use of boundary information for salient object segmentation. Specifically, we come up with a novel loss function, i.e., Contour Loss, which leverages object contours to guide models to perceive salient object boundaries. Such a boundary-aware network can learn boundary-wise distinctions between salient objects and background, hence effectively facilitating the salient object segmentation. Yet the Contour Loss emphasizes the boundaries to capture the contextual details in the local range. We further propose the hierarchical global attention module (HGAM), which forces the model hierarchically to attend to global contexts, thus captures the global visual saliency. Comprehensive experiments on six benchmark datasets show that our method achieves superior performance over state-of-the-art ones. Moreover, our model has a real-time speed of 26 fps on a TITAN X GPU.

Recently, contour information largely improves the performance of saliency detection. However, the discussion on the correlation between saliency and contour remains scarce. In this paper, we first analyze such correlation and then propose an interactive two-stream decoder to explore multiple cues, including saliency, contour and their correlation. Specifically, our decoder consists of two branches, a saliency branch and a contour branch. Each branch is assigned to learn distinctive features for predicting the corresponding map. Meanwhile, the intermediate connections are forced to learn the correlation by interactively transmitting the features from each branch to the other one. In addition, we develop an adaptive contour loss to automatically discriminate hard examples during learning process. Extensive experiments on six benchmarks well demonstrate that our network achieves competitive performance with a fast speed around 50 FPS. Moreover, our VGG-based model only contains 17.08 million parameters, which is significantly smaller than other VGG-based approaches. Code has been made available at: https://github.com/moothes/ITSD-pytorch.