Derived from diffusion models, MangaNinja specializes in the task of reference-guided line art colorization. We incorporate two thoughtful designs to ensure precise character detail transcription, including a patch shuffling module to facilitate correspondence learning between the reference color image and the target line art, and a point-driven control scheme to enable fine-grained color matching. Experiments on a self-collected benchmark demonstrate the superiority of our model over current solutions in terms of precise colorization. We further showcase the potential of the proposed interactive point control in handling challenging cases (e.g., extreme poses and shadows), cross-character colorization, multi-reference harmonization, etc., beyond the reach of existing algorithms. Our code and model could be found here.

This study focuses on a novel task in text-to-image (T2I) generation, namely action customization. The objective of this task is to learn the co-existing action from limited data and generalize it to unseen humans or even animals. Experimental results show that existing subject-driven customization methods fail to learn the representative characteristics of actions and struggle in decoupling actions from context features, including appearance. To overcome the preference for low-level features and the entanglement of high-level features, we propose an inversion-based method Action-Disentangled Identifier (ADI) to learn action-specific identifiers from the exemplar images. ADI first expands the semantic conditioning space by introducing layer-wise identifier tokens, thereby increasing the representational richness while distributing the inversion across different features. Then, to block the inversion of action-agnostic features, ADI extracts the gradient invariance from the constructed sample triples and masks the updates of irrelevant channels. To comprehensively evaluate the task, we present an Action-Bench that includes a variety of actions, each accompanied by meticulously selected samples. Both quantitative and qualitative results show that our ADI outperforms existing baselines in action-customized T2I generation. Our project page is at https://adi-t2i.github.io/ADI.

The isocitrate dehydrogenase (IDH) gene mutation is an essential biomarker for the diagnosis and prognosis of glioma. It is promising to better predict glioma genotype by integrating focal tumor image and geometric features with brain network features derived from MRI. Convolutional neural networks show reasonable performance in predicting IDH mutation, which, however, cannot learn from non-Euclidean data, e.g., geometric and network data. In this study, we propose a multi-modal learning framework using three separate encoders to extract features of focal tumor image, tumor geometrics and global brain networks. To mitigate the limited availability of diffusion MRI, we develop a self-supervised approach to generate brain networks from anatomical multi-sequence MRI. Moreover, to extract tumor-related features from the brain network, we design a hierarchical attention module for the brain network encoder. Further, we design a bi-level multi-modal contrastive loss to align the multi-modal features and tackle the domain gap at the focal tumor and global brain. Finally, we propose a weighted population graph to integrate the multi-modal features for genotype prediction. Experimental results on the testing set show that the proposed model outperforms the baseline deep learning models. The ablation experiments validate the performance of different components of the framework. The visualized interpretation corresponds to clinical knowledge with further validation. In conclusion, the proposed learning framework provides a novel approach for predicting the genotype of glioma.

Video surveillance is gaining increasing popularity to assist in railway intrusion detection in recent years. However, efficient and accurate intrusion detection remains a challenging issue due to: (a) limited sample number: only small sample size (or portion) of intrusive video frames is available; (b) high inter-scene dissimilarity: various railway track area scenes are captured by cameras installed in different landforms; (c) high intra-scene similarity: the video frames captured by an individual camera share a same background. In this paper, an efficient few-shot learning solution is developed to address the above issues. In particular, an enhanced model-agnostic meta-learner is trained using both the original video frames and segmented masks of track area extracted from the video. Moreover, theoretical analysis and engineering solutions are provided to cope with the highly similar video frames in the meta-model training phase. The proposed method is tested on realistic railway video dataset. Numerical results show that the enhanced meta-learner successfully adapts unseen scene with only few newly collected video frame samples, and its intrusion detection accuracy outperforms that of the standard randomly initialised supervised learning.

Electromagnetic source localization is a technique that enables the study of neural dynamical activities on a millisecond timescale using Magnetoencephalography (MEG) or Electroencephalography (EEG) data. It aims to reveal neural activities in the brain cortical region which cannot be seen with imaging methods that operate on a slower timescale such as fMRI. In this paper, we model the problem under a Bayesian multi-target tracking framework. A multi-target detection and particle filtering algorithm is developed to estimate the dipolar source dynamics, and a minimum norm (MN) based estimation method is incorporated to construct the birth-death move for the dynamical number of dipolar sources. The algorithm is tested using both simulated and experimental data1. The results demonstrate that the proposed algorithm performs better than that in previous works in terms of both localization accuracy and computational cost.

Node localization is an important task in the context of wire less sensor networks. Although algorithms already exist to carry out localization using measurements of received signal strength (RSS), none of these methods take into account the attenuating and scattering effects of objects which lie within or around the network and which therefore affect the obtained RSS measurements. In this paper, we use a map of the attenuations seen over a given area in order to reinterpret the measured RSS values, thereby refining RSS-based node localization and providing significant improvements over existing algorithms. Simulation results are presented to demonstrate the performance gains which can be achieved using our method.

Radio Frequency (RF) tomographic tracking is the process of tracking moving targets by analyzing changes of attenuation in wireless transmissions. This paper presents a novel sequential Monte Carlo (SMC) method for RF tomographic tracking of a single target using a wireless sensor network. The algorithm incorporates on-line Expectation Maximization (EM) to estimate model parameters. Based on experimental measurements, we introduce a new measurement model for the attenuation caused by a target. We assess performance through numerical simulation and demonstrate that it significantly outperforms previous RF tomographic tracking procedures.