Unsupervised action segmentation (UAS) aims to identify action boundaries in long, untrimmed videos without the use of annotations. This involves learning discriminative frame features and applying a segmentation mechanism to organize frames into coherent action segments. However, most of the common approaches ignore the importance of multi-scale temporal interactions within the video sequence, resulting in a limited frame representation capability and inaccurate action boundary detection. In this paper, we propose MulSclTE, a novel UAS framework that incorporates multi-scale temporal interactions across global, clip, and frame levels to enhance the overall performance. To address the limited representation capability, we first present global-level interaction enhancement by implementing a bi-directional temporal encoding mechanism, designed to capture comprehensive information across the entire sequence. Then, we devise a hierarchical self-supervised loss function equipped with a clip-level interaction constraint that aims to bring temporally adjacent clips closer while separating non-adjacent ones. To precisely identify action boundaries, we provide comprehensive information by integrating frame-level prediction errors and similarity scores to alleviate the under-segmentation issue, and present a refinement mechanism to mitigate the over-segmentation issue. Extensive experiments on Breakfast, YouTube Instructions, 50Salads, and EPIC-KITCHENS show that MulSclTE attains leading or second-best performance across all datasets, and even exceeds some supervised methods in MoF and F1 metrics, underscoring its robustness and effectiveness.

Text-conditioned motion generation has significant applications across various domains. However, generating natural motion remains challenging due to the vast solution space and the accumulation of errors during motion generation. To address these challenges, a novel hierarchical motion intention decoding-based motion synthesis model named Hi-Motion is proposed, which disentangles human motion into temporal intents of pivot joints and skeleton synthesis guided by intention from a new perspective. Specifically, Hi-Motion first parameterizes pivot joint motion with high-order Bézier curves and constructs a Bézier decoder to generate their trajectories, which serve as motion intention to guide skeleton generation. Secondly, we formulate the generation of skeletons as a graph node transformation problem under the condition of determined edge connections. By incorporating hierarchical joint motion intentions into the graph node features, the spatial details of each frame can be precisely synthesized. The proposed Hi-Motion effectively decouples motion generation into temporal and spatial dimensions through hierarchical motion intention decoding, ensuring coordination and naturalness in the generated motion. Extensive experiments on HumanML3D and KIT-ML datasets substantiate the motion generation capabilities of Hi-Motion. Further analysis demonstrates that Hi-Motion can accurately predict the motion intention of pivot joints and synthesize skeletal details.

Controlling the cyborg rat with the guidance of an unmanned aerial vehicle (UAV) is important yet challenging. UAV can provide a mobile bird’s eye view (BEV) with multiple shooting angles to expand the motion space of the cyborg rat. However, uncertainty is a critical problem arising from the obscure and blurry imaging from UAV and the locomotion willingness of cyborg rats. To solve this problem, we propose the collaborative Rat-UAV navigation (RUN) paradigm. The uncertainty in RUN is formulated using the partially observed Markov decision processes (POMDP) in a perceptual-control framework. First, perceptual uncertainties in rat pose estimation and environmental modeling are reduced. Second, control policies are delivered to maximize rewards and minimize control uncertainties until the cyborg rat reaches the target. To achieve this, RUN can accurately detect tiny rats with fewer parameters, dynamically plan paths with partial observations from the UAV, and stably control the rat using a locomotion willingness transition graph in a large practical arena. Our framework is tested in real-world experiments. For the first time, a cyborg rat successfully completed a navigation task with the guidance of a UAV.Note to Practitioners—This paper studies, for the first time, the problem of navigating a cyborg rat with the guidance of an unmanned aerial vehicle (UAV). Most existing researches on the autonomous navigation of cyborg rats are conducted in fully observed environments with fixed cameras, disregarding uncertainties in complex environmental perception and rat control. In this paper, we propose a collaborative Rat-UAV navigation (RUN) paradigm, which formulates uncertainties using the partially observed Markov decision processes (POMDP). Specifically, RUN enhances the cyborg rat’s interaction with the UAV in the navigation environment. First, it narrows the gap between the rat’s perceived and actual states by utilizing keypoint detection, dynamic environment modeling, and locomotion willingness estimation from the view of the UAV. Secondly, RUN constructs a policy generation module to maximize the reward of controlling the cyborg rat, based on a locomotion willingness transition graph. Experimental results demonstrate that our framework effectively guides the cyborg rat through a large maze, followed by six targets, with the assistance of the UAV. In future researches, we will investigate the real-time prediction of rat locomotion willingness using multimodal neuroethological recordings, towards a bidirectional brain computer interface between rats and UAVs.

Animal pose estimation is often constrained by the scarcity of annotations and the diversity of scenarios and species. The pseudo-label generation based unsupervised domain adaptation paradigm, which discriminates the predicted keypoints of unlabeled data based on the skeleton position consistency, has demonstrated effectiveness for such problems. However, existing methods generate pseudo-labels with massive false positives, because they cannot effectively distinguish sample pairs with the same errors. In this study, we propose a cross-domain animal pose estimation model from a novel perspective of skeleton anomaly learning. We construct a graph contrastive learning mechanism to acquire the skeleton anomaly-aware knowledge, which enables the generation of accurate pseudo-labels for target domain and imposes graph constraint on unlabeled data. And a skeleton anomaly-feedback based domain adaptation framework is designed to facilitate implicit alignment of object-specific features and joint training of cross-domain. Besides, we propose a novel rat pose dataset named UDARP-9.4K to address the gap of small-sized animal pose datasets encompassing diverse experimental scenarios. The related datasets are reviewed and evaluated in detail. Extensive experiments are conducted on UDARP-9.4K and two public datasets to demonstrate the superiority of the proposed model in cross-scenarios and cross-species animal pose estimation tasks. Further analysis reveals the effectiveness of the proposed model for skeleton structure feature learning.The UDARP-9.4K dataset is available here.

Channel attention mechanisms endeavor to recalibrate channel weights to enhance representation abilities of networks. However, mainstream methods often rely solely on global average pooling as the feature squeezer, which significantly limits the overall potential of models. In this paper, we investigate the statistical moments of feature maps within a neural network. Our findings highlight the critical role of high-order moments in enhancing model capacity. Consequently, we introduce a flexible and comprehensive mechanism termed Extensive Moment Aggregation (EMA) to capture the global spatial context. Building upon this mechanism, we propose the Moment Channel Attention (MCA) framework, which efficiently incorporates multiple levels of moment-based information while minimizing additional computation costs through our Cross Moment Convolution (CMC) module. The CMC module via channel-wise convolution layer to capture multiple order moment information as well as cross channel features. The MCA block is designed to be lightweight and easily integrated into a variety of neural network architectures. Experimental results on classical image classification, object detection, and instance segmentation tasks demonstrate that our proposed method achieves state-of-the-art results, outperforming existing channel attention methods.

Curvilinear structures, which include line-like continuous objects, are fundamental geometrical elements in image-based applications. Reconstructing these structures from images constitutes a pivotal research area in computer vision. However, the complex topology and ambiguous image evidence render this process a challenging task. In this paper, we introduce DeepBranchTracer, a novel method that learns both external image features and internal geometric characteristics to reconstruct curvilinear structures. Firstly, we formulate the curvilinear structures extraction as a geometric attribute estimation problem. Then, a curvilinear structure feature learning network is designed to extract essential branch attributes, including the image features of centerline and boundary, and the geometric features of direction and radius. Finally, utilizing a multi-feature fusion tracing strategy, our model iteratively traces the entire branch by integrating the extracted image and geometric features. We extensively evaluated our model on both 2D and 3D datasets, demonstrating its superior performance over existing segmentation and reconstruction methods in terms of accuracy and continuity.

Accurately parsing (i.e., segmenting and recognizing) muscles of freely-moving animals such as Drosophila larva in light-sheet fluorescence microscopy images is necessary to study the relationship between muscle activity and animal motions. However, this task is challenging due to the large inter-class similarity and intra-class variance of muscles, as well as the in-homogeneous intensity and blurred boundaries of neighboring muscles. Existing semantic and instance segmentation methods cannot effectively overcome these challenges, resulting in poor segmentation and unreliable classification. In this work, we propose a novel framework named MuscleParseNet that explicitly utilizes sequential and spatial contexts to address these challenges. MuscleParseNet contains a deformable muscle candidate detector (D-CMD) to detect candidate muscles, and a sequential and spatial context-based fine muscle parser (SS-FMP) to refine the candidates. D-CMD boosts Mask RCNN with deformable convolutions to capture shape variations for more accurate muscle segmentation. Moreover, SS-FMP re-classifies the detected candidates by establishing a global spatial context to explicitly reflect spatial relative location, then optimizes the classification using the sequential associations of candidates in adjacent frames, which significantly improves muscle recognition accuracy. Experiments on the synchronized muscle-motion dataset of nearly freely-moving larvae show that MuscleParseNet produces promising results, outperforming state-of-the-art semantic and instance segmentation methods.

Currently, three-way decision with neighborhood rough sets (3WDNRS) is widely used in many fields. The core of 3WDNRS is to calculate threshold pairs to divide a neighborhood space into three pair-wise disjoint regions. The majority of research on 3WDNRS mainly aims to calculate thresholds with the given risk parameters to minimize the misclassification cost. However, in practical applications, risk parameters are often subjectively determined based on expert experience. This makes it challenging to accurately obtain the thresholds in 3WDNRS. To solve this problem, fuzziness is introduced into 3WDNRS to provide a new perspective on 3WD theory. First, a shadowed set framework is constructed, named three-way approximations based on shadowed sets (3WA-SS). Based on 3WA-SS, a datadriven adapted neighborhood (DAN) is constructed. Then, an improved fuzziness-based 3WDNRS (F′ -3WDNRS) is further proposed and optimized by minimizing uncertainty change to obtain more reasonable threshold pair based on DAN. Finally, extensive experiments are conducted on our proposed model, and the results show that F′ -3WDNRS is effective and reliable for making decisions.

People live in a 3D world. However, existing works on person re-identification (re-id) mostly consider the semantic representation learning in a 2D space, intrinsically limiting the understanding of people. In this work, we address this limitation by exploring the prior knowledge of the 3D body structure. Specifically, we project 2D images to a 3D space and introduce a novel parameter-efficient omni-scale graph network (OG-Net) to learn the pedestrian representation directly from 3D point clouds. OG-Net effectively exploits the local information provided by sparse 3D points and takes advantage of the structure and appearance information in a coherent manner. With the help of 3D geometry information, we can learn a new type of deep re-id feature free from noisy variants, such as scale and viewpoint. To our knowledge, we are among the first attempts to conduct person re-id in the 3D space. We demonstrate through extensive experiments that the proposed method: (1) eases the matching difficulty in the traditional 2D space; 2) exploits the complementary information of 2D appearance and 3D structure; 3) achieves competitive results with limited parameters on four large-scale person re-id datasets; and 4) has good scalability to unseen datasets. Our code, models, and generated 3D human data are publicly available at https://github.com/layumi/person-reid-3d.

Existing machine learning (ML) models are often fragile in open environments because the data distribution frequently shifts. To address this problem, domain generalization (DG) aims to explore underlying invariant patterns for stable prediction across domains. In this work, we first characterize that this failure of conventional ML models in DG attributes to an inadequate identification of causal structures. We further propose a novel invariant Directed Acyclic Graph (dubbed iDAG) searching framework that attains an invariant graphical relation as the proxy to the causality structure from the intrinsic data-generating process. To enable tractable computation, iDAG solves a constrained optimization objective built on a set of representative class-conditional prototypes. Additionally, we integrate a hierarchical contrastive learning module, which poses a strong effect of clustering, for enhanced prototypes as well as stabler prediction. Extensive experiments on the synthetic and real-world benchmarks demonstrate that iDAG outperforms the state-of-the-art approaches, verifying the superiority of causal structure identification for DG. The code of iDAG is available at https://github.com/lccurious/iDAG.

When deep learning is applied to cervical cell screening, it encounters challenges related to the inefficient labeling of data and a shortage of annotators. To mitigate these challenges, this study introduces AL-Annotator, an active learning-based cervical cell annotation system. This system consists of three key components: an interactive interface designed for efficient pathologists’ review, a front-end and back-end separated cervical cell annotation system, and an active learning-based annotation method. Compared to traditional classification-based annotation methods, our annotation system achieves a 17.74% reduction in annotation quantity while improving the accuracy by 1.92%. This annotation system is crucial for developing and testing new machine learning and artificial intelligence algorithms for cervical screening, thereby facilitating advancements in diagnostic tools and methodologies.

With the rapid advancement of artificial intelligence, the complexity and depth of deep neural networks continues to grow, placing higher demands on computational power. To meet these requirements, various manufacturers have developed specialized computing processors for the training process of deep learning, such as Huawei’s Ascend Neural Processing Unit (NPU). In order to fully leverage the capabilities of the Ascend NPU, Huawei has introduced the MindSpore deep learning framework. The computational performance of deep learning frameworks plays a critical role for developers. However, there is a lack of comprehensive research on the analysis of performance and optimization in MindSpore framework on the Ascend NPU, leading to a scarcity of relevant references for deep learning development utilizing MindSpore on the Ascend NPU. To address this gap, this study examined the performance and optimization of MindSpore on Ascend NPUs through detailed experiments involving diverse workloads and multiple analysis metrics. The analysis is conducted at three levels: operations, models, and techniques, investigating the operations configurations, performance bottleneck, along with techniques selections, within the training process of DNNs. Consequently, this study offers significant insights and guidance for DL researchers and practitioners using MindSpore on Ascend NPUs.

The development of deep learning depends on the support of deep learning processing units and frameworks. HUAWEI's Ascend Neural Processing Unit (Ascend NPU), as a chip designed specifically for neural network computation acceleration, not only provides support for its self-developed framework MindSpore, but also provides adaptation for PyTorch. However, there is a lack of comparative evaluation research on MindSpore and other frameworks on Ascend NPU, making it difficult to understand their actual performance. Therefore, comprehensive evaluation experiments in both model level and operator level on MindSpore and PyTorch based on Ascend NPU are conducted in this paper, analyzing and comparing the performance of the two frameworks. With the conclusions analyzed on the model evaluation experiments and operator evaluation experiments, we provide references and suggestions for deep learning researchers and practitioners in terms of framework selection and optimization strategies for Ascend NPU.

Understanding the dynamics of time series data typically requires identifying the unique latent factors for data generation, a.k.a., latent processes identification. Driven by the independent assumption, existing works have made great progress in handling single-view data. However, it is a non-trivial problem that extends them to multi-view time series data because of two main challenges: (i) the complex data structure, such as temporal dependency, can result in violation of the independent assumption; (ii) the factors from different views are generally overlapped and are hard to be aggregated to a complete set. In this work, we propose a novel framework MuLTI that employs the contrastive learning technique to invert the data generative process for enhanced identifiability. Additionally, MuLTI integrates a permutation mechanism that merges corresponding overlapped variables by the establishment of an optimal transport formula. Extensive experimental results on synthetic and real-world datasets demonstrate the superiority of our method in recovering identifiable latent variables on multi-view time series. The code is available on https://github.com/lccurious/MuLTI.

In the field of pose estimation, keypoint representations can take the form of Gaussian heatmaps, classification vectors, or direct coordinates. However, the current networks suffer from a lack of consistency with these keypoint representations. They only accommodate these representations in the final layer, resulting in suboptimal efficiency and requiring a high number of parameters or computational resources. In this paper, we propose a simple yet efficient plug-and-play module, named the Implicit Decouple Module (IDM), which decouples features into two parts along the x-y axes and aggregates features in a direction-aware manner. This approach implicitly fuses direction-specific coordinate information, improving the consistency with the keypoint representations, especially in vector form. Furthermore, we introduce a fully convolutional backbone network, named the Implicit Decouple Network (IDN), which incorporates IDM without the need to maintain high-resolution features, dense multi-level feature fusion, or lots of repeated stages, while still achieving high performance. In experiments on the COCO dataset, our basic IDN without pre-training can outperform HRNet (28.5M) by 2.4 AP with 18.2M parameters, and even surpass some transformer-based methods. In the lightweight model scenario, our model outstrips Lite-HRNet by 3.9 AP with only 2.5M parameters. We also evaluate our model on the person instance segmentation task and other datasets, demonstrating its generality and effectiveness. http(s)://znk.ink/su/mm23idn.

The deformability and high degree of freedom of mollusks bring challenges in mathematical modeling and synthesis of motions. Traditional analytical and statistical models are limited by either rigid skeleton assumptions or model capacity, and have difficulty in generating realistic and multi-pattern mollusk motions. In this work, we present a large-scale dynamic pose dataset of Drosophila larvae and propose a motion synthesis model named Path2Pose to generate a pose sequence given the initial poses and the subsequent guiding path. The Path2Pose model is further used to synthesize long pose sequences of various motion patterns through a recursive generation method. Evaluation analysis results demonstrate that our novel model synthesizes highly realistic mollusk motions and achieves state-of-the-art performance. Our work proves high performance of deep neural networks for mollusk motion synthesis and the feasibility of long pose sequence synthesis based on the customized body shape and guiding path.

Domain adaptation methods reduce domain shift typically by learning domain-invariant features. Most existing methods are built on distribution matching, e.g., adversarial domain adaptation, which tends to corrupt feature discriminability. In this paper, we propose Discriminative Radial Domain Adaptation (DRDA) which bridges source and target domains via a shared radial structure. It's motivated by the observation that as the model is trained to be progressively discriminative, features of different categories expand outwards in different directions, forming a radial structure. We show that transferring such an inherently discriminative structure would enable to enhance feature transferability and discriminability simultaneously. Specifically, we represent each domain with a global anchor and each category a local anchor to form a radial structure and reduce domain shift via structure matching. It consists of two parts, namely isometric transformation to align the structure globally and local refinement to match each category. To enhance the discriminability of the structure, we further encourage samples to cluster close to the corresponding local anchors based on optimal-transport assignment. Extensively experimenting on multiple benchmarks, our method is shown to consistently outperforms state-of-the-art approaches on varied tasks, including the typical unsupervised domain adaptation, multi-source domain adaptation, domain-agnostic learning, and domain generalization.

Recent years have witnessed a flurry of research activity in graph matching, which aims at finding the correspondence of nodes across two graphs and lies at the heart of many artificial intelligence applications. However, matching heterogeneous graphs with partial overlap remains a challenging problem in real-world applications. This paper proposes the first practical learning-to-match method to meet this challenge. The proposed unsupervised method adopts a novel partial optimal transport paradigm to learn a transport plan and node embeddings simultaneously. In a from-one-to-all manner, the entire learning procedure is decomposed into a series of easy-to-solve sub-procedures, each of which only handles the alignment of a single type of nodes. A mechanism for searching the transport mass is also proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art graph matching methods.

Reconstructing neuron morphologies from fluorescence microscope images plays a critical role in neuroscience studies. It relies on image segmentation to produce initial masks either for further processing or final results to represent neuronal morphologies. This has been a challenging step due to the variation and complexity of noisy intensity patterns in neuron images acquired from microscopes. Whereas progresses in deep learning have brought the goal of accurate segmentation much closer to reality, creating training data for producing powerful neural networks is often laborious. To overcome the difficulty of obtaining a vast number of annotated data, we propose a novel strategy of using two-stage generative models to simulate training data with voxel-level labels. Trained upon unlabeled data by optimizing a novel objective function of preserving predefined labels, the models are able to synthesize realistic 3D images with underlying voxel labels. We showed that these synthetic images could train segmentation networks to obtain even better performance than manually labeled data. To demonstrate an immediate impact of our work, we further showed that segmentation results produced by networks trained upon synthetic data could be used to improve existing neuron reconstruction methods.

Although recent work has achieved great progress in human pose estimation (HPE), most methods show limitations in either inference speed or accuracy. In this paper, we propose a fast and accurate end-to-end HPE method, which is specifically designed to overcome the commonly encountered jitter box, defective box and ambiguous box problems of box-based methods, e.g. Mask R-CNN. Concretely, 1) we propose the ROIGuider to aggregate box instance features from all feature levels under the guidance of global context instance information. Further, 2) the proposed Center Line Branch is equipped with a Dichotomy Extended Area algorithm to adaptively expand each instance box area, and Ambiguity Alleviation strategy to eliminate duplicated keypoints. Finally, 3) to achieve efficient multi-scale feature fusion and real-time inference, we design a novel Trapezoidal Network (TNet) backbone. Experimenting on the COCO dataset, our method achieves 68.1 AP at 25.4 fps, and outperforms Mask-RCNN by 8.9 AP at a similar speed. The competitive performance on the HPE and person instance segmentation tasks over the state-of-the-art models show the promise of the proposed method. The source code will be made available at https://github.com/zlcnup/CGANet.

Recently, a large number of channel attention blocks are proposed to boost the representational power of deep convolutional neural networks (CNNs). These approaches commonly learn the relationship between global contexts and attention activations by fully-connected layers or linear transformations. However, we empirically find that though many parameters are introduced, these attention blocks may not learn the relationship well. In this paper, we hypothesize that the relationship is predetermined. Based on this hypothesis, we propose a simple yet extremely efficient channel attention block, called Gaussian Context Transformer (GCT), which achieves contextual feature excitation using a Gaussian function that satisfies the presupposed relation-ship. According to whether the standard deviation of the Gaussian function is learnable, we develop two versions of GCT: GCT-B0 and GCT-B1. GCT-B0 is a parameter-free channel attention block by fixing the standard deviation. It directly maps global contexts to attention activations with-out learning. In contrast, GCT-B1 is a parameterized version, which adaptively learns the standard deviation to enhance the mapping ability. Extensive experiments on ImageNet and MS COCO benchmarks demonstrate that our GCTs lead to consistent improvements across various deep CNNs and detectors. Compared with a bank of state-of-the-art channel attention blocks, such as SE [17] and ECA [42], our GCTs are superior in effectiveness and efficiency.

A variety of computer vision tasks benefit significantly from increasingly powerful deep convolutional neural networks. However, the inherently local property of convolution operations prevents most existing models from capturing long-range feature interactions for improved performances. In this paper, we propose a novel module, called Spatially-Aware Context (SAC) block, to learn spatially-aware contexts by capturing multi-mode global contextual semantics for sophisticated long-range dependencies modeling. We enable customized non-local feature interactions for each spatial position through re-weighted global context fusion in a non-normalized way. SAC is very lightweight and can be easily plugged into popular backbone models. Extensive experiments on COCO, ImageNet, and HICO-DET benchmarks show that our SAC block achieves significant performance improvements over existing baseline architectures while with a negligible computational burden increase. The results also demonstrate the exceptional effectiveness and scalability of the proposed approach on capturing long-range dependencies for object detection, segmentation, and image classification, outperforming a bank of state-of-the-art attention blocks.

Squeeze-and-Excitation (SE) block presents a channel attention mechanism for modeling global context via explicitly capturing dependencies across channels. However, we are still far from understanding how the SE block works. In this work, we first revisit the SE block, and then present a detailed empirical study of the relationship between global context and attention distribution, based on which we propose a simple yet effective module, called Linear Context Transform (LCT) block. We divide all channels into different groups and normalize the globally aggregated context features within each channel group, reducing the disturbance from irrelevant channels. Through linear transform of the normalized context features, we model global context for each channel independently. The LCT block is extremely lightweight and easy to be plugged into different backbone models while with negligible parameters and computational burden increase. Extensive experiments show that the LCT block outperforms the SE block in image classification task on the ImageNet and object detection/segmentation on the COCO dataset with different backbone models. Moreover, LCT yields consistent performance gains over existing state-of-the-art detection architectures, e.g., 1.5∼1.7% APbbox and 1.0%∼1.2% APmask improvements on the COCO benchmark, irrespective of different baseline models of varied capacities. We hope our simple yet effective approach will shed some light on future research of attention-based models.

In this paper, a novel stratified sampling strategy is designed to accelerate the mini-batch SGD. We derive a new iteration-dependent surrogate which bound the stochastic variance from above. To keep the strata minimizing this surrogate with high probability, a stochastic stratifying algorithm is adopted in an adaptive manner, that is, in each iteration, strata are reconstructed only if an easily verifiable condition is met. Based on this novel sampling strategy, we propose an accelerated mini-batch SGD algorithm named SGD-RS. Our theoretical analysis shows that the convergence rate of SGD-RS is superior to the state-of-the-art. Numerical experiments corroborate our theory and demonstrate that SGD-RS achieves at least 3.48-times speed-ups compared to vanilla minibatch SGD.

In machine learning research, many emerging applications can be (re)formulated as the composition optimization problem with nonsmooth regularization penalty. To solve this problem, traditional stochastic gradient descent (SGD) algorithm and its variants either have low convergence rate or are computationally expensive. Recently, several stochastic composition gradient algorithms have been proposed, however, these methods are still inefficient and not scalable to large-scale composition optimization problem instances. To address these challenges, we propose an asynchronous parallel algorithm, named Async-ProxSCVR, which effectively combines asynchronous parallel implementation and variance reduction method. We prove that the algorithm admits the fastest convergence rate for both strongly convex and general nonconvex cases. Furthermore, we analyze the query complexity of the proposed algorithm and prove that linear speedup is accessible when we increase the number of processors. Finally, we evaluate our algorithm Async-ProxSCVR on two representative composition optimization problems including value function evaluation in reinforcement learning and sparse mean-variance optimization problem. Experimental results show that the algorithm achieves significant speedups and is much faster than existing compared methods.

Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local features and their multi-mode statistics. We show that the learned local feature patterns are more generic and transferable and a further local feature distribution matching enables fine-grained feature alignment. In this paper, we present a method for learning domain-invariant local feature patterns and jointly aligning holistic and local feature statistics. Comparisons to the state-of-the-art unsupervised domain adaptation methods on two popular benchmark datasets demonstrate the superiority of our approach and its effectiveness on alleviating negative transfer.

Bioimage Informatics is a growing area that aims to extract biological knowledge from microscope images of biomedical samples automatically. Its mission is vastly challenging, however, due to the complexity of diverse imaging modalities and big scales of multi-dimensional images. One major challenge is automatic image segmentation, an essential step towards high-level modeling and analysis. While progresses in deep learning have brought the goal of automation much closer to reality, creating training data for producing powerful neural networks is often laborious. To provide a shortcut for this costly step, we propose a novel two-stage generative model for simulating voxel level training data based on a specially designed objective function of preserving foreground labels. Using segmenting neurons from LM (Light Microscopy) image stacks as a testing example, we showed that segmentation networks trained by our synthetic data were able to produce satisfactory results. Unlike other simulation methods available in the field, our method can be easily extended to many other applications because it does not involve sophisticated cell models and imaging mechanisms.

Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the input features, due to the unavailability of target domain labels. We show that domain shift may still exist via label distribution shift at the classifier, thus deteriorating model performances. To alleviate this issue, we propose an approximate joint distribution matching scheme by exploiting prediction uncertainty. Specifically, we use a Bayesian neural network to quantify prediction uncertainty of a classifier. By imposing distribution matching on both features and labels (via uncertainty), label distribution mismatching in source and target data is effectively alleviated, encouraging the classifier to produce consistent predictions across domains. We also propose a few techniques to improve our method by adaptively reweighting domain adaptation loss to achieve nontrivial distribution matching and stable training. Comparisons with state of the art unsupervised domain adaptation methods on three popular benchmark datasets demonstrate the superiority of our approach, especially on the effectiveness of alleviating negative transfer.

Cyborg insects have attracted great attention as the flight performance they have is incomparable by micro aerial vehicles and play a critical role in supporting extensive applications. Approaches to construct cyborg insects consist of two major issues: 1) the stimulating paradigm and 2) the control policy. At present, most cyborg insects are constructed based on invasive methods, requiring the implantation of electrodes into neural or muscle systems, which would harm the insects. As the control policy is basically manual control, the shortcomings of which lie in the requirement of excessive amount of experiments and focused attention. This paper presents the design and implementation of a noninvasive and much safer cyborg insect system based on visual stimulation. The tethered paradigm is adopted here and we look at controlling the flight behavior of bumblebees, especially the abdominal-waving behavior, in the context of a model-free reinforcement learning problem. The problem is formulated as a finite and deterministic Markov decision process, where the agent is designed to change the abdominal-waving behavior from the initial state to the target state. Sarsa with transformed reward function which can speed up the learning process is employed to learn the optimal control policy. Learned policies are compared to the stochastic one by evaluating the results of ten bumblebees, demonstrating that abdominal-waving state can be modulated to approximate the target state quickly with small deviation.

In recent years, skeleton based action recognition is becoming an increasingly attractive alternative to existing video-based approaches, beneficial from its robust and comprehensive 3D information. In this paper, we explore an unsupervised representation learning approach for the first time to capture the long-term global motion dynamics in skeleton sequences. We design a conditional skeleton inpainting architecture for learning a fixed-dimensional representation, guided by additional adversarial training strategies. We quantitatively evaluate the effectiveness of our learning approach on three well-established action recognition datasets. Experimental results show that our learned representation is discriminative for classifying actions and can substantially reduce the sequence inpainting errors.

This letter proposes an efficient algorithmic solution to diagnose multiple mechanical faults in wind turbine gearbox through source number estimation using an empirical mode decomposition (EMD) and singular value decomposition (SVD) joint approach, and source signal recovery based on short-time Fourier transform (STFT), fuzzy C-means clustering and l1 norm decomposition. The effectiveness of the solution is validated using real wind turbine measurements under multifault scenarios.

The matrix completion problem is restoring a given matrix with missing entries when handling incomplete data. In many existing researches, rank minimization plays a central role in matrix completion. In this paper, noticing that the locally linear reconstruction can be used to approximate the missing entries, we view the problem from a new perspective and propose an algorithm called locally linear approximation (LLA). The LLA method tries to keep the local structure of the data space while restoring the missing entries from row angle and column angle simultaneously. The experimental results have demonstrated the effectiveness of the proposed method.

Attribute selection is considered as the most characteristic result in rough set theory to distinguish itself to other theories. However, existing attribute selection approaches can not handle partially labeled data. So far, few studies on attribute selection in partially labeled data have been conducted. In this paper, the concept of discernibility pair based on rough set theory is raised to construct a uniform measure for the attributes in both supervised framework and unsupervised framework. Based on discernibility pair, two kinds of semisupervised attribute selection algorithm based on rough set theory are developed to handle partially labeled categorical data. Experiments demonstrate the effectiveness of the proposed attribute selection algorithms.

Electronic textiles are regarded as one of the most important computation platforms for future computer-assisted health-monitoring applications. In these novel systems, multiple batteries are used in order to prolong their operational lifetime, which is a significant metric for system usability. However, due to the nonlinear features of batteries, computing systems with multiple batteries cannot achieve the same battery efficiency as those powered by a monolithic battery of equal capacity. In this paper, we propose an algorithm aiming to maximize battery efficiency globally for the computer-assisted health-care systems with multiple batteries. Based on an accurate analytical battery model, the concept of weighted battery fatigue degree is introduced and the novel battery-scheduling algorithm called predicted weighted fatigue degree least first (PWFDLF) is developed. Besides, we also discuss our attempts during search PWFDLF: a weighted round-robin (WRR) and a greedy algorithm achieving highest local battery efficiency, which reduces to the sequential discharging policy. Evaluation results show that a considerable improvement in battery efficiency can be obtained by PWFDLF under various battery configurations and current profiles compared to conventional sequential and WRR discharging policies.

Electronic textiles (e-textiles), known as computational fabrics, offer an emerging platform for constructing ambient intelligent applications. Computational nodes in e-textiles are driven by batteries. Unlike wireless sensor networks, not each computational node in e-textiles has its own battery. Instead, many computational nodes in e-textiles share a battery. Existing e-textiles use one fixed battery to drive a fixed set of computation nodes (or power consuming electronic components). The fixed battery-component connection may result in electronic components stopping functioning and/or energy waste in batteries when link connection problems occur. In this paper, we propose a new infrastructure of the power networks for e-textiles: flexible power network (FPN). Under the FPN infrastructure, a power consuming node (PCN) is not just connected to one single fixed battery. Instead, it is connected to multiple batteries and can obtain power energy from one of the available battery nodes (BNs) with the help of a battery selector. The electrical features of battery selectors and overcurrent protectors that protect the batteries from wasting the charge when short-circuit faults occur are illustrated. Moreover, by modeling the number of fault occurrence at conductive wires and nodes stochastically, an evaluation algorithm is proposed to analyze the reliability of FPN and to compare the metrics of different design schemes under the perspective of both the BNs and the PCNs. Experimental results show that our FPN is more dependable than some common e-textile electric networks published before with the occurrence of short- and/or open-circuit faults.

An increasing number of real-time applications like railway signaling control systems and medical electronics systems require high quality of security to assure confidentiality and integrity of information. Therefore, it is desirable and essential to fulfill security requirements in security-critical real-time systems. This paper addresses the issue of optimizing quality of security in real-time systems. To meet the needs of a wide variety of security requirements imposed by real-time systems, a group-based security service model is used in which the security services are partitioned into several groups depending on security types. While services within the same security group provide the identical type of security service, the services in the group can achieve different quality of security. Security services from a number of groups can be combined to deliver better quality of security. In this study, we seamlessly integrate the group-based security model with a traditional real-time scheduling algorithm, namely earliest deadline first (EDF). Moreover, we design and develop a security-aware EDF schedulability test. Given a set of real-time tasks with chosen security services, our scheduling scheme aims at optimizing the combined security value of the selected services while guaranteeing the schedulability of the real-time tasks. We study two approaches to solve the security-aware optimization problem. Experimental results show that the combined security values are substantially higher than those achieved by alternatives for real-time tasks without violating real-time constraints.

Electronic textiles, known as computational fabrics, offer an emerging method for constructing wearable and large area applications. Because e-textiles are battery-driven and fault-prone systems, there is a need for developing a dependable infrastructure of the electric networks for e-textiles. In this paper, a new infrastructure of the power networks for e-textiles, flexible power network (FPN), is presented. Instead of drawing power from a fixed battery as in the conventional electric networks, the power consuming nodes in a FPN can obtain power energy from one of the choices of batteries available with the help of the battery selectors. We also introduce the over current protectors into the battery nodes (BN) to protect the batteries from wasting the charge when short-circuit faults occur. The electric features of battery selectors and over current protectors, the two types of important electric devices used in FPNs, are illustrated in the paper. We have performed simulation experiments and the results show that our FPNs are more dependable than some common electric networks published before in the cases of short- and open-circuit faults.