We study Policy-extended Value Function Approximator (PeVFA) in Reinforcement Learning (RL), which extends conventional value function approximator (VFA) to take as input not only the state (and action) but also an explicit policy representation. Such an extension enables PeVFA to preserve values of multiple policies at the same time and brings an appealing characteristic, i.e., value generalization among policies. We formally analyze the value generalization under Generalized Policy Iteration (GPI). From theoretical and empirical lens, we show that generalized value estimates offered by PeVFA may have lower initial approximation error to true values of successive policies, which is expected to improve consecutive value approximation during GPI. Based on above clues, we introduce a new form of GPI with PeVFA which leverages the value generalization along policy improvement path. Moreover, we propose a representation learning framework for RL policy, providing several approaches to learn effective policy embeddings from policy network parameters or state-action pairs. In our experiments, we evaluate the efficacy of value generalization offered by PeVFA and policy representation learning in several OpenAI Gym continuous control tasks. For a representative instance of algorithm implementation, Proximal Policy Optimization (PPO) re-implemented under the paradigm of GPI with PeVFA achieves about 40% performance improvement on its vanilla counterpart in most environments.

Learning to collaborate is critical in Multi-Agent Reinforcement Learning (MARL). Previous works promote collaboration by maximizing the correlation of agents’ behaviors, which is typically characterized by Mutual Information (MI) in different forms. However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. PMIC uses a new collaboration criterion measured by the MI between global states and joint actions. Based on this criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating better collaborations while avoiding falling into sub-optimal ones. Experiments on a wide range of MARL benchmarks show the superior performance of PMIC compared with other algorithms.

Practicing gymnastics activities at home with online resources has become an increasingly popular choice due to its convenience and accessibility. However, without face-to-face guidance by a trainer, a major challenge is how to assess the quality of performed gymnastics activities, effectively and fairly. Existing intrusive assessing approaches usually require live cameras or wearable sensors, which usually generate privacy and feasibility concerns. There is a lacking of accurate approaches to assess the quality of the activities. To address these challenges, a gymnastics activity assessment approach is proposed in this article, and Wi-Gym, an effective first-of-its-kind gymnastics activity assessment system is developed utilizing commodity Wi-Fi. Wi-Gym is designed to compare the activity-induced channel state information (CSI) dynamics by an exerciser and that of a trainer utilizing dynamic time warping (DTW). The comparison results are provided by a fuzzy inference system (FIS). To make Wi-Gym robust to the changes in the environment, domain adaptation is leveraged to mitigate the data distribution imbalance caused by the environment changes. Extensive experimental studies have been conducted using Wi-Gym, acoustic, and video-based sensing systems. The experimental results validate the effectiveness and robustness of the proposed approach.

Change detection and coregistration problems, essential tasks in remote sensing image processing, can identify the differences in images of the same scene (e.g., buildings or forests) captured at two distinct time points. These images may be detected by the same or different sensors. Recently, new methods have been developed for such detection tasks, including fully convolutional networks and attention mechanisms, with favorable results. However, in fully convolutional networks, some useless information may be extracted from the multidimensional information of an image. Real-time processing is difficult because of location discrepancies, the large scale of images, and the fusion cropping problem; thus, change detection may be time-consuming and require a substantial amount of hardware resources. In our study, a novel network model was designed to overcome these problems by using high- and low-frequency image information for feature extraction. By using similarity and homography measurements, our proposed system solved the change detection and coregistration problems efficiently. A batch change detection dataset and an established discrepancy dataset were used for evaluation. The results revealed that the proposed method achieved high accuracy in detecting changes in real-time.

Abstract Line drawing with colorization is a popular art format and tool for architectural illustration. The goal of this research is toward generating a high-quality and natural-looking colorization based on an architectural line drawing. This paper presents a new Generative Adversarial Network (GAN)-based method, named ArchGANs, including ArchColGAN and ArchShdGAN. ArchColGAN is a GAN-based line-feature-aware network for stylized colorization generation. ArchShdGAN is a lighting effects generation network, from which the building depiction in 3D can benefit. In particular, ArchColGAN is able to maintain the important line features and the correlation property of building parts as well as reduce the uneven colorization caused by sparse lines. Moreover, we proposed a color enhancement method to further improve ArchColGAN. Besides the single line drawing images, we also extend our method to handle line drawing image sequences and achieve rotation animation. Experiments and studies demonstrate the effectiveness and usefulness of our proposed method for colorization prototyping.

Abstract Multi‐dimensional data exploration is a classic research topic in visualization. Most existing approaches are designed for identifying record patterns in dimensional space or subspace. In this paper, we propose a visual analytics approach to exploring subset patterns. The core of the approach is a subset embedding network (SEN) that represents a group of subsets as uniformly‐formatted embeddings. We implement the SEN as multiple subnets with separate loss functions. The design enables to handle arbitrary subsets and capture the similarity of subsets on single features, thus achieving accurate pattern exploration, which in most cases is searching for subsets having similar values on few features. Moreover, each subnet is a fully‐connected neural network with one hidden layer. The simple structure brings high training efficiency. We integrate the SEN into a visualization system that achieves a 3‐step workflow. Specifically, analysts (1) partition the given dataset into subsets, (2) select portions in a projected latent space created using the SEN, and (3) determine the existence of patterns within selected subsets. Generally, the system combines visualizations, interactions, automatic methods, and quantitative measures to balance the exploration flexibility and operation efficiency, and improve the interpretability and faithfulness of the identified patterns. Case studies and quantitative experiments on multiple open datasets demonstrate the general applicability and effectiveness of our approach.

Architectural illustration using line drawing with colorization is an important tool and art format. In this paper, in order to generate a natural-looking and high quality watercolorlike colorization for architectural line drawing, we propose a novel Generative Adversarial Network (GAN) approach, namely ArchGANs. The proposed ArchGANs unifies a line-feature-aware stylized colorization network (ArchColGAN), which can learn, predict and generate the coloring based on a dataset, as well as a shading generation network (ArchShdGAN), which augments the illustration with controllable lighting effects for better depicting building in 3D. Specifically, ArchColGAN can preserve the essential line features and building part correlation property, it also tackles the uneven colorization problem caused by the sparse lines. Experimental results demonstrate our proposed method is effective and suitable for colorization prototyping.

Abstract Analyzing the geometric and semantic properties of 3D point cloud data via the deep learning networks is still challenging due to the irregularity and sparsity of samplings of their geometric structures. In our study, the authors combine the advantage of voxels and point clouds by presenting a new data form of voxel models, called Layer‐Ring data. This data type can retain the fine description of the 3D data, and keep the high efficiency of feature extraction. After that, based on the Layer‐Ring data, a modern network architecture, called VoxPoint Annular Network (VAN), works on the Layer‐Ring data for the feature extraction and object category prediction. The design idea is based on the edge‐extraction and the coordinate representation for each voxel on the separated layer With the flexible design, our proposed VAN can adapt to the layer's geometric variability and scalability Finally, the extensive experiments and comparisons demonstrate that our approach obtained the notable results with the state‐of‐the‐art methods on a variety of standard benchmark datasets (e.g., ModelNet10, ModelNet40). Moreover, the tests also proved that 3D shape features could learn efficiently and robustly. All relevant codes will be available at https://github.com/helloFionaQ/Vox‐PointNet .

Mobile devices have become an enormous platform for augmented reality (AR) technology to meet the need of users to experience the charm of AR technology in anytime and anywhere. However, the development of mobile AR technology is currently in a dilemma. Both extensibility and real-time are often challenging to be taken into account. In our study, we explored cloud-based solutions and designed two systems, S-MARC, and D-MARC. The first system boldly tried the idea of cloud technology supporting mobile AR and explored more possibilities for combining location information. The cloud-based mobile AR design optimization scheme is proposed in our study details. Besides, we invited some users to carry out experimental studies, and their user evaluations showed that the mobile AR cloud solution was implemented in both systems and received excellent feedback in the review. The results show that the proposed two schemes are superior to others in the performance and efficiency, and help to improve the current dilemma of mobile AR and build up users' confidence in the application of mobile AR.

Osteoporosis is the most common reason that causes the fracture among the elderly. For the purpose of convenience and safety, 2D texture analysis has been used to diagnose osteoporosis. In this study, a supervised method using proposed texture features to identify osteoporotic cases from healthy was proposed. We designed two groups of new features, Encoded GLCM and Encoded LBP, each of which contains two subgroups through encoding the Gabor and Hessian information into the Gray Level Co-Occurrence Matrix (GLCM) features and Local Binary Patterns (LBP) features respectively. These two groups of features, together with the raw feature group containing the GLCM and LBP features, totally 560 features, were categorized into various groups and used to train the Random Forest classifier. Classification performances using these features were compared inter-and intra-groups/subgroups. And the performance using each individual feature was also provided. We conducted feature selection based on Recursive Feature Elimination (RFE) inside a voting scheme to further increase the efficiency. The inter-and intra-groups/subgroups results indicate that the Encoded GLCM and Encoded LBP, are more discriminative than the raw GLCM and LBP features for the identification of the osteoporosis; The best individual feature is from the Encoded LBP group and can achieve 70% of balanced accuracy; Furthermore, using only ten of the proposed features through feature selection, the balanced accuracy can even be improved from 60% to 71%. This shows that the proposed method is promising to assist the early diagnosis of osteoporosis.

This paper presents an approach of automatically generating abstract paintings of Malevich Style. Through parameterized program and flexible production processes, we successfully generated images mimicking the styles of the abstract artist Kazimir Malevich, having classified different styles of his paintings. The generated results using the basic components in Malevich's style based on his art theory are shown and the reason why some colors are chosen are also discussed.