Timely and personalized treatment decisions are essential across a wide range of healthcare settings where patient responses can vary significantly and evolve over time. Clinical data used to support these treatment decisions are often irregularly sampled, where missing data frequencies may implicitly convey information about the patient's condition. Existing Reinforcement Learning (RL) based clinical decision support systems often ignore the missing patterns and distort them with coarse discretization and simple imputation. They are also predominantly model-free and largely depend on retrospective data, which could lead to insufficient exploration and bias by historical behaviors. To address these limitations, we propose medDreamer, a novel model-based reinforcement learning framework for personalized treatment recommendation. medDreamer contains a world model with an Adaptive Feature Integration module that simulates latent patient states from irregular data and a two-phase policy trained on a hybrid of real and imagined trajectories. This enables learning optimal policies that go beyond the sub-optimality of historical clinical decisions, while remaining close to real clinical data. We evaluate medDreamer on both sepsis and mechanical ventilation treatment tasks using two large-scale Electronic Health Records (EHRs) datasets. Comprehensive evaluations show that medDreamer significantly outperforms model-free and model-based baselines in both clinical outcomes and off-policy metrics.

In this work, we investigate the joint optimization strategy in the scenario of video coding for machines (VCM). We formulated two kinds of joint optimization strategies, Opt_JA and Opt_JH , and compared them with the separate optimization strategy Opt_S. The three optimization strategies are illustrated in Fig. 1 . In Opt_S , we separately train the feature compression network with mean squared error (MSE). In Opt_JA , we optimize all modules jointly toward the person re-identification task. In Opt_JH , only the aggregation module and feature compression module are jointly optimized. The feature compression consists of two fully-connected (FC) layers and two batch normalization (BN) layers. Specifically, we set five compression ratios (CR): 256, 128, 64, 32, and 16.

Single-step retrosynthesis is the cornerstone of retrosynthesis planning, which is a crucial task for computer-aided drug discovery. The goal of single-step retrosynthesis is to identify the possible reactants that lead to the synthesis of the target product in one reaction. By representing organic molecules as canonical strings, existing sequence-based retrosynthetic methods treat the product-to-reactant retrosynthesis as a sequence-to-sequence translation problem. However, most of them struggle to identify diverse chemical reactions for a desired product due to the deterministic inference, which contradicts the fact that many compounds can be synthesized through various reaction types with different sets of reactants. In this work, we aim to increase reaction diversity and generate various reactants using discrete latent variables. We propose a novel sequence-based approach, namely RetroDVCAE, which incorporates conditional variational autoencoders into single-step retrosynthesis and associates discrete latent variables with the generation process. Specifically, RetroDVCAE uses the Gumbel-Softmax distribution to approximate the categorical distribution over potential reactions and generates multiple sets of reactants with the variational decoder. Experiments demonstrate that RetroDVCAE outperforms state-of-the-art baselines on both benchmark dataset and homemade dataset. Both quantitative and qualitative results show that RetroDVCAE can model the multi-modal distribution over reaction types and produce diverse reactant candidates.

Deep graph neural networks (GNNs) have been shown to be expressive for modeling graph-structured data. Nevertheless, the overstacked architecture of deep graph models makes it difficult to deploy and rapidly test on mobile or embedded systems. To compress over-stacked GNNs, knowledge distillation via a teacher-student architecture turns out to be an effective technique, where the key step is to measure the discrepancy between teacher and student networks with predefined distance functions. However, using the same distance for graphs of various structures may be unfit, and the optimal distance formulation is hard to determine. To tackle these problems, we propose a novel Adversarial Knowledge Distillation framework for graph models named GraphAKD, which adversarially trains a discriminator and a generator to adaptively detect and decrease the discrepancy. Specifically, noticing that the well-captured inter-node and inter-class correlations favor the success of deep GNNs, we propose to criticize the inherited knowledge from node-level and class-level views with a trainable discriminator. The discriminator distinguishes between teacher knowledge and what the student inherits, while the student GNN works as a generator and aims to fool the discriminator. Experiments on nodelevel and graph-level classification benchmarks demonstrate that GraphAKD improves the student performance by a large margin. The results imply that GraphAKD can precisely transfer knowledge from a complicated teacher GNN to a compact student GNN.

Generalization across different environments with the same tasks is critical for successful applications of visual reinforcement learning (RL) in real scenarios. However, visual distractions---which are common in real scenes---from high-dimensional observations can be hurtful to the learned representations in visual RL, thus degrading the performance of generalization. To tackle this problem, we propose a novel approach, namely Characteristic Reward Sequence Prediction (CRESP), to extract the task-relevant information by learning reward sequence distributions (RSDs), as the reward signals are task-relevant in RL and invariant to visual distractions. Specifically, to effectively capture the task-relevant information via RSDs, CRESP introduces an auxiliary task---that is, predicting the characteristic functions of RSDs---to learn task-relevant representations, because we can well approximate the high-dimensional distributions by leveraging the corresponding characteristic functions. Experiments demonstrate that CRESP significantly improves the performance of generalization on unseen environments, outperforming several state-of-the-arts on DeepMind Control tasks with different visual distractions.

Graph convolutional networks (GCNs)—which are effective in modeling graph structures—have been increasingly popular in knowledge graph completion (KGC). GCN-based KGC models first use GCNs to generate expressive entity representations and then use knowledge graph embedding (KGE) models to capture the interactions among entities and relations. However, many GCN-based KGC models fail to outperform state-of-the-art KGE models though introducing additional computational complexity. This phenomenon motivates us to explore the real effect of GCNs in KGC. Therefore, in this paper, we build upon representative GCN-based KGC models and introduce variants to find which factor of GCNs is critical in KGC. Surprisingly, we observe from experiments that the graph structure modeling in GCNs does not have a significant impact on the performance of KGC models, which is in contrast to the common belief. Instead, the transformations for entity representations are responsible for the performance improvements. Based on the observation, we propose a simple yet effective framework named LTE-KGE, which equips existing KGE models with linearly transformed entity embeddings. Experiments demonstrate that LTE-KGE models lead to similar performance improvements with GCN-based KGC methods, while being more computationally efficient. These results suggest that existing GCNs are unnecessary for KGC, and novel GCN-based KGC models should count on more ablation studies to validate their effectiveness. The code of all the experiments is available on GitHub at https://github.com/MIRALab-USTC/GCN4KGC.

Rate adaption is one of the decisive factors for the applications of video compression. However, previous deep video compression methods are usually optimized for a single fixed rate-distortion (R-D) tradeoff. While they can achieve multiple bitrates by training multiple independent models, the realized bitrates are limited to several discrete points on the R-D curve and the storage cost increases proportionally to the number of models. In this paper, we propose a variable-rate scheme for deep video compression, which can achieve continuously variable rate by a single model, i.e., it can reach any point on the R-D curve. In our scheme, two deep auto-encoders are used to compress the residual and the motion vector field respectively, which directly generate the final bitstream. The basic rate adaptation can be achieved by using the R-D tradeoff parameter to deeply modulate all the internal feature maps of the auto-encoders. However, other modules in our scheme, notably motion estimation and motion compensation, also affect the final bitrate indirectly. We further use the R-D tradeoff parameter to modulate them via a conditional map, which effectively improves the compression efficiency. We use a multi-rate-distortion loss function together with a step-by-step training strategy to optimize the entire scheme. Our experiments show that the proposed scheme achieves continuously variable rate by a single model with almost the same compression efficiency as multiple fixed-rate models. The additional parameters and computation of our model are negligible when compared with a single fixed-rate model.

We propose a convolutional neural network-based coefficients prediction (CNNCP) method for intra-predicted residues in the High Efficiency Video Coding (HEVC) standard. In HEVC, discrete cosine transform (DCT) or discrete sine transform (DST) is adopted to convert the intra-predicted residues in the spatial domain into coefficients in the frequency domain. Each coefficient is scalar quantized and entropy coded into the bitstream. As DCT or DST is non-optimal linear transform, there still exist linear and non-linear correlations among different coefficients after the transform. In addition, there exist coefficients' correlations between current block and neighboring blocks, as these correlations cannot be completely exploited in the intra prediction. We thus propose to perform coefficients prediction to further reduce the redundancy among coefficients. The coefficients prediction is achieved using trained convolutional neural networks (CNNs), as CNNs can build complex relationship between input and output by training with a lot of data. In addition, a flag that signals whether to perform coefficients prediction or not at the coding unit level is transmitted to decoder. The proposed CNNCP method is implemented upon the HEVC reference software. Experimental results show that the proposed method achieves on average 1.8%, 4.1%, and 4.5% BD-rate reduction ratios in Y, U, V, respectively, compared with the HEVC baseline in all-intra configuration. In particular, the average BD-rate reduction ratios for 4K test sequences are 2.9%, 6.5%, and 6.6%.

This paper proposes an image upsampling scheme by introducing a new gradient field sharpening transform that converts the blurry gradient field of upsampled low-resolution (LR) image to a much sharper gradient field of original high-resolution (HR) image. Different from the existing methods that need to figure out the whole gradient profile structure and locate the edge points, we derive a new approach that sharpens the gradient field adaptively only based on the pixels in a small neighborhood. To maintain image contrast, image gradient is adaptively scaled to keep the integral of gradient field stable. Finally the HR image is reconstructed by fusing the LR image with the sharpened HR gradient field. Experimental results demonstrate that the proposed algorithm can generate more accurate gradient field and produce super-resolved images with better objective and visual qualities. Another advantage is that the proposed gradient sharpening transform is very fast and suitable for low-complexity applications.

Different from conventional profiles in the state-of-the-art video coding standard HEVC and related optimization methods, we focus on building the optimally scalable algorithms under constrained and varying computational capacity to take full advantages of HEVC as far as possible in order to meet the growing demands of computational capacity adaptive applications such as real-time video communication and video coding on different mobile devices. We propose a video coding framework based on priority order for a special profile and give the general thoughts of designing algorithms by utilizing cost-performance as priority. For the framework, we invent a feasible solution by introducing a three-level coding structure and some novel features that express the relationship between video contents and their priorities. Experimental results partially prove our framework may nearly achieve the optimal coding efficiency under the continuously changeable computing limitations at every time with negligible extra time consuming.

Markov decision processes (MDPs) provide a rich framework for planning under uncertainty. However, exactly solving a large MDP is usually intractable due to the “curse of dimensionality”— the state space grows exponentially with the number of state variables. Online algorithms tackle this problem by avoiding computing a policy for the entire state space. On the other hand, since online algorithm has to find a near-optimal action online in almost real time, the computation time is often very limited. In the context of reinforcement learning, MAXQ is a value function decomposition method that exploits the underlying structure of the original MDP and decomposes it into a combination of smaller subproblems arranged over a task hierarchy. In this article, we present MAXQ-OP—a novel online planning algorithm for large MDPs that utilizes MAXQ hierarchical decomposition in online settings. Compared to traditional online planning algorithms, MAXQ-OP is able to reach much more deeper states in the search tree with relatively less computation time by exploiting MAXQ hierarchical decomposition online. We empirically evaluate our algorithm in the standard Taxi domain—a common benchmark for MDPs—to show the effectiveness of our approach. We have also conducted a long-term case study in a highly complex simulated soccer domain and developed a team named WrightEagle that has won five world champions and five runners-up in the recent 10 years of RoboCup Soccer Simulation 2D annual competitions. The results in the RoboCup domain confirm the scalability of MAXQ-OP to very large domains.

Conventional image and video communication systems are usually designed with the objective being to maximize the fidelity of reconstructed images measured by mean square errors (MSE). It is well known that the fidelity metric MSE may not reflect the visual quality perceived by human eyes. Recent advancements in image quality assessment tell us that the structural similarity (SSIM), especially the gradient similarity, reveals the perceptual fidelity of images more reliably. Inspired by this observation, this paper proposes a new image communication approach, which conveys the visual information in an image by transmitting the image gradients and recovers the image from the received gradient data at decoder side using statistical image prior knowledge. In particular, we designed a gradient-based image SoftCast scheme for wireless scenarios. Experimental results show that the proposed scheme can produce reconstruction images with much better perceptual quality. The advantage in perceptual quality is verified by the quality improvement measured by the metrics SSIM and gradient signal-to-noise ratio (GSNR).

Video broadcasting is a popular application of wireless network. However, the existing layered approaches can hardly accommodate users with diverse channel conditions as analog communication can do. The newly emerged `soft cast' approach, utilizing soft broadcast, provides smooth multicast performance but is not very efficient in inter frame compression. In this work, we propose a motion-aligned wireless video multicast scheme DCAST. Instead of using conventional close loop prediction (CLP), DCAST is based on distributed source coding (DSC) theory. This helps DCAST to avoid error propagation but still achieve high compression efficiency in inter frame coding. DCAST outperforms soft cast 5dB in video PSNR while maintaining the similar graceful degradation feature as soft cast.

In lifting-based directional wavelet transforms, different subsampling patterns may show significant difference for directional signals in image coding. This paper investigates the influence of subsampling in directional wavelet transform. We show that the best subsampling depends on the direction and the directionality strength of the signal. To improve the coding performance, we further propose a subsampling-adaptive directional wavelet transform, which can use different subsampling patterns adaptively and according to the local characteristics of the image. To handle the boundary transition when subsampling changes, a phase completion process is applied to ensure that wavelet transform with various subsampling can be performed without introducing boundary effects and performance loss. Experimental results show that the proposed transform can achieve significant coding gain in image coding compared to other existing directional wavelet transforms.

Visual summarization is an attractive new scheme to summarize web pages, which can help achieve a more friendly user experience in search and re-finding tasks by allowing users quickly get the idea of what the web page is about and helping users recall the visited web page. In this paper, we perform a careful study on the recently proposed visual summarization approaches, including the thumbnail of the web page snapshot, the internal image in the web page which is representative of the content in the page, and the visual snippet which is a synthesized image based on the internal image, the title, and the logo found in the web page. Moreover, since the internal image based summarization approach hardly works when the representative internal images are unavailable, we propose a new strategy, which retrieves the representative image from the external to summarize the web page. The experimental results suggest that the various summarization approaches have respective advantages on different types of web pages. While internal images and thumbnails can provide a reliable summarization on web pages with dominant images and web pages with simple structure respectively, the external images are regarded as a useful information to complement the internal images and are demonstrated very useful in helping users understanding new web pages. The visual snippet performs well on the re-finding tasks since it incorporates the title and logo which are advantageous on identifying the visited web pages.

In this paper we propose an alternative decoding method for inverse wavelet transform when only partial coefficients are available. We have been inspired by the recently developed compressive sensing (CS) decoding, which is capable in recovering sparse signals from a few linear and non-adaptive measurements. Let x be a sparse signal with N entries and only K out of them are non-zero, and y be its approximation coefficients. Classic CS decoding such as l1-minimization can be applied to decode x from y, and it indeed provides better reconstruction of sparse signals than direct inverse transform, as demonstrated by our simulation results. When coefficients have been quantized, the performance of CS decoding decreases more severely compared with direct inverse transform, but still better than the latter once the signal is sparse enough.

Summary form only given. This paper studies how to better exploit intra correlation to compress images. In general, edge and texture areas of images exhibit strong anisotropic property. The correlation among samples is determined by not only their distance but also the link orientation. Traditional transforms are not efficient on handling this anisotropic correlation. Therefore, in this paper we propose a directional filtering transform (dFT, in order to distinguish from the common usage on DFT) to exploit local anisotropic correlation among samples. Similar to directional prediction in H.264 intra-frame coding, but it adopts the hierarchal structure to decrease the distance between samples to be predicted and that are used for prediction. From another viewpoint, the dFT prediction resembles the directional wavelet transform without update, which can take both intra-block and inter-block correlations into account.

We investigate the usage of edge-based inpainting as an intra prediction method in block-based image compression. The joint utilization of edge information and the well-known Laplace equation yields a simple and effective inpainting algorithm. As for intra prediction, the edge-based inpainting is a uniform solution, yet adaptive to local image features. During the integration of edge-based inpainting into a block-based coding scheme, edge extraction and coding are jointly considered to achieve the rate-distortion optimization. Our proposed schemes are compared with JPEG2000, and experimental results demonstrate that both PSNR gain and visible quality improvement are achieved.

This paper proposes a new image compression scheme by introducing visual patterns to nonlinear interpolative vector quantization (IVQ). Input images are first distorted by a generic down-sampling so that some details are removed before compression. Then, the distorted images are compressed lossly by traditional image coding scheme and transmitted to the decoder. In the decoder side, VQ indices are extracted from the decoded images to reproduce the removed details from a pre-trained codebook. One of main contributions in this paper is, we introduce visual patterns on designing the codebook, where only removed details that contain visual patterns and their original counterparts as pairs are trained. Experimental results show: (1) visual pattern blocks are easy to form clusters than original blocks; (2) the proposed scheme achieves much better performance over JPEG in terms of visual quality and PSNR.

This paper presents a scheme to design directional lapped transforms. Lapped transforms can be factorized into lifting steps. By introducing directional operator into each lifting step, the directional lapped transform is constructed. The directional lapped transform proposed not only preserves the advantages of lapped transforms, it also can represent directional signals more efficiently. An image coding scheme using the directional lapped transform is also described. Compared to the state-of-the-art image coding using lapped transform, HD photo, the proposed scheme shows more than 20 dB's gain for artificial images with strong directional correlations. And for natural images, up to 1.5 dB's gain can also be observed.

Considering that quincunx lattice is a more efficient spatial sampling scheme than square lattice, we investigate a new approach of image coding for quincunx sample arrangement. The key findings are: 1) adaptive directional lifting is particularly suited to decorrelate samples on quincunx lattice, and 2) quincunx samples can be processed by a 2D piecewise autoregressive model to reproduce the image of conventional square pixel grid, while preserving high frequency spatial features well. By incorporating these two techniques into the encoder and decoder respectively, we are able to improve the performance of JPEG 2000 image codec at low to modest bit rates. Since an image can be easily split into quincunx segments, this work has significance for multiple description image/video coding as well

Summary form only given. A sprite which is an image composed of pixels belonging to a video object visible throughout a video segment is a very important concept proposed by MPEG4. Because of the search region limitation in the global motion estimation, the performance of traditional sprite coding technology is not satisfactory in the case of fast camera motion. Only enlarging the search region is difficult to ensure the right motion estimation. An improved algorithm is proposed with enlarging the search region, predicting the motion of the current VOP (video object plane) and shortening the iterative time. Three main techniques are adopted in the new algorithm. They are: (1) enlarging search region; (2) weighting sum of absolute difference (SAD); and (3) shortening iteration. Two group experiments present a comparison between the original algorithm and the improved algorithm. The first group experiment shows the improved algorithm gets the same performance as the original algorithm in coding the general motion sequences. In the second group experiment, the Stefan background sequences which frame rates of 15 Hz, 10 Hz, 7.5 Hz, and 6 Hz are encoded in various transformations. The results given in a table show the coding performances of the algorithm are significantly improved.