Although diffusion-based models can generate high-quality and high-resolution video sequences from textual or image inputs, they lack explicit integration of geometric cues when controlling scene lighting and visual appearance across frames. To address this limitation, we propose IllumiCraft, an end-to-end diffusion framework accepting three complementary inputs: (1) high-dynamic-range (HDR) video maps for detailed lighting control; (2) synthetically relit frames with randomized illumination changes (optionally paired with a static background reference image) to provide appearance cues; and (3) 3D point tracks that capture precise 3D geometry information. By integrating the lighting, appearance, and geometry cues within a unified diffusion architecture, IllumiCraft generates temporally coherent videos aligned with user-defined prompts. It supports the background-conditioned and text-conditioned video relighting and provides better fidelity than existing controllable video generation methods.

Language has emerged as a natural interface for image editing. In this paper, we introduce a method for region- based image editing driven by textual prompts, without the need for user-provided masks or sketches. Specifically, our approach leverages an existing pre-trained text-to-image model and introduces a bounding box generator to iden-tify the editing regions that are aligned with the textual prompts. We show that this simple approach enables flex-ible editing that is compatible with current image generation models, and is able to handle complex prompts featuring multiple objects, complex sentences, or lengthy para- graphs. We conduct an extensive user study to compare our method against state-of-the-art methods. The experiments demonstrate the competitive performance of our method in manipulating images with high fidelity and realism that correspond to the provided language descriptions. Our project webpage can be found at: https://yuanzelin.me/LearnableRegions_page.

This paper presents evaluation results of Low Complexity Enhancement Video Codec (LCEVC) using its reference implementation (LTM-5.4.1) when combined with HEVC (HM-16.25) and VVC (VTM-19.0) as base layer codecs. Tests were performed for 4K content, following JVET Random Access (RA) Common Testing Conditions (CTC). The anchor is the result of single layer coding, i.e., encoding 4K videos by VTM or HM only, and the test is the result of base layer codec, which encodes quarter-resolution version of 4K videos by using VTM or HM as base layer, followed by enhancement layer coded using L TM-5.4.1, where enhancement layer is obtained as a difference between original4K video and upsampled reconstructed frames from the base layer. PSNR BD-rate was used for objective evaluation. PSNR results show that LCEVC is introducing significant losses. Also, a viewing session was performed to evaluate visual quality of LCEVC when combined with VTM, which showed that LCEVC does not provide any visual benefits for 4K content.

Transition metal oxides as the next generation of 2-D materials have attracted widespread attention because of their unique electrochemical properties. In this study, the preparation of metal oxides (MO) involves a two-step process of a hydrothermal reaction with subsequent air annealing. First, the transition metal dichalcogenides (TMD) nanosheets were prepared by a hydrothermal method. Subsequently, the TMD nanosheets were converted to MO nanosheets via the thermal annealing method. In addition, Au@NFs nanocomposites were also prepared using an efficiently combined approach of hydrothermal process and in situ chemical synthesis method. Both XRD and TEM results illustrated that MO nanosheets and Au@NFs nanocomposites were successfully prepared. For the first time, a novel electrochemical immunosensor modified with Au@NFs/MO nanocomposites was fabricated by the solution-casting method for detecting the calreticulin (CRT) biomarkers. This construction step provides a suitable and simple method for the covalent attachment of modified SPCE and anti-CRT molecules. Bio-affinity interactions between CRT biomarkers and anti-CRT molecules were investigated by CV and EIS techniques. The CV and EIS results indicated that L-Cysteine and EDC/NHS modified-Au@NFs/MO/SPCE could increase the electron transfer ability. When EN-LC/Au@NFs/MO/SPCE was sequentially modified with anti-CRT, BSA, and CRT, the Rct value gradually increased to$4433.71\ \varOmega$, indicating that these non-conductive biomolecules were successfully immobilized. This result indicated that the CRT-immunosensors were successfully fabricated. In the future, the CRT-immunosensors are expected to detect real samples.

The most accredited mechanisms of Meniere's disease (MD) and related frequency components reduction of transient evoked otoacoustic emission (TEOAE) are dysfunctions of outer hair cells. The clinical TEOAE parameters lack quantitative physiological information. This study explored the feasibility of sample entropy (SampEn) of TEAOE for quantitative diagnosis of MD based on frequency components reduction. Nineteen normal (38 ears) and 40 (40 ears) unilateral definitive MD participants were recruited. Based on combinations of different data size$N$, template length$m$and similarity tolerance$r$, TEOAE SampEn were estimated. The results showed that TEOAE SampEn using the parameters of ($N=772,\ m=2,\ r=0.2$) resulted in the maximum area under receiver operating characteristic curve = 0.7982 in diagnosing MD. TEOAE SampEn provides a good diagnostic performance and demonstrates a great potential for understanding physiology of MD.

Traditional treatment strategy for knee cartilage injuries (such as osteochondrosis dissecans, early degenerative arthritis, femoral condyle necrosis) includes mosaicplasty, which usually requires surgeons to harvest healthy cartilage tissues from other non-weight bearing joints. The development of surface modification techniques have brought a major paradigm shift for clinical bone tissue regeneration applications. In this study, we modified the surface of calcium silicate scaffolds (CS) with lithium ions (Li) via a simple immersion technique and evaluated its capabilities for bone regeneration. Li has been reported to have anti-inflammatory, osteogenic and chondrogenic capabilities via the promotion of several intracellular signalling pathways. Our results showed that Li ions could be easily coated onto the surfaces of CS scaffolds without affecting the microstructural properties of CS itself. In addition, the modifications did not affect printing capabilities of CS and porous scaffolds could be fabricated via the extrusion method. Furthermore, the presence of Li showed improvements in surface roughness and hydrophilicity, thus leading to enhanced secretion of osteochondral-related regeneration factors such as ALP, BSP and Col II proteins. Subsequent in vivo studies, including histological and micro-CT analysis confirmed that our Li-modified CS scaffolds were able to promote osteochondral regeneration. From our NGS analysis, the enhanced osteo-chondrogenic capabilities of our scaffolds were hypothesised to be influenced by paracrine exosomes. Taken together, we hoped that our study could inspire more osteochondral regeneration studies using the surface modification techniques.

Fucoidan has various biological activities, such as antioxidant, antibacterial, anti-inflammatory, antiviral, anticoagulant, antitumor and immunomodulatory activities. In view of the role of fucoidan in regulating transforming growth factor$\beta 1$related to wound repair, and helping the formation of new blood vessels and fibrous collagen matrix, it has the potential to help skin repair. Therefore, this study applied fucoidan to the development of a three-dimensional sponge dressing for skin damage repair. At present, the manufacturing process and formulation of the three-dimensional sponge dressing containing fucoidan have been developed, and its physical properties and biocompatibility have been evaluated. It is expected to contribute to the development of skin damage treatment in the future.

This study aimed to investigate the effects of human chondrocytes-laden auxetic scaffold under the cyclic tension stimulation by carrying out the decellularized extracellular matrix of rabbit meniscus (dECM) and light-curable gelatin (FGelMA) as the material substrate. The prepared photocurable bioink provided cells proliferation and morphological alterations. There was a trend to elevate cell numbers significantly compared with static culturing, and a wider spread of cell expansion can also be found at the edge of the auxetic structure. Remarkably, the scaffold designed as an auxetic structure through digital light processing and combined with cyclic tension stimulation to further promote the synergetic effect on chondrogenic-related ECM markers of chondrocytes.

The biofabricate technologies has allowed us to manufacture complex novel scaffolds for tissue regeneration. In this study, we demonstrated the incorporation of different concentrations of ceramic powder into fish gelatin methacrylate (FGelMa) bioink for the fabrication of CS/FGelMa auxetic bioscaffolds using bioprinting technology. Our results indicated that ceramic could be successfully incorporated into FGelMa bioink without effecting the structural components of FGelMa. Furthermore, it conveyed that ceramic modifications both the mechanical properties and degradation rates of the scaffolds were improved in accordance with the concentrations of ceramic upon modifications of ceramic. In addition, the presence of ceramic promoted the adhesion and proliferation of human periodontal ligament cells (hPDLs) cultured in the scaffold. Further osteogenic evaluation also confirmed that ceramic was able to enhance the osteogenic capabilities via activation of downstream intracellular factors such as pFAK/FAK and pERK/ERK. More interestingly, it was noted that the application of extrinsic biomechanical stimulation to the auxetic scaffolds further enhanced the proliferation and differentiation of hPDLs cells and secretion of osteogenicrelated markers when compared to CS/FGelMa hydrogels without tensile stimulation. This prompted us to explore the related mechanism behind this interesting phenomenon. Subsequent studies showed that biomechanical stimulation works via YAP, which is a biomechanical cue. Taken together, our results showed that novel auxetic scaffolds could be fabricated by combining different aspects of science and technology, in order to improve the future chances of clinical applications for bone regeneration.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) cause severe outbreak of coronavirus disease 2019 (COVID-19). Even though vaccination, the spread of SARS-CoV-2 is still continue. It is urgent to have a model that can efficiently evaluate potential therapeutic agents to counteract SARS-CoV-2 infection. Iron is an essential molecule for maintaining homeostasis. Supplement of iron significantly to affect virus infection. But the detailed mechanisms of iron on regulating SARS-CoV-2 infection are still unveiled. The three-dimensional (3D) model is a promising system for drug screening and disease progression analysis. Organoid is a typical 3D culture system that recapitulates genetic characteristics and phenotypic features of organs within body. Vasculature is prevalent for all various organs or tumors in the body which transport nutrients, oxygen and metabolites to maintain cellular homeostasis. Thus, we have established a 3D model of vascularized organoid to evaluate the effects of iron on infectivity of SARS-CoV-2 pseudovirus to provide the novel therapeutic strategy in coping SARS-CoV-2 infection.

This paper proposes one new In-loop filtering technique cross-component sample adaptive offset (CCSAO) for further coding efficiency improvement beyond Versatile Video Coding (VVC). The CCSAO reduces the sample distortion by 1) utilizing the strong correlation between luma and chroma components to classify the reconstructed samples into different categories and 2) deriving one offset for each category and adding the offset to the samples in the category. The offset of each category is properly derived at encoder and signaled to decoder. To keep the design at low complexity, only band information of reconstructed samples is considered for the sample classification of the CCSAO. To verify the performance, the proposed CCSAO is implemented on top of the enhanced compression model (ECM) for the joint video exploration team (JVET)'s exploratory work of future video coding technologies beyond VVC. Simulation results show that the CCSAO achieves average {0.20%, 2.83%, 2.98%} and {0.41%, 7.36%, 7.36%} Bjentegaard delta (BD)-rate savings for {Y, U, V} components under the Random Access and Low Delay B configuration, with negligible complexity impacts on encoding and decoding complexity. The proposed CCSAO scheme has been adopted to the ECM-2.0 software platform.

Single image reflection removal is a crucial technique which can improve the performance of object detection, semantic segmentation, and various computer vision applications. In this paper, we present a novel reflection removal algorithm using bi-channel priors (i.e., the dark channel prior and the bright channel prior). We observe that the values of dark channel pixels are not near 0, and those of bright channel pixels are not closer to 1 under the reflection scenario. We first demonstrate these phenomena statistically and mathematically. Then, we apply these properties as the constraints in optimizing the proposed reflection removal process. Extensive experiments on several well-known benchmarks demonstrate that our approach achieves desirable reflection suppression results compared with other methods.

Video salient object detection aims to find the most visually distinctive objects in a video. To explore the temporal dependencies, existing methods usually resort to recurrent neural networks or optical flow. However, these approaches require high computational cost, and tend to accumulate inaccuracies over time. In this paper, we propose a network with attention modules to learn contrastive features for video salient object detection without the high computational temporal modeling techniques. We develop a non-local self-attention scheme to capture the global information in the video frame. A co-attention formulation is utilized to combine the low-level and high-level features. We further apply the contrastive learning to improve the feature representations, where foreground region pairs from the same video are pulled together, and foreground-background region pairs are pushed away in the latent space. The intra-frame contrastive loss helps separate the foreground and background features, and the inter-frame contrastive loss improves the temporal consistency. We conduct extensive experiments on several benchmark datasets for video salient object detection and unsupervised video object segmentation, and show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.

Abstract Referring expression comprehension aims to localize objects identified by natural language descriptions. This is a challenging task as it requires understanding of both visual and language domains. One nature is that each object can be described by synonymous sentences with paraphrases, and such varieties in languages have critical impact on learning a comprehension model. While prior work usually treats each sentence and attends it to an object separately, we focus on learning a referring expression comprehension model that considers the property in synonymous sentences. To this end, we develop an end-to-end trainable framework to learn contrastive features on the image and object instance levels, where features extracted from synonymous sentences to describe the same object should be closer to each other after mapping to the visual domain. We conduct extensive experiments to evaluate the proposed algorithm on several benchmark datasets, and demonstrate that our method performs favorably against the state-of-the-art approaches. Furthermore, since the varieties in expressions become larger across datasets when they describe objects in different ways, we present the cross-dataset and transfer learning settings to validate the ability of our learned transferable features.

In image processing, smoke may degrade visibility and deteriorate the performance of high-level vision applications. Therefore, single image smoke removal is crucial for computer vision. Currently, existing smoke removal algorithms mainly leverage handcrafted priors. Moreover, these methods usually apply haze removal methods to perform smoke removal due to the similarity between smoke and haze. However, these methods cannot sufficiently address the degradation of thick smoke and may suffer from residual smoke and color distortion problems due to the non-global and non-homogeneous distribution of smoke. In this paper, to solve the aforementioned problems, an end-to-end deep neural network called DesmokeNet is proposed. We construct a two-stage recovered pipeline to remove the smoke in different thicknesses. The light and thick smoke is first removed locally by the smoke removal network (SRN). The missing pixels in the thick smoke are then recovered by the pixel compensation network (PCN). Moreover, we proposed the thickness-aware pixel loss and the dark channel loss to suppress the residual smoke. To further increase the discriminative ability of the DesmokeNet, we proposed self-attentive feature consensus loss and multi-level contrastive regularization loss to improve the performance of smoke removal. Finally, to train the proposed method, we construct the first large-scale dataset containing synthetic and real-world data. Extensive experiments show that the proposed method outperforms favorably against other state-of-the-art methods quantitatively and qualitatively.

We address the problem of text-guided video temporal grounding, which aims to identify the time interval of a certain event based on a natural language description. Different from most existing methods that only consider RGB images as visual features, we propose a multi-modal framework to extract complementary information from videos. Specifically, we adopt RGB images for appearance, optical flow for motion, and depth maps for image structure. While RGB images provide abundant visual cues of certain events, the performance may be affected by background clutters. Therefore, we use optical flow to focus on large motion and depth maps to infer the scene configuration when the action is related to objects recognizable with their shapes. To integrate the three modalities more effectively and enable inter-modal learning, we design a dynamic fusion scheme with transformers to model the interactions between modalities. Furthermore, we apply intra-modal self-supervised learning to enhance feature representations across videos for each modality, which also facilitates multi-modal learning. We conduct extensive experiments on the Charades-STA and ActivityNet Captions datasets, and show that the proposed method performs favorably against state-of-the-art approaches.

MOTIVATION: While promoter methylation is associated with reinforcing fundamental tissue identities, the methylation status of distant enhancers was shown by genome-wide association studies to be a powerful determinant of cell-state and cancer. With recent availability of long reads that report on the methylation status of enhancer-promoter pairs on the same molecule, we hypothesized that probing these pairs on the single-molecule level may serve the basis for detection of rare cancerous transformations in a given cell population. We explore various analysis approaches for deconvolving cell-type mixtures based on their genome-wide enhancer-promoter methylation profiles. RESULTS: To evaluate our hypothesis we examine long-read optical methylome data for the GM12878 cell line and myoblast cell lines from two donors. We identified over 100 000 enhancer-promoter pairs that co-exist on at least 30 individual DNA molecules. We developed a detailed methodology for mixture deconvolution and applied it to estimate the proportional cell compositions in synthetic mixtures. Analysis of promoter methylation, as well as enhancer-promoter pairwise methylation, resulted in very accurate estimates. In addition, we show that pairwise methylation analysis can be generalized from deconvolving different cell types to subtle scenarios where one wishes to resolve different cell populations of the same cell-type. AVAILABILITY AND IMPLEMENTATION: The code used in this work to analyze single-molecule Bionano Genomics optical maps is available via the GitHub repository https://github.com/ebensteinLab/Single_molecule_methylation_in_EP.

This paper provides an in-depth overview of the adaptive color transform (ACT) tool that is adopted into the emerging versatile video coding (VVC) standard. With the ACT, prediction residuals in the original color space are adaptively converted into another color space to reduce the correlation among the three color components of video sequences in 4:4:4 chroma format. The residuals after color space conversion are then transformed, quantized and entropy-coded, following the VVC framework. YCgCo-R transforms, which can be easily implemented with shift and addition operations, are selected as the ACT core transforms to do the color space conversion. Additionally, to facilitate its implementations, the ACT is disabled in certain cases where the three color components do not share the same block partition, e.g. under separate tree partition mode or intra sub-partition prediction mode. Simulation results based on the VVC reference software show that ACT may provide significant coding gains with negligible impact on encoding and decoding runtime.

Affine Motion Compensation (AMC) is a promising coding tool in Joint Exploration Model (JEM) developed by the Joint Video Exploration Team (JVET). AMC in JEM employs a 4-parameter affine model between the current block and its reference block. With this model, Motion Vectors (MV) of each sub-block can be derived from the MVs at two control points. In this paper, we present a practical framework to further improve the AMC in JEM. First, we introduce a multi-model AMC approach, which allows the encoder to select either the 4-parameter affine model or the 6-paramter affine model adaptively. Second, we improve the affine inter-mode in two aspects. For the normative part, we present an efficient affine motion coding method, which replaces the affine MV Prediction (MVP) candidates in JEM with more accurate but simpler ones, and employs a second-order MVP. For the non-normative part, we enhance the motion estimation process for AMC, by regulating the optimization algorithm. Finally, we propose to unify the affine merge-mode and the normal merge-mode into a unified merge-mode, which combine affine merge candidates and normal merge candidates in a single merge candidate list. Partial of these methods have been adopted into the next generation video coding standard named Versatile Video Coding (VVC). Simulation results show that the proposed methods can achieve 1.67% BD rate savings in average for the random access configurations.

The rapid development of cloud computing goes along with the explosion of internal network traffic in the cloud. In this paper, we analyzed the usage of fields of internal packet of cloud, and customize the packets based on the statistical results. We remove unneeded fields of header in order to compress the packet size. Because other machines can not read customized packets, we modify the network portion of the Linux kernel, and make it have capability to send and receive customized packets. The result of experiment shows that the compression ratio is 1.9, space-saving rate is about 47.4%.

Computed tomography (CT) can rapidly provide high-resolution cross-section images for clinical diagnosis. The image contrast of the CT strongly influences the visibility of lesions in the images. However, the low material-dependent characteristic of the Compton scattering (CS) lowers the image contrast. In this study, a novel physical model-based method was proposed to enhance the contrast of CT images. At first, relationships between CT number and tissue parameters were determined using the CT images and elemental composition of tissue equivalent rods. Then, the CT images to be enhanced were converted to tissue parameter maps using pre-determined relationships. By using a classical parametric fit model, partial attenuation images with enhanced image contrast can be calculated. A phantom CT image and an abdominal CT image were used to evaluate the performance of the proposed method. For the phantom CT image, the image contrast between rods and background solid water were enhanced. For the abdominal CT image, the visibility of a low-attenuation lesion in the right lobe of the liver was improved. In conclusion, the proposed method could be applied in clinical diagnosis to improve the visibility of CT images.

Palette mode is the new coding tool that has been adopted in the Screen Content Coding Extensions of High Efficiency Video Coding (HEVC SCC). Palette mode can represent colour clusters for screen content efficiently and can be summarized into two parts: palette coding tools and colour index map coding tools. This paper proposes two techniques to improve colour index map coding: transition copy and prediction across coding unit boundary. The former is for exploring the correlation between an index and all of the previously coded indices; the latter for utilizing the correlation between the content along the coding unit boundary. Experimental results reportedly show that, compared with HEVC SCC, the proposed techniques can achieve 3.2%, 2.2%, and 1.7% BD-rate savings compared with HEVC SCC for “YUV, text & graphics with motion, 1080p & 720p sequences” under all intra, random access, and low-delay B common test conditions, respectively.

Palette mode, as a new coding tool, is adopted in the Screen Content Coding Extensions of High Efficiency Video Coding (HEVC SCC) that is being defined by the Joint Collaborative Team on Video Coding (JCT-VC). In the palette mode, pixels in a coding unit (CU) are represented by selected representative colours according to the characteristics of screen contents in which pixel values usually concentrate on few colour values. This paper introduces the palette mode in HEVC SCC, and our contributions to the palette mode are also presented. Experimental results show that disabling the palette mode suffers 20.3%, 13.0%, and 7.5% BD-rate increases for “YUV, text & graphics with motion, 1080p & 720p sequences” under all intra (AI), random access (RA), and low-delay B (LB) common test conditions, respectively.

As the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC) has been finalized in January 2013. To support the coding of multiple views and associated depth data, the development of the HEVC extension for 3D video coding is also about to be finalized. New coding unit (CU) level coding tools are added to the HEVC design to improve the compression capabilities for both video views and depth data. In this paper, we introduce a new coding mode termed as “Single Depth Intra Coding Mode” to efficiently code the smooth area within a depth map. The concept of single depth mode is to simply reconstruct a coding block with a single depth value based on block merging scheme under the HEVC quad-tree based block partitioning. The proposed single depth mode has been adopted into the working draft and 3D-HEVC test model 12 (HTM-12.0). The experimental results evaluated under the common test conditions (CTC) for 3D-HEVC show that the proposed technique achieves up to 5% bit-rate saving for depth coding and on average 0.3% bit-rate saving regarding the synthesis results over the texture and depth bit-rate with decoding time reduced by 4%.

The 3D extension of High Efficiency Video Coding (3D-HEVC) is a new international video coding standard that has been developed by the Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V). It aims at improving the coding efficiency of 3D and multi-view videos by introducing new coding tools to utilize the correlations between views and between texture and depth components. In this paper, we propose an inter-view motion prediction (inter-view merge candidate) and an inter-component motion prediction (texture merge candidate) to explore the inter-view and the inter-component redundancies. The proposed schemes were adopted in 3D-HEVC and the experimental results demonstrate that the proposed inter-view merge candidate and texture merge candidate achieve significant BD-rate reductions of 20% for dependent texture views and 9% for the synthesized texture views under the common test conditions used for 3D-HEVC standardization.

The target of 3D video coding is to compress Multiview Video plus Depth (MVD) format data, which consist of a texture image and its corresponding depth map. In the MVD format, the depth map plays an important role for successful services in 3D video applications, because it enables the user to experience 3D by generating arbitrary intermediate views. The depth map has a strong correlation with its associated texture data, so it can be utilized to improve texture coding efficiency. This paper introduces a novel and efficient depth-based texture coding scheme. It includes depth-based motion vector prediction, block-based view synthesis prediction, and adaptive luminance compensation, which were adopted in an AVC-compatible 3D video coding standard. Simulation results demonstrate that the proposed scheme reduces the total coding bitrates of texture and depth by 19.06% for the coded PSNR and 17.01% for the synthesized PSNR in a P-I-P view prediction structure, respectively.

This paper presents a novel inter-view motion prediction technique used in 3D-HEVC which provides efficient compression for motion vectors. The 3D extension of HEVC (High Efficiency Video Coding) namely 3D-HEVC is under development in JCT-3V for coding multi-view video and depth data. Inter-view motion prediction takes benefit of the inter-view correlation between views by inferring the motion information of a view from the already coded motion information in another view as well as the local disparity between two views. Experimental results show that inter-view motion prediction in 3D-HEVC provides an average bit-rate savings of 15% for the dependent views.

A part and parcel of any automatic speech recognition (ASR) system is language modeling (LM), which helps to constrain the acoustic analysis, guide the search through multiple candidate word strings, and quantify the acceptability of the final output hypothesis given an input utterance. Despite the fact that the n-gram model remains the predominant one, a number of novel and ingenious LM methods have been developed to complement or be used in place of the n-gram model. A more recent line of research is to leverage information cues gleaned from pseudo-relevance feedback (PRF) to derive an utterance-regularized language model for complementing the n-gram model. This paper presents a continuation of this general line of research and its main contribution is two-fold. First, we explore an alternative and more efficient formulation to construct such an utterance-regularized language model for ASR. Second, the utilities of various utterance-regularized language models are analyzed and compared extensively. Empirical experiments on a large vocabulary continuous speech recognition (LVCSR) task demonstrate that our proposed language models can offer substantial improvements over the baseline n-gram system, and achieve performance competitive to, or better than, some state-of-the-art language models.

With the exponential proliferation of multimedia associated with spoken documents, research on spoken document retrieval (SDR) has emerged and attracted much attention in the past two decades. Apart from much effort devoted to developing robust indexing and modeling techniques for representing spoken documents, a recent line of thought targets at the improvement of query modeling for better reflecting the user's information need. Pseudo-relevance feedback is by far the most commonly-used paradigm for query reformulation, which assumes that a small amount of top-ranked feedback documents obtained from the initial round of retrieval are relevant and can be utilized for this purpose. Nevertheless, simply taking all of the top-ranked feedback documents obtained from the initial retrieval for query modeling (reformulation) does not always work well, especially when the top-ranked documents contain much redundant or non-relevant information. In the view of this, we explore in this paper an interesting problem of how to effectively glean useful cues from the top-ranked documents so as to achieve more accurate query modeling. To do this, different kinds of information cues are considered and integrated into the process of feedback document selection so as to improve query effectiveness. Experiments conducted on the TDT (Topic Detection and Tracking) task show the advantages of our retrieval methods for SDR.

In the High Efficiency Video Coding (HEVC) based 3D video coding, 3D-HEVC, the disparity vector (DV) derivation is critical for inter-view motion prediction, inter-view residual prediction, disparity-compensated prediction (DCP) or any other tools exploiting inter-view correlation. In HTM-5.0.1, the DV is derived from some spatial and temporal neighbors to locate a corresponding block in another view. This paper advocates modifications of the DV derivation to reduce the complexity and to achieve slightly better coding efficiency. Firstly, we propose to remove the additional temporal block, unify the DV searching order for all views, and to impose restrictions on the temporal blocks for memory access bandwidth and complexity reduction in DV derivation. We also propose an improved DV searching order for slightly better coding performance. These proposed methods were adopted into the 3D-HEVC standard in the 3rd JCT-3V meeting in Jan. 2013.

Visualization of uncertainty in datasets is a new field of research, which aims to represent incomplete data for analysis in real scenarios. In many cases, datasets, especially multi-dimensional datasets, often contain either errors or uncertain values. To address this challenge, we may treat these uncertainties as scalar values like probability. For visual representation in parallel coordinates, we draw a small "circle" to temporarily define a dummy vertex for an uncertain value of a data item, at the crossing point between polylines and the axis of certain dimension. Furthermore, these temporary positions of uncertainty could be permuted to achieve visual effectiveness. This feature provides a great opportunity by optimizing the order of uncertain values to tackle another important challenge in information visualization: clutter reduction. Visual clutter always obscures the visualizing structure even in small datasets. In this paper, we apply Sugiyama's layered directed graph drawing algorithm into parallel coordinates visualization to minimize the number of edge crossing among polylines, which has significantly improved the readability of visual structure. Experiments in case studies have shown the effectiveness of our new methods for clutter reduction in parallel coordinates visualization. These experiments also imply that besides visual clutter, the number of uncertain values and the type of multi-dimensional data are important attributes that affect visualization performance in this field.

Ever-increasing amounts of publicly available multimedia associated with speech information have motivated spoken document retrieval (SDR) to be an active area of intensive research in the speech processing community. Much work has been dedicated to developing elaborate indexing and modeling techniques for representing spoken documents, but only little to improving query formulations for better representing the information needs of users. The latter is critical to the success of a SDR system. In view of this, we present in this paper a novel use of a relevance language modeling framework for SDR. It not only inherits the merits of several existing techniques but also provides a principled way to render the lexical and topical relationships between a query and a spoken document. We further explore various ways to glean both relevance and non-relevance cues from the spoken document collection so as to enhance query modeling in an unsupervised fashion. In addition, we also investigate representing the query and documents with different granularities of index features to work in conjunction with the various relevance and/or non-relevance cues. Empirical evaluations performed on the TDT (Topic Detection and Tracking) collections reveal that the methods derived from our modeling framework hold good promise for SDR and are very competitive with existing retrieval methods.

This paper adapts overlapped block motion compensation (OBMC) to suit variable block-size motion partitioning. The motion vectors (MVs) for various partitions are formalized as motion samples taken on an irregular grid. From this viewpoint, determining OBMC weights to associate with these samples becomes an under-determined problem since a distinct solution has to be sought for each prediction pixel. In this paper, we tackle this problem by expressing the optimal weights in closed form based on parametric signal assumptions. In particular, the computation of this solution requires only the geometric relations between the prediction pixel and its nearby block centers, leading to a generic framework capable of reconstructing temporal predictors from any irregularly sampled MVs. A modified implementation is also proposed to address the MV location uncertainty and to reduce computational complexity. Experimental results demonstrate that our scheme performs better than similar previous works, and when compared to the recently proposed Quadtree-based adaptive loop filter and enhanced adaptive interpolation filter, show a comparable gain. Furthermore, the combination of it with either of them gives a combined effect that is almost the sum of their separate improvements.

This paper introduces a bi-prediction scheme with only a motion overhead as for unidirectional prediction. It combines motion vectors found by template and block matchings with the overlapped block motion compensation (OBMC). An optimal window function is designed based on a model-based framework. Additionally, the concept of adaptive motion merging is incorporated to enable a template-matching-free implementation. Three algorithms featuring different performance and complexity trade-offs are implemented using the TMuC-0.9_HM software and tested with the common test conditions. Relative to the anchor, the best of them achieves an average BD-rate saving of 2.2%, with a minimum of 0.2% and a maximum of 4.7%.

High Efficiency Video Coding (HEVC) is a new international video coding standard that has been developed by the Joint Collaborative Team on Video Coding (JCT-VC). In this paper, an overview of the motion vector coding techniques for HEVC is presented. Our three proposed coding tools for the motion vector predictor (MVP) in the Inter, Skip and Merge modes of HEVC are also presented, which includes a new location of the temporal MVP, a priority-based derivation method of spatial MVPs, and a derivation method of temporal MVPs. A combination of these three tools can achieve on average 1.3%, 1.8%, 1.2% and 2.2% bit rate reductions for high efficiency random access, low complexity random access, high efficiency low delay, and low complexity low delay, respectively.

This paper addresses the problem of improving motion sampling efficiency for motion-compensated prediction (MCP). We provide a theoretical framework for analyzing the effect of motion sampling structure on MCP efficiency. It is shown that the sampling grid in-duced by the quadtree partition in H.264/AVC is suboptimal. To improve sampling efficiency, we propose a new pattern, which pro-vides sampling points at both the center and the top-left corner of a macroblcok. When contrasted with conventional NxN/2 block par-tition, the proposed scheme performs consistently and significantly better in subjective and objective quality. Index Terms — Motion Sampling, H.264/AVC, OBMC 1.

Template matching prediction (TMP), which estimates the motion for a target block by using its surrounding pixels, has been observed to perform efficiently in inter-frame coding. In this paper, we expose, from a more theoretical viewpoint, the factors that determine the prediction efficiency of TMP. It is shown that the motion estimate found by template matching tends to be the motion associated with the template centroid and that TMP consistently outperforms SKIP prediction, but hardly competes with block motion compensation (BMC) unless both the motion and intensity fields are less random or have high spatial correlation. We also demonstrate how template and block motion estimates can jointly be applied in a parametric overlapped block motion compensation (OBMC) framework to further improve temporal prediction. Preliminary results show that combining TMP with OBMC can yield 2-16% reductions in mean-square prediction error, as compared with the single use of OBMC. The gain is even higher (18%) when the performance is compared with that of the standard BMC.

This paper addresses the problem of refining depth information from the received reference and depth images within the MPEG FTV framework. An analytical model is first developed to approximate the per-pixel synthesis distortion (caused by depth-image compression) as a function of depth-error variances, intensity variations, ground-truth depth and virtual camera locations. We then follow the model to detect unreliable depth pixels by inspecting intensity gradients and to refine their values with a candidate-based block disparity search. Additional side information is transmitted to make both operations robust against compression effects. Experimental results show that our scheme offers an average PSNR improvement of 1.2 dB over MPEG FTV and consistently outperforms the state-of-the-art methods. Moreover, it can remove synthesis artifacts to a great extent, producing a result that is very close in appearance to the ground-truth view image.

This paper addresses the problem of adapting overlapped block motion compensation (OBMC) windows for use with variable blocksize motion estimates. We tackle the problem by using a parametric window design, which expresses, based on a statistical motion model, the optimal weights as a function of the distances between the predicted pixel and its nearby block centers. The formula enables both prediction weights and prediction order to be adapted on a pixelbypixel basis. Extensive experiments have been conducted using JM 12.4. Compared with conventional block motion compensation, our scheme shows a bitrate saving of 18% (5% on average) while maintaining the same or even higher PSNR (0.1 dB). It also provides a competitive advantage to variable block size motion compensation. Additionally, a hybrid of the two techniques achieves a further bitrate reduction of 13%. The result, nevertheless, provides only a lower bound on what is achievable since both motion estimation and mode decision were accomplished without considering OBMC. Further improvement is expected by incorporating iterative methods.

In this paper, we propose a system that can automatically segment a basketball video into several clips on the basis of a GOP-based scene change detection method. The length of each clip and the number of dominant color pixels of each frame are used to classify shots into close-up view, medium view, and full court view. Full court view shots are chosen to do advanced analyses such as ball tracking and parameter extracting for the transformation from a 3D real-world court to a 2D image. After that, we map points in the 2D image to the corresponding coordinates in a real-world court by some physical properties of the 3D shooting trajectory, and compute the statistics of all shooting positions. Eventually we can obtain the information about the most possible shooting positions of a professional basketball team, which is useful for opponents to adopt appropriate defense tactics.

MOTIVATION: The most commonly utilized microarrays for mRNA profiling (Affymetrix) include 'probe sets' of a series of perfect match and mismatch probes (typically 22 oligonucleotides per probe set). There are an increasing number of reported 'probe set algorithms' that differ in their interpretation of a probe set to derive a single normalized 'signal' representative of expression of each mRNA. These algorithms are known to differ in accuracy and sensitivity, and optimization has been done using a small set of standardized control microarray data. We hypothesized that different mRNA profiling projects have varying sources and degrees of confounding noise, and that these should alter the choice of a specific probe set algorithm. Also, we hypothesized that use of the Microarray Suite (MAS) 5.0 probe set detection p-value as a weighting function would improve the performance of all probe set algorithms. RESULTS: We built an interactive visual analysis software tool (HCE2W) to test and define parameters in Affymetrix analyses that optimize the ratio of signal (desired biological variable) versus noise (confounding uncontrolled variables). Five probe set algorithms were studied with and without statistical weighting of probe sets using the MAS 5.0 probe set detection p-values. The signal-to-noise ratio optimization method was tested in two large novel microarray datasets with different levels of confounding noise, a 105 sample U133A human muscle biopsy dataset (11 groups: mutation-defined, extensive noise), and a 40 sample U74A inbred mouse lung dataset (8 groups: little noise). Performance was measured by the ability of the specific probe set algorithm, with and without detection p-value weighting, to cluster samples into the appropriate biological groups (unsupervised agglomerative clustering with F-measure values). Of the total random sampling analyses, 50% showed a highly statistically significant difference between probe set algorithms by ANOVA [F(4,10) > 14, p < 0.0001], with weighting by MAS 5.0 detection p-value showing significance in the mouse data by ANOVA [F(1,10) > 9, p < 0.013] and paired t-test [t(9) = -3.675, p = 0.005]. Probe set detection p-value weighting had the greatest positive effect on performance of dChip difference model, ProbeProfiler and RMA algorithms. Importantly, probe set algorithms did indeed perform differently depending on the specific project, most probably due to the degree of confounding noise. Our data indicate that significantly improved data analysis of mRNA profile projects can be achieved by optimizing the choice of probe set algorithm with the noise levels intrinsic to a project, with dChip difference model with MAS 5.0 detection p-value continuous weighting showing the best overall performance in both projects. Furthermore, both existing and newly developed probe set algorithms should incorporate a detection p-value weighting to improve performance. AVAILABILITY: The Hierarchical Clustering Explorer 2.0 is available at http://www.cs.umd.edu/hcil/hce/ Murine arrays (40 samples) are publicly available at the PEPR resource (http://microarray.cnmcresearch.org/pgadatatable.asp http://pepr.cnmcresearch.org Chen et al., 2004).

Data analysis and visualization is strongly influenced by noise and noise filters. There are multiple sources of "noise" in microarray data analysis, but signal/noise ratios are rarely optimized, or even considered. Here, we report a noise analysis of a novel 13 million oligonucleotide dataset - 25 human U133A (/spl sim/500,000 features) profiles of patient muscle biopsies. We use our recently described interactive visualization tool, the hierarchical clustering explorer (HCE) to systemically address the effect of different noise filters on resolution of arrays into "correct" biological groups (unsupervised clustering into three patient groups of known diagnosis). We varied probe set interpretation methods (MAS 5.0, RMA), "present call" filters, and clustering linkage methods, and investigated the results in HCE. HCE's interactive features enabled us to quickly see the impact of these three variables. Dendrogram displays showed the clustering results systematically, and color mosaic displays provided a visual support for the results. We show that each of these three variables has a strong effect on unsupervised clustering. For this dataset, the strength of the biological variable was maximized, and noise minimized, using MAS 5.0, 10% present call filter, and average group linkage. We propose a general method of using interactive tools to identify the optimal signal/noise balance or the optimal combination of these three variables to maximize the effect of the desired biological variable on data interpretation.

BACKGROUND: We provide a systematic study of the sources of variability in expression profiling data using 56 RNAs isolated from human muscle biopsies (34 Affymetrix MuscleChip arrays), and 36 murine cell culture and tissue RNAs (42 Affymetrix U74Av2 arrays). RESULTS: We studied muscle biopsies from 28 human subjects as well as murine myogenic cell cultures, muscle, and spleens. Human MuscleChip arrays (4,601 probe sets) and murine U74Av2 Affymetrix microarrays were used for expression profiling. RNAs were profiled both singly, and as mixed groups. Variables studied included tissue heterogeneity, cRNA probe production, patient diagnosis, and GeneChip hybridizations. We found that the greatest source of variability was often different regions of the same patient muscle biopsy, reflecting variation in cell type content even in a relatively homogeneous tissue such as muscle. Inter-patient variation was also very high (SNP noise). Experimental variation (RNA, cDNA, cRNA, or GeneChip) was minor. Pre-profile mixing of patient cRNA samples effectively normalized both intra- and inter-patient sources of variation, while retaining a high degree of specificity of the individual profiles (86% of statistically significant differences detected by absolute analysis; and 85% by a 4-pairwise comparison survival method). CONCLUSIONS: Using unsupervised cluster analysis and correlation coefficients of 92 RNA samples on 76 oligonucleotide microarrays, we found that experimental error was not a significant source of unwanted variability in expression profiling experiments. Major sources of variability were from use of small tissue biopsies, particularly in humans where there is substantial inter-patient variability (SNP noise).

BACKGROUND: We describe the development, validation, and use of a highly redundant 120,000 oligonucleotide microarray (MuscleChip) containing 4,601 probe sets representing 1,150 known genes expressed in muscle and 2,075 EST clusters from a non-normalized subtracted muscle EST sequencing project (28,074 EST sequences). This set included 369 novel EST clusters showing no match to previously characterized proteins in any database. Each probe set was designed to contain 20-32 25 mer oligonucleotides (10-16 paired perfect match and mismatch probe pairs per gene), with each probe evaluated for hybridization kinetics (Tm) and similarity to other sequences. The 120,000 oligonucleotides were synthesized by photolithography and light-activated chemistry on each microarray. RESULTS: Hybridization of human muscle cRNAs to this MuscleChip (33 samples) showed a correlation of 0.6 between the number of ESTs sequenced in each cluster and hybridization intensity. Out of 369 novel EST clusters not showing any similarity to previously characterized proteins, we focused on 250 EST clusters that were represented by robust probe sets on the MuscleChip fulfilling all stringent rules. 102 (41%) were found to be consistently "present" by analysis of hybridization to human muscle RNA, of which 40 ESTs (39%) could be genome anchored to potential transcription units in the human genome sequence. 19 ESTs of the 40 ESTs were furthermore computer-predicted as exons by one or more than three gene identification algorithms. CONCLUSION: Our analysis found 40 transcriptionally validated, genome-anchored novel EST clusters to be expressed in human muscle. As most of these ESTs were low copy clusters (duplex and triplex) in the original 28,000 EST project, the identification of these as significantly expressed is a robust validation of the transcript units that permits subsequent focus on the novel proteins encoded by these genes.

The main contribution of this paper is the design of several efficient algorithms for modified run-length chain coding and for computing shape's moments on arrays with reconfigurable optical buses. We first propose two constant time algorithms for boundary extraction and run-length chain coding. Based on the modified run-length chain coding, and the advantages of both optical transmission and electronic computation, a constant time parallel algorithm for computing shape's moments using N/spl times/N processors is proposed. Based on the product of time and the number of processors used, the proposed parallel algorithms are time and cost optimal.