Causal inference has emerged as a promising approach for identifying decisive semantic factors and eliminating spurious correlations in visual representation learning. However, most existing methods rely on latent, data-driven confounder modeling, normally attributing the source of bias to background information while neglecting object-level semantic confusions that commonly occur in complex scenes. This limits their effectiveness in disentangling causal factors from confounding semantics. To address this challenge, we propose an explicit modeling approach for both causal factors and confounders, termed Explicit Modeling Causal Model (EMCM). The proposed framework consists of three key components. The Features Stability Estimation module explicitly models the relationship between visual semantics and class labels by leveraging clustering patterns to perform class-aware separation of causal and confounding factors. It produces class-specific causal factors and confounding factors linked to ambiguous categories. Subsequently, the Discriminative Features Enhancing module integrates causal factors into fused patch features via front-door intervention for stable semantics. In parallel, the Explicit Confounder Modeling and Debiasing Module learns confounders under clear label guidance and derives debiased context features by TDE modeling. This framework leverages two complementary causal perspectives to construct a unified semantic representation that facilitates improved generalization. Extensive experiments on two datasets demonstrate that EMCM effectively disentangles causal and confounding factors in complex scenarios, consistently outperforming state-of-the-art causal debiasing methods and text-guided methods in all metrics.

Video classification requires event-level representations of objects and their interactions. Existing methods typically rely on data-driven approaches, which either learn such features from whole frames or object-centric visual regions. Therefore, the modeling of spatiotemporal interactions among objects is usually overlooked. To address this issue, this paper presents a Decomposition of Synergistic, Unique, and Redundant Causal Representations Learning (SurdCRL) model for video classification, which introduces a newly-proposed SURD causal theory to model the spatiotemporal features of both object dynamics and their in- and cross-frame interactions. Specifically, SurdCRL employs three modules to model the object-centric spatiotemporal dynamics using distinct types of causal components, where the first module Spatial-Temporal Entity Modeling decouples the frame into object and context entities, and employs a temporal message passing block to capture object state changes over time, generating spatiotemporal features as basic causal variables. Second, the Dual-Path Causal Inference module mitigates confounders among causal variables by front-door and back-door interventions, thus enabling the subsequent causal components to reflect their intrinsic effects. Finally, the Causal Composition and Selection module employs the compositional structure-aware attention to project the causal variables and their high-order interactions into the synergistic, unique, and redundant components. Experiments on two benchmarking datasets verify that SurdCRL better captures event-relevant object-centric representation by decomposing spatiotemporal object interactions into three types of causal components.

Recommender system (RS) aims to capture personalized preferences from massive user behaviors, making them pivotal in the era of information explosion. However, the presence of “information cocoons,” interaction sparsity, cold-start problem, and feedback loops inherent in RS make users interact with a limited number of items. Conventional recommendation algorithms typically focus on the positive historical behaviors, while neglecting the essential role of negative feedback in user preference understanding. As a promising but easy-to-ignored area, negative sampling is proficient in revealing the genuine negative aspect inherent in user behaviors, emerging as an inescapable procedure in RS. In this survey, we first discuss existing user feedback, the critical role of negative sampling and the optimization objectives in RS, and thoroughly analyze challenges that consistently impede its progress. Then, we conduct an extensive literature review on the existing negative sampling strategies in RS and classify them into five categories with their discrepant techniques. Finally, we detail the insights of the tailored negative sampling strategies in diverse RS scenarios and outline an overview of the prospective research directions toward which the community may engage and benefit.

Incorporating tagging information to regularize the representation learning of images usually leads to improved performance in image classification by aligning the visual features with the textual ones of higher discriminative power. Existing methods typically follow the predictive approach, which uses tags as the semantic labels for visual input to make predictions. However, they typically face the problem of handling the heterogeneity between modalities. In order to learn accurate visual-semantic mapping, this paper presents a visual-semantic causal association modeling framework termed VSCNet. It aligns visual regions with tags, uses a pre-learned hierarchy of visual and semantic exemplars to refine tag predictions and constructs an augmented heterogeneous graph to perform causal intervention. Specifically, the fine-grained visual-semantic alignment (FVA) module adaptively locates the semantic-intensive regions corresponding to tags. The heterogeneous association refinement (HAR) module associates the visual regions, semantic elements and pre-learned visual prototypes in a heterogeneous graph to filter the error predictions and enrich the information. The causal inference with graphical masking (CIM) module applies self-learned masks to discover the causal nodes and edges in the heterogeneous graph to address the spurious association, forming robust causal representations. Experimental results from two benchmarking datasets show that VSCNet effectively builds the visual-semantic associations from images and leads to better performance than the state-of-the-art methods with enriched predictive information.

Data imbalance across clients in federated learning often leads to different local feature space partitions, harming the global model's generalization ability. Existing methods either employ knowledge distillation to guide consistent local training or performs procedures to calibrate local models before aggregation. However, they overlook the ill-posed model aggregation caused by imbalanced representation learning. To address this issue, this paper presents a cross-silo feature space alignment method (FedFSA), which learns a unified feature space for clients to bridge inconsistency. Specifically, FedFSA consists of two modules, where the in-silo prototypical space learning (ISPSL) module uses predefined text embeddings to regularize representation learning, which can improve the distinguishability of representations on imbalanced data. Subsequently, it introduces a variance transfer approach to construct the prototypical space, which aids in calibrating minority classes feature distribution and provides necessary information for the cross-silo feature space alignment (CSFSA) module. Moreover, the CSFSA module utilizes augmented features learned from the ISPSL module to learn a generalized mapping and align these features from different sources into a common space, which mitigates the negative impact caused by imbalanced factors. Experimental results from three datasets verified that FedFSA improves the consistency between diverse spaces on imbalanced data, which results in superior performance compared to existing methods.

Indoor scene synthesis aims to automatically produce plausible, realistic, and diverse 3D indoor scenes, especially given arbitrary user requirements. Recently, the promising generalization ability of pre-trained large language models (LLM) assist in open-vocabulary indoor scene synthesis. However, the challenge lies in converting the LLM-generated outputs into reasonable and physically feasible scene layouts. In this paper, we propose to generate hierarchically structured scene descriptions with LLM and then compute the scene layouts. Specifically, we train a hierarchy-aware network to infer the fine-grained relative positions between objects and design a divide-and-conquer optimization to solve for scene layouts. The advantages of using hierarchically structured scene representation are two-fold. First, the hierarchical structure provides a rough grounding for object arrangement, which alleviates contradictory placements with dense relations and enhances the generalization ability of the network to infer fine-grained placements. Second, it naturally supports the divide-and-conquer optimization, by first arranging the sub-scenes and then the entire scene, to more effectively solve for a feasible layout. We conduct extensive comparison experiments and ablation studies with both qualitative and quantitative evaluations to validate the effectiveness of our key designs with the hierarchically structured scene representation. Our approach can generate more reasonable scene layouts while better aligned with the user requirements and LLM descriptions. We also present open-vocabulary scene synthesis and interactive scene design results to show the strength of our approach in the applications.

Multimodal recommendation (MMRec) aims to integrate multimodal information of items to address the inherent data sparsity issue in collaborative-based recommendation. Traditional MMRec methods typically capture the structure-level item representations from the observed user behaviors within the multimodal graph, overlooking the potential impact of negative instances for personalized preference understanding. In light of the outstanding generative ability and step-by-step inference characteristic of Diffusion Models (DMs), we propose a Curriculum Conditioned Diffusion framework for Multimodal Recommendation (CCDRec), which precisely excavates the modality-aware distribution-level correlation among multi-modalities and elegantly integrates the reverse phase of DMs into negative sampling to highlight the most suitable instances in a curricular manner. Specifically, CCDRec proposes the Diffusion-controlled Multimodal Aligning module (DMA) to align multimodal knowledge with collaborative signals by capturing the fine-grained relationships among multi-modalities in the probabilistic distribution space. Furthermore, CCDRec designs the Negative-sensitive Diffusive Inferring module (NDI) to progressively synthesize the negative sample pool with diverse hardness to support the following knowledge-aware negative sampling. To gradually ramp up the training complexity, CCDRec further introduces a Curricular Negative Sampler (CNS) to tally the curriculum learning paradigm with the reverse phase of DMA, thereby adaptively sampling the gold-standard negative instances to enhance optimization. Extensive experiments on three datasets with four diverse backbones demonstrate the effectiveness and robustness of our CCDRec. The visualization analyses also clarify the underlying mechanism of our DMA in multimodal representation alignment and CNS in curricular negative discovery. The code and the corresponding dataset will be uploaded in the Appendix.

Indoor positioning with 5G technologies depends on accurate Channel State Information (CSI) for high-precision location services, but limited data, especially in low signal-to-noise ratio (SNR) environments, presents a significant challenge. Traditional deep learning methods, relying on "Learning + Finetuning", require large datasets at specific SNR levels, limiting their effectiveness. This paper introduces a novel CSI image generation and restoration method designed for small-sample conditions, allowing data from various SNR levels to be utilized to improve downstream task performance. We propose a new CSI generation model that leverages graph neural networks and a multi-level loss function to accurately generate CSI images corresponding to specific SNRs. Our approach addresses the unique characteristics of CSI signals that traditional image generation methods fail to capture. Experiments show that our approach significantly improves prediction accuracy, outperforming existing methods, particularly in low-SNR scenarios. Our core code is publically available at https://github.com/kaqiz/DARC.

Remote Sensing Change Detection (RSCD) is essential for identifying alterations within geographical landscapes based on dual-temporal imagery. Current methods often enhance global modeling through the use of Transformers or integrate global and local features in a coarse-grained manner. The former tends to overlook critical details, such as change boundaries and textures, while the latter defines detailed features based solely on the receptive field, with feature fusion methods often operating as a black box. In this paper, we propose a novel feature refinement methodology guided by relation preferences, specifically designed for RSCD tasks. Our approach decomposes feature extraction into components that emphasize detailed and global relationships, employing 3D small window partitioning and information-preserving extraction techniques. By minimizing mutual information between refined features and their base counterparts, we enhance feature separability, reducing information coupling. This allows for effective change detection using a simple MLP structure. Experimental results on three datasets demonstrate that our method surpasses previous state-of-the-art models, achieving 89.01% IoU, 63.29% IoU, and 84.73% IoU on the Google, CLCD, and PRELEVIR datasets, respectively, while maintaining superior performance and computational efficiency. Code can be available: https://github.com/mxx123321/remote_sensing.

Fifth-generation communication technology enables advanced indoor positioning with its high bandwidth and frequency capabilities. However, indoor environment variability causes signal propagation fluctuations, making existing models inadequate for accurate location estimation. In this paper, we demonstrate that integrating SNR recognition training with the positioning network can enhance localization performance. Specifically, by leveraging a general neural network to extract common features from 5G Channel State Information (CSI), we employ two carefully designed downstream networks for signal-to-noise ratio (SNR) estimation and location prediction. In order to improve localization performance, we utilize mutual information-based metric to direct each network head towards its specific task, thereby refining and enhancing the features pertinent to each task. Moreover, for enhanced adaptive positioning accuracy, the coefficients linked to the features used in SNR and position estimation are determined based on the hidden features of the corresponding identification head. These coefficients are then utilized to appropriately weigh the hidden features of the location regression module. Experimental results using datasets from real-world scenarios demonstrate that our proposed method significantly improves positioning accuracy over state-of-the-art (SOTA) methods. Code can be available at: https://github.com/mxx123321/indoor_snr.

Artistic typography visualizes the meaning of input characters by harmonizing font and imagery. However, current methods face significant challenges in balancing artistic expression with readability, precise control, and with limited support for non-Latin scripts. To address these issues, we propose ArtTypo, a novel multimodal guided, multi-level controlled framework for artistic typography with iterative feedback. To improve artistic expression, we follow the principles of art design, implement a Chain-of-Thought approach through Multimodal Large Language Models to integrate user intentions, and a feedback module for iterative refinement of outputs. For precise and multi-level control, we introduce auto path match to extract vector paths aligned with multimodal input. Additionally, we develop a texture with background preserving diffusion process, ensuring clean outputs and artistic expression as well. Experiments demonstrate that ArtTypo effectively generates diverse artistic typography, consistently producing visually appealing and contextually sensitive results.

Attribute skew in federated learning leads local models to focus on learning non-causal associations, guiding them towards inconsistent optimization directions, which inevitably results in performance degradation and unstable convergence. Existing methods typically leverage data augmentation to enhance sample diversity or employ knowledge distillation to learn invariant representations. However, the instability in the quality of generated data and the lack of domain information limit their performance on unseen samples. To address these issues, this paper presents a global intervention and distillation method, termed FedGID, which utilizes diverse attribute features for backdoor adjustment to break the spurious association between background and label. It includes two main modules, where the global intervention module adaptively decouples objects and backgrounds in images, injects background information into random samples to intervene in the sample distribution, which links backgrounds to all categories to prevent the model from treating background-label associations as causal. The global distillation module leverages a unified knowledge base to guide the representation learning of client models, preventing local models from overfitting to client-specific attributes. Experimental results on three datasets demonstrate that FedGID enhances the model’s ability to focus on the main subjects in unseen data and outperforms existing methods in collaborative modeling.

For individuals in situations where their hands are occupied, no-handed touch interaction represents a significant yet underexplored area. In this paper, we investigate touch performance on screens using body parts (the nose, tongue, chin, elbow, and toe) rather than the hand. Experiments were conducted with 37 participants to analyze their touch interaction, where they selected targets using various body parts under different conditions. We employed the Dual-Gaussian model and Ternary-Gaussian model to describe the touch endpoints of these body parts. Based on the statistical criterion derived from the models, we proposed a no-handed assistive technology, which achieved up to a 24.4% improvement in tapping speed and a 58.47% increase in tapping accuracy. Additionally, this paper provides HCI developers with design guidelines for optimizing no-handed touch interaction interfaces.

The 3D perception of satellites, including both their shape and pose, is a key foundation for robotic on-orbit servicing. However, the demanding space environment-such as intense and dim illumination-presents significant challenges. Previous non-cooperative methods focus on specific geometric features like solar panel brackets or docking rings, overlooking the satellite's overall shape and increasing the risk of collisions during grasping. Additionally, satellites are often weakly textured, limiting the accuracy of 3D perception. To address these issues, we propose, for the first time, a 3D perceptionbased visual servo system of non-cooperative satellites. This system combines reconstruction and tracking to enhance shape perception and pose estimation accuracy in orbital conditions. Specifically, we employ an alternating iterative strategy to simultaneously reconstruct and track the satellite and introduce a novel constraint to fuse different cues under extreme conditions. Further, we develop a simulation environment platform, a dualarm microgravity grasping system, and an online monitoring module to enhance system capabilities for on-orbit servicing. Synthetic and real-world datasets from the simulation environment are also created for experimental validation. Results show that each module of our system achieves state-of-the-art performance.

Cross-modal alignment is an effective approach to improving visual classification. Existing studies typically enforce a one-step mapping that uses deep neural networks to project the visual features to mimic the distribution of textual features. However, they typically face difficulties in finding such a projection due to the two modalities in both the distribution of class-wise samples and the range of their feature values. To address this issue, this paper proposes a novel Semantic-Space-Intervened Diffusive Alignment method, termed SeDA, models a semantic space as a bridge in the visual-to-textual projection, considering both types of features share the same class-level information in classification. More importantly, a bi-stage diffusion framework is developed to enable the progressive alignment between the two modalities. Specifically, SeDA first employs a Diffusion-Controlled Semantic Learner to model the semantic feature space of visual features by constraining the interactive features of the diffusion model and the category centers of visual features. In the later stage of SeDA, the Diffusion-Controlled Semantic Translator focuses on learning the distribution of textual features from the semantic space. Meanwhile, the Progressive Feature Interaction Network introduces stepwise feature interactions at each alignment step, progressively integrating textual information into mapped features. Experimental results show that SeDA achieves stronger cross-modal feature alignment, leading to superior performance over existing methods across multiple scenarios.

Attribute bias in federated learning (FL) typically leads local models to optimize inconsistently due to the learning of non-causal associations, resulting degraded performance. Existing methods either use data augmentation for increasing sample diversity or knowledge distillation for learning invariant representations to address this problem. However, they lack a comprehensive analysis of the inference paths, and the interference from confounding factors limits their performance. To address these limitations, we propose the Federated Deconfounding and Debiasing Learning (FedDDL) method. It constructs a structured causal graph to analyze the model inference process, and performs backdoor adjustment to eliminate confounding paths. Specifically, we design an intra-client deconfounding learning module for computer vision tasks to decouple background and objects, generating counterfactual samples that establish a connection between the background and any label, which stops the model from using the background to infer the label. Moreover, we design an inter-client debiasing learning module to construct causal prototypes to reduce the proportion of the background in prototype components. Notably, it bridges the gap between heterogeneous representations via causal prototypical regularization. Extensive experiments on 2 benchmarking datasets demonstrate that FedDDL significantly enhances the model capability to focus on main objects in unseen data, leading to 4.5% higher Top-1 Accuracy on average over 9 state-of-the-art existing methods.

Causal inference has emerged as a promising approach to mitigate long-tail classification by handling the biases introduced by class imbalance. However, along with the change of advanced backbone models from Convolutional Neural Networks (CNNs) to Visual Transformers (ViT), existing causal models may not achieve an expected performance gain. This paper investigates the influence of existing causal models on CNNs and ViT variants, highlighting that ViT's global feature representation makes it hard for causal methods to model associations between fine-grained features and predictions, which leads to difficulties in classifying tail classes with similar visual appearance. To address these issues, this paper proposes TSCNet, a two-stage causal modeling method to discover fine-grained causal associations through multi-scale causal interventions. Specifically, in the hierarchical causal representation learning stage (HCRL), it decouples the background and objects, applying backdoor interventions at both the patch and feature level to prevent model from using class-irrelevant areas to infer labels which enhances fine-grained causal representation. In the counterfactual logits' bias calibration stage (CLBC), it refines the optimization of model's decision boundary by adaptive constructing counterfactual balanced data distribution to remove the spurious associations in the logits caused by data distribution. Extensive experiments conducted on various long-tail benchmarks demonstrate that the proposed TSCNet can eliminate multiple biases introduced by data imbalance, which outperforms existing methods.

In recent years, the demand for indoor positioning services has surged due to urbanization and the widespread use of mobile devices. Unlike outdoor environments, where GNSS signals struggle to penetrate indoor spaces, traditional RSSI-based fingerprinting methods encounter challenges posed by severe signal attenuation caused by multipath effects. Channel State Information (CSI) emerges as a promising alternative, offering fine-grained signal propagation characteristics that are advantageous for indoor positioning. Despite the potential of CSI-based positioning, challenges remain in effectively capturing features from CSI matrices with high aspect ratios. Deep learning methods have gained traction in addressing indoor positioning challenges, yet existing approaches often overlook the correlation between different subcarriers in CSI. To address these issues, this paper proposes a lightweight indoor positioning model comprising a convolutional reparameterization-based feature fusion module and a spatial modeling module based on graph neural networks. Experimental results demonstrate the effectiveness of the proposed model in achieving high-precision indoor positioning.

The clustering performance of Fuzzy Adaptive Resonance Theory (Fuzzy ART) is highly dependent on the preset vigilance parameter, where deviations in its value can lead to significant fluctuations in clustering results, severely limiting its practicality for non-expert users. Existing approaches generally enhance vigilance parameter robustness through adaptive mechanisms such as particle swarm optimization and fuzzy logic rules. However, they often introduce additional hyperparameters or complex frameworks that contradict the original simplicity of the algorithm. To address this, we propose Iterative Refinement Fuzzy Adaptive Resonance Theory (IR-ART), which integrates three key phases into a unified iterative framework: (1) Cluster Stability Detection: A dynamic stability detection module that identifies unstable clusters by analyzing the change of sample size (number of samples in the cluster) in iteration. (2) Unstable Cluster Deletion: An evolutionary pruning module that eliminates low-quality clusters. (3) Vigilance Region Expansion: A vigilance region expansion mechanism that adaptively adjusts similarity thresholds. Independent of the specific execution of clustering, these three phases sequentially focus on analyzing the implicit knowledge within the iterative process, adjusting weights and vigilance parameters, thereby laying a foundation for the next iteration. Experimental evaluation demonstrates that IR-ART improves tolerance to suboptimal vigilance parameter values while preserving the parameter simplicity of Fuzzy ART. Case studies visually confirm the algorithm’s self-optimization capability through iterative refinement, making it particularly suitable for non-expert users in resource-constrained scenarios.

The personalized text-to-image generation has rapidly advanced with the emergence of Stable Diffusion. Existing methods, which typically fine-tune models using embedded identifiers, often struggle with insufficient stylization and inaccurate image content due to reduced textual controllability. In this paper, we propose style refinement and content preservation strategies. The style refinement strategy leverages the semantic information of visual reasoning prompts and reference images to optimize style embeddings, allowing a more precise and consistent representation of style information. The content preservation strategy addresses the content bias problem by preserving the model’s generalization capabilities, ensuring enhanced textual controllability without compromising stylization. Experimental results verify that our approach achieves superior performance in generating consistent and personalized text-to-image outputs.

Federated Class Incremental Learning (FCIL) aims to collaboratively process continuously increasing incoming tasks across multiple clients. Among various approaches, data replay has become a promising solution, which can alleviate forgetting by reintroducing representative samples from previous tasks. However, their performance is typically limited by class imbalance, both within the replay buffer due to limited global awareness and between replayed and newly arrived classes. To address this issue, we propose a class-wise balancing data replay method for FCIL (FedCBDR), which employs a global coordination mechanism for class-level memory construction and reweights the learning objective to alleviate the aforementioned imbalances. Specifically, FedCBDR has two key components: 1) the global-perspective data replay module reconstructs global representations of prior task knowledge in a privacy-preserving manner, which then guides a class-aware and importance-sensitive sampling strategy to achieve balanced replay; 2) Subsequently, to handle class imbalance across tasks, the task-aware temperature scaling module adaptively adjusts the temperature of logits at both class and instance levels based on task dynamics, which reduces the model’s overconfidence in majority classes while enhancing its sensitivity to minority classes. Experimental results verified that FedCBDR achieves balanced class-wise sampling under heterogeneous data distributions and improves generalization under task imbalance between earlier and recent tasks, yielding a 2%-15% Top-1 accuracy improvement over six state-of-the-art methods.

Federated Prompt Learning (FPL) enables communication-efficient adaptation by tuning lightweight prompts on top of frozen pre-trained models. Existing FPL methods typically rely on global information, which is only available after the second training round, to facilitate collaboration among client models. Therefore, they are inherently dependent on multi-round communication to fully exhibit their strengths. Moreover, existing one-shot federated learning methods typically focus on fitting seen tasks, but lack cross-task generalization. To bridge this gap, we propose the global prompt refinement with non-interfering attention masking (GPR-NIAM) method for one-shot FPL. The core idea is to design a masking mechanism that restricts excessive interaction between the original text embeddings and the learnable prompt embeddings. GPR-NIAM achieves this through the collaboration of two key modules. Firstly, the attention isolation module suppresses attention from the learnable prompt tokens to the original text tokens, and reweights the reverse attention which preserves generalization across tasks. Secondly, the cross-silo collaborative refinement module integrates decentralized visual knowledge into a unified base and calibrates the global prompt through multi-source cross-modal knowledge alignment, further mitigating the inconsistency caused by data heterogeneity. Extensive experiments conducted on ten benchmark datasets under two tasks show that GPR-NIAM outperforms eight state-of-the-art methods in both class-level and domain-level generalization.

Soft shadows play a crucial role in enhancing visual realism in real-time rendering. Although traditional shadow mapping techniques offer high efficiency, they often suffer from artifacts and limited quality. In contrast, ray tracing can produce high-fidelity soft shadows but incurs substantial computational cost. In this paper, we propose a general-purpose, real-time soft shadow generation method based on neural networks. To encode shadow geometry, we employ the hard shadows via shadow mapping as input to our network, which effectively captures the spatial layout of shadow positions and contours. A lightweight U-Net architecture then refines this input to synthesize high-quality soft shadows in real time. The generated shadows closely approximate ray-traced references in visual fidelity. Compared to existing learning-based methods, our approach produces higher-quality soft shadows and offers improved generalization across diverse scenes. Furthermore, it requires no scene-specific precomputation, making it directly applicable to practical real-time rendering scenarios.

With the deepening exploitation of marine resources, the application of underwater communication is becoming increasingly widespread, and the noise in the underwater channel will severely impact the reliability of underwater communication. Therefore, researchers have focused on improving the accuracy and reliability of underwater communication through denoising. Currently, denoising methods are widely used based on algorithms, such as Fourier transform, wavelet transform, convolutional neural network, and time-space domain filtering. Still, they all have some areas for improvement in the application of underwater communication. In this regard, inspired by quantum mechanical tools and adaptive algorithms, this article proposes a quantum adaptive chunk denoising algorithm applied in underwater communication. First, this article proposes an adaptive chunking algorithm to dynamically adjust the chunk size according to the local signal characteristics to capture the signal details better. Second, the chunked signal and image are taken as potentials in the discrete Schrödinger equation and solved for the eigenvalues and eigenvectors of the discrete Hamiltonian matrix constructed by the adaptive chunking algorithm. Finally, the solved values are subjected to total variational denoising and reconstruction after denoising. The algorithm proposed in this article is compared with wavelet denoising based on hard and soft thresholding, Denoising Algorithm Based on an ideal low-pass adaptive filter, sparsity-assisted signal smoothing (SASS) algorithm and Quantum Mechanics Adaptive Basis Denoising Algorithm. The experimental results show that the quantum adaptive chunking denoising algorithm proposed in this article can effectively remove underwater noise from signals and images under the condition of low-signal-to-noise ratio, which improves the reliability and signal quality of underwater communication and meets the denoising standard of underwater communication.

Remote sensing change detection (RSCD) is essential for monitoring surface changes, requiring features that capture both categorical and fine boundary details at high resolution. Deep learning methods typically fuse upsampled coarse features from deep layers with high-resolution shallow features to form comprehensive semantic representations. However, the fixed grid structure in Euclidean space limits the handling of irregular boundaries and edges, often resulting in blurred boundaries, especially near ambiguous thresholds. To overcome these limitations, we propose a GNN-based feature fusion method that integrates high- and low-frequency information with a mutual information-driven semantic enhancement module. Specifically, we leverage non-Euclidean space to enable flexible spatial transformations during feature extraction. In our framework, low-frequency extraction captures global structure and background features for intra-class consistency, while high-frequency extraction preserves edge details for boundary clarity. After fusing these complementary features, we further optimize their mutual information to suppress background noise and irrelevant changes, enabling the model to focus on true change regions and thereby improving detection accuracy and overall performance. The code is available at https://github.com/sundaobo/GFMNet.

Federated learning aims to facilitate collaborative training among multiple clients with data heterogeneity in a privacy-preserving manner, which either generates the generalized model or develops personalized models. However, existing methods typically struggle to balance both directions, as optimizing one often leads to failure in another. To address the problem, this article presents a method named personalized federated learning via cross silo prototypical calibration (pFedCSPC) to enhance the consistency of knowledge of clients by calibrating features from heterogeneous spaces, which contributes to enhancing the collaboration effectiveness between clients. Specifically, pFedCSPC employs an adaptive aggregation method to offer personalized initial models to each client, enabling rapid adaptation to personalized tasks. Subsequently, pFedCSPC learns class representation patterns on clients by clustering, averages the representations within each cluster to form local prototypes, and aggregates them on the server to generate global prototypes. Meanwhile, pFedCSPC leverages global prototypes as knowledge to guide the learning of local representation, which is beneficial for mitigating the data imbalanced problem and preventing overfitting. Moreover, pFedCSPC has designed a cross-silo prototypical calibration (CSPC) module, which utilizes contrastive learning techniques to map heterogeneous features from different sources into a unified space. This can enhance the generalization ability of the global model. Experiments were conducted on four datasets in terms of performance comparison, ablation study, in-depth analysis, and case study, and the results verified that pFedCSPC achieves improvements in both global generalization and local personalization performance via calibrating cross-source features and strengthening collaboration effectiveness, respectively.

Service level agreements (SLAs) formulated by cloud rendering service providers and users are varied, as users may have diverse performance requirements for their own jobs. This leads to a complex issue that cloud resources need to be allocated to rendering jobs in an appropriate and effective manner to satisfy users' diverse SLAs. To address this issue, in this paper, we propose a novel fair and efficient resource allocation algorithm, which aims to maximize the execution efficiency of rendering service applications while satisfying users' diverse SLAs. Firstly, to satisfy users' diverse SLAs, we propose a rigorous definition ofweighted acceleration ratio fairness, whose guiding principle is that the execution speed of a rendering job should be proportional to its weight determined by users' SLAs. Then, under the guidance of the proposed principle of acceleration ratio fairness, we formulate a new algorithm to fairly allocate resources to rendering jobs. Lastly, to improve execution efficiency and coordinate efficiency and fairness, we propose a fair and efficient resource allocation algorithm with relaxing fairness in resource competitive and non-competitive situations separately for rendering service applications. With extensive experiments that involve real rendering application workloads, we validate the effectiveness of our algorithms in improving execution efficiency and satisfying users' diverse SLAs.

With technological advancements, virtual reality (VR), once limited to high-end professional applications, is rapidly expanding into entertainment and broader consumer domains. However, the inherent contradiction between mobile hardware computing power and the demand for high-resolution, high-refresh-rate rendering has intensified, leading to critical bottlenecks, including frame latency and power overload, which constrain large-scale applications of VR systems. This study systematically analyzes four key technologies for efficient VR rendering: (1) foveated rendering, which dynamically reduces rendering precision in peripheral regions based on the physiological characteristics of the human visual system (HVS), thereby significantly decreasing graphics computation load; (2) stereo rendering, optimized through consistent stereo rendering acceleration algorithms; (3) cloud rendering, utilizing object-based decomposition and illumination-based decomposition for distributed resource scheduling; and (4) low-power rendering, integrating parameter-optimized rendering, super-resolution technology, and frame-generation technology to enhance mobile energy efficiency. Through a systematic review of the core principles and optimization approaches of these technologies, this study establishes research benchmarks for developing efficient VR systems that achieve high fidelity and low latency while providing further theoretical support for the engineering implementation and industrial advancement of VR rendering technologies.

Scene sketch-to-image synthesis is a challenging task, especially when the sketches contain multiple objects of different classes. Existing methods interfere between different classes of objects when generating images from scene sketches, making it difficult to synthesis images with accurate object classes. In this paper, we propose a scene sketch-to-image generation method based on multi-object control, which can generate high-quality and class-accurate images from scene sketches and text prompts. We propose a sampling strategy based on segmentation mask and independent denoising, which can accurately control the classes of foreground objects and make foreground objects and background more harmonized. Our method is based on a pre-trained diffusion model without additional training overhead. Experiments on SketchyCOCO and SketchyScene datasets demonstrate that our method’s capacity to generate realistic complex images from scene sketches and text prompts.

Neonatal seizures are a prevalent clinical manifestation of neurological disorders and can potentially impact the neurodevelopment of the infant’s brain. Accurate and timely detection of neonatal epilepsy is crucial for early diagnosis and treatment. However, because of the complexity of newborn brains and signal instability, current seizure detection methods often produce false positives and negatives. In this work, we propose an automated scheme for detecting neonatal seizures, namely MSHENet. Through a multi-scale hierarchical modeling approach, the model constructs deep learning models at various levels, including the time dimension, channel dimension, and local details, to differentiate between normal brain electrical activity and epileptic seizures. Experiments conducted on a comprehensive real public neonatal electroencephalogram (EEG) dataset demonstrate that our method achieves higher accuracy in detection and a lower rate of false positives and negatives.

Out-of-distribution issues lead to different optimization directions between clients, which weakens collaborative modeling in federated learning. Existing methods aims to decouple invariant features in the latent space to mitigate attribute bias. However, their performance is limited by suboptimal decoupling capabilities in complex latent spaces. To address this problem, this paper presents a method, termed FedAKD, that adaptively identifies meaningful visual regions in images to guide the model in learning causal features. It includes two main modules, where the attentive modeling module adaptively locates critical regions to mitigate the negative impact of irrelevant elements, which are considered significant contributors to distribution heterogeneity. The attention-guided representation learning module leverages attentive knowledge to guide the local model to pay more attention to important regions, which acts as a soft attention regularizer to mitigate the trade-off between capturing category-relevant information and irrelevant contextual information in images. Experiments were conducted on four datasets, including performance comparison, ablation study, and case study. The results demonstrate that FedAKD can effectively enhance attention to causal features, which leads to superior performance compared with the state-of-the-art methods. The source codes have been released at https://github.com/qizhuang-qz/FedAKD.

In recent research literature, the layout to image domain has gained significant traction. Research based on GANs can generate complete images, but there are still issues with insufficient details and overall quality not being high. Diffusion models, on the other hand, challenges still exist in ensuring the quality of image details in complex scene regions, as well as ensuring the smooth expression of the overall semantic information. To address these challenges, we propose LayoutDM based on multi-scale diffusion, which employs the Parallel Sampling Module to enhance local precision and ensures global semantic coherence through the Semantic Coherence Module. Significantly, our approach generates within the visible space, progressively revealing more details and semantic information. Simultaneously, our method enhances layout part handling via parallel region-wise clip guidance, achieving strong zero-shot generation without direct training samples. Tests on COCO-stuff and VG datasets confirm that our approach achieves both fine-grained object generation and overall visual effectiveness.

Ultra reliable low latency communication (URLLC) has emerged as a crucial element in various communication services due to its ability to provide highly dependable and real-time connections. However, existing time series models utilized in URLLC scenarios often overlook the interconnections among delay cycles, trends, and bursts. Therefore, they fail to effectively capture the temporal periodic statistical characteristics exhibited by real data. To address this limitation, we propose a network delay prediction method that utilizes graph neural networks to aggregate different characteristic trends. Specifically, we employ channels with independently learnable weights of varying magnitudes to capture the inherent periodic patterns of cycles, trends, and short fluctuations. To obtain a generalized feature representation from the combined features, we map them into a non-Euclidean space and leverage mature graph convolution techniques to aggregate structural features. Results on real world URLLC datasets show that our approach can reduce the MSE error by up to 50.1% compared to current methods.

Leveraging the enhanced bandwidth and the advantages conferred by Massive Multiple-Input Multiple-Output (MIMO) technology, 5G New Radio (NR) extends unprecedented prospects for high-fidelity indoor positioning systems. However, the expensive equipment required for 5G CSI collection and the complex workforce involved often lead to a scarcity of "CSI-position" samples, hindering the deployment of deep learning (DL)-based fingerprint positioning methods. In this paper, a novel semi-supervised contrastive learning method is proposed for few-shot 5G fingerprint positioning to fully leverage the untapped potential of unlabeled 5G CSI data in modeling the "CSI to position" representation. Specifically, we encourage consistent representations of the same CSI under different channel conditions and propose a straightforward yet effective technique to simulate positive samples from diverse channel conditions. Additionally, we introduce range encoding to facilitate subcarrier strength position awareness for 5G CSI. Experimental results on 1000 location samples in a training-unknown positioning setting demonstrate that our method, with just fine-tuning an MLP, can achieve performance comparable to fully supervised approaches. Our code can be accessed at https://github.com/mxx123321/few-shot-learning-for-5G-indoor-positioning.

In recent years, diffusion models have dominated the field of image generation with their outstanding generation quality. However, pre-trained large-scale diffusion models are generally trained using fixed-size images, and fail to maintain their performance at different aspect ratios. Existing methods for generating arbitrary-size images based on diffusion models face several issues, including the requirement for extensive finetuning or training, sluggish sampling speed, and noticeable edge artifacts. This paper presents the InstantAS method for arbitrary-size image generation. This method performs non-overlapping minimum coverage segmentation on the target image, minimizing the generation of redundant information and significantly improving sampling speed. To maintain the consistency of the generated image, we also proposed the Inter-Domain Distribution Bridging method to integrate the distribution of the entire image and suppress the separation of diffusion paths in different regions of the image. Furthermore, we propose the dynamic semantic guided cross-attention method, allowing for the control of different regions using different semantics. Experimental results show that InstantAS has better fusion capabilities compared to previous arbitrary-size image generation methods and is far ahead in sampling speed compared to them.

Classifying videos differs from that of images in the need to capture the information on what has happened, instead of what is in the frames. Conventional methods typically follow the data-driven approach, which uses transformer-based attention models to extract and aggregate the features of video frames as the representation of the entire video. However, this approach tends to extract the object information of frames and may face difficulties in classifying the classes talking about events, such as "fixing bicycle". To address this issue, This paper presents an Event-level Causal Representation Learning (ECRL) model for the spatio-temporal modeling of both the in-frame object interactions and their cross-frame temporal correlations. Specifically, ECRL first employs a Frame-to-Video Causal Modeling (F2VCM) module, which simultaneously builds the in-frame causal graph with the background and foreground information and models their cross-frame correlations to construct a video-level causal graph. Subsequently, a Causality-aware Event-level Representation Inference (CERI) module is introduced to eliminate the spurious correlations in contexts and objects via the back- and front-door interventions, respectively. The former involves visual context de-biasing to filter out background confounders, while the latter employs global-local causal attention to capture event-level visual information. Experimental results on two benchmarking datasets verified that ECRL may better capture the cross-frame correlations to describe videos in event-level features. The source codes have been released at https://github.com/wyqcrystal/ECRL.

Remote sensing image change detection (RSCD) is an active research area in remote sensing. Existing convolutional neural networks (CNN) and Transformer methods operate on images in Euclidean space, limiting flexible perception and interaction among different positions. Recent advancements in various fields have focused on transferring feature processing to non-Euclidean space, allowing for more flexible feature learning. Mapping different locations of high-resolution remote sensing images into non-Euclidean space enables more flexible perception of individual pixels. Motivated by this, we have developed a method solely based on GNN, termed Pure GNN-based Method with the Mutual Information Sensing (PGNet), specifically for detecting changes in remote sensing. In the PGNet framework, Graph Convolutional Networks (GCNs) are employed to derive graph-level characteristics. By aggregating information from neighboring nodes in the graph, we capture richer and more detailed features. Subsequently, by fusing the shallow and deep distributions of local and global features of bi-temporal remote sensing images, we achieve an effective combination of fine-grained and macro-level information. Finally, in order to make explicit semantic perception of bi-temporal images, we propose a Mutual Information Sensing module, which encourages bi-temporal feature pairs to enhance their understanding of each other's semantic information. To confirm the efficacy of PGNet, we carried out a series of tests with advanced techniques across multiple datasets. The outcomes show that our model surpasses existing CNN and Transformer-based methods in precision.

Abstract Freehand sketch‐to‐image (S2I) is a challenging task due to the individualized lines and the random shape of freehand sketches. The multi‐class freehand sketch‐to‐image synthesis task, in turn, presents new challenges for this research area. This task requires not only the consideration of the problems posed by freehand sketches but also the analysis of multi‐class domain differences in the conditions of a single model. However, existing methods often have difficulty learning domain differences between multiple classes, and cannot generate controllable and appropriate textures while maintaining shape stability. In this paper, we propose a style‐guided multi‐class freehand sketch‐to‐image synthesis model, SMFS‐GAN, which can be trained using only unpaired data. To this end, we introduce a contrast‐based style encoder that optimizes the network's perception of domain disparities by explicitly modelling the differences between classes and thus extracting style information across domains. Further, to optimize the fine‐grained texture of the generated results and the shape consistency with freehand sketches, we propose a local texture refinement discriminator and a Shape Constraint Module, respectively. In addition, to address the imbalance of data classes in the QMUL‐Sketch dataset, we add 6K images by drawing manually and obtain QMUL‐Sketch+ dataset. Extensive experiments on SketchyCOCO Object dataset, QMUL‐Sketch+ dataset and Pseudosketches dataset demonstrate the effectiveness as well as the superiority of our proposed method.

Automatic font generation is of great benefit to improving the efficiency of font designers. Few-shot font generation aims to generate new fonts from a few reference samples, and has recently attracted a lot of attention from researchers. This is valuable but challenging, especially for ideograms with high diversity and complex structures. Existing models based on convolutional neural networks (CNNs) struggle to generate glyphs with accurate font style and stroke details in the few-shot setting. This paper proposes the TransFont, exploiting the long-range dependency modeling ability of the Vision Transformer (ViT) for few-shot font generation. For the first time, we empirically show that the ViT is better at glyph image generation than CNNs. Furthermore, based on the observation of the high redundancy in the glyph feature map, we introduce the glyph self-attention module for mitigating the quadratic computational and memory complexity of the pixel-level glyph image generation, along with several new techniques, i.e., multi-head multiple sampling, yz axis convolution, and approximate relative position bias. Extensive experiments on two Chinese font libraries show the superiority of our method over existing CNN-based font generation models, the proposed TransFont generates glyph images with more accurate font style and stroke details.

The prevalence of long-tailed distributions in real-world data often results in classification models favoring the dominant classes, neglecting the less frequent ones. Current approaches address the issues in long-tailed image classification by rebalancing data, optimizing weights, and augmenting information. However, these methods often struggle to balance the performance between dominant and minority classes because of inadequate representation learning of the latter. To address these problems, we introduce descriptional words into images as cross-modal privileged information and propose a cross-modal enhanced method for long-tailed image classification, referred to as CMLTNet. CMLTNet improves the learning of intraclass similarity of tail-class representations by cross-modal alignment and captures the difference between the head and tail classes in semantic space by cross-modal inference. After fusing the above information, CMLTNet achieved an overall performance that was better than those of benchmark long-tailed and cross-modal learning methods on the long-tailed cross-modal datasets, NUS-WIDE and VireoFood-172. The effectiveness of the proposed modules was further studied through ablation experiments. In a case study of feature distribution, the proposed model was better in learning representations of tail classes, and in the experiments on model attention, CMLTNet has the potential to help learn some rare concepts in the tail class through mapping to the semantic space.

A freehand sketch is geometrically distorted and lacks colors, textures, and other visual details. This leads to challenges in freehand sketch-to-image generation tasks. Many existing works take a multistage strategy to generate images from sketches, and they depend on the shape of the input sketch badly and ignore the shape adjustment of the generated image. More importantly, the generated images are of low quality with distorted textures and colors, and shape deformation. In order to overcome the above challenges, we propose an end-to-end method to accomplish the freehand sketch-to-image task, and the proposed architecture is based on an unsupervised network. Our key insight is to propose a shape discriminator to provide shape constraints for the generated image. Besides, we introduce Image-Global Attention(IGA) and Focal Frequency Loss(FFL). IGA and FFL mainly focuses on the whole image and every patch of an image, respectively. We also extend a new dataset called NewGiraffe based on the giraffe class of SketchyCOCO, and our approach is validated on two datasets: Shoes and NewGiraffe. Through qualitative and quantitative results, we demonstrate our method’s ability to generate realistic images from freehand sketches.

Cross-training has become a promising strategy to handle data heterogeneity problem in federated learning, which re-train a local model across different clients to improve its generalization capability in a privacy-preserving manner. Its main idea is to make the local models to fit the data of all clients. However, the heterogeneity between data sources may lead the local models to quickly forget the knowledge learned in several rounds of cross-training. To address the problem, this paper presents a novel prototype guided cross training mechanism, termed PGCT, to regularize the change of class-level data representations across clients. It includes two main modules, where the prototype guided representation learning module employs client-aware prototypes of data patterns learned by clustering to guide the learning of consistency representation across feature spaces. This maintains the similar decision boundary across different clients. The prototype-based feature augmentation module uses prototypes as soft attention regularizers to further aggregate rich information to enhance the discrimination of historical features. Experiments were conducted on four datasets in terms of performance comparison, ablation study and case study, and the results verified that PGCT can learn discriminative features with different classes under the guidance of prototypes, which leads to better performance than the state-of-the-art methods.

Few-shot font generation (FFG) has received increasing attention recently. However, due to the complex structure of Chinese characters, most existing methods suffer from missing component-level content details and inaccurate capture of content-independent style representations. In this paper, we proposed a novel generative adversarial network for FFG by learning component information and inter-class glyph style representation. Specifically, we proposed a Content-Component Aware Module (CCAM) to help the model learn the accurate content representation by using component images, which is a brand new perspective. In addition, we employed a Glyph Style Contrastive Enhance (GSCE) strategy to help the encoder learn the differences of inter-class glyph style while ignoring the influence of reference character content. Both qualitative and quantitative experiments show our method achieves state-of-the-art performance in terms of the content structure integrity and style accuracy of the glyph.

Scene text editing(STE) aims to modify the text in the scene image to the target text while retaining the original style. Existing models are based on GAN, where the source image and the target text are input only once during the generation process, and this approach could not fully obtain the style of the source image and content of the target text. In this paper, we propose an STE method based on the classifier-free guidance diffusion model. To our best knowledge, our model is the first work that developed diffusion models to handle the STE task. Specifically, we divide the STE task into multiple steps and extract style information and text content information in each step. In addition, we introduce the letter embedding method as guidance. We experimentally prove that our method outperforms other STE models in terms of overall realism and maintaining glyphs.

Recently, many researchers have made remarkable achievements in the field of artistic font synthesis, with impressive glyph style and effect style in the results. However, due to less exploration in style disentanglement, it is difficult for existing methods to envision a kind of unseen style (glyph-effect) compositions of artistic font, and thus can only learn the seen style compositions. To solve this problem, we propose a novel compositional zero-shot artistic font synthesis gan (CAFS-GAN), which allows the synthesis of unseen style compositions by exploring the visual independence and joint compatibility of encoding semantics between glyph and effect. Specifically, we propose two contrast-based style encoders to achieve style disentanglement due to glyph and effect intertwining in the image. Meanwhile, to preserve more glyph and effect detail, we propose a generator based on hierarchical dual styles AdaIN to reorganize content-styles representations from structure to texture gradually. Extensive experiments demonstrate the superiority of our model in generating high-quality artistic font images with unseen style compositions against other state-of-the-art methods. The source code and data is available at moonlight03.github.io/CAFS-GAN/.

Multimedia recommendation aims to fuse the multi-modal information of items for feature enrichment to improve the recommendation performance. However, existing methods typically introduce multi-modal information based on collaborative information to improve the overall recommendation precision, while failing to explore its cold-start recommendation performance. Meanwhile, these above methods are only applicable when such multi-modal data is available. To address this problem, this paper proposes a recommendation framework, named Cross-modal Content Inference and Feature Enrichment Recommendation (CIERec), which exploits the multi-modal information to improve its cold-start recommendation performance. Specifically, CIERec first introduces image annotation as the privileged information to help guide the mapping of unified features from the visual space to the semantic space in the training phase. And then CIERec enriches the content representation with the fusion of collaborative, visual, and cross-modal inferred representations, so as to improve its cold-start recommendation performance. Experimental results on two real-world datasets show that the content representations learned by CIERec are able to achieve superior cold-start recommendation performance over existing visually-aware recommendation algorithms. More importantly, CIERec can consistently achieve significant improvements with different conventional visually-aware backbones, which verifies its universality and effectiveness.

Multimodal video classification aims to incorporate semantic information to regularize the visual representation learning of videos. Conventional methods typically focus on analyzing all information extracted from different modals rather than key information. However, they usually face the problem of handling the redundant video frames of little categorical information. To address this problem, this paper proposes a novel approach that employs multi-channel weighting of visual frames to mitigate the interference of redundant information. Specifically, the proposed algorithm, termed MCA-WF, includes two main modules, where the multi-channel attentive weighting of video frames (McAW) module performs the multi-granularity and multi-channel frame weighting mechanism based on visual self-attention, contrastive attention and cross-modal attention constraints to filter visual noise and redundant information. The visual frame selection (VFS) module explores the combination of multi-channel attention mechanisms to select the key visual information in the video. Experiments were conducted on MSR-VTT and ActivityNet Captions datasets in terms of performance comparison, ablation study, in-depth analysis, and case studies. The results verified that MCA-WF can notice the key information in the classification and effectively improve the ability of information complementation and integration between modals, which leads to better performance than the state-of-the-art methods.

Representation learning for images has been advanced by recent progress in more complex neural models such as the Vision Transformers and new learning theories such as the structural causal models. However, these models mainly rely on the classification loss to implicitly regularize the class-level data distributions, and they may face difficulties when handling classes with diverse visual patterns. We argue that the incorporation of the structural information between data samples may improve this situation. To achieve this goal, this paper presents a framework termed Class-level Structural Relation Modeling and Smoothing for Visual Representation Learning (CSRMS), which includes the Class-level Relation Modelling, Class-aware Graph Sampling, and Relational Graph-Guided Representation Learning modules to model a relational graph of the entire dataset and perform class-aware smoothing and regularization operations to alleviate the issue of intra-class visual diversity and inter-class similarity. Specifically, the Class-level Relation Modelling module uses a clustering algorithm to learn the data distributions in the feature space and identify three types of class-level sample relations for the training set; Class-aware Graph Sampling module extends typical training batch construction process with three strategies to sample dataset-level sub-graphs; and Relational Graph-Guided Representation Learning module employs a graph convolution network with knowledge-guided smoothing operations to ease the projection from different visual patterns to the same class. Experiments demonstrate the effectiveness of structured knowledge modelling for enhanced representation learning and show that CSRMS can be incorporated with any state-of-the-art visual representation learning models for performance gains. The source codes and demos have been released at https://github.com/czt117/CSRMS.

Federated Learning aims to learn a global model on the server side that generalizes to all clients in a privacy-preserving manner, by leveraging the local models from different clients. Existing solutions focus on either regularizing the objective functions among clients or improving the aggregation mechanism for the improved model generalization capability. However, their performance is typically limited by the dataset biases, such as the heterogeneous data distributions and the missing classes. To address this issue, this paper presents a cross-silo prototypical calibration method (FedCSPC), which takes additional prototype information from the clients to learn a unified feature space on the server side. Specifically, FedCSPC first employs the Data Prototypical Modeling (DPM) module to learn data patterns via clustering to aid calibration. Subsequently, the cross-silo prototypical calibration (CSPC) module develops an augmented contrastive learning method to improve the robustness of the calibration, which can effectively project cross-source features into a consistent space while maintaining clear decision boundaries. Moreover, the CSPC module's ease of implementation and plug-and-play characteristics make it even more remarkable. Experiments were conducted on four datasets in terms of performance comparison, ablation study, in-depth analysis and case study, and the results verified that FedCSPC is capable of learning the consistent features across different data sources of the same class under the guidance of calibrated model, which leads to better performance than the state-of-the-art methods. The source codes have been released at https://github.com/qizhuang-qz/FedCSPC.

Deep learning has been proven to be effective in image classification tasks. However, existing methods may face difficulties in distinguishing complex images due to the distraction caused by diverse image content. To overcome this challenge, we propose a class-aware convolution and attentive aggregation (CA-Net) framework that improves the effectiveness of representation learning and reduces the influence of irrelevant background. CA-Net includes three main modules: the discrete representation learning (DRL) module that uses a group learning method to learn discriminative representations, the class-aware score of discrete representation (CSDR) module that infers class-aware scores to generate weights for representation learners, and the class-aware representation fusion module(CRF) that aggregates class-aware representations using the class-aware scores as a guide. Our experimental results on three benchmarking datasets show that CA-Net improves the performance of state-of-the-art backbones and enhances feature extraction robustness.

The automatic generation of artistic fonts is a challenging task that attracts many research interests. Previous methods specifically focus on glyph or texture style transfer. However, we often come across creative fonts composed of objects in posters or logos. These fonts have proven to be a challenge for existing methods as they struggle to generate similar designs. This paper proposes a novel method for generating creative artistic fonts using a pre-trained text-to-image diffusion model. Our model takes a shape image and a prompt describing an object as input and generates an artistic glyph image consisting of such objects. Specifically, we introduce a novel heatmap-based weak position constraint method to guide the positioning of objects in the generated image, and we also propose the Latent Space Semantic Augmentation Module that improves other information while constraining object position. Our approach is unique in that it can preserve the object’s original shape while constraining its position. And our training method requires only a small quantity of generated data, making it an efficient unsupervised learning approach. Experimental results demonstrate that our method can generate various glyphs, including Chinese, English, Japanese, and symbols, using different objects. We also conducted qualitative and quantitative comparisons with various position control methods for the diffusion model. The results indicate that our approach outperforms other methods in terms of visual quality, innovation, and user evaluation.

Long Sequence Time-Series Forecasting (LSTF) is an important and challenging research with broad applications. Recent studies have shown that Transformer-based models can be effective in solving correlation problems in time-series data, but they also introduce issues of quadratic time and memory complexity, which make them unsuitable for LSTF problems. As a response, we investigate the impact of the long-tail distribution of attention scores on prediction accuracy and propose a Bis-Attention mechanism based on the mean measurement to bi-directionally sparse the self-attention matrix as a way to enhance the differentiation of attention scores and to reduce the complexity of the Transformer-based models from$O(L^{2})$to$O((logL)^{2})$. Moreover, we reduce memory consumption and optimize the model architecture through the use of a shared-QK method. The effectiveness of the proposed method is verified by theoretical analysis and visualisation. Extensive experiments on three benchmarks demonstrate that our method achieves better performance compared to other state-of-the-art methods, including an average reduction of 19.2% in MSE and 12% in MAE compared to Informer.

Ultra-Reliable Low Latency Communication (URLLC) has become crucial in various communication services. It encompasses a broad range of fields, including Internet of Things (IoT) and Internet of Vehicles (IoV), where precise network latency prediction analysis is required to ensure reliability. However, current network latency forecasting models are insufficient in capturing the temporal periodicity and statistical features of network latency. Therefore, we propose a Patching-based Method for Network Latency Forecasting (PatchNF) in the URLLC scenarios. Specifically, to preserve local semantic information, we segment the latency sequence into subsequence-level patches that are used as input tokens for the encoder. Next, we employ dilated convolutions to capture different characteristics of network delay periodicity by utilizing varying receptive fields. To make full use of features with different network delay semantics and improve prediction performance, we introduce a hierarchical approach that exploits network delay semantics at different levels before the execution of the fully-connected mapping. Experimental results on real datasets collected in a URLLC scenario indicate that the proposed method outperforms TS2Vec by 55.2% and Informer by 181.9% in terms of mean squared error (MSE). These results confirm the significant advantage of PatchNF in network latency forecasting.

In recent years, the problem of discovering influence communities has received much attention. However, in practical applications, vertices are often associated with attributes that are significant for understanding community. This paper explores the problem of computing top-r influential communities in attributed networks. We present a new community model called AAC that aims to promote cohesiveness in both structure and attributes. In order to measure the impact of a community$C$, we have developed an influential score function called$\text{iScore}(C)$by balancing attribute and structure cohesiveness. We propose two baseline approaches with effective pruning techniques and an index-based approach to efficiently report communities in an attributed network. The experimental results on real attributed networks indicate the effectiveness of our model and the efficiency of our proposed online and index-based algorithms.

Thanks to the rapid development of machine learning and deep learning, the fingerprint localization community has achieved high localization performance through advanced models. However, current approaches mainly focus on machine learning or pure convolutions algorithms for example k-Nearest Neighbor (KNN) and Convolutional Neural Network (CNN). Despite their success, their localization ability is mainly derived from the development of model learning ability without in-depth study of the physical meaning of Channel State Information (CSI), such as the varying degrees of correlation and interference between subcarriers of antennas located at different locations in a wireless communication system. In particular, we develop a novel graph neural network (GNN)-based method to embed subcarriers from different antennas at different positions of the CSI as nodes and connect them to K-nearest neighbours (KNNs) to obtain a cohesive graph structure. Furthermore, we introduce a dilated KNN to perform graph-level learning using graph convolution, effectively modeling the correlation and interference between subcarriers. Finally, to prevent the degradation of positioning performance caused by reduced node feature diversity, we introduce a feedforward neural network (FFN) module for node feature transformation. Experimental results on real datasets indicate that our method achieves an average localization error of 0.17m, which is almost 39.3% better than the existing optimal baseline.

Various temporal denoising methods have been proposed to clean up the noise for real-time ray tracing (RTRT). These methods rely on the temporal correspondences of pixels between the current and previous frames, i.e. per-pixel screen-space motion vectors. However, the state-of-the-art temporal reuse methods with traditional motion vectors cause artifacts in motion occlusions. We accordingly propose a novel neural temporal denoising method for indirect illumination of Monte Carlo (MC) ray tracing at 1 sample per pixel. Based on end-to-end multi-scale kernel-based reconstruction, we apply temporally reliable dual motion vectors to facilitate better reconstruction of the occlusions, and also introduce additional motion occlusion loss to reduce ghosting artifacts. Experiments show that our method significantly reduces the over-blurring and ghosting artifacts while generating high-quality images at real-time rates.

Recently, the artistic font generation has made significant progress. However, existing methods typically treat the style of artistic font as a whole. Their performance is usually limited to the artistic fonts with complex style elements in glyph and text effect. To solve these problems, this paper presents a disentangled style encoding network, termed DSE-Net, to synthesize artistic fonts. In order to obtain the disentangled text effect features, we introduce a perspective transformation network. We propose a cross-layer fusion mechanism to improve the artistic fonts' structure and texture according to their different representations in CNN. Notably, encoding different style elements for artistic font generation is a new task, so there is no publicly-accessible dataset. Therefore, a new dataset, termed SSAF, has been constructed. Extensive experiments demonstrate that our model significantly outperforms the state-of-the-art methods, with more fine-grained text effect and accurate stroke details.

In this paper, we focus on generating photo-realistic images from given text descriptions. Current methods first generate an initial image and then progressively refine it to a high-resolution one. These methods typically indiscriminately refine all granularity features output from the previous stage. However, the ability to express different granularity features in each stage is not consistent, and it is difficult to express precise semantics by further refining the features with poor quality generated in the previous stage. Current methods cannot refine different granularity features independently, resulting in that it is challenging to clearly express all factors of semantics in generated image, and some features even become worse. To address this issue, we propose a Hierarchical Disentangled Representations Generative Adversarial Networks (HDR-GAN) to generate photo-realistic images by explicitly disentangling and individually modeling the factors of semantics in the image. HDR-GAN introduces a novel component called multi-granularity feature disentangled encoder to represent image information comprehensively through explicitly disentangling multi-granularity features including pose, shape and texture. Moreover, we develop a novel Multi-granularity Feature Refinement (MFR) containing a Coarse-grained Feature Refinement (CFR) model and a Fine-grained Feature Refinement (FFR) model. CFR utilizes coarse-grained disentangled representations (e.g., pose and shape) to clarify category information, while FFR employs fine-grained disentangled representations (e.g., texture) to reflect instance-level details. Extensive experiments on two well-studied and publicly available datasets (i.e., CUB-200 and CLEVR-SV) demonstrate the rationality and superiority of our method.

Haze classification has gained much attention recently as a cost-effective solution for air quality monitoring. Different from conventional image classification tasks, it requires the classifier to capture the haze patterns of different severity degrees. Existing efforts typically focus on the extraction of effective haze features, such as the dark channel and deep features. However, it is observed that the light-haze images are often mis-classified due to the presence of diverse background scenes. To address this issue, this paper presents an unsupervised contrastive masking (UCM) algorithm to segment the haze regions without any supervision, and develops a dual-channel model-agnostic framework, termed magnifier neural network (MagNet), to effectively use the segmented haze regions to enhance the learning of haze features by conventional deep learning models. Specifically, MagNet employs the haze regions to provide the pixel- and feature-level visual information via three strategies, including Input Augmentation, Network Constraint, and Feature Enhancement, which work as a soft-attention regularizer to alleviates the trade-off between capturing the global scene information and the local information in the haze regions. Experiments were conducted on two datasets in terms of performance comparison, parameter estimation, ablation studies, and case studies, and the results verified that UCM can accurately and rapidly segment the haze regions, and the proposed three strategies of MagNet consistently improve the performance of the state-of-the-art deep learning backbones.

Traditional Chinese painting is a unique form of artistic expression. Compared with western art painting, it pays more attention to the verve in visual effect, especially ink painting, which makes good use of lines and pays little attention to information such as texture. Some style transfer methods have recently begun to apply traditional Chinese painting style (such as ink wash style) to photorealistic. Ink stylization of different types of real-world photos in a dataset using these style transfer methods has some limitations. When the input images are animal types that have not been seen in the training set, the generated results retain some semantic features of the data in the training set, resulting in distortion. Therefore, in this paper, we attempt to separate the feature representations for styles and contents and propose a style-woven attention network to achieve zero-shot ink wash painting style transfer. Our model learns to disentangle the data representations in an unsupervised fashion and capture the semantic correlations of content and style. In addition, an ink style loss is added to improve the learning ability of the style encoder. In order to verify the ability of ink wash stylization, we augmented the publicly available dataset $ChipPhi$. Extensive experiments based on a wide validation set prove that our method achieves state-of-the-art results.

Abstract Although some progress has been made in the layout‐to‐image generation of complex scenes with multiple objects, object‐level generation still suffers from distortion and poor recognizability. We argue that this is caused by the lack of feature encodings for edge information during image generation. In order to solve these limitations, we propose a novel edge‐enhanced Generative Adversarial Network for layout‐to‐image generation (termed EL‐GAN). The feature encodings of edge information are learned from the multi‐level features output by the generator and iteratively optimized along the generator's pipeline. Two new components are included at each generator level to enable multi‐scale learning. Specifically, one is the edge generation module (EGM), which is responsible for converting the output of the multi‐level features by the generator into images of different scales and extracting their edge maps. The other is the edge fusion module (EFM), which integrates the feature encodings refined from the edge maps into the subsequent image generation process by modulating the parameters in the normalization layers. Meanwhile, the discriminator is fed with frequency‐sensitive image features, which greatly enhances the generation quality of the image's high‐frequency edge contours and low‐frequency regions. Extensive experiments show that EL‐GAN outperforms the state‐of‐the‐art methods on the COCO‐Stuff and Visual Genome datasets. Our source code is available at https://github.com/Azure616/EL-GAN .

Abstract Normal map‐based microstructure rendering can generate both glint and scratch appearance accurately. However, the extra high‐resolution normal map that defines every microfacet normal may incur high storage and computation costs. We present an example‐based real‐time rendering method for arbitrary microstructure materials, which significantly reduces the required storage space. Our method takes a small‐size normal map sample as input. We implicitly synthesize a high‐resolution normal map from the normal map sample and construct MIP‐mapped 4D position‐normal Gaussian lobes. Based on the above MIP‐mapped 4D lobes and a LUT (lookup table) data structure for the synthesized high‐resolution normal map, an efficient Gaussian query method is presented to evaluate ‐NDFs (position‐normal distribution functions) for shading. We can render complex scenes with glint and scratch surfaces in real time ( 30 fps) with a full high‐definition resolution, and the space required for each microstructure material is decreased to 30 MB.

The interaction between light and materials is key to physically-based realistic rendering. However, it is also complex to analyze, especially when the materials contain a large number of details and thus exhibit “glinty” visual effects. Recent methods of producing glinty appearance are expected to be important in next-generation computer graphics. We provide here a comprehensive survey on recent glinty appearance rendering. We start with a definition of glinty appearance based on microfacet theory, and then summarize research works in terms of representation and practical rendering. We have implemented typical methods using our unified platform and compare them in terms of visual effects, rendering speed, and memory consumption. Finally, we briefly discuss limitations and future research directions. We hope our analysis, implementations, and comparisons will provide insight for readers hoping to choose suitable methods for applications, or carry out research.

The curvature gain of redirected walking method enables users to explore virtual spaces that are larger than real spaces. The estimation of human sensitivity for curvature gain is important for redirected walking. Herein, we conduct two experiments under different path conditions. By adopting the psychophysical “method of limits” to be a new approach, the first experiment re-estimates the sensitivity for a curved path and further proves that people are more sensitive for right-curved paths than left-curved paths. The second experiment investigates the characteristics of preorder paths on the sensitivity to postorder paths, and finds that the existing of preorder paths significantly increases the sensitivity for the postorder path. Moreover, we find an interesting moderation effect on the direction consistency of preorder and postorder paths: the larger curved preorder path can significantly decrease the sensitivity for postorder path when they have the same direction. If their directions diverge, the effect of preorder path is not significant.

The automatic generation of Chinese fonts is challenging due to the large quantity and complex structure of Chinese characters. When there are insufficient reference samples for the target font, existing deep learning-based methods cannot avoid overfitting caused by too few samples, resulting in blurred glyphs and incomplete strokes. To address these problems, this paper proposes a novel deep meta-learning-based font generation method (MLFont) for few-shot Chinese font generation, which leverages existing fonts to improve the generalization capability of the model for new fonts. Existing deep meta-learning methods mainly focus on few-shot image classification. To apply meta-learning to font generation, we present a meta-training strategy based on Model-Agnostic Meta-Learning (MAML) and a task organization method for font generation. The meta-training makes the font generator easy to fine-tune for new font generation tasks. Through random font generation tasks and extraction of glyph content and style separately, the font generator learns the prior knowledge of character structure in the meta-training stage, and then quickly adapts to the generation of new fonts with a few samples by fine-tuning of adversarial training. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods with more complete strokes and less noise in the generated character images.

LAMMPS is one of the most popular Molecular Dynamic (MD) packages and is widely used in the field of physics, chemistry and materials simulation. Layered Materials Force Field (LMFF) is our expansion of the LAMMPS potential function based on the Tersoff potential and inter-layer potential (ILP) in LAMMPS. LMFF is designed to study layered materials such as graphene and boron hexanitride. It is universal and does not depend on any platform. We have also carried out a series of optimizations on LMFF and the optimization work is carried out on the new generation of Sunway supercomputer, called SWLMFF. Experiments show that our implementation is efficient, scalable and portable. When generic LMFF is ported to Intel Xeon Gold 6278C, 2X performance improvement is achieved. For the optimized SWLMFF, the overall performance improvement is nearly 200--330X compared to the original ILP and Tersoff potentials. And SWLMFF has good parallel efficiency of 95%-100% under weak scaling with 2.7 million atoms on a single process. The maximum atomic system simulated by SWLMFF is close to 231 atoms. And nanosecond simulations in one day can be realized.

Rendering participating media is important to the creation of photorealistic images. Participating media has a translucent aspect that comes from light being scattered inside the material. For materials with a small mean-free-path (mfp), multiple scattering effects dominate. Simulating these effects is computationally intensive, as it requires tracking a large number of scattering events inside the material. Existing approaches precompute multiple scattering events inside the material and store the results in a table. During rendering time, this table is used to compute the scattering effects. While these methods are faster than explicit scattering computation, they incur higher storage costs. In this paper, we present a new representation for double and multiple scattering effects that uses a neural network model. The scattering response from all homogeneous participating media is encoded into a neural network in a preprocessing step. At run time, the neural network is then used to predict the double and multiple scattering effects. We demonstrate the effects combined with Virtual Ray Lights (VRL), although our approach can be integrated with other rendering algorithms. Our algorithm is implemented on GPU. Double and multiple scattering effects for the entire participating media space are encoded using only 23.6 KB of memory. Our method achieves 50 ms per frame in typical scenes and provides results almost identical to the reference.

With the rapid development of next-generation sequencing (NGS) technologies, high throughput sequencing platforms continuously produce large amounts of short read DNA data at low cost. Read mapping is a performance-critical task, being one of the first stages required for many different types of NGS analysis pipelines. We present SWMapper — a scalable and efficient read mapper for the Sunway TaihuLight supercomputer. A number of optimization techniques are proposed to achieve high performance on its heterogeneous architecture which are centered around a memory-efficient succinct hash index data structure including seed filtration, duplicate removal, dynamic scheduling, asynchronous data transfer, and overlapping I/O and computation. Furthermore, a vectorized version of the banded Myers algorithm for pairwise alignment with 256-bit vector registers is presented to fully exploit the computational power of the SW26010 processor. Our performance evaluation shows that SWMapper using all 4 compute groups of a single Sunway TaihuLight node outperforms S-Aligner on the same hardware by a factor of 6.2. In addition, compared the state-of-the-art CPU-based mappers RazerS3, BitMapper2, and Hobbes3 running on a 4-core Xeon W-2123v3 CPU, SWMapper achieves speedups of 26.5, 7.8, and 2.6, respectively. Our optimizations achieve an aggregated speedup of 11 compared to the naïve implementation on one compute group of an SW26010 processor as well as a strong scaling efficiency of 74% on 128 compute groups.

Style transfer between Chinese fonts is challenging due to both the complexity of Chinese characters and the significant difference between fonts. Existing algorithms for this task typically learn a mapping between the reference and target fonts for each character. Subsequently, this mapping is used to generate the characters that do not exist in the target font. However, the characters available for training are unlikely to cover all fine-grained parts of the missing characters, leading to the overfitting problem. As a result, the generated characters of the target font may suffer problems of incomplete or even radicals and dirty dots. To address this problem, this paper presents a multi-task adversarial learning approach, termed MTfontGAN, to generate more vivid Chinese characters. MTfontGAN learns to transfer a reference font to multiple target ones simultaneously. An alignment is imposed on the encoders of different tasks to make them focus on the important parts of the characters in general style transfer. Such cross-task interactions at the feature level effectively improve the generalization capability of MTfontGAN. The performance of MTfontGAN is evaluated on three Chinese font datasets. Experimental results show that MTfontGAN outperforms the state-of-the-art algorithms in a single-task setting. More importantly, increasing the number of tasks leads to better performance in all of them.

The Weather Research and Forecasting (WRF) Model is one of the widely-used mesoscale numerical weather prediction system and is designed for both atmospheric research and operational forecasting applications. However, it is an extremely time-consuming application: running a single simulation takes researchers days to weeks as the simulation size scales up and computing demands grow. In this paper, we port and optimize the whole WRF model to the Sunway TaihuLight supercomputer at a large scale. For the dynamic core in WRF, we present a domain-specific tool, namely, SWSLL, which is a directive-based compiler tool for the Sunway many-core architecture to convert the stencil computation into optimized parallel code. We also apply a decomposition strategy for SWSLL to improve the memory locality and decrease the number of off-chip memory accesses. For physical parameterizations, we explore the thread-level parallelization using OpenACC directives via reorganizations of data layouts and loops to achieve high performance. We present the algorithms and implementations and demonstrate the optimizations of a real-world complicated atmospheric modeling on the Sunway TaihuLight supercomputer. Evaluation results reveal that for the widely used benchmark with a horizontal resolution of 2.5 km, the speedup of 4.7 can be achieved by using the proposed algorithm and optimization strategies for the whole WRF model. In terms of strong scalability, our implementation scales well to hundreds of thousands of heterogeneous cores on Sunway TaihuLight.

Weak association model indicates that distraction caused by the disjunction between the primary task and interactive artifacts may be a key factor directly leading to weak association between flow and task performance in virtual reality (VR) activities. To test the idea, this paper proposed a VR visual search paradigm based on which we constructed a VR oceanic treasure hunting system. Experiment 1 proved that distraction caused by the incongruence was indeed a direct antecedent of weak association. Next, we slightly adjusted the system by providing visual cues to achieve task-oriented selective attention. Experiment 2 found this helped enhance the task performance without damaging flow experience.

This paper presents a toolkit called Rotbav for easily constructing mixed reality (MR) apps that can be experienced in real time in large indoor physical space via HoloLens. It resolves the problem that existing MR devices, e.g. HoloLens, are unable to scan and model an entire large scene with several rooms at once. We introduce a custom data structure called VorPa, based on the Voronoi diagram, to implement path editing, accelerated rendering and location effectively. Our experiments and applications show that the toolkit is convenient and easy to use for constructing MR apps targeting large indoor physical spaces, in which users can roam in real time.

Many studies conducted in non-virtual activities have shown that flow significantly influences performance, yet studies in virtual activities often reveal only a weak association. This paper begins by building a theoretical explanatory model, and then conducts 3 empirical studies to explore this question. Study 1 exams the mechanism of weak association in two virtual activities. Study 2 tests the effectiveness of a potential approach to strengthen this association. In Study 3 we applied our proposed model and design approach to optimize a VR tennis game. Results show that the influence of flow on performance was not significant in those virtual activities where the primary task and the operation of interactive artifacts were less congruent such that the artifacts can lead to flow experience that is independently of the primary task. Our research offers a theoretical and empirical basis on how to optimize virtual environment design and maximize positive effect of the flow experience.

Stereoscopic display is a standard display mode for virtual reality environments. Typical 3D projection provides only a single stereoscopic video stream; thus co-located users cannot correctly perceive the virtual scene based on their own position and view. Several works devoted to developing multi-user stereoscopic display, but the number of users is very limited or the technical implementation is complicated. In this paper we put forward two flexible and simple projection-based multi-user stereoscopic display systems. The first one, named TPA, is based on a triple-projector array and provides a 120Hz active stereo for three users. Two TPAs can be combined to form a six-user system. The second one, named DPA, is a dual-projector and easy-implemented system providing individual stereoscopic video stream for two to six users. Finally, a co-located multi-user virtual fireman simulation training system and a virtual tennis simulation system were created to verify the effectiveness of our systems.

Modern high throughput sequencing platforms can produce large amounts of short read DNA data at low cost. Error correction is an important but time-consuming initial step when processing this data in order to improve the quality of downstream analyses. In this paper, we present a Scalable Parallel Error CorrecToR designed to improve the throughput of DNA error correction for Illumina reads on various parallel platforms. Our design is based on a k-spectrum approach where a Bloom filter is frequently probed as a key operation and is optimized towards AVX-512-based multi-core CPUs, Xeon Phi many-cores (both KNC and KNL), and heterogeneous compute clusters. A number of architecture-specific optimizations are employed to achieve high performance such as memory alignment, vectorized Bloom filter probing, and a stack-based iteration to eliminate recursion. Our experiments show that our optimizations result in speedups of up to 2.8, 5.2, and 9.3 on a CPU (Xeon W-2123), a KNC-based Xeon Phi (31S1P), and a KNL-based Xeon Phi (7210), respectively, compared to a multi-threaded CPU reference implementation for the error correction stage. Furthermore, when executed on the same hardware, SPECTR achieves a speedup of up to 1.7, 2.1, 2.4, and 6.4, compared to the state-of-the-art tools Lighter, BLESS2, RECKONER, and Musket, respectively. In addition, our MPI implementation exhibits an efficiency of around 86% when executed on 32 nodes of the Tianhe-2 supercomputer. SPECTR is available at https://github.com/Xu-Kai/SPECTR.

Performance analysis is important for service clouds serving composite service application jobs containing parallelizable tasks, for optimizing the degree of parallelism (DOP) and resource allocation schemes could improve performance obviously. In this paper, we describe a novel tandem queuing network with a parallel multi-station multi-server system as an analytical model for service clouds serving composite service application jobs. We design a partition method (termed the 'pleasing partition') to help us propose an analytical model for parallelizable service which is the vital fraction of composite service. After that, we could obtain a complete probability distribution of response time, waiting time and other important performance metrics calculated by our proposed analytical model. Thus, to use this model, cloud operators could determine proper job configurations and resource allocation schemes, for achieving specific QoS (Quality of Service). Extensive simulations are conducted to validate that our analytical model has high accuracy in predicting performance metrics of composite service application jobs.

In this paper, we present a method of view-dependent stereoscopic projection on curved screen. It allows the user to walk around with the correct perspective view of the virtual scene consistent with his/her location. To solve the problem of distortion and drift of virtual objects when projecting the view-dependent scene images on curved screen, we operate a dynamic parallax adjustment of the stereoscopic images according to viewpoints. User evaluation shows that our proposed approaches are effective on improving visual experience.

Most existing virtual machine introspection (VMI) technologies analyze the status of a target virtual machine under the assumption that the operating system (OS) version and kernel structure information are known at the hypervisor level. In this paper, we propose a model of virtual machine (VM) security monitoring based on memory introspection. Using a hardware-based approach to acquire the physical memory of the host machine in real time, the security of the host machine and VM can be diagnosed. Furthermore, a novel approach for VM memory forensics based on the virtual machine control structure (VMCS) is put forward. By analyzing the memory of the host machine, the running VMs can be detected and their high-level semantic information can be reconstructed. Then, malicious activity in the VMs can be identified in a timely manner. Moreover, by mutually analyzing the memory content of the host machine and VMs, VM escape may be detected. Compared with previous memory introspection technologies, our solution can automatically reconstruct the comprehensive running state of a target VM without any prior knowledge and is strongly resistant to attacks with high reliability. We developed a prototype system called the VEDefender. Experimental results indicate that our system can handle the VMs of mainstream Linux and Windows OS versions with high efficiency and does not influence the performance of the host machine and VMs.

In this work, we propose an original scheme for generic content-based retrieval of marker-based motion capture data. It works on motion capture data of arbitrary subject types and arbitrary marker attachment and labelling conventions. Specifically, we propose a novel motion signature to statistically describe both the high-level and the low-level morphological and kinematic characteristics of a motion capture sequence, and conduct the content-based retrieval by computing and ordering the motion signature distance between the query and every item in the database. The distance between two motion signatures is computed by a weighted sum of differences in separate features contained in them. For maximum retrieval performance, we propose a method to pre-learn an optimal set of weights for each type of motion in the database through biased discriminant analysis, and adaptively choose a good set of weights for any given query at the run time. Excellence of the proposed scheme is experimentally demonstrated on various data sets and performance metrics.

With the fast development of virtual reality games, one of the key research questions is how players may express their creativity and participate in the process of game design. In this paper, we present a new game genre which combines user-controlled game design in physical space with game play in virtual space on a mobile device. The new system supports authoring by anyone, creating virtual reality games that can be easily modified or developed for physical space, and be used anywhere by novice end-users without any knowledge of tracking technology. We present the design and implementation of the system, as well as a user experiment. Findings illustrate that the proposed system promotes participation and provides a richer, more interactive and engaging experience.

Mobile cloud computing is now emerging as a promising way to enlarge the capabilities of mobile devices by computation offloading. One of the critical challenges faced by the mobile cloud providers today is how to increase the profitability of their cloud services. In this paper, we deal with the problem of scheduling parallelizable computation jobs offloaded by mobile users in public cloud to maximize cloud providers' profit. We propose an efficient dynamic-priority based and profit-driven scheduling mechanism. We first compute the inherent profitability of a job based on the marginal effect theory and use it as the initial priority. Then, we update the priorities of waiting jobs based on the proposed time-dependent dynamic-priority calculation model as the elapse of time. The results of numerical experiments and simulations show that our approach are efficient in scheduling these kind of jobs in cloud data centers external to mobile devices and considerable profit improvements can be achieved by our proposed dynamic-priority based scheduling mechanism.

Current Infrastructure-as-a-Service (IaaS) clouds offer both on-demand and reservation instance purchasing options. Users can combine these two options dynamically to serve time-varying demands while minimizing their instance acquisition costs. However, when future demands are unknown, it is far from trivial for cloud users to make optimal instance purchasing decisions. To deal with this problem, a carefully designed online algorithm can be employed to guide users in acquiring instances without any prior knowledge of future demands while guaranteeing a competitive ratio. In this paper, we propose an instance reselling model, in which a cloud user can temporarily rent out its idle reserved instances to other users through pay-as-you-go model. We also design online instance acquisition strategies which achieve a better competitive ratio than previous methods. Through extensive simulations based on both synthetic data and real-world traces, we show that our online algorithm under the proposed reselling model can outperform previous models and achieve significant cost savings.

In virtual learning environment, both personality and emotional features of animated pedagogical agents (APAs) may influence learning. To investigate this question, we developed four APAs with two distinct personality types and two sets of gestures expressing distinct emotional feedback. Effects of APAs' personality types and emotional feedback strategies on learning experiences and performance were assessed experimentally using a virtual Tai Chi training system. Fifty six participants completed the experiment. Results showed that positive emotional feedback strategy led to better learning experiences and performance than negative feedback strategy. Moreover, personality type had significant effect on learning. Choleric APAs led to better performance than Phlegmatic APAs. Personality types moderated the effect of emotional feedback on learning satisfaction. Our study demonstrates that APAs with distinct personality types and emotional feedback are important design parameters for virtual learning environments.

With the development of intelligent manufacturing technology, it can be foreseen that time series data generated by smart devices will raise to an unprecedented level. For time series with high amount, high dimension and renewal speed characteristics, resulting in difficult data mining and presentation on the original time series data. This paper presented a piecewise linear representation based on importance data points for time series data, which called PLR_IDP for short. The method finds importance data points by calculating the fitting error of single point and piecewise, and then represents time series approximately by linear composed of the importance data points. Results from theoretical analysis and experiments show that PLR_IDP reduces the dimensionality, holds the main characteristic with small fitting error of segments and single points.

Today, there has been a strong demand of distributed collaboration in design and manufacturing, due to the acceleration of economic globalization and the popularity of virtual enterprises (VE) model. Because of the characteristics of cloud computing, such as elasticity and on-demand computing, it is promising to deploy and execute collaborative workflows that contain multiple tasks and services such as Computer-Aided Design (CAD) software components on cloud resources for supporting collaboration across enterprises. Specifically, how to cost-effectively select appropriate services to execute workflows within deadlines while without violating multiple constraints becomes an important issue. In this paper, through investigating the practical requirements of collaborative design workflow, we first formulate the issue of the cost-optimal cloud service selection for collaborative workflow executions as a multi-dimensional optimization problem with multiple constraints. Then we propose an effective approach based on genetic algorithms to address this problem for obtaining near-optimal solutions. Based on workload data derived from real-world systems, we conduct experiments which show that our approach outperforms traditional greedy algorithms in finding better solutions and it also provides real-time performance guarantees in real-world cloud computing environments.

Progressive photon mapping [Hachisuka et al. 2008] (PPM) obtains increasingly accurate results with progressive visualization, but it is problematic when results are obtained through thousands of iterations. An uniform photon distribution is critical for the accurate result. In this work, we use the sample elimination [Yuksel 2015] (SE) in PPM to achieve optimal results and accelerate the iterations. Sample elimination can produce sample sets with more pronounced blue noise characteristics. We apply the feature of this elimination method to the progressive iterations.

Memory forensics plays an important role in security and forensic investigations. Hence, numerous studies have investigated Windows memory forensics, and considerable progress has been made. In contrast, research on Linux memory forensics is relatively sparse, and the current knowledge does not meet the requirements of forensic investigators. Existing solutions are not especially sophisticated, and their complicated operation and limited treatment range are unsatisfactory. This paper describes an adaptive approach for Linux memory analysis that can automatically identify the kernel version and recovery symbol information from an image. In particular, given a memory image or a memory snapshot without any additional information, the proposed technique can automatically reconstruct the kernel code, identify the kernel version, recover symbol table files, and extract live system information. Experimental results indicate that our method runs satisfactorily across a wide range of operating system versions.

Global illumination is the core part of photo-realistic rendering. The photon mapping algorithm is an effective method for computing global illumination with its obvious advantage of caustic and color bleeding rendering. It is an active research field that has been developed over the past two decades. The deficiency of precise details and efficient rendering are still the main challenges of photon mapping. This report reviews recent work and classifies it into a set of categories including radiance estimation, photon relaxation, photon tracing, progressive photon mapping, and parallel methods. The goals of our report are giving readers an overall introduction to photon mapping and motivating further research to address the limitations of existing methods.

Abstract Point‐Based Global Illumination (PBGI) is a popular rendering method in special effects and motion picture productions. This algorithm provides a diffuse global illumination solution by caching radiance in a mesh‐less hierarchical data structure during a preprocess, while solving for visibility over this cache, at rendering time and for each receiver, using microbuffers, which are localized depth and color buffers inspired from real time rendering environments. As a result, noise free ambient occlusion, indirect soft shadows and color bleeding effects are computed efficiently for high resolution image output and in a temporally coherent fashion. We propose an evolution of this method to address the case of non‐diffuse inter‐reflections and refractions. While the original PBGI algorithm models radiance using spherical harmonics, we propose to use wavelets parameterized on the direction space to better localize the radiance representation in the presence of highly directional reflectance. We also propose a new importance‐driven adaptive microbuffer model to capture accurately incoming radiance at a point. Furthermore, we evaluate outgoing radiance using a fast wavelet radiance product and contain the induced larger memory footprint by encoding hierarchically the wavelets in the PBGI tree. As a result, our algorithm can handle non‐lambertian BSDF in the light transport simulation, reproducing caustics and multiple reflections/refractions bounces with a similar quality to bidirectional path tracing in a large number of cases and for only a fraction of its computation time. Our approach is simple to implement and easy to integrate into any existing PBGI framework, with an intuitive control on the approximation error. We evaluate it on a collection of example scenes.

Temporal coherence is one of the central challenges for rendering a stylized line. It is especially difficult for stylized contours of coarse meshes or non-uniformly sampled models, because those contours are polygonal feature edges on the models with no continuous correspondences between frames. We describe a novel and simple technique for constructing a 2D brush path along a 3D contour. We also introduce a 3D parameter propagation and re-parameterization procedure to construct stroke paths along the 2D brush path to draw coherently stylized feature lines with a wide range of styles. Our method runs in real-time for coarse or non-uniformly sampled models, making it suitable for interactive applications needing temporal coherence.

With the popularity of cloud computing and SaaS, enterprises are willing to rent various cloud services and use them in their daily work. However, on-premise applications are still widely used inside enterprises. That is to say, to achieve the goal of business collaboration, a cloud service usually need to obtain business data from multiple sources, which include the relevant on-premise applications and other cloud services. In this paper, we study and summarize data organization and management patterns for cloud enterprise applications from different aspects. Based on these patterns, we propose an innovative cloud-based data organization model (CEDM). Compared with traditional ones, it highlights the characters of resource sharing and reuse. Besides, this model is more convenient for tackling complex data synchronization issues in cloud environment.

This paper presents a method to render smooth stylized contours of 3D meshes with temporal coherence. Contour triangles that contain countour curves perform two functions: they ensure that smooth contour curves are coherent and reconstructed faithfully during the motion of the viewpoint and they help to establish the corresponding relationship of contours which can propagate parameters on contours between frames against sliding effects. Our method can also deal with zig-zags and short segments for coarse meshes and nonuniform sampling meshes in a wide range of styles very well.

In this paper, we present a novel method for realtime synthesis of human motion under external perturbations. The proposed method is data-driven and based on angular momentum analysis. When an external force is applied on the virtual human body, we analyze the change in the joints' angular momentums in a short period of time, predict the human body response, find an appropriate motion sequence from the pre-built motion capture (MoCap) database, and make a smooth transition between the current and the retrieved motion sequences to obtain the synthesized motion. The most important contributions of our method include that we propose a complete momentum analysis solution for the human body and that we make effective MoCap data organization based on the major characteristics of the body motion and the external force. As a result, realistic and real-time human motion synthesis is achieved, as experimentally demonstrated with the walking, the running and the jumping sequences.

Nowadays, environment pollution is one of the most serious problems facing mankind, and more and more people are concerned about it. To solve it, we should get to know it first. In this paper, we present the method of tree-shaped multivariate visualization for hierarchical data, to reveal changes in the data and make comparisons among different pollutants and the time. The technique is based on a botanical tree metaphor. The resulting tree-like visualization can display many properties of the data and we can learn some useful information through the interactions. We also use traditional parallel coordinates as an auxiliary method and a complement to analyze the result.

In this paper, we propose a new method of visualizing hierarchy and time-series data. We use the node-linked technology to show hierarchy structures, rectangles from the left to right to represent the time-series data and a pie chart to represent statistical information about the time-series data. The method is designed to display and compare the corresponding data of each layer, and then observe the differences between the nodes at each layer and the trends of the thing. We applied this method in the urban air quality data visualization and achieved good results.

Abstract The Point‐Based Global Illumination (PBGI) algorithm is composed of two major steps: a caching step and a multiview rasterization step. At caching time, a dense point‐sampling of the scene is shaded and organized in a spatial hierarchy, with internal nodes approximating the radiance of their subtrees using spherical harmonics. At rasterization time, a microbuffer is instantiated at the unprojected position of each image pixel (receiver). Then, a view‐adaptive level‐of‐detail of the scene is extracted in the form of a tree cut and rasterized in the receiver's microbuffer, solving for visibility using a local variant of the z‐buffer. Finally, the pixel color is computed by convolving its filled microbuffer with the surface BRDF. This noise‐free indirect lighting method is widely used in the industry and captures several critical lighting effects, including ambient occlusion, color bleeding, (indirect) soft‐shadows and environment lighting. However, we observe a large redundancy in this algorithm, both in cuts and receivers'microbuffers, which stems from their relatively low resolution. In this paper, we propose an evolution of PBGI which exploits spatial coherence to reduce these redundant computations. Starting from a similarity‐based variational clustering of the receivers, we compute a single tree cut and rasterize a single microbuffer for each cluster. This per‐cluster microbuffer provides a faithful approximation of the incident radiance for distant nodes and is composited over a receiver‐specific microbuffer rasterizing only the closest nodes of the cluster's cut. This factorized approach is easy to integrate in any existing PBGI implementation and offers a significant rendering speed‐up for a negligible and controllable approximation error.

Many automated negotiation models have been developed to solve the conflict in many distributed computational systems. However, the problem of finding win–win outcome in multiattribute negotiation has not been tackled well. To address this issue, based on an evolutionary method of multiobjective optimization, this paper presents a negotiation model that can find win–win solutions of multiple attributes, but needs not to reveal negotiating agents’ private utility functions to their opponents or a third‐party mediator. Moreover, we also equip our agents with a general type of utility functions of interdependent multiattributes, which captures human intuitions well. In addition, we also develop a novel time‐dependent concession strategy model, which can help both sides find a final agreement among a set of win–win ones. Finally, lots of experiments confirm that our negotiation model outperforms the existing models developed recently. And the experiments also show our model is stable and efficient in finding fair win–win outcomes, which is seldom solved in the existing models.

ABSTRACT We present a rigid‐motion‐inspired method for animating liquid characters in this paper. Our method allows an animator to control the motion of liquid characters with motion capture data that is widely used in rigid body animation. It animates the most visual interesting part of liquid character, that is, to preserve character's shape as well as produce enough liquid details. To this end, we build a two‐layer model to represent the character by two coaxial layers: the rigid kernel and the liquid shell. Different control paradigms are used for the two layers instead of applying homogeneous force that is common in previous approaches. By embedding the control algorithm to the Navier–Stokes equations, we compute the fluid velocity that drives the motion of the liquid character. Results show that the method is easy and intuitive to use while incurring little additional cost.Copyright © 2013 John Wiley & Sons, Ltd.

We propose a real-time rendering system for automatically creating a caricature drawing, i.e., an exaggerated portrait, of a human face, based on simultaneous use of a range image (or 3D mesh) and a registered photograph of the same face. Combining these information sources provides complementary information. Significant geometric lines such as occluding contours and suggestive contours are extracted from the range data, while textured areas corresponding to shading features are extracted from the photograph. These are combined, and then distorted to produce the final caricature. The final output may be produced using a choice of non-photorealistic rendering styles. Our system method works well for low resolution range images; for these it is fast enough to allow the viewpoint to be chosen in real time. The final output combines significant lines, textured areas, and optional shading, giving a pleasing result which preserves not only the shape cues of the geometric description, but also other essential visual characteristics of the facial image that cannot be deduced from geometry alone.

A concept of shape coordinates system for visualization of data set is proposed in this paper. First, the visualization data set, visualization graphics, visualization process and visualization space are defined. Then, the definition, mapping, operation, theorems, properties and algorithms of shape coordinates system are described. Finally an example to visualize the data set as a tree shape graphics in the visualization space by using the shape coordinates system is given.

Existing video object cutout systems can only deal with limited cases. They usually require detailed user interactions to segment real-life videos, which often suffer from both inseparable statistics (similar appearance between foreground and background) and temporal discontinuities (e.g. large movements, newly-exposed regions following disocclusion or topology change). In this paper, we present an efficient video cutout system to meet this challenge. A novel directional classifier is proposed to handle temporal discontinuities robustly, and then multiple classifiers are incorporated to cover a variety of cases. The outputs of these classifiers are integrated via another classifier, which is learnt from real examples. The foreground matte is solved by a coherent matting procedure, and remaining errors can be removed easily by additive spatio-temporal local editing. Experiments demonstrate that our system performs more robustly and more intelligently than existing systems in dealing with various input types, thus saving a lot of user labor and time.

Point based rendering can simulate global illumination phenomenon fast and is widely used in movie production. This paper proposes a faster point based global illumination algorithm compared with the standard one. During our algorithm, the shading points generated by ray tracing are clustered in to groups. When doing point based rendering, the indirect illumination of each group's center point is computed instead of each shading point. We use an error strategy to make the indirect illumination compute adaptively. This method is several times faster than the standard point based global illumination method, while preserves high rendering quality. What's more, we implement our algorithm on GPU. In order to improve parallelism, we design a GPU-based cluster method and then implement the point based rendering process on GPU. The speed is times faster compared with the CPU one.

In this paper, we present a novel BVH tracing method on GPU, which can achieve better SIMD utilization than traditional method. In the traditional way, thread usually sticks to a ray until the closest hit is found. When the threads of the same warp follow very divergent ray paths, SIMD utilization drops significantly. The idea of our method is to redefine the way of work distribution by binding the ray and the data to be tested together, in order to spread the computation of the single ray to multi threads. We also separate the tracing process into three steps to collect the work units of the same type and process them in a stream-like manner. The first step is ray traversal whose task is to do ray-box testing for the ray-node pairs. Its output is an stack of ray-triangle pairs, which is then fed to intersecting step to form an stack of ray-hit pairs. The last step is to use ray-hit pairs to update the closest hits for each ray of the same warp. The experiment shows our method can efficiently improve the SIMD utilization and result in less tracing time.

As the competitions among enterprises become more and more serious, and the production processes become increasingly complex, finding an efficient way to realize collaboration between different processes of industries is necessary for enterprises. Therefore, in order to support business collaboration, we propose an Integrated Multi-channel Messaging Model (IM3) in this paper, which can be used as a stand-alone web-service in many different business systems. The main feature of our model of work is that it can integrate e-mail, SMS and instant message formats into a general purpose message format, and then send it out according to the requirements of users. The general purpose message format is transformed by the use of XSLT (extensible style language transformation), and is also extensible to more message formats. Besides, our model of work has functions of providing message alerts, the coordination between various business processes, the support of business process execution etc. It is an efficient model that provides an open, shared, distributed but centralized collaborative working environment.

Manufacturing industry is now being challenged with globalization and grid technology provides a reliable and distributed computing solution. In this paper, firstly, a service-oriented architecture of virtual enterprise (VE) is proposed. Then three main modules, which are project management, partner management and business process management, are described how to construct a VE based on manufacturing grid. The project management controls the projects distributed in different enterprises, the partner management maps roles in entity enterprises to that of VE, and the business process management inspects the VE process flows coming from different entity enterprises.

Cloud Storage is an important part of Cloud Computing, and it provides a way to achieve large scale storage architecture. And virtual storage is a strategy for Cloud Storage. A load balancing virtual storage strategy (LBVS) is proposed in this paper. The contribution of this strategy is that, it provides a large scale net data storage model and Storage as a Service model based on Cloud Storage. Three layers architecture is used to achieve storage virtualization, and two load balancing modules are used to balance systems load. This paper describes details of LBVS strategy, the model of virtual storage as well as how to implement the LBVS with iRODS and two load balancing algorithms.

In this paper, through the analysis of regional air pollutant concentration data in spatial area, using particle system to model and render the air pollutant concentration in the region at different height levels. We establish the corresponding relationship between pollutant concentration of a grid and the particle radius, and use different scattering models to render the particles, the results show that the rendering of the translucent matter in spatial region has better visual effect when the method is used.

This paper introduces a software as a service (SaaS) application which is designed and delivered in high level maturity model. In order to realize the configurability, metadata is used to define all the variability points of the application. Meanwhile, JMX is used to manage the metadata so that changed metadata can be hot deployed immediately during runtime. Scalability is discussed both in application layer and data layer. Integration requirements and roadmap between SaaS application and on-premise applications are introduced. What's more, service surrogate extended from SCA and a message engine are used to meet the integration requirements in business processes.

In this paper we design and implement a digital media service oriented digital museum platform based on Grid. We construct the whole system based on the Open Grid Service Architecture (OGSA), which supports flexibly constructing and deploying various digital media applications. The system shield the heterogeneity of resources via data access service so that the enormous dispersed resources can be effectively accessed and integrated. We also propose an on-demand application constructing method based on service composition. Using this method, we can implement digital media applications by dynamic service binding. Moreover, we implement digital media service applications based on this platform, such as 3D virtual museum walkthrough system.

Model resizing is a useful technique for model reuse. Introduced by Kraevoy et al., non-homogeneous model resizing is able to preserve important geometric features during anisotropic scaling. However, many practical objects contain various geometric textures. Different from similar objects without geometric textures, such objects seem as if extremely difficult to resize even if the underlying surfaces are relatively simple. In this paper, we present an automatic model resizing method based on geometric texture transfer. Geometric textures are separated from the underlying surfaces and reproduced on the non-homogenously scaled surfaces using geometric texture synthesis. By utilizing the natural correspondence between the surfaces before and after resizing, surfaces with multiple geometric textures can be resized and geometric texture recovered automatically. Experimental results show that our method effectively and automatically preserves geometric textures during the model resizing process.

As researches of service science have gradually come into practical use, architecture of practical service system has arisen as an important problem. To solve this problem, the paper proposes a kind of service system integration architecture based on UI (user interface) integration technology. Important modules of this architecture and their constructing process are particularly introduced. In the end of this paper, the main characteristics of this architecture are enumerated to summarize the whole architecture.

Coarse piecewise linear approximation of surfaces causes undesirable polygonal appearance of silhouettes. We present an efficient method for smoothing the silhouettes of coarse triangle meshes using efficient 3D curve reconstruction and simple local re-meshing. It does not assume the availability of a fine mesh and generates only moderate amount of additional data at run time. Furthermore, polygonal feature edges are also smoothed in a unified framework. Our method is based on a novel interpolation scheme over silhouette triangles and this ensures that smooth silhouettes are faithfully reconstructed and always change continuously with respect to continuous movement of the view point or objects. We speed up computation with GPU assistance to achieve real-time rendering of coarse meshes with the smoothed silhouettes. Experiments show that this method outperforms previous methods for silhouette smoothing.

Current smart mobile devices such as Pocket PCs have become common accepted daily-life personal assistants. Such mobile devices provide much less support for visualization compared to desktop computers. When developing mobile visualization applications, one should take the limited screen size, currently available interaction tools and relatively poor computation power into consideration. We describe a visual lens toolkit integrated with Focus+Context technique and it can be incorporated into various applications on Pocket PCs. Our technique creates a distorted display view of target object in a high resolution focus area that is smoothly embedded into the distorted context. We have tested the toolkit in two applications as stylistic lens and sketch lens separately and received fine experiment results. In remote line drawing of 3D model system, the lens toolkit embedded into the client side mainly serve as stylistic lens which provide the users with different presentation styles of the model according to the current viewpoint and focus. In mobile sketch system, the lens toolkit performs like a refinement filter which adoptively turns the loose sketch or SVG file into finer image. Both tested applications.

As the need for application service becomes larger and larger, turning software into a service has been addressed as a matter of urgency. There are two kinds of application delivery models, ASP and SaaS. This paper describes a method for turning software into a service using the ASP model, and use conference management system as an example to illustrate the idiographic process. According to the pertinences, software systems can be classified to two types. One type is user-independent, and the other is organization-centralized. This paper describes how to turn the organization-centralized software to a large-granularity application service.

Selection of correct partner is one of the most crucial factors in organizing virtual enterprises. Approaches of partner selection do not consider the semantic features in building a virtual enterprise. In this paper, we describe a system that can select partners effectively through semantic attributes with naive Bayesian algorithm. We show that semantic features can be successfully extracted by applying naive Bayesian technology and build a knowledge base of potential partners in manufacturing enterprises. An example of machine tool industry is used to describe the approach.

The exploration in the grid technology and semantic web service promotes the development of modern manufacturing technology. Grid applications need resources and services to be discovered quickly and efficiently. The greatest obstacle we have, however, lies in the difficulties in appropriately describing resources and services. To easily find the needed services, ontology is used to describe resources and services in our Manufacturing Grid (MG) architecture. We extend UDDI in order to enhance the precision of service discovering through embedding service category ontology into service descriptions. In addition, the extended UDDI can support service publishing and service retrieving. We built a prototype to implement resource description based on ontology in MG.

In a multi-domain collaboration environment, an enterprise should authorize different access rights for sensitive information to partners according to its security policies and relationships with them, which may be changed dynamically with the development of transaction and business rules. So, it is emerging as one of the major concerns to effectively manage the authorizations while supporting flexible multi-level collaboration. In this work, we propose an active authorization model for multi-domain cooperation, which introduces the notions of business rules and context parameters to update security policies automatically and satisfy the dynamic context requirements. The algorithms of handling authorization queries and roles mapping are also presented. The system architecture is discussed in detail to implement this model and support interoperation among heterogeneous platforms.

The service-oriented integration approach is presented to integrate many geographically and logically distributed, heterogeneous industrial simulation codes. The hierarchy is decomposed into four layers: the infrastructure layer, concrete web service layer, business service layer and application layer. The paper presents an approach to encapsulate industrial simulation codes into services following Web Service Resource Framework (WSRF) specification which can make them offer their services and functionality in a standard web service environment. To hide the complexity, changeability and the technical details (such as web services' WSDL address) of concrete web services for business users, these concrete web services are virtualized as business services. Three kinds of virtualization patterns are presented in the paper. Engineers can compose these business services in a high level to complete virtual product development. At last, we give a use case to validate our method and put forward the future work.

In manufacturing grid, a lot of applications and resources in manufacturing enterprises are encapsulated as Web services and a cooperative work environment is provided. Applying Web service composition and workflow technologies, services are integrated into a composite service to represent a cooperative business process. Therefore the enterprise cooperation can be implemented by executing and monitoring the composite service. According to the characteristics of manufacturing, this paper presents a composite service modeling solution based on BPEL4WS and discusses how this solution support BPEL4WS during the whole modeling process. The solution includes dynamic service discovery and service selection to support a flexible binding with the partners' services. This paper also describes how to execute and monitor the composite service

The combined exploration of grid technology and science of data visualization is a new paradigm in cross-domain science research. GVis presented in the paper is a distributed and collaborative Web service-based visualization environment that we developed to support automated availability and allocation of heterogeneous visualization resources. It provides a flexible framework for sharing of visualization components by geologically distributed collaborative users. The framework based on grid/Web services specified by OGSA model uses visualization resources broker (VisBroker) for the availability and allocation of remote components. Visualization grid services (VGS) runs as the core layer of GVis which splits and distributes visualization components on the grid following distributed visualization pipeline mapping model. Data driven mechanism is used for updating data and pipeline of collaborative users

The complexities of manufacturing resources greatly increase the difficulty of managing them. In this paper, with the help of dynamic manufacturing job management which is designed to invoke various types of services, a feasible approach is presented. According to the different mechanisms of service invoking, the invoking engine is divided into three categories: composite service invoking engine, Web service invoking engine and OGSA-DAI service invoking engine. Furthermore, through the use of composite service invoking engine, not only Web services but also flows which are composed of several manufacturing services can be invoked and controlled in a uniform way

Security and flexibility are two important issues in workflow management systems. The RBAC flexible workflow model is proposed recently which captures above needs, however there exist an open issue of its role hierarchies. With the numbers of users and roles increasing, how to reduce down the complexity of management is emerging concernful. A novel method of role management is presented in this paper that introduce the notation of partial inheritance into the role hierarchies. Compared with other dominating methods, it can efficiently achieve the objectives of role management inflexible workflow

Today, it has become more difficult for individual designer to fulfill the complicated design work due to the complexity of modern products. Grid provides a supporting environment for collaborative design. Cooperating work is the key to ensure the success of products design. Developing within the grid-based environment that is suitable to the demand of manufacturing design on special area is a challenge to the traditional methods of software development. In this paper, it creates a framework based manufacturing grid that uses peer-to-peer technology. According to this framework, there is a cooperating design environment, in which designers can make designing of machine tools. It also discusses the schedule of cooperating.

According to the CSCW cooperative mechanism, this paper presents the implementation of the distributed collaborative design platform based on pattern design, which is an application of CSCW in CAD. The architecture platform and protocol implementation is provided. It also describes the push and pulls functions used to perform data integrity and maintenance consistency. The applications show that the platform has features of CSCW to ensure that it operates well in real world conditions.

Digital museums become necessary when we have to present the precious collections to more and more visitors while preventing the state of preservation of the collections deteriorating over the time. It has significance to share and protect expensive museum resources. However, most current digital museum systems present the vast amount information in a static and primitive fashion and are not fully capable of dealing with generally heterogeneous and isolated resources. In this paper, we propose a grid-enabled digital museum system to improve the quality, availability and variety of the information presentation and sharing by incorporating grid-computing technologies. By designing cooperative grid services, we could efficiently and effectively organize, present and share the resources over the grid. The framework and workflow of the system are illustrated and the implementation details are also given.

With the fast increase of electronic commerce, more and more enterprises and organizations are facilitating their business processes by workflow. To protect information secure and meet the requirement of frequent business changes over time, the security and flexibility become two of the most important aspects that attract attention both from academy and industry. Many related research work on flexible workflow and secure RBAC model were presented respectively. Unfortunately, the analysis and implementation of enforcing RBAC into Web-based flexible workflow have not been mentioned The intention of this paper is to extend RBAC framework further to flexible workflow to support the security, flexibility and expansibility of organization business. A model and its corresponding mechanism are introduced for establishment, dynamical customization and run-time management of the RBAC workflow. A practical system for Property Right Exchange (PRES) based on this model is implemented.

In the paper, we are devoted to resolve the problem of resource organizing in manufacturing grid. A manufacturing grid (MG) architecture is first introduced which adopts the thoughts of peer-to-peer system and Web service. Then the issue to organize the resource in the manufacturing grid is mainly introduced. To describe the method of resource organizing, we introduce processes of resource publishing, resource searching and resource transfer or duplication when the hub joins in or exits from the grid are introduces seriatim. A distributed hashing algorithm is also presented. Then a prototype system of manufacturing grid is built to validate the way to organize resources.

With the development of grid technology, manufacturing grid (MG) has been proposed to standardize networked manufacturing modes in industry. As manufacturing resources are complicated, heterogeneous, geographically distributed, and owned by different enterprises, manufacturing resource sharing is much more complicated in MG. Resource owners and users have respective requirements, and then MG should provide capability to trade between them. This paper presents a trading supported manufacturing resources sharing model. Organized in a P2P pattern, Hubs, computer nodes deployed core services required, make up the backbone of the model and responsible for aggregating manufacturing resource. In the model, the resource selection process includes two phases: resource discovering and resource trading. Firstly, discover all the potential appreciate resources by multi-keys hashing; Secondly, gain the best appreciate resources by resource trading to make the benefits of both sides best. This model supports many kinds of economic models. The resources owners and users can use any one or some of these models or even combinations of them in meeting their objectives. The procedure of the competitive bidding trade and the method of evaluating tenders are discussed in detail.

A semantic Web based model for product cooperative design is proposed by enabling engineering knowledge to be published, discovered, reused, and integrated. Three features of the model are described in detail. The first is the tools for Web design, which enable a design process to be published as a service and then to be reused as a design resource in other design processes; the second is the ontology of design resources, which helps the implementation of intelligent retrieval by semantic inference; the third is design workflows, which enable designers to construct complex design with the existing simple design instances.

This paper mainly discusses how to design a virtual museum system used by many users at the same time, and how to manage the scene data. To a 3D indoor scene, the empty scene (only the building) without other objects such as collections can be created by sweeping a 2D polygon along an axis which is perpendicular to the horizontal plane. We can create the scene polygon's Voronoi diagram, which subdivides the polygon into some cells called Voronoi regions by the proximity characteristic. An edge of the polygon corresponding to a wall of the building has its own Voronoi region where all the points are nearest to the edge than others. Hence, every user can select one Voronoi region to lay out collections such as cultural relics on the exhibition platform and every collection is saved by the Voronoi region in which it is laid out. Based on the scene polygon's Voronoi diagram, the system can also support multi-user cooperative interaction in the created scene very well. The work also can be used to create virtual art gallery, product exhibition hall and so on.

This paper describes a system framework of 3D virtual museum supporting multi-user cooperative interaction. To a 3D virtual museum, the building (empty scene) can be projected to a 2D polygon. Hence, we can manage the data of collections and users based on the scene polygon's Voronoi diagram. We can get the Voronoi diagram by subdividing the polygon into some cells using the proximity characteristic. We also establish some communication rules between users. In this article, how to compute visibility and detect collisions based on polygon's Voronoi diagram is discussed and this method is aimed to improve the reality and coherence of cooperative interaction in virtual museum.