Livestreaming has become increasingly prevalent in modern visual communication, where automatic camera quality tuning is essential for delivering superior user Quality of Experience (QoE). Such tuning requires accurate blind image quality assessment (BIQA) to guide parameter optimization decisions. Unfortunately, the existing BIQA models typically only predict an overall coarse-grained quality score, which cannot provide fine-grained perceptual guidance for precise camera parameter tuning. To bridge this gap, we first establish FGLive-10K, a comprehensive fine-grained BIQA database containing 10,185 high-resolution images captured under varying camera parameter configurations across diverse livestreaming scenarios. The dataset features 50,925 multi-attribute quality annotations and 19,234 fine-grained pairwise preference annotations. Based on FGLive-10K, we further develop TuningIQA, a fine-grained BIQA metric for livestreaming camera tuning, which integrates human-aware feature extraction and graph-based camera parameter fusion. Extensive experiments and comparisons demonstrate that TuningIQA significantly outperforms state-of-the-art BIQA methods in both score regression and fine-grained quality ranking, achieving superior performance when deployed for livestreaming camera tuning.

Recent years have witnessed remarkable achievements in perceptual image restoration (IR), creating an urgent demand for accurate image quality assessment (IQA), which is essential for both performance comparison and algorithm optimization. Unfortunately, the existing IQA metrics exhibit inherent weakness for IR task, particularly when distinguishing fine-grained quality differences among restored images. To address this dilemma, we contribute the first-of-its-kind fine-grained image quality assessment dataset for image restoration, termed FGRestore, comprising 18,408 restored images across six common IR tasks. Beyond conventional scalar quality scores, FGRestore was also annotated with 30,886 fine-grained pairwise preferences. Based on FGRestore, a comprehensive benchmark was conducted on the existing IQA metrics, which reveal significant inconsistencies between score-based IQA evaluations and the fine-grained restoration quality. Motivated by these findings, we further propose FGResQ, a new IQA model specifically designed for image restoration, which features both coarse-grained score regression and fine-grained quality ranking. Extensive experiments and comparisons demonstrate that FGResQ significantly outperforms state-of-the-art IQA metrics.

Semantic change detection (SCD) endeavors to identify land-cover changes from multitemporal remote sensing images, providing essential information for various applications. Nevertheless, conventional supervised SCD methods necessitate extensive pixel-level annotations, limiting their applicability. The capability of self-supervised methods to learn feature representations with large amounts of unlabeled data and minimal annotation, and to achieve superior performance, has made them one of the hot topics in remote sensing. However, most self-supervised methods in remote sensing are primarily designed to learn general semantic representations of images, which limits their effectiveness for tasks like SCD that require the analysis of complex semantic transformations. To address this, we propose a multistage, multitask, and multilevel self-supervised network, named S3CD, that learns semantic changes from bi-temporal remote sensing images across scene, pixel, and prototype levels in two stages. In particular, in Stage 2, the network enhances the robustness of SCD by learning semantic consistency within the semantic stable categories across different temporal and capturing the temporal patterns of semantic change categories. We evaluate S3CD on two widely used remote sensing change detection (CD) datasets, where it outperformed state-of-the-art self-supervised and supervised SCD methods. Notably, in the binary CD (BCD) task (i.e., detecting the locations of changes), S3CD also outperforms most supervised learning methods. Therefore, this approach facilitates the application of self-supervised learning in the field of remote sensing CD.

Recent works of music representation learning mainly focus on learning acoustic music representations with unlabeled audios or further attempt to acquire multi-modal music representations with scarce annotated audio-text pairs. They either ignore the language semantics or rely on labeled audio datasets that are difficult and expensive to create. Moreover, merely modeling semantic space usually fails to achieve satisfactory performance on music recommendation tasks since the user preference space is ignored. In this paper, we propose a novel Hierarchical Two-stage Contrastive Learning (HTCL) method that models similarity from the semantic perspective to the user perspective hierarchically to learn a comprehensive music representation bridging the gap between semantic and user preference spaces. We devise a scalable audio encoder and leverage a pre-trained BERT model as the text encoder to learn audio-text semantics via large-scale contrastive pre-training. Further, we explore a simple yet effective way to exploit interaction data from our online music platform to adapt the semantic space to user preference space via contrastive fine-tuning, which differs from previous works that follow the idea of collaborative filtering. As a result, we obtain a powerful audio encoder that not only distills language semantics from the text encoder but also models similarity in user preference space with the integrity of semantic space preserved. Experimental results on both music semantic and recommendation tasks confirm the effectiveness of our method.

Obtaining timely and reliable built-up area (BUA) information across extensive geographical zones holds crucial significance for understanding environmental change and human activities. BUAs often exhibit detailed textures and structures in high-resolution imagery but also present strong heterogeneity. Current methods for BUA extraction primarily relied on planar information from single-view imagery, struggling to effectively capture the 3-D attributes of urban landscapes. Therefore, to address this challenge, this article proposes a cross-angle propagation network (CAPNet) based on multiview remote sensing stereo observation imagery. Our contributions are threefold: 1) we propose the cross-angle fusion module (CAFM) to exploit BUA’s complementary spatial-spectral-angular context across different viewing angles. This module leverages attention mechanisms for the automated acquisition of multiangle feature representation learning from diverse angle combinations. 2) We propose a multiangular propagation decoder (MAPD) that pioneers the exploration of gradually propagating multiangle disparity information through bidirectional-adjacent feature fusion across hierarchical levels. 3) We construct a large-scale, high-resolution multiview BUA (MVBA) dataset over China’s 41 major cities based on the ZY-3 satellites. Extensive experiment results on MVBA and the public WV-3 multiview semantic stereo datasets verify CAPNet’s superiority to existing state-of-the-art (SOTA) models, on preserving overall BUA shape, edge, and internal structures. The dataset and the source code of CAPNet will be publicly available athttps://github.com/zuo-ux/Cross-Angle-Propagation-Network.

In e-commerce platforms, the relevant recommendation is a unique scenario providing related items for a trigger item that users are interested in. However, users' preferences for the similarity and diversity of recommendation results are dynamic and vary under different conditions. Moreover, individual item-level diversity is too coarse-grained since all recommended items are related to the trigger item. Thus, the two main challenges are to learn fine-grained representations of similarity and diversity and capture users' dynamic preferences for them under different conditions. To address these challenges, we propose a novel method called the Dynamic Preference-based and Attribute-aware Network (DPAN) for predicting Click-Through Rate (CTR) in relevant recommendations. Specifically, based on Attribute-aware Activation Values Generation (AAVG), Bi-dimensional Compression-based Re-expression (BCR) is designed to obtain similarity and diversity representations of user interests and item information. Then Shallow and Deep Union-based Fusion (SDUF) is proposed to capture users' dynamic preferences for the diverse degree of recommendation results according to various conditions. DPAN has demonstrated its effectiveness through extensive offline experiments and online A/B testing, resulting in a significant 7.62% improvement in CTR. Currently, DPAN has been successfully deployed on our e-commerce platform serving the primary traffic for relevant recommendations.

Different from large-scale platforms such as Taobao and Amazon, CVR modeling in small-scale recommendation scenarios is more challenging due to the severe Data Distribution Fluctuation (DDF) issue. DDF prevents existing CVR models from being effective since 1) several months of data are needed to train CVR models sufficiently in small scenarios, leading to considerable distribution discrepancy between training and online serving; and 2) e-commerce promotions have significant impacts on small scenarios, leading to distribution uncertainty of the upcoming time period. In this work, we propose a novel CVR method named MetaCVR from a perspective of meta learning to address the DDF issue. Firstly, a base CVR model which consists of a Feature Representation Network (FRN) and output layers is designed and trained sufficiently with samples across months. Then we treat time periods with different data distributions as different occasions and obtain positive and negative prototypes for each occasion using the corresponding samples and the pre-trained FRN. Subsequently, a Distance Metric Network (DMN) is devised to calculate the distance metrics between each sample and all prototypes to facilitate mitigating the distribution uncertainty. At last, we develop an Ensemble Prediction Network (EPN) which incorporates the output of FRN and DMN to make the final CVR prediction. In this stage, we freeze the FRN and train the DMN and EPN with samples from recent time period, therefore effectively easing the distribution discrepancy. To the best of our knowledge, this is the first study of CVR prediction targeting the DDF issue in small-scale recommendation scenarios. Experimental results on real-world datasets validate the superiority of our MetaCVR and online A/B test also shows our model achieves impressive gains of 11.92% on PCVR and 8.64% on GMV.