Large visual-language models (LVLMs) have achieved great success in multiple applications. However, they still encounter challenges in complex scenes, especially those involving camouflaged objects. This is primarily due to the lack of samples related to camouflaged scenes in the training dataset. To mitigate this issue, we construct the MM-CamObj dataset for the first time, comprising two subsets: CamObj-Align and CamObj-Instruct. Specifically, CamObj-Align contains 11,363 image-text pairs, and it is designed for VL alignment and injecting rich knowledge of camouflaged scenes into LVLMs. CamObj-Instruct is collected for fine-tuning the LVLMs with improved instruction-following capabilities, and it includes 11,363 images and 68,849 conversations with diverse instructions. Based on the MM-CamObj dataset, we propose the CamObj-Llava, an LVLM specifically designed for addressing tasks in camouflaged scenes. To facilitate our model's effective acquisition of knowledge about camouflaged objects and scenes, we introduce a curriculum learning strategy with six distinct modes. Additionally, we construct the CamObj-Bench to evaluate the existing LVLMs' capabilities of understanding, recognition, localization and count in camouflage scenes. This benchmark includes 600 images and 7 tasks, with a total of 9,449 questions. Extensive experiments are conducted on the CamObj-Bench with CamObj-Llava, 8 existing open-source and 3 closed-source LVLMs. Surprisingly, the results indicate that our model achieves a 25.84% improvement in 4 out of 7 tasks compared to GPT-4o.

Multi-Task Learning (MTL) enables a single model to learn multiple tasks simultaneously, leveraging knowledge transfer among tasks for enhanced generalization, and has been widely applied across various domains. However, task imbalance remains a major challenge in MTL. Although balancing the convergence speeds of different tasks is an effective approach to address this issue, it is highly challenging to accurately characterize the training dynamics and convergence speeds of multiple tasks within the complex MTL system. To this end, we attempt to analyze the training dynamics in MTL by leveraging Neural Tangent Kernel (NTK) theory and propose a new MTL method, NTKMTL. Specifically, we introduce an extended NTK matrix for MTL and adopt spectral analysis to balance the convergence speeds of multiple tasks, thereby mitigating task imbalance. Based on the approximation via shared representation, we further propose NTKMTL-SR, achieving training efficiency while maintaining competitive performance. Extensive experiments demonstrate that our methods achieve state-of-the-art performance across a wide range of benchmarks, including both multi-task supervised learning and multi-task reinforcement learning. Source code is available at https://github.com/jianke0604/NTKMTL.

In realistic open-set scenarios where labels of a part of testing data are totally unknown, when vision-language (VL) prompt learning methods encounter inputs related to unknown classes (i.e., not seen during training), they always predict them as one of the training classes. The exhibited label bias causes difficulty in open set recognition (OSR), in which an image should be correctly predicted as one of the known classes or the unknown one. To achieve this goal, we propose a vision-language prompt tuning method with mitigated label bias (M-Tuning). It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Besides, inspired by the observation that classifying directly on large datasets causes a much higher false positive rate than on small datasets, we propose a Combinatorial Tuning and Testing (CTT) strategy for improving performance. CTT decomposes M-Tuning on large datasets as multiple independent group-wise tuning on fewer classes, then makes accurate and comprehensive predictions by selecting the optimal sub-prompt. Finally, given the lack of VL-based OSR baselines in the literature, especially for prompt methods, we contribute new baselines for fair comparisons. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.

Practical image recognition often encounters samples whose labels either are totally unknown or belong to new classes outside the training set. The first problem refers to the open-set recognition (OSR), in which unknown classes are recognized as one with no more semantic information. While the latter is called zero-shot learning (ZSL), in which new classes are usually predefined. The existing literature mostly addresses these two problems separately. In this paper, we take the ambition for solving the combination of these two problems to fulfill semantically recognizing the unknown classes detected in OSR by zero-shot prediction. We propose the Contrastive multimodal prompt tuning for Hierarchical Open-set Zero-shot recognition (CoHOZ). Specifically, we firstly build a global and compatible hierarchical label tree with all downstream datasets aligned, which lays foundations for other modules. To detect unknown classes, we propose the contrastive continuous prompt tuning, which introduces additional negative classes from the fine level of the built hierarchy for prompt learning. To generate candidate classes for zero-shot prediction on the unknown data using prompt, we combine the built hierarchy to collect candidate classes from coarse to fine. In our experiments, when following the standard OSR protocol regarding all the unknown classes as a single class, CoHOZ achieves a new state-of-the-art performance both in unknown detection and open-set recognition. Few-shot tuning by the CoHOZ also shows competitive performance on them. In addition, the detailed semantic information of unknown classes are well explored, which has also been verified in experiments.

In Synthetic Aperture Radar (SAR) image classification tasks, the performance depends on both the classifier and the dataset itself. However, in comparison with plenty of SAR classification methods, there is little work aimed at analyzing the distinguishability of the dataset. In the classification dataset, some classes are semantically different but their distinguishability is low, the classes are hard to be classified especially in some more practical cases that there are unknown classes without supervision exist. Referring to open set recognition (OSR), in this paper, we proposed the SAR Distinguishability Analysor (SAR-DA) to evaluate the distinguishability of the OpenSARUrban dataset. By modeling each class as a multivariate Gaussian distribution in latent space, SAR-DA can not only classify the classes having been seen in training phase, but also can recognize unknown samples if a test sample is out of each known distribution. Each class in OpenSARUr-ban is set unknown in turn, then we apply the SAR-DA on the split dataset in OSR and supervised setting. The distinguishability can be reflected by the unknown recognition recall rate. The experimental results show that the unknown recognition recall rate in OSR setting significantly decreased compared with those in supervised setting, indicating that even though the classes in OpenSARUrban are semantically different from each other, the latent distributions of some classes are quite similar and hard to be classified, thus these classes are of low distinguishability.

Image retargeting technology has been widely studied to adapt images for the devices with heterogeneous screen resolutions. Meanwhile effective objective retargeting quality assessment algorithms are also very important for optimizing and selecting favorable retargeting methods. Unlike previous assessment algorithms which rely on image local structure features and unidirectional prediction of information loss, we propose a bi-directional natural salient scene distortion model (BNSSD) including image natural scene statistics (NSS) measurement, salient global structure distortion measurement, and bi-directional salient information loss measurement. First, we propose a new NSS model in log-Gabor domain and verify its effectiveness in reflecting nature scene statistical distortions introduced during the retargeting process. Second, the concept of salient global structure distortion is proposed to measure the global structure uniformity in the corresponding salient regions between original and retargeted images. Finally, we propose a bidirectional salient information loss metric to measure the information loss between salient areas in original image and retargeted image. The effectiveness of the BNSSD model is verified on two widely recognized public databases, and the experimental results show that our method outperforms the state-of-the-art algorithms under different statistical assessment criteria.

Content-aware image retargeting has attracted substantial research interests in the related research community. However, so far there is still no method can preserve important image contents and structure well without introducing deformation. To address this problem, we propose a Saliency & Structure Preserving Multi-operator (SSPM) method. SSPM classifies images into three categories utilizing SIFT density to improve performance of saliency preservation, helping to mitigate negative influence from center-bias property of most existing saliency detection models. SSPM also employs different principles to improve structure preservation performance, including Earth Mover's Distance (EMD) and Gray-Level Cooccurrence Matrix (GLCM) to get optimal operator sequences for smart content-aware image retargeting. SSPM method not only can well preserve salient contents and structure, but also can greatly improve deformation resilience. Experimental results demonstrated that our method outperforms state-of-art image retargeting methods.

Visual Quality Assessment of 3D/stereoscopic video (3D VQA) is significant for both quality monitoring and optimization of the existing 3D video services. In this paper, we build a 3D video database based on the latest 3D-HEVC video coding standard, to investigate the relationship among video quality, depth quality, and overall quality of experience (QoE) of 3D/stereoscopic video. We also analyze the pivotal factors to the video and depth qualities. Moreover, we develop a No-Reference 3D-HEVC bitstream-level objective video quality assessment model, which utilizes the key features extracted from the 3D video bitstreams to assess the perceived quality of the stereoscopic video. The model is verified to be effective on our database as compared with widely used 2D Full-Reference quality metrics as well as a state-of-the-art 3D FR pixel-level video quality metric.

As the demand and supply for 3D technologies grows, the quality assessment of stereoscopic content is both demanding and increasingly urgent. This paper proposes a novel 3D images quality assessment (IQA) algorithm which is based on the Binocular Rivalry model and the Summation and Difference model. Both the influence of distorted disparity information on perceptual quality and the binocular rivalry mechanism of HVS are considered in our algorithm. Since disparity map is not utilized in our algorithm, it saves the complex computation of disparity map and improves the robustness of our algorithm. Experiment results shows that our algorithm outperforms well-known 3D IQA metrics on LIVE database and delivers competitive performance on Waterloo database. The cross database validation indicates that our algorithm is stable on different database.

No-reference bitstream-layer video quality assessment is very important and practical for monitoring the perceptual experience of end users and facilitating network maintenance. For pervasive Internet Protocol Television and mobile streaming services, in addition to quality degradation due to lossy compression, the unreliable transmission mechanism (i.e., User Datagram Protocol/IP) often leads to quality degradation due to packet loss. Different technical solutions bring in different types of visual artifacts. In this paper, we proposed a hybrid distortion ranking (HDR)-based bitstream-layer quality assessment model, whose artifact combination framework is based on the ranked linear combination operation. The model can predict the perceived quality of a video with sufficient accuracy when the video is distorted by compression artifacts, slicing artifacts, freezing (with frame skipping) artifacts, or their combinations. The core algorithms of the model were adopted into ITU-T Recommendations, P.1202.1 and P.1202.2. Furthermore, with respect to the three different types of artifacts, we compared the proposed no-reference HDR model with some state-of-the-art full-reference perceptual quality assessment models including Video Quality Model (i.e., ITU-T Rec. J.144), structural similarity (SSIM), multiscale SSIM, visual information fidelity, and the widely used metric, peak signal-to-noise ratio. We also compared our HDR model with the top performing no-reference models including Blind/Referenceless Image Spatial Quality Evaluator and video Blind Prediction of Natural Video Quality. The experiment results demonstrate the efficiency of our HDR model.

Packet-layer video quality assessment model is a light-weight model that is useful for application scenarios like inservice video quality monitoring and network service planning, which is under standardization in ITU-T. In this paper we first differentiate the requirements for video quality assessment model from these two different application scenarios. Then different criteria and methods are analyzed and used to construct two types of test datasets for model building and evaluation. Finally, a novel packet-layer video quality assessment model dealing with video quality monitoring is proposed by incorporating the estimation of the spatiotemporal complexity of video content. The interaction between content features and the error concealment effects plus error propagation effects is considered. Experiment results demonstrate that the proposed model achieves preferable and robust performance improvement compared with the existing models in both datasets for scenarios of video quality planning and monitoring. Especially, larger Pearson correlation increase from 0.75 to 0.93 and RMSE decrease from 0.56 to 0.31 are obtained in dataset for video quality monitoring.

IEEE802.16e based mobile WiMAX network provides a promising technology for broadband wireless access. Among all types of services supported by WiMAX, real time video multicast has attracted much attention. However, because of the absence of feedback channel for multicast service, it is challenging to provide link adaptation and resource allocation under the situation with diverse receiver distribution. The legacy method - using the most robust modulation mode for multicasting service, or select a leader as the representative - can not accurately reflect the variety of channel condition. In this paper, we firstly analyze the overhead problem caused by the periodic downlink channel quality report, then propose a light-weighted terminal triggered feedback mechanism to reduce the signaling overhead. Furthermore, with the combined consideration of layered video coding and adaptive modulation and coding, a utility-based resource allocation approach is proposed. The utility function is relating with the reception satisfaction and spectrum efficiency. The scheduling algorithm dynamically adjusts layer number and its bandwidth according to receiver distribution. Simulation results show the proposed feedback mechanism can greatly reduce the signaling overhead while maintain the accurate channel condition report. The scheduling algorithm can not only improve the link adaptation capability following the receivers' expectation and program popularity, but also outperform the fixed allocation method in term of system utility.

WiMAX technology is a good candidate to provide broadcast TV services for mobile users because it supports QoS based multicasting functionality. To provide multicast-based TV services over the mobile WiMAX network, a multicast session setup involves both MAC-layer and IP-layer multicast management. However, in mobile environment, IP-layer multicast management entails service interruption due to unreliable wireless transmission, and disrupts mobile terminal's power-saving mode. In this paper, we propose a cross-layer multicast service management scheme that can not only avoid the service interruption and power-saving mode disruption but also reduce the bandwidth consumption for IP-layer multicast signaling. Simulation results demonstrate that the scheme can eliminate the unnecessary power consumption caused by IP-layer multicast signaling, and improve the wireless uplink throughput as well. We also study, by simulation, the influence of the multicast management signaling procedures on channel switching time that is a major performance indication for mobile TV services.

Hybrid ARQ is deployed in error prone wireless network to provide reliable unicast transmission with the positive or negative acknowledgement report. However, the feedback implosion is a challenging problem for multicast service, wherein multiple receivers send out feedback in a short period of interval. The traditional representative-based or timer-based feedback suppression method can not provide adequate accuracy or latency guarantee; and the approach to monitor packet loss rate from each receiver has poor performance in scalability. This paper investigates the relation between feedback overhead and average recovery rate for multicast service in Time Division Multiple Access (TDMA) based wireless network. A scalable aggregate feedback mechanism is proposed to corporate with either fix-rate or rate-less channel coding scheme. Receivers who can not recover the data block are assigned to send identical NACK message at the same uplink feedback time slot, so that the sender can acquire a global view of the data block reception condition, an adaptive sliding window is defined in accordance with the previous NACK statistics, dynamic number of recovery parity symbols are delivered in each window following the sequence of exponent or arithmetical serial. Comparing with the conventional feedback suppression mechanism, the simulation results show the proposed mechanism can not only adjust the traffic of parity packets to match the actual reception condition among all receivers, but also greatly reduce the feedback overhead and improve the scalability.