Incremental object detection (IOD), surpassing simple classification, requires the simultaneous overcoming of catastrophic forgetting in both recognition and localization tasks, primarily due to the significantly higher feature space complexity. Integrating Knowledge Distillation (KD) would mitigate the occurrence of catastrophic forgetting. However, the challenge of knowledge shift caused by invisible previous task data hampers existing KD-based methods, leading to limited improvements in IOD performance. This paper aims to alleviate knowledge shift by enhancing the accuracy and granularity in describing complex high-dimensional feature spaces. To this end, we put forth a novel higher-dimension-prototype learning approach for KD-based IOD, enabling a more flexible, accurate, and fine-grained representation of feature distributions without the need to retain any previous task data. Existing prototype learning methods calculate feature centroids or statistical Gaussian distributions as prototypes, disregarding actual irregular distribution information or leading to inter-class feature overlap, which is not directly applicable to the more difficult task of IOD with complex feature space. To address the above issue, we propose a Gaussian Mixture Distribution-based Prototype (GMDP), which explicitly models the distribution relationships of different classes by directly measuring the likelihood of embedding from new and old models into class distribution prototypes in a higher dimension manner. Specifically, GMDP dynamically adapts the component weights and corresponding means/variances of class distribution prototypes to represent both intra-class and inter-class variability more accurately. Progressing into a new task, GMDP constrains the distance between the distribution of new and previous task classes, minimizing overlap with existing classes and thus striking a balance between stability and adaptability. GMDP can be readily integrated into existing IOD methods to enhance performance further. Extensive experiments on the PASCAL VOC and MS-COCO show that our method consistently exceeds four baselines by a large margin and significantly outperforms other SOTA results under various settings.

Simulation speed is crucial for virtual reality simulators that simulate real-time cutting of deformable objects with haptic feedback, such as surgical simulators. This type of simulator combines visual feedback and haptic feedback, and therefore can be considered as a type of Multimedia Applications. To increase simulation speed, improvements are made in this paper to a previous deformable cutting method which divides a deformable object’s surface mesh into an interface mesh (including exterior surface mesh and interior surface mesh between different materials) constructed on a fine level linked voxel grid and a cut surface mesh constructed on a coarse level linked voxel grid. Our method changes the fine level linked voxel grid from a uniform grid to an octree. The algorithms for constructing and incrementally updating the object surface mesh and the collision proxy (an approximation of the object surface mesh used for collision processing) are changed accordingly. A new algorithm is proposed to resolve inconsistencies between partially cut and fully cut parts using visibility tests. Simulation tests show that our proposed method can moderately increase simulation speed during cutting and reduce CPU memory usage with almost imperceptible reductions in rendering qualities.

In the realm of multi-person pose estimation, bottom-up approaches often tackle the task of identifying human keypoints for individuals at various scales within a given image. However, in practical scenarios, algorithms tend to perform better in recognizing larger individuals. This is primarily attributed to the increased pixel count and richer feature information available. Conversely, recognizing smaller-scale individuals poses a notably more challenging task.To address this challenge, we propose an instance-level data augmentation strategy. This strategy involves applying transformations to individual instances rather than the entire image. Its primary objectives are to enhance dataset diversity, refine the distribution of different human scale samples in the training data, and augment the representation of medium-sized human instances in the training set. The goal of this augmentation strategy is to empower the model to better recognize finer details.Our extensive experiments, conducted on the HigherHRNet benchmark model, demonstrate the effectiveness of our approach in improving accuracy, particularly in the recognition of mediumsized individuals. Importantly, these improvements are achieved without introducing additional model complexity or requiring additional image collection.

This article proposes a two-stage image segmentation method based on the MS model, aiming to enhance the segmentation accuracy of images with complex structure and background. In the first stage, in order to obtain the smooth approximate solution of the image by minimizing the energy functional, an anisotropic regularization term formed by the combination of the gradient operator and an adaptive weighted matrix is introduced. Different weights in both horizontal and vertical directions can be provided by the adaptive weighting matrix according to the gradient information, so that the curve diffuses along the directions of local feature tangents of the objects. In addition, information irrelevant to the image target can be filtered out by the adaptive weighting matrix, thus reducing the interference of complex background. The alternating direction method of multipliers (ADMMs) is employed to solve the convex optimization problem in the first stage. In the second stage, the smoothed image obtained in the first stage is segmented by the deep learning method. By comparing with some traditional methods and deep learning methods, the results demonstrate that not only has good perceptual quality been achieved by this segmentation method, but also superior evaluation metrics have been obtained.

Detecting oriented objects along with estimating their rotation information is one crucial step for image analysis, especially for remote sensing images. Despite that many methods proposed recently have achieved remarkable performance, most of them directly learn to predict object directions under the supervision of only one (e.g., the rotation angle) or a few (e.g., several coordinates) groundtruth (GT) values individually. Oriented object detection would be more accurate and robust if extra constraints, with respect to proposal and rotation information regression, are adopted for joint supervision during training. To this end, we propose a mechanism that simultaneously learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner, via naive geometric computing, as one additional steady constraint. An oriented center prior guided label assignment strategy is proposed for further enhancing the quality of proposals, yielding better performance. Extensive experiments on six datasets demonstrate the model equipped with our idea significantly outperforms the baseline by a large margin and several new state-of-the-art results are achieved without any extra computational burden during inference. Our proposed idea is simple and intuitive that can be readily implemented. Source codes are publicly available at: https://github.com/wangWilson/CGCDet.git.

With the outbreak of COVID-19 around the world, the frequency of video conferencing at home is increasing. Therefore, a segmentation architecture that can quickly carry out close-range portrait segmentation has become a current need. However, the current portrait segmentation architectures cannot meet the requirements of lightweight and edge-friendly. We built architecture with 0.06G FLOPs and 0.02M parameters to overcome this phenomenon. This lightweight architecture can be better embedded and run on mobile devices that only support CPU computing. Our network achieves an FPS of 39.02 on CPU, which is more than three times faster than other networks. In addition, we pay special attention to the enhancement of edge features. The independent edge feature enhancement is embedded, and the edge-optimized attention mechanism (EOAM) is designed to collect specific edge areas for the bottom features and the high-level features in the process of feature fusion. Codes and results are publicly available at https://github.com/XinyueZhangqdu/ESPS.

In this work, we propose a flexible and efficient instance segmentation framework, termed DISF (Dynamic Instance Segmentation with Semantic Features), which is a more novel two-stage instance segmentation framework. Firstly, we divide the image into multiple regions of the same size and directly classify the pixels in different regions, which converts the instance-level segmentation task into the pixel-level classification task within the region. We make full use of the location and size information of objects to distinguish different instances of the same category and obtain relatively coarse instance segmentation results. Secondly, we decouple the prediction of instance masks into convolution kernel prediction and instance features prediction. The instance masks are dynamically generated by convolution operations between the predicted convolution kernel and instance features. The segmentation results in this way do not contain redundant information. Thirdly, a parallel branch of semantic segmentation is added to refine instance segmentation results further. Semantic features provide global information about the image from a higher level. Semantic features and instance features are sent to the Features Fusion Module (FFM) to optimize the relatively coarse instance segmentation results generated in the previous stage. The experimental results reveal the promising potential of DISF in instance-level recognition.

Transformers, which are famous for their attention mechanisms, have a strong ability to extract global information. The advantage of CNNs is parameter sharing, which can extract local information well. Both global information and local information are important for pose recognition. However, the existing human pose estimation methods based on Transformers or their variants can not extract the local information of images very well in mid-sized datasets (See [1] for more details). Therefore, we propose a novel Transformer framework to solve the problem of human pose estimation, termed ATPose (Aggregation Transformer for Human Pose Estimation): (1) We embed the convolution operations into the decoder of the Transformer to extract the local information. The attention module of Transformer first extracts global information from feature maps, and then the convolution layers perform convolution operations on feature maps to focus on local information. Thus, the Transformer can extract global information and local information at the same time. (2) We introduce the sparse attention mechanism and multi-scale attention mechanism into the Transformer. The sparse attention mechanism allows the feature pixels to only interact with sampled pixels, rather than all pixels, which reduces the computation cost. Multi-scale attention mechanism can calculate attention on feature maps with different resolutions to better extract small target features. (3) The Keypoint Head module is added to the decoder. The Keypoint Head can refine the prediction results of the model, making the predicted coordinates more accurate, and can also guide the training to prevent the model from deviating. The experimental results show ATPose has achieved state-of-the-art performance and become a new baseline in regression-based methods.

Previous video salient object detection (VSOD) approaches have mainly focused on the perspective of network design for achieving performance improvements. However, with the recent slowdown in the development of deep learning techniques, it might become increasingly difficult to anticipate another breakthrough solely via complex networks. Therefore, this paper proposes a universal learning scheme to obtain a further 3% performance improvement for all state-of-the-art (SOTA) VSOD models. The major highlight of our method is that we propose the ‘motion quality’, a new concept for mining video frames from the ‘buffered’ testing video stream for constructing a fine-tuning set. By using our approach, all frames in this set can all well-detect their salient object by the ‘target SOTA model’ — the one we want to improve. Thus, the VSOD results of the mined set, which were previously derived by the target SOTA model, can be directly applied as pseudolearning objectives to fine-tune a completely new spatial model that has been pretrained on the widely used DAVIS-TR set. Since some spatial scenes in the buffered testing video stream are shown, the fine-tuned spatial model can perform very well for the remaining unseen testing frames, outperforming the target SOTA model significantly. Although offline model fine tuning requires additional time costs, the performance gain can still benefit scenarios without speed requirements. Moreover, its semisupervised methodology might have considerable potential to inspire the VSOD community in the future.

Underwater captured images are usually degraded by low contrast, hazy, and blurry due to absorbing and scattering, which limits their analyses and applications. To address these problems, a red channel prior guided variational framework is proposed based on the complete underwater image formation model (UIFM). Unlike most of the existing methods that only consider the direct transmission and backscattering components, we additionally include forward scattering component into the UIFM. In the proposed variational framework, we successfully incorporate the normalized total variation item and sparse prior knowledge of blur kernel together. In addition, we perform the estimation of blur kernel by varying image resolution in a coarse-to-fine manner to avoid local minima. Moreover, for solving the generated non-smooth optimization problem, we employ the alternating direction method of multipliers (ADMM) to accelerate the whole progress. Experimental results demonstrate that the proposed method has a good performance on dehazing and deblurring. Extensive qualitative and quantitative comparisons further validate its superiority against the other state-of-the-art algorithms. The code is available online at:https://github.com/Hou-Guojia/UNTV

Single image dehazing and denoising models can simultaneously remove haze and noise with high efficiency. Here, the authors propose three variational models combining the celebrated dark channel prior (DCP) and total variations (TV) models for image dehazing and denoising. The authors firstly estimate the transmission map associated with depth using DCP, then design three variational models for colour image dehazing and denoising based on this estimation and the layered total variation (LTV) regulariser, multichannel total variation (MTV) regulariser, and colour total variation (CTV) regulariser, respectively. In order to improve the computation efficiency of the three models, the authors design their fast split Bregman algorithms via introducing some auxiliary variables and the Bregman iterative parameters. Numerous experiments are presented to compare their denoising effects, edge‐preserving properties, and computation efficiencies. To demonstrate the merits of the proposed models, the authors also conduct some comparisons with several existing state‐of‐the‐art methods. Numerical results further prove that the LTV‐based model is fastest, and the CTV model is the best for denoising with edge‐preserving, and it also leads to the best visually haze‐free and noise‐free images.

In this study, a novel underwater colour image enhancement approach based on hue preserving is presented by combining hue–saturation–intensity (HSI) and HS–value (HSV) colour models. In this study, the proposed wavelet‐domain filtering (WDF) and constrained histogram stretching (CHS) algorithms are operated on HSI and HSV colour models, respectively. The degraded image is first converted from red–green–blue colour model into the HSI colour model, wherein the hue component H is preserved and WDF algorithm is executed on the S and I components. Similarly, the image is further converted into the HSV colour model, wherein H component is kept invariant as well and CHS algorithm is applied on the S and V components. The authors' key contribution is that the H preserving method can improve image quality in terms of contrast, colour rendition, non‐uniform illumination, and denoising. In addition, experimental results show that the proposed approach outperforms several other state‐of‐the‐art algorithms.

It is common that textures occur in real-word color image, moreover, textures could cause difficulties in image segmentation. For the purpose of solving those difficulties, we put forward a new model. In this model we only need the structural and oscillating components’ information of the real color image. This model is based on the VO model, MTV and active contour models. We will use the fast Split Bregman algorithm to solve this model. The results of our model is mentioned in numerical experiments.

In this paper, we present a system to detect and count the number of vehicles in traffic surveillance videos based on Fast Region-based Convolutional Network (Fast R-CNN). Fast R-CNN is a state-of-the-art object detection network, which takes an entire image and a set of object proposals as input, produces bounding-box positions with probability estimates over object classes as output. First, we fine-tune a pre-trained Fast R-CNN net with images captured from traffic videos for accuracy improvement. Second, we define a series of rules of bounding boxes screening for vehicle counting. The proposed system takes around 3 seconds per image to count vehicles on a GTX970 GPU, and then records the corresponding number of vehicles into a database for traffic flow analysis. Experimental results demonstrated that the proposed system can provide significant improvements on the detection accuracy. In addition, experiments on challenging videos with occlusions or full of vehicles show that the proposed system works effectively.

OTSU method is considered to be the best algorithm for image segmentation of threshold selection. It's very simple and it's regardless of image brightness and contrast effects. Therefore it has been widely used in digital image processing. However, in the actual image, because of the influence of noise etc., traditional OTSU algorithm cannot be obtained an accurate segmentation results. In this paper, we divide the CT images of the liver by the combination of the PM filter and local gray stretch. Then we get binary images of vascular. Finally we reconstruct the venous system in liver by the use of Visualization Toolkit (VTK) and 2D segmentation results.

The variational level set model for piecewise constant multiphase image segmentation on the plane and the related Split-Augmented-Lagrangian Projection Method (SALPM) are investigated in this paper. On the analysis of the current problems based on the variational level set method for image segmentation, we also design a rapid SALPM method for two-phase image segmentation model, getting the general model of multiple phase level set in order to facilitate the generic design and program. In addition, the concrete formula of the rapid split algorithms for multiphase image segmentation with level set model and the calculation steps are given, and simultaneously taking liver tumor CT image as examples of the multiphase segmentation. Moreover, by comparing with the traditional methods, the experiments show that our algorithm presented in this paper have higher computational efficiency and accuracy, and better the extraction of liver contour.

Motion deblur is a very hot and hard research topic currently because it is an ill-posed problem. In this paper, we proposed using normalized nonlinear diffusion regularization for motion deblurring. To reduce the complexity of solving the deblurring equation, a fast method called Split method is used. Because the result derived from the energy function give the lowest cost, our method doesn't need any auxiliary method for solving the energy function besides multiscale implementation. Using the estimated kernel, the final clear image can be got by total variation method. Experiments demonstrate the validity of the proposed method.