Class-incremental semantic segmentation (CISS) faces significant challenges due to catastrophic forgetting and background shift problems. While existing approaches typically rely on labeled data to progressively finetune models for new tasks, these methods become impractical when only multiple pre-trained models are available or when labeled data cannot be accessed for either old or new tasks. To address this challenge, we propose Multiple Model Consensus (MMC), which leverages the outputs from multiple pre-trained models on unlabeled data. MMC selectively distills trustworthy knowledge by identifying consensus among the models' predictions. We evaluate our method using Pascal VOC2012 as the training dataset and MS COCO2017 as a source of unlabeled auxiliary data. Experimental results demonstrate the effectiveness of our approach, achieving 6.3% and 15.55% improvement in mean Intersection over Union (mIoU) compared to existing methods after incorporating three consecutive tasks.

This paper presents an incremental learning method that aims to address the challenges of retraining deep neural networks with old data for new categories in remote sensing image classification. While deep learning models can quickly update the existing model with new data, they face challenges such as "catastrophic forgetting" and the "stability-plasticity dilemma." To address these challenges, this study proposes an incremental learning method that uses feature map-based knowledge distillation to enable the new model to memorize the old model's knowledge. The discrepancy loss between multiple classifiers is calculated to enhance performance on new tasks. The proposed method is compared to the classical incremental learning method Learning without Forgetting (LwF) on the RESISC45 dataset. The results show that our method improves the overall accuracy by 3.2% and reduces the forgetting rate by 2.0% compared to LwF after continuously adding one to five tasks. The proposed method can effectively reduce the degree of catastrophic forgetting and provide a feasible solution for remote sensing image classification problems.

Most of the deep neural networks require a large amount of data for training to achieve good results. However, in practical anomaly detection applications, we usually only have few labeled anomalous samples that have various types. This study proposed an innovative hybrid architecture that aims to detect anomalies with a small number of labeled anomalous samples. The proposed method consists of two stages. First, the network learns the distribution of normal data with a generative adversarial network (GAN). Second, the discriminator of the network is combined with a classifier and the training process updates different network components depending on whether the labeled samples are normal or anomalies. From the experiments on CIFAR-10 and UC Merced datasets, we demonstrate that our method yields significant performance improvements then another GAN-based approach even when few labeled samples are provided.

A traffic flow estimation mechanism is proposed for highway surveillance systems with scenes tampered by raindrops. To detect rain-drop tampered scenes, features are extracted via salient region detection and block segmentation. Feature selection is performed to select more discriminative features. For traffic flow estimation, daytime and night time models are performed separately to adapt to the characteristics of the surveillance scenes. Finally, an effective graph-based mapping method is designed to map the vehicle count sequences to per minute traffic flow. The system is tested with a highly challenging dataset. The accuracy of the traffic flow analysis is satisfying with low mean absolute errors even when the cameras are seriously tampered by rain.

This paper proposes a framework to automatically create cartoon images with low computation resources and small training datasets. The system segments and reassembles regions according to the topologies learned from example images. Region relationship trees are constructed for training images with no requirement of manual labeling. An enhanced clustering mechanism with no prior knowledge of cluster number is designed to effectively group components into desired groups for image creation. Compared with methods based on Generative Adversarial Networks, the proposed framework which performs automatic reasoning, clustering and reassembling regions of cartoon images can create better images with a very small amount of training samples.

In this study, we proposed a new operator known as the synchronized rotation local ternary pattern (SRLTP) for texture classification. The proposed SRLTP descriptor improves on the local ternary pattern (LTP) method with an additional process on the generated lower and upper LTPs. The lower and upper patterns are encoded to a rotation invariant pattern histogram and a uniform pattern histogram, respectively. Thus, the feature vector can utilize the advantages offered by the rotation invariant pattern histogram while retaining the original information in the uniform pattern histogram. Moreover, in this study, a two-dimensional discrete wavelet transform (DWT) and a discrete Fourier transform (DFT) enhanced the robustness of the texture classification. The experimental results demonstrate that the performance of the SRLTP descriptor is better than those of the existing descriptors.

This paper proposes a system to automatically perform classification and data cleaning on location images in a database contributed by arbitrary users. Since human inspection is not feasible for large-scale databases, the ability to detect incorrect scenes uploaded by users is very important to maintain the correctness of the database. In this work, we compare different feature extractors using deep convolutional networks trained by massive datasets. Also, a detector is designed to identify incorrect scenes that can overcome the challenges of large intra-cluster distances. The experiments have validated the effectiveness of the proposed approach on a very challenging dataset.

In this work, an accurate solar irradiance now-casting system is presented. The proposed system utilizes information from all-sky images to complement insufficient temporal and spatial resolution of satellite images and help improve prediction accuracy. Based on the history irradiance and the all-sky image features, irradiance are predicted ten to fifteen minutes ahead to allow Photovoltaics operators to schedule and allocate energy resources more effectively. To capture the relationship between clouds and the sun, an enhanced cloud detection algorithm using multi-scale neighborhood features is proposed. Then, feature points on the clouds are tracked to predict if the sun will be occluded by the clouds. According to the cloud tracking results, ramp-down events are forecasted and the predicted solar irradiance is refined. The proposed system is validated using a challenging dataset and exhibits superior performance compared with existing works.

In this work, we propose a self-diagnosing intelligent highway surveillance system and design effective solutions different lighting and weather conditions. If tracking algorithms could work properly, performing tracking should be preferred in intelligent surveillance systems. However, it is unrealistic to segment and track each individual vehicle under all circumstances. Under congestion conditions, we propose a mechanism to estimate the traffic flow parameter via regression analysis. The experimental results have shown that the self-diagnosis ability and the modules designed for the system make the proposed system robust and reliable.

In this paper, we propose an intelligent highway surveillance system that performs self-diagnosis and detects conditions when the camera is seriously tampered by raindrops at night. The system also provides solutions to analyze the traffic flow under the challenging nighttime raindrop-tampered conditions. To deal with the challenging scenes, we extract effective features via salient region detection and block segmentation. The extracted features are used to train a support vector machine to achieve self-diagnosis. For traffic flow analysis, we use the extracted features in the region of interest and construct a regression model to get an estimated vehicle count for each frame. The vehicle counts in consecutive frames form a vehicle count sequence. We propose a mapping model to acquire the desired per-minute traffic flow from the vehicle count sequence. The model utilizes state transfer likelihoods and takes into account the length of the segmented vehicle count sequence. The experiments on highly challenging data sets have demonstrated that the proposed system can effectively estimate the traffic flow for raindrop-tampered highway surveillance cameras at night.

The proposed system provides a solution to analyze the traffic flow under challenging nighttime conditions when the surveillance camera is raindrop tampered. To deal with the challenging scenes, we extract effective features via salient region detection and block segmentation. We use the extracted features in the region of interest to construct a regression model to get an estimated vehicle number for each frame. The vehicle numbers in consecutive frames form a vehicle number sequence. A mapping model utilizing state transition likelihoods is proposed to acquire the desired per minute traffic flow from the vehicle number sequence. The experiments on highly challenging datasets have demonstrated that the proposed system can effectively estimate the traffic flow for rain-drop tampered highway surveillance cameras at night.

This paper presents a reliable and robust palmprint verification approach that involves using a bi-feature, biometric, palmprint feature-point number (FPN) and a histogram of oriented gradient (HOG). The bi-feature was fused and verified using a support vector machine (SVM) at the feature level. The approach has the advantages of capturing palm images in pegless scenarios with a low cost and low-resolution (100 dpi) digital scanner, and one sensor can capture palmprint bi-feature information. The low-resolution images result in a smaller database. Nine thousand palmprint images were collected from 300 people to verify the validity of the proposed approach. The results showed an accurate classification rate of 99.04%. The experimental results demonstrated that the proposed approach is feasible and effective in palmprint verification. Our findings will help extend palmprint verification technology to security access control systems.

We present an automatic vehicle detection system for aerial surveillance in this paper. In this system, we escape from the stereotype and existing frameworks of vehicle detection in aerial surveillance, which are either region based or sliding window based. We design a pixelwise classification method for vehicle detection. The novelty lies in the fact that, in spite of performing pixelwise classification, relations among neighboring pixels in a region are preserved in the feature extraction process. We consider features including vehicle colors and local features. For vehicle color extraction, we utilize a color transform to separate vehicle colors and nonvehicle colors effectively. For edge detection, we apply moment preserving to adjust the thresholds of the Canny edge detector automatically, which increases the adaptability and the accuracy for detection in various aerial images. Afterward, a dynamic Bayesian network (DBN) is constructed for the classification purpose. We convert regional local features into quantitative observations that can be referenced when applying pixelwise classification via DBN. Experiments were conducted on a wide variety of aerial videos. The results demonstrate flexibility and good generalization abilities of the proposed method on a challenging data set with aerial surveillance images taken at different heights and under different camera angles.

In this paper, we propose a self-diagnosing intelligent highway surveillance system and design effective solutions for both daytime and nighttime traffic surveillance. For daytime surveillance, vehicles are detected via background modeling. For nighttime videos, headlights of vehicles need to be located and paired for vehicle detection. An algorithm based on likelihood computation is developed to pair the headlights of vehicles at night. Moreover, to balance between the robustness and abundance of acquired information, the proposed system adapts different strategies under different traffic conditions. Performing tracking would be preferred when traffic is smooth. However, under congestion conditions, it is better to obtain traffic parameters by estimation. We utilize a time-varying adaptive system state transition matrix in Kalman filter for better prediction in a traffic surveillance scene when performing tracking. We also propose a mechanism for estimating the traffic flow parameter via regression analysis. The experimental results have shown that the self-diagnosis ability and the modules designed for the system make the proposed system robust and reliable.

Falsification of human faces using face photos has been an arising problem for face recognition and verification systems. In this paper, we propose a system to distinguish face photos from true faces by their motion models. In order to enhance the difference between the two classes, we design an enhanced optical flow method which generates a larger difference between the motion model of true faces and that of face photos. The feature vector we adopted is the dense optical flow field across a short period of time. An LDA-based training method is adopted to separate the projection of the training data into two classes, and a Bayes classifier is used to classify the testing samples. Under the specified motion of true faces and face photos, our proposed method can effectively distinguish the two classes with high verification rate. Even if the motion is arbitrary for both classes, the proposed system can also report satisfying results.

In this paper, we propose a fully automated approach for tracking of multiple objects across multiple cameras with overlapping and non-overlapping views in a unified framework without initial training. For single camera cases, Kalman filter and adaptive particle sampling are integrated for multiple objects tracking. When extended to multiple cameras cases, the relations between adjacent cameras are learned systematically by using image registration techniques for consistent handoff of tracking-object labels across cameras. In addition, object appearance measurement is employed to validate the labeling results. Experimental results demonstrate the performance of our approach on real video sequences for cameras with overlapping and non-overlapping views.

This paper presents a hierarchical lane detection system with the ability to deal with both structured and unstructured roads. The proposed system classifies the environment first before applying suitable algorithms for different types of roads. For environment classification, pixels with lane- marking colors are extracted as feature points. Eigenvalue decomposition regularized discriminant analysis is utilized in model selection and maximum likelihood estimation of Gaussian parameters in high dimensional feature space. For structured roads, the extracted feature points are reused for lane detection. For unstructured roads, mean-shift segmentation is applied to divide the scene into regions. Possible road boundary candidates are selected, and Bayes rule is used to choose the most probable boundary pairs. The experimental results show that the system is able to robustly find the boundaries of the lanes on different types of roads and various weather conditions.

A multiple-target tracking system aimed at analyzing crossroad traffic systematically is proposed in this paper. The proposed mechanism is based on Kalman filtering and modified probabilistic data association. Unlike traditional Kalman filtering tracking, the proposed mechanism constructs candidate measurement lists by matching the sizes of the measurements and the targets first. When the sizes do not match, object matching within a limited area is performed. Also, we modify the classical probabilistic data association method to enhance its performance and make it more suitable for vision-based systems. The proposed mechanism, which can serve as the foundation for automatic traffic event detection, can solve the occlusion problems effectively without incurring too much computational complexity.

We developed a system for automatic 3-D human body motion in video surveillance sequences. As the first step, the human body that will be used for the analysis is extracted and tracked based on video object (VO) segmentation technique. Once human body is extracted, this 2-D object, which consists of the merging of several regions, is then analyzed for activity and body part interpretation. The specific 2-D human body shape along with the information derived from maximum probable explanation (MPE) of dynamic Bayesian modeling is then effectively utilized to obtain 3-D body model frame-by-frame so that the 3-D motion trajectory of the human object can be derived for any future event analysis, which is independent of the viewing perspective of the 2-D video

A lane-detection method aimed at handling moving vehicles in the traffic scenes is proposed in this brief. First, lane marks are extracted based on color information. The extraction of lane-mark colors is designed in a way that is not affected by illumination changes and the proportion of space that vehicles on the road occupy. Next, for vehicles that have the same colors as the lane marks, we utilize size, shape, and motion information to distinguish them from the real lane marks. The mechanism effectively eliminates the influence of passing vehicles when performing lane detection. Finally, pixels in the extracted lane-mark mask are accumulated to find the boundary lines of the lane. The proposed algorithm is able to robustly find the left and right boundary lines of the lane and is not affected by the passing traffic. Experimental results show that the proposed method works well on marked roads in various lighting conditions

Personal Number (PN) service or Follow-me service allows a user to access telecommunication services with any terminal (e.g., fixed telephones or mobile phones) in any location within the service area. To provide this feature, the PN user needs to manually register with a phone number every time he/she enters a new location. If the user forgets to register the new phone number, the incoming calls will be misrouted. To provide user-friendly follow-me service, this paper proposes an automatic follow-me service (AFS) approach that automatically updates the PN records in the Follow-me database. The significance of our approach is that AFS can be integrated with existing Follow-me databases to automate PIN services offered by different PSTN service providers. We show how AFS can be implemented by using the Voice over IP and Bluetooth technologies. Then, we propose an analytic model to investigate the performance of AFS. The analytic results are validated by simulation experiments. Our study suggests how to select polling frequency to optimize the AFS performance.