Current methodologies for incremental object detection (IOD) primarily rely on Faster R-CNN or DETR series detectors; however, these approaches do not accommodate the real-time YOLO detection frameworks. In this paper, we first identify three primary types of knowledge conflicts that contribute to catastrophic forgetting in YOLO-based incremental detectors: foreground-background confusion, parameter interference, and misaligned knowledge distillation. Subsequently, we introduce YOLO-IOD, a real-time Incremental Object Detection (IOD) framework that is constructed upon the pretrained YOLO-World model, facilitating incremental learning via a stage-wise parameter-efficient finetuning process. Specifically, YOLO-IOD encompasses three principal components: 1) Conflict-Aware Pseudo-Label Refinement (CPR), which mitigates the foreground-background confusion by leveraging the confidence levels of pseudo labels and identifying potential objects relevant to future tasks. 2) Importance-based Kernel Selection (IKS), which identifies and updates the pivotal convolution kernels pertinent to the current task during the current learning stage. 3)Cross-Stage Asymmetric Knowledge Distillation (CAKD), which addresses the misaligned knowledge distillation conflict by transmitting the features of the student target detector through the detection heads of both the previous and current teacher detectors, thereby facilitating asymmetric distillation between existing and newly introduced categories. We further introduce LoCo COCO, a more realistic benchmark that eliminates data leakage across stages. Experiments on both conventional and LoCo COCO benchmarks show that YOLO-IOD achieves superior performance with minimal forgetting.

Spiking Neural Networks (SNNs) transmit information via discrete spike events, offering advantages in energy efficiency and computational cost. However, current SNN-based object detectors suffer from limited feature expression and inefficient fusion due to temporal sparsity and the asynchronous nature of spike features. To address these challenges, we propose SpikeGate-YOLO, a spiking object detection architecture optimized for spike-driven processing. Specifically, we introduce the Reparam-Spike Gating (RSG) block to enhance feature expressiveness while maintaining computational efficiency. We also design the Spike Multi-Granularity Difference-aware Feature Harmonizer (SpikeMDFH), which improves multi-scale feature fusion through dynamic attention and biologically inspired gating mechanisms, preserving spike sparsity. Experiments on both the COCO and Gen1 datasets show that SpikeGate-YOLO achieves state-of-the-art results, reaching 63.4% mAP@50 and 46.3% mAP@50:95 on COCO, and 69.3% mAP@50 and 42.9% mAP@50:95 on Gen1. These results confirm the effectiveness of our architecture in overcoming spike-specific limitations in feature representation and fusion for object detection.

The affinity graph is regarded as a mathematical representation of the local manifold structure. The performance of locality-preserving projections (LPPs) and its variants is tied to the quality of the affinity graph. However, there are two drawbacks in current approaches. First, the pre-designed graph is inconsistent with the actual distribution of data. Second, the linear projection way would cause damage to the nonlinear manifold structure. In this article, we propose a nonlinear dimensionality reduction model, named deep locality-preserving projections (DLPPs), to solve these problems simultaneously. The model consists of two loss functions, each employing deep autoencoders (AEs) to extract discriminative features. In the first loss function, the affinity relationships among samples in the intermediate layer are determined adaptively according to the distances between samples. Since the features of samples are obtained by nonlinear mapping, the manifold structure can be kept in the low-dimensional space. Additionally, the learned affinity graph is able to avoid the influence of noisy and redundant features. In the second loss function, the affinity relationships among samples in the last layer (also called the reconstruction layer) are learned. This strategy enables denoised samples to have a good manifold structure. By integrating these two functions, our proposed model minimizes the mismatch of the manifold structure between samples in the denoising space and the low-dimensional space, while reducing sensitivity to the initial weights of the graph. Extensive experiments on toy and benchmark datasets have been conducted to verify the effectiveness of our proposed model.

The ability to support multitasking becomes more and more important in the development of graphic processing unit (GPU). GPU multitasking methods are classified into three types: temporal multitasking, spatial multitasking, and simultaneous multitasking (SMK). This article first introduces the features of some commercial GPU architectures to support multitasking and the common metrics used for evaluating the performance of GPU multitasking methods, and then reviews the GPU multitasking methods supported by hardware architecture (i.e., hardware GPU multitasking methods). The main problems of each type of hardware GPU multitasking methods to be solved are illustrated. Meanwhile, the key idea of each previous hardware GPU multitasking method is introduced. In addition, the characteristics of hardware GPU multitasking methods belonging to the same type are compared. This article also gives some valuable suggestions for the future research. An enhanced GPU simulator is needed to bridge the gap between academia and industry. In addition, it is promising to expand the research space with machine learning technologies, advanced GPU architectural innovations, 3D stacked memory, etc. Because most previous GPU multitasking methods are based on NVIDIA GPUs, this article focuses on NVIDIA GPU architecture, and uses NVIDIA's terminology. To our knowledge, this article is the first survey about hardware GPU multitasking methods. We believe that our survey can help the readers gain insights into the research field of hardware GPU multitasking methods.

Connected Component labeling (CCL) is usually time-consuming, so a dedicated hardware accelerator of CCL is essential in the embedded vision and multimedia system. In this paper, we propose a single-scan resource-efficient parallel CCL algorithm. Our CCL method scans two adjacent rows simultaneously to extract runs and detect equivalent runs; an equivalent label set is used to resolve equivalences. After each row scan, the finished objects are output in time, and the freed memory resources are reused to reduce memory requirements. Both pixel-based labeled image (PLI) and run-based labeled image (RLI) can be generated by our CCL method. In addition, the steps of our CCL method are executed concurrently to improve labeling performance. The hardware architecture based on our CCL method is implemented with Verilog. The evaluation results illustrate that our CCL architecture can label more than 40 2048 × 1536 benchmark images per second on average, and outperforms previous CCL architectures in terms of labeling performance or memory resource consumption.

In this paper, we introduce a hardware architecture to accelerate connected component labeling (CCL) for embedded systems. The proposed CCL architecture scans the given binary image only once, and during the raster scan, the input binary image is compressed with run-length encoding to extract runs. The equivalences between runs are resolved efficiently by merging equivalent label lists, and all the intermediate data generated while labeling the images are stored in on-chip memory to avoid frequent access to off-chip memory. The finished connected components are determined and then output directly to free onchip memory resources early. These freed memory resources can be reused, which saves memory. Our CCL architecture is implemented with Verilog, and a quantitative comparison of memory cost shows that the proposed CCL architecture is memory-efficient and requires significantly fewer memory resources compared to other methods. In addition, our CCL architecture can process more than 25 2048 × 1536 benchmark images per second when it works at 300 MHz.

Entity matching (EM), also known as entity resolution, fuzzy join, and record linkage, refers to the process of identifying records corresponding to the same real-world entities from different data sources. It is an important and long-standing problem in data integration and data mining. So far progresses have been made mainly in the form of model improvements, where models with better accuracy are developed when large amounts of training data is available. In real-world applications we find that advanced approaches can often require too many labeled examples that is expensive to obtain, which has become a key obstacle to wider adoption.

Following the advances in technology scaling, graphics processing units (GPUs) incorporate an increasing amount of computing resources and it becomes difficult for a single GPU kernel to fully utilize the vast GPU resources. One solution to improve resource utilization is concurrent kernel execution (CKE). Early CKE mainly targets the leftover resources. However, it fails to optimize the resource utilization and does not provide fairness among concurrent kernels. Spatial multitasking assigns a subset of streaming multiprocessors (SMs) to each kernel. Although achieving better fairness, the resource underutilization within an SM is not addressed. Thus, intra-SM sharing has been proposed to issue thread blocks from different kernels to each SM. However, as shown in this study, the overall performance may be undermined in the intra-SM sharing schemes due to the severe interference among kernels. Specifically, as concurrent kernels share the memory subsystem, one kernel, even as computing-intensive, may starve from not being able to issue memory instructions in time. Besides, severe L1 D-cache thrashing and memory pipeline stalls caused by one kernel, especially a memory-intensive one, will impact other kernels, further hurting the overall performance. In this study, we investigate various approaches to overcome the aforementioned problems exposed in intra-SM sharing. We first highlight that cache partitioning techniques proposed for CPUs are not effective for GPUs. Then we propose two approaches to reduce memory pipeline stalls. The first is to balance memory accesses of concurrent kernels. The second is to limit the number of inflight memory instructions issued from individual kernels. Our evaluation shows that the proposed schemes significantly improve the weighted speedup of two state-of-the-art intra-SM sharing schemes, Warped-Slicer and SMK, by 24.6% and 27.2% on average, respectively, with lightweight hardware overhead.

Object recognition is widely used in vision computing for various applications. Traditional CPU and application specific integrated circuit for vision computing cannot provide high performance and enough flexibility, which limit the use of vision systems. In this paper, a hierarchical and parallel pipelined heterogeneous chip for object recognition is proposed to achieve high flexibility, high performance, and area efficiency. In addition, a reformulation of 3D position estimation is proposed. The method uses single precision to achieve the short computing time and accuracy requirement. The hardware resource is small. Application-specific components, such as connected component information extractor and information extraction accelerator, are designed for high performance. Reconfiguration processors and application-specific instruction set processor are introduced to improve flexibility. These components are connected to hierarchical parallel buses. The chip is fabricated in 180-nm CMOS technology and occupies 72.25 mm2with 1.09M bits on-chip memory. It delivers 204 GOPS + 665M FLOPS operations. The results show that this hierarchical and parallel pipelined heterogeneous chip is suitable for embedded vision systems.

In this study, we demonstrate that the performance may be undermined in the state-of-the-art intra-SM sharing schemes for concurrent kernel execution (CKE) on GPUs, due to the interference among concurrent kernels. We highlight that cache partitioning techniques proposed for CPUs are not effective for GPUs. Then we propose to balance memory accesses and limit the number of inflight memory instructions issued from concurrent kernels to reduce memory pipeline stalls. Our proposed schemes significantly improve the performance of two state-of-the-art intra-SM sharing schemes, Warped-Slicer and SMK.

GPUs are widely used to accelerate general purpose applications, and could hide memory latency through massive multithreading. But multithreading can increase contention for the L1 data caches (L1D). This problem is exacerbated when an application contains irregular memory references which would lead to un-coalesced memory accesses. In this paper, we propose a simple yet effective GPU cache Bypassing scheme for Un-Coalesced Loads (BUCL). BUCL makes bypassing decisions at two granularities. At the instruction-level, when the number of memory accesses generated by a non-coalesced load instruction is bigger than a threshold, referred as the threshold of un-coalescing degree (TUCD), all the accesses generated from this load will bypass L1D. The reason is that the cache data filled by un-coalesced loads typically have low probabilities to be reused. At the level of each individual memory access, when the L1D is stalled, the accessed data is likely with low locality, and the utilization of the target memory sub-partition is not high, this memory access may also bypass L1D. Our experiments show that BUCL achieves 36% and 5% performance improvement over the baseline GPU for memory un-coalesced and memory coherent benchmarks, respectively, and also significantly outperforms prior GPU cache bypassing and warp throttling schemes.

LightNet is a lightweight, versatile, purely Matlab-based deep learning framework. The idea underlying its design is to provide an easy-to-understand, easy-to-use and efficient computational platform for deep learning research. The implemented framework supports major deep learning architectures such as Multilayer Perceptron Networks (MLP), Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). The framework also supports both CPU and GPU computation, and the switch between them is straightforward. Different applications in computer vision, natural language processing and robotics are demonstrated as experiments.