The advent of edge computing has made real-time intelligent video analytics feasible. Previous works, based on traditional model architecture (e.g., CNN, RNN, etc.), employ various strategies to filter out non-region-of-interest content to minimize bandwidth and computation consumption but show inferior performance in adverse environments. Recently, visual foundation models based on transformers have shown great performance in adverse environments due to their amazing generalization capability. However, they require a large amount of computation power, which limits their applications in realtime intelligent video analytics. In this paper, we find visual foundation models like Vision Transformer (ViT) also have a dedicated acceleration mechanism for video analytics. To this end, we introduce Arena, an end-to-end edge-assisted video inference acceleration system based on ViT. We leverage the capability of ViT that can be accelerated through token pruning by only offloading and feeding Patches-of-Interest to the downstream models. Additionally, we design an adaptive keyframe inference switching algorithm tailored to different videos, capable of adapting to the current video content to jointly optimize accuracy and bandwidth. Through extensive experiments, our findings reveal that Arena can boost inference speeds by up to 1.58×, 1.82× and 1.98× on average while consuming only 47%, 31% and 27% of the bandwidth, respectively, all with high inference accuracy.

With the advancement of computer vision, the recently emerged 3D Gaussian Splatting (3DGS) has increasingly become a popular scene analytics algorithm due to its outstanding performance. Existing cloud-based 3DGS architectures overlook the challenges in real-world environments when handling large-scale scene analysis. This exposes issues such as inefficiency, low security, lack of privacy, and limited scalability. In this paper, we propose Radiant, a hierarchical framework for large scene analytics in a heterogeneous cloud-edge-device system, which jointly considers high efficiency, privacy and security, and scalability. Via extensive empirical study, we find that it is crucial to partition the regions for each edge appropriately and allocate varying camera positions to each device for image collection and training. The core of Radiant is partitioning regions based on heterogeneous environment information and allocating workloads to each device accordingly. Furthermore, we provide a 3DGS model aggregation algorithm that enhances the quality and ensures the continuity of models' boundaries. Finally, we develop a testbed, and experiments demonstrate that Radiant improved reconstruction quality by up to 25.7% and reduced up to 79.6% end-to-end latency.

Object detection in aerial images faces domain adaptive challenges, such as changes in shooting height, viewing angle, and weather. These changes constitute a large number of fine-grained domains that place greater demands on network’s generalizability. To tackle these challenges, we propose a submodule named Fine-grained Feature Disentanglement which decomposes image features into domain-invariant and domain-specific using practical imaging condition parameters. The composite feature can improve the domain generalization and single domain accuracy compared to the conventional fine-grained domain detection method. The proposed algorithm is compared with state-of-the-art fine-grained domain detectors on the UAVDT and VisDrone datasets. The results show that it achieves an average detection precision improvement of 5.7 and 2.4, respectively.

Object detection in aerial images is an important task for many applications such as reconnaissance, surveillance, search, and rescue. At present, convolution neural network-based aerial image object detection algorithms mainly focus on rotation invariance and scale invariance, but ignore an important characteristic of the aerial image that the image captured height is an important prior knowledge. At the same captured height, the target has a clear scale range. In this letter, a scale-aware network is proposed to determine the scale of predefined anchors, which can effectively reduce the scale search range, reduce the risk of overfitting, and improve the detection accuracy and speed in aerial images. Experiments on the VisDrone data set show that the proposed method can not only improve the detection speed by 18% but also improve the average accuracy by 1.6%.

Ellipse detection is an important task in machine vision and can be used to solve other high-level visual problems. In the past, many studies have been conducted by researchers at high precision ellipse fitting, but most of the proposed solutions are not fast enough to be applied in real-time detection for larger images or with limited hardware resources. This paper improves the accuracy of ellipse detection by improving the candidate ellipse scoring method and geometric constraints. Combined with ellipse fitting algorithm, the proposed method can perform real-time ellipse and semi ellipse detection. The efficiency of the algorithm is proven by using three popular datasets which shows the state of the art ellipse and semiellipse detection performance.

Depth estimation from single image is an important component of many vision systems, including robot navigation, motion capture and video surveillance. In this paper, we propose to apply a structure forest framework to infer depth information from single RGB image. The core idea of our approach is to exploit the structure properties exhibit in local patches of depth map to learn the depth level for each pixel. We formulate the problem of depth estimation in a structured learning framework based on random decision forests. Each trained forest infers a patch of structured labels that are accumulated across the image to obtain the final depth map. Moreover, we systematically investigate a variety of depth-relevant features and the regression forest framework automatically determines the best feature combination and uses as input of structure forest. Our approach achieves quasi real-time performance that is orders of magnitude faster than state-of-the-art approaches, while also achieving state-of-the-art depth estimation results on the Make3D dataset.