This study introduces a framework to the camera calibration problem in broadcast soccer videos, focusing on accurately localizing field line endpoints. These endpoints are crucial for homography estimation. Our proposed framework, a unique two-stage process, involves semantic segmentation-based endpoint prediction and homography computation for estimating camera parameters. We adopt the transformer-based Seg-former-B3 model, fine-tuned on high-resolution field images, and train it using the PyTorch-Lightning framework. To overcome limitations in raw segmentation outputs, we design a dedicated post-processing pipeline that integrates Hough Transform to enforce line-level geometric consistency and morphological operations to eliminate noise and restore spatial continuity of field markings. This hybrid approach significantly improves endpoint precision and results in more reliable camera calibration.

In recent years, deep learning-based image compression has achieved significant success. Most schemes adopt an end-to-end trained compression network with a specifically designed entropy model. Inspired by recent advances in conditional video coding, in this work, we propose a novel transformer-based conditional coding paradigm for learned image compression. Our approach first compresses a low-resolution version of the target image and up-scales the decoded image using an off-the-shelf super-resolution model. The super-resolved image then serves as the condition to compress and decompress the target high-resolution image. Experiments demonstrate the superior rate-distortion performance of our approach compared to existing methods.

This paper illustrates a group photographing guidance system that enables to detect the eyes if closed and the face if occluded by another subject in group photo. To consider the practical issue, the proposed auxiliary system is designed to remind the photographer when any face of subject is occluded, or eyes closed. We perform eyes-closed detection on every detected face region. If the eyes of subject were closed during the shot, system will notice the photographer automatically. In this study, the Multi-task Cascaded Convolutional Networks (MTCNN) is applied for face detection. MTCNN is efficient for the group photo with a large amount subject. Compared with media-pipe contributed by Google, the number of human and bounding box detected by MTCNN is more accurate and complete. For eyes detection, a simple convolution neural network is used. In the experiments, we demonstrated that the results well-performed in terms of the occluded ratio of every covered face as well as whose eyes are closed.

This paper we proposed a vision-based square puzzle reconstruction method using the color compatibility feature. Using similarity equation via Hausdorff-based border compatibility, the best piece of a puzzle can be obtained to form the initial combination by calculating the similarity of adjacent pieces and completing the puzzles after three combinations. Based on the experimental results tested on 25 images where classified into 35-piece and 70-piece puzzles, the average success rate reached 87.1% and 17 of the 35-piece puzzles were solved with 100% accuracy. For the class of 70 pieces, 77.8% was obtained on average, and 13 of them were solved with 100% accuracy.

In this study, we proposed a new operator known as the synchronized rotation local ternary pattern (SRLTP) for texture classification. The proposed SRLTP descriptor improves on the local ternary pattern (LTP) method with an additional process on the generated lower and upper LTPs. The lower and upper patterns are encoded to a rotation invariant pattern histogram and a uniform pattern histogram, respectively. Thus, the feature vector can utilize the advantages offered by the rotation invariant pattern histogram while retaining the original information in the uniform pattern histogram. Moreover, in this study, a two-dimensional discrete wavelet transform (DWT) and a discrete Fourier transform (DFT) enhanced the robustness of the texture classification. The experimental results demonstrate that the performance of the SRLTP descriptor is better than those of the existing descriptors.

We present a method for reconstruction of fragmented images. All nonoverlapping and randomly placed pieces can be determined and repositioned correctly by using a robotic arm. The proposed method has potential applications in numerous fields including art restoration, industrial automation, archeology, and robot vision.

Sports data analysis is becoming increasingly large scale, diversified, and shared, but difficulty persists in rapidly accessing the most crucial information. Previous surveys have focused on the methodologies of sports video analysis from the spatiotemporal viewpoint instead of a content-based viewpoint, and few of these studies have considered semantics. This paper develops a deeper interpretation of content-aware sports video analysis by examining the insight offered by research into the structure of content under different scenarios. On the basis of this insight, we provide an overview of the themes particularly relevant to the research on content-aware systems for broadcast sports. Specifically, we focus on the video content analysis techniques applied in sportscasts over the past decade from the perspectives of fundamentals and general review, a content hierarchical model, and trends and challenges. Content-aware analysis methods are discussed with respect to object-, event-, and context-oriented groups. In each group, the gap between sensation and content excitement must be bridged using proper strategies. In this regard, a content-aware approach is required to determine user demands. Finally, this paper summarizes the future trends and challenges for sports video analysis. We believe that our findings can advance the field of research on content-aware video analysis for broadcast sports.

This paper proposes a novel automatic reference color selection (ARCS) scheme for the adaptive mathematical morphology (MM) method, and is specifically designed for color image segmentation applications. Because of the main advantages of being intuitive and simple, in the past decade, it has contributed to the growing popularity of binary and gray-scale MM processing. However, the MM process typically neglects the details of reference color determination. Applying other ordering methods, which select only black as the reference color for sorting pixels, result in the problem in which the scope of the distance measurement is not optimal. The proposed ARCS scheme is used for determining the ideal reference color for MM and for color image segmentation application. In addition, we use both 1D histogram-based modeling scheme binning from 3D color spaces, such as red-green-blue and hue-saturation-intensity, and 2D color models, such as (H, S), (Cb, Cr), and (I, By). According to the results of the quartile analysis, the threshold determination reacts with less sensitivity to the context variations of the images tested. The experiments focused on color-based image segmentation using the proposed ARCS scheme for color MM processing through a bottom-up scenario. To evaluate the system, four quantitative indices were utilized for an ARCS comparison using advanced segmentation methods in the experiments. The cross validation with different system parameters and a comparison of the morphological gradient operation with different color models are also presented.

Nowadays, electronic toll collection (ETC) is used extensively in many countries and places, but many drivers evade detection by covering, altering or otherwise obscuring their license plates. In order to detect vehicles attempting to evade paying tolls, this study aims to identify vehicles without requiring the license plate information. Unlike traditional vehicle license plate recognition, in this study vehicles are matched according to the vehicles' appearance information. The proposed system builds a complete vehicle model using the training database. In the experiments, we demonstrate the robustness of the proposed system for the highway ETC application.

This paper describes a robust template matching algorithm undergoing rotation-scaling-translation (RST) variations via our proposed spiral projection model (SPM). Compared to other model-based methods, the proposed algorithm provides the structural and statistical information about the template in a more general and easier to comprehend format. The well-defined spiral-based projection provides high sensitivity and high scanning efficiency compared with ring projection [1]. It also preserves the inherent spatial information better than the radial projection [2]. In addition, we introduced a novel image warping scheme called the spiral aggregation map (SAM). It not only provides an efficient method for generating the desired projection profiles for matching, it also enables us to determine the rotation angle, and is invariant to scale changes. The experimental evaluation shows that the properties of the algorithm achieved very attractive results.

This paper presents a novel key-frame detection method that combines the visual saliency-based attention features with the contextual game status information for sports videos. First, it describes the approach of extracting the object oriented visual attention map and illustrates the algorithm for determining the contextual excitement curve. Semantic contextual inference is used to simulate how the video content attracts the subscribers. Second, it presents the fusion methodology of visual and contextual attention analysis based on the characteristics of human excitement. Finally, the experimental results demonstrate the efficiency and the robustness of our system by means of some baseball game videos.

The attention analysis of multimedia data is challenging since different models have to be constructed according to different attention characteristics. This paper analyzes how people are excited about the watched video content and proposes a content-driven attention ranking strategy which enables client users to iteratively browse the video according to their preference. The proposed attention rank (AR) algorithm, which is extended from the Google PageRank algorithm that sorts the websites based on the importance, can effectively measure the user interest (UI) level for each video frame. The degree of attention is derived by integrating the object-based visual attention model (VAM) with the contextual attention model (CAM), which not only can more reliably take advantage of the human perceptual characteristics, but also can effectively identify which video content may attract users' attention. The information of users' feedback is utilized in re-ranking procedure to further improve the retrieving accuracy. The proposed algorithm is specifically evaluated on broadcasted baseball videos.

This paper proposes a semantics understanding system to interpret the superimposed caption box (SCB) in sports videos of which the template of SCB is presumably not a priori. The embedded captions in sports video programs represent digested key information of the video content. Most of the previous studies assume that the SCB template and the character bitmaps are known. Nevertheless, the representative character bitmaps are required for recognizing the captions. This paper has the following novelty: (1) SCB extraction and identification, (2) symbol extraction, and (3) semantic interpretation of the identified captions and symbols. Experimental results show that the algorithm performs the SCB contents understanding of several commercial sports video programs.

In this paper, we propose new video attention modeling and content-driven mining strategies which enable client users to browse the video according to their preference. By integrating the object-based visual attention model (V'AM) with the contextual attention model (CAM), the proposed scheme not only can more reliably take advantage of the human perceptual characteristics but also effectively discriminate which video contents may attract users' attention. In addition, extended from the Google PageRank algorithm which sorts the websites based on the importance, we introduce the so-call content-based attention rank (AR) to effectively measure the user interest (UI) level of each video frame. The information of users' feedback is treated as the enhanced query data to further improve the retrieving accuracy. The proposed algorithm is evaluated on commercial baseball game sequences and produces promising results.

This paper presents a novel semantic-oriented video analysis system for the basketball game videos. Based on Bayesian belief network (BBN), it may bridge this gap between the low-level features describing image/video structure and the high-level knowledge. We apply the support vector machine (SVM) to identify and track the ball, the shooter, and the basket as the low-level features. Based on these features, our BBN framework can identify four categories of shot event such as short shot, medium shot, long shot, free throw, and the scoring event. In the experiments, we demonstrate that our system may interpret the video shots in terms of four different shot events and one scoring event effectively

This paper proposes a semantics understanding system to interpret the superimposed caption box (SCB) in sports videos of which the template of SCB is presumably not a priori. The semantics in SCB represent the digested key information of the sports video content. Most of the previous studies assume that the SCB template is known as a priori, and require the representative character bitmaps. The SCB understanding system consists of SCB extraction, character extraction, digit/text character recognition, digital object interpretation using labeling mechanism. In the experiments, we will illustrate that the system can interpret the SCB in basketball and baseball game videos successfully

Video semantic analysis is formulated based on the low-level image features and the high-level knowledge which is encoded in abstract, nongeometric representations. This paper introduces a semantic analysis system based on Bayesian network (BN) and dynamic Bayesian network (DBN). It is validated in the particular domain of soccer game videos. Based on BN/DBN, it can identify the special events in soccer games such as goal event, corner kick event, penalty kick event, and card event. The video analyzer extracts the low-level evidences, whereas the semantic analyzer uses BN/DBN to interpret the high-level semantics. Different from previous shot-based semantic analysis approaches, the proposed semantic analysis is frame-based for each input frame, it provides the current semantics of the event nodes as well as the hidden nodes. Another contribution is that the BN and DBN are automatically generated by the training process instead of determined by ad hoc. The last contribution is that we introduce a so-called temporal intervening network to improve the accuracy of the semantics output.

This paper proposes a novel semantics-based content analysis system for reliable media highlight extraction using dynamic Bayesian network (DBN). It extracts the low-level evidences and then converts the input video to high-level semantic meaning. Specific domains contain rich spatial and temporal transitional structures that help the transformation process. We introduce a robust audio-visual low-level evidence extraction scheme, and develop the so-called temporal intervening network to improve the performance of our system. In experiments, we show that our system can detect soccer events such as goal event, corner kick event, penalty kick event, and card event effectively.

This paper presents a real time vision-based human motion capturing and recognition system using two calibrated CCD cameras. We propose a simple but effective method to estimate the motion parameters (BAPs) of the human object by analyzing the vertical projection profile and the horizontal projection profile in each view to identify different arm and leg postures. With the identified postures, we can apply the Kalman filtering to capture the motion parameters (joint angles). Our method is divided into macro motion analysis and micro motion analysis. The former identifies certain well-defined postures and the latter traces the variation of joint angle or BAPs. In the experiments, we test 22 different arm and leg postures and show the errors of the estimated BAPs.

The paper introduces a highlight detection method that consists of: (1) an inference scheme for semantic feature extraction; (2) a semantic highlight detection scheme using a multi-level semantic network (MSN). MSN is basically a Bayesian belief network (BBN) which consists of the low-level feature nodes connected to the mid-level semantic nodes and finally the high-level semantic nodes. A probabilistic structure can be combined by visual features and can be applied on highlight detection. Satisfactory results are shown.

The exploitation of semantic information in videos is difficult because of the large difference in representations, levels of knowledge and abstract episodes. Traditional image/video understanding and indexing is formulated in terms of low-level features describing image/video structure and intensity, while high-level knowledge such as common sense and human perceptual knowledge are encoded. This paper attempts to bridge this gap through the integration of image/video analysis algorithms with multi-level semantic network to interpret the baseball video.