ABSTRACT Multi‐focus image fusion (MFIF) aims to synthesise an all‐in‐focus image from multiple source images captured with different focal depths. However, effectively distinguishing focused regions from defocused backgrounds remains challenging due to the smooth transitions and lack of ground‐truth supervision. MFIF synthesises an all‐in‐focus image by selecting sharp regions from images captured at different focal planes. However, many deep methods ignore the gradient discrepancy between focused and defocused areas during multi‐scale extraction, treat features without quantifying their reliability, and build decision maps using only horizontal and vertical gradients, leading to detail loss and structural misclassification. We propose MSFE‐Net, a gradient‐aware multi‐scale feature enhancement network comprising three core components. First, a gradient‑aware multi‑scale feature enhancement module (GradientAwareMFE) balances fine detail and contextual cues by fusing dual atrous spatial pyramid pooling branches with small and large dilation rates under a learned gradient gate aligned to the Sobel gradient magnitude. Second, focus‐reliability attention converts local‐variance statistics into a spatial reliability mask and applies squeeze‐and‐excitation to modulate channels. Third, enhanced spatial frequency fusion integrates four‐directional gradients with local context and morphological refinement to produce a robust decision map for pixel‐wise fusion. Trained in a self‐supervised manner on Microsoft Common Objects in Context (MS‐COCO) using mean squared error, structural similarity, and gradient‐consistency losses, and evaluated on Lytro, MFFW, and MFI‐WHU datasets, MSFE‐Net attains state‐of‐the‐art or competitive results on most metrics, delivering sharper edges and fewer artefacts.

Serverless computing is renowned for its computation elasticity, yet its full potential is often constrained by the requirement for functions to operate within local and dedicated background environments, resulting in limited memory elasticity. To address this limitation, this paper introduces TrEnv, a co-designed integration of the serverless platform with the operating system and CXL/RDMA-based remote memory pools in two key areas. Firstly, TrEnv introduces repurposable sandboxes, which can be shared across different functions and hence, substantially decrease the overhead associated with creating isolation sandboxes. Secondly, it augments the OS with "memory templates" that enable rapid restoration of function states stored on remote memory. These innovations allow TrEnv to facilitate rapid transitions between instances of different functions and enable memory sharing across multiple nodes. Our evaluations using a variety of representative and real-world workloads demonstrate that TrEnv can initiate a container within 10 milliseconds, achieving up to a 7× speedup in P99 end-to-end latency and reducing memory usage by 48% on average compared to state-of-the-art on-demand restoring systems.

In this paper, we propose a long-term motion model for visual object tracking. In crowded street scenes, persistent occlusions are a frequent challenge for tracking algorithm and a robust, long-term motion model could help in these situations. Motivated by progresses in robot motion planning, we propose to construct a set of `plausible' plans for each person, which are composed of multiple long-term motion prediction hypotheses that do not include redundancies, unnecessary loops or collisions with other objects. Constructing plausible plan is the key step in utilizing motion planning in object tracking, which has not been fully investigate in robot motion planning. We propose a novel method of efficiently constructing disjoint plans in different homotopy classes, based on winding numbers and winding angles of planned paths around all obstacles. As the goals can be specified by winding numbers and winding angles, we can avoid redundant plans in the same homotopy class and multiple whirls or loops around a single obstacle. We test our algorithm on a challenging, real-world dataset, and compare our algorithm with Linear Trajectory Avoidance and a simplified linear planning model. We find that our algorithm outperforms both algorithms in most sequences.

This article proposes an active basis model, a shared sketch algorithm, and a computational architecture of sum-max maps for representing, learning, and recognizing deformable templates. In our generative model, a deformable template is in the form of an active basis, which consists of a small number of Gabor wavelet elements at selected locations and orientations. These elements are allowed to slightly perturb their locations and orientations before they are linearly combined to generate the observed image. The active basis model, in particular, the locations and the orientations of the basis elements, can be learned from training images by the shared sketch algorithm. The algorithm selects the elements of the active basis sequentially from a dictionary of Gabor wavelets. When an element is selected at each step, the element is shared by all the training images, and the element is perturbed to encode or sketch a nearby edge segment in each training image. The recognition of the deformable template from an image can be accomplished by a computational architecture that alternates the sum maps and the max maps. The computation of the max maps deforms the active basis to match the image data, and the computation of the sum maps scores the template matching by the log-likelihood of the deformed active basis.

In this paper, we present a novel algorithm based on flow velocity field estimation to count the number of pedestrians across a detection line or inside a specified region. We regard pedestrians across the line as fluid flow, and design a novel model to estimate the flow velocity field. By integrating over time, the dynamic mosaics are constructed to count the number of pixels and edges passed through the line. Consequentially, the number of pedestrians can be estimated by quadratic regression, with the number of weighted pixels and edges as input. The regressors are learned off line from several camera tilt angles, and have taken the calibration information into account.We use tilt-angle-specific learning to ensure direct deployment and avoid overfitting while the commonly used scene-specific learning scheme needs on-site annotation and always trends to overfitting. Experiments on a variety of videos verified that the proposed method can give accurate estimation under different camera setup in real-time.

This article proposes a method for learning object templates composed of local sketches and local textures, and investigates the relative importance of the sketches and textures for different object categories. Local sketches and local textures in the object templates account for shapes and appearances respectively. Both local sketches and local textures are extracted from the maps of Gabor filter responses. The local sketches are captured by the local maxima of Gabor responses, where the local maximum pooling accounts for shape deformations in objects. The local textures are captured by the local averages of Gabor filter responses, where the local average pooling extracts texture information for appearances. The selection of local sketch variables and local texture variables can be accomplished by a projection pursuit type of learning process, where both types of variables can be compared and merged within a common framework. The learning process returns a generative model for image intensities from a relatively small number of training images. The recognition or classification by template matching can then be based on log-likelihood ratio scores. We apply the learning method to a variety of object and texture categories. The results show that both the sketches and textures are useful for classification, and they complement each other.

This paper proposes an algorithm for abandoned object detection based on generative model of low level features. First, suspected blobs are detected by foreground detection and pixel variance thresholding. Then several low level features on blobs are calculated to remove false alarms, which include pixel variance over time, edge intensity score, edge variance over time, foreground completeness, histogram contrast with respect to surrounding background, bag color model and priors of height and width. The last two features are used as threshold. For other features, their log probability ratio between distributions on positive examples and negative ones are tted by sigmoid function. At last, the fitted log probability ratios are weighted to construct a classifier. The algorithm has been verified in 29 challenging scenes and produces very low false alarms and missing detection.

This paper presents a novel integrated background model for video surveillance. Our model uses a primal sketch representation for image appearance and 3D scene geometry to capture the ground plane and major surfaces in the scene. The primal sketch model divides the background image into three types of regions - flat, sketchable and textured. The three types of regions are modeled respectively by mixture of Gaussians, image primitives and LBP histograms. We calibrate the camera and recover important planes such as ground, horizontal surfaces, walls, stairs in the 3D scene, and use geometric information to predict the sizes and locations of foreground blobs to further reduce false alarms. Compared with the state-of-the-art background modeling methods, our approach is more effective, especially for indoor scenes where shadows, highlights and reflections of moving objects and camera exposure adjusting usually cause problems. Experiment results demonstrate that our approach improves the performance of background/foreground separation at pixel level, and the integrated video surveillance system at the object and trajectory level.

In this paper, based on active appearance model (AAM), we present an easy-to-use framework for facial image composition, which can automatically exchange the source image's face or facial features onto the target image. The manual interaction is simple and the user only needs to input semantic information of ROI (region of interest) to be exchanged, such as 'face' or 'eyes'. Our framework mainly consists of two steps: model fitting and component compositing. Model fitting is designed to interpret each input image and obtain a synthesized model face of the image. Then by using component compositing, visual pleasing result is generated by solving Poisson equation with the boundary condition, produced automatically from model fitting. Furthermore, we propose a solution for eliminating the artifacts when part of the target face is occluded by hair, glasses, etc. The visually satisfactory results demonstrate the effectiveness of our facial image composition system.

An interactive scene annotation tool for video surveillance is presented in this paper. The annotation process is divided into three stages. (1) camera rough calibration;(2) calibration refinement; (3) major surfaces annotation. Inputs are then rendered in a 3D environment, which again help users check calibration accuracy and annotation correctness. Experiments show that this tool is easy to use and attains acceptable annotation accuracy. The interactive procedure helps users without knowledge in computer vision to complete camera calibration as well as surface annotation.

This paper proposes a hybrid model for deformable template which combines alignable and non-alignable sketches. These sketches are subject to slight or considerable translations in different images. For slight translations, Wu et al proposed active basis model to capture them, where each sketch is allowed to shift in position and orientation. For larger translations of sketches, assumed that they follow the same distribution as sketches of natural image ensembles, which need not be explicitly modeled. But in fact, for a specified object class, the unaligned sketches follow a totally different distribution from those of natural images. We summarize these sketches by their means in the foreground mask. We treat the mean value in each direction as independent features and fit their marginal distributions on object ensemble and natural image ensemble using Gaussian distribution. The marginal distributions are combined with Active Basis into a joint probability ratio to distinguish foreground object from natural background. Experiments are conducted on 14 object classes, most of which show considerable improvement in ROC.

In this paper, a novel background model on spatio-temporal patches is introduced for video surveillance, especially for night outdoor scene, where extreme lighting conditions often cause troubles. The spatio-temporal patch, called brick, is presented to simultaneously capture spatio-temporal information in surveillance video. The set of bricks of a given background patch, under all possible lighting conditions, lies in a low-dimensional subspace, which can be learned by online subspace learning. The proposed method can efficiently model the background and detect the appearance and motion variance caused by foreground. Experimental results on real data show that the proposed method is insensitive to dramatic lighting changes and achieves superior performance to two classical methods.

Most of spectral embedding algorithms such as Isomap, LLE and Laplacian Eigenmap only give map on training samples. One main problem of these methods is to find the embedding of new samples, which is known as the outof-sample problem of spectral embedding. In this paper, we propose a neural network based method to solve this problem. Neural network is used to train and perform both the forward map from high dimensional image space to low dimensional embedding space, and the backward map in the reverse direction. Additionally, combining the forward and backward network, this method is able to build auto-association model to retrieve high dimensional data, and cross association model to learn high dimensional correspondences. Experiments are conducted on real images for forward and backward map, auto-association and cross association.

This paper presents a novel scheme for manifold learning. Different from the previous work reducing data to Euclidean space which cannot handle the looped manifold well, we map the scattered data to its intrinsic parameter manifold by semisupervised learning. Given a set of partially labeled points, the map to a specified parameter manifold is computed by an iterative neighborhood average method called anchor points diffusion procedure (APD). We explore this idea on the most frequently used close formed manifolds, Stiefel manifolds whose special cases include hyper sphere and orthogonal group. The experiments show that APD can recover the underlying intrinsic parameters of points on scattered data manifold successfully.

Scale space is a natural way to handle multi-scale problem. Yang and Ma considered the correspondence between scales, and proposed optical flow in the scale space. In this paper, we generalized Yang and Ma's work to generic images. We first generalize the Horn-Schunck algorithm to multidimensional multichannel image sequence. Since the global smoothness constraint for regularization is no longer suitable in general cases, we introduce localized smoothness weight regularization. At last, we apply the proposed method in color image scale-space pullback, together with another localized smoothness trick considering flow density.