Weakly supervised tumor localization in whole-slide images (WSIs) remains challenging due to the absence of region level annotations and the difficulty of capturing diagnostically relevant cellular organization under gigapixel resolution. Most existing multiple instance learning (MIL) methods rely primarily on patch-level appearance features, overlooking the nuclear structural patterns that underlie pathological interpretation. We propose Nuclear Graph-Guided Multiple Instance Learning (NGG-MIL), a weakly supervised framework that explicitly incorporates nuclear-level structural information into WSI analysis. Within each patch, nuclei are represented as a Direction Weighted Nuclear Graph, where edges encode both spatial proximity and nuclear orientation consistency. Graph Neural Network (GNN) is employed to learn structure-aware patch representations, which are subsequently aggregated through Attention Based MIL for slide-level prediction. The learned patch attention scores are spatially reprojected to generate interpretable tumor probability maps without requiring region-level supervision. Extensive experiments on three public breast cancer WSI datasets (CAMELYON16, TCGA-BRCA, and BRACS) demonstrate that NGG-MIL consistently surpasses strong MIL baselines in both slide-level classification performance and interpretability of localization maps, achieving consistent AUC improvements and competitive F1 scores across datasets.

Spectral videos (SVs) capture spatio-temporal-spectral information from dynamic scenes, but their acquisition traditionally requires expensive and complex systems, motivating the development of compressive spectral video sensing (CSVS). It typically employs the coded aperture snapshot spectral imager (CASSI) to acquire compressed measurements, from which SVs are reconstructed via model-driven or learning-based algorithms. However, two major limitations remain in current CASSI-based reconstruction methods: 1) conventional model-driven algorithms rely on iterative optimization, which limits their representational capacity in complex scenes and results in slow reconstruction; 2) existing deep learning-based approaches overlook the joint modeling of spatial, temporal, and spectral correlations, failing to fully exploit the multi-dimensional dependencies. Hence, we propose a principled compressive spectral video sensing unfolding framework (CSVSUF) in a CASSI system for spectral video reconstruction. Moreover, we develop a novel spatio-temporal-spectral prior-learning Transformer (STS-PLT) to capture the multi-dimensional correlations within each unfolding stage. By treating STS-PLT as a Gaussian denoiser for the prior term in CSVSUF, we establish a deep unfolding-based method for CSVS. Extensive experiments demonstrate that our method consistently outperforms existing approaches in both reconstruction accuracy and visual quality, validating the benefit of combining physics-guided modeling with deep prior learning in CSVS. Code is available at https://github.com/zli1024/CSVSUF.

Knowing expected milk yield can help dairy farmers in better decision-making and management.The objective of this study was to build and compare predictive models to forecast daily milk yield over a long duration.A machine-learning pipeline was provided and five baseline models as well as a novel stacking model were developed for the prediction of milk yield on the CowNflow dataset using 414 Holstein cattle records collected from 1983 to 2019.Four different feature selection methods were performed to evaluate the essential biological characteristics and feeding-related features which affect milk yield.The results showed that the overall performance of predictive models improved after proper feature selection, with an R 2 value increased to 0.811, and a root mean squared error (RMSE) decreased to 3.627.The stacking model achieved the best performance with an R 2 value of 0.85, a mean absolute error (MAE) of 2.537 and an RMSE of 3.236.This research provides benchmark information for the prediction of milk yield on the CowNflow dataset and identifies useful factors such as dry matter (DM) intake and lactation month in long-term milk yield prediction.

Video understanding has been enhanced by the use of multi-modal networks. However, recent multi-modal video analysis models have limited applicability to sports videos due to their specialised nature. This paper proposes a novel attention-based multi-modal neural network for sports event classification featuring a multi-stage fusion training strategy. The proposed multi-modal neural network integrates three modalities, including an image sequence modality, an audio modality and a newly proposed sports formation modality, to improve the sports video classification performance. Empirical results show that the proposed model outperforms the state-of-the-art transformer-based video method by 4.43% on top-1 accuracy on Soccernet-V2 dataset.

In this paper we introduce a novel multi-object tracker based on the tracking-by-detection paradigm. This tracker utilises a Dynamic Bayesian Network for predicting objects’ positions through filtering and updating in real-time. The algorithm is trained and then tested using the MOTChallenge1 challenge benchmark of video sequences. After initial testing, a state-of-the-art residual neural network for extracting feature descriptors is used. This ResNet feature extractor is integrated into the tracking algorithm for object similarity estimation to further enhance tracker performance. Finally, we demonstrate the effects of object detection on tracker performance using a custom trained state of the art You Only Look Once (YOLO) V5 object detector. Results are analysed and evaluated using the MOTChallenge Evaluation Kit, followed by a comparison to state-of-the-art methods.

Human action recognition (HAR) from RGB videos is essential and challenging in the computer vision field due to its wide range of real-world applications in fields of human behaviour analysis, human-computer interactions, robotics and surveillance etc. Since the breakthrough and fast development of deep learning technology, the performance of HAR based on deep neural networks has been significantly improved in this decade. In this survey, we discuss the growing use of deep learning for HAR, such as representative two-stream and 3D CNNs, and particularly highlight most recent success achieved by using attention and transformers. We will provide our perspective on the new trend of designing innovative deep learning methods. In addition, we also present popular HAR datasets developed in recent years and benchmark accuracy achieved by current advancement in deep learning. This draws research attention to the challenges of HAR by identifying performance gaps when applying the deep learning methods on large HAR datasets. Further, this survey sheds light on the development of new methods and facilitates qualitative comparison with state of the art.

In 3-D attitude angle estimation, monocular vision-based methods are often utilized for the advantages of short-time and high efficiency. However, the limitations of these methods lie in the complexity of the algorithm and the specificity of the scene, which needs to match the characteristics of the cooperation object and the scene. In this article, we propose a fully connected attitude detection network (FADN), which combines neural network and traditional algorithms for 3-D attitude angle estimation. FADN provides a whole process from the input of a single frame image in the industrial video stream to the output of the corresponding 3-D attitude angle estimation. Benefiting from the end-to-end estimation framework, FADN avoids tedious matching algorithms and thus has certain portability. A series of comparative experiments based on the rendering software 3-D Studio Max (3d Max) have been carried out to evaluate the performance of FADN. The experimental results show that FADN has high estimation accuracy and fast running speed. At the same time, the simulation results reliably prove the feasibility of FADN, and also promote the research in real scenarios.

We propose an efficient and robust segmentation network for automated seagrass region detection. The proposed network has a simple architecture to save computational demands as well as inference energy cost. More importantly, the scale of network can be feasibly adjusted, to balance the network computational demands and segmentation accuracy. Experimental results show that our proposed network is robust to segment the various seagrass patterns with 90.66% mIoU (mean Intersection over Union) accuracy. It had achieved 200 frames per second (FPS, 1.42 times faster than the second-best network GCN) on desktop GPU, and 18 FPS on NVIDIA Jetson TX2. It also has 3.45M parameters and 0.587 GMACs FLOPs (FLoating Point OPerations), only 14.6% and 10.8% of those in GCN respectively. To segment a single image on the Jetson TX2, our architecture requires an average energy of 0.26 Joule. This energy cost is only 46% of DeepLab, which shows the proposed network to be an energy efficient one. The proposed network demonstrates accurate and real-time segmentation capability, and it can be deployed to low-energy embedded AUVs for sea habitat protection.

An image is compressed or stretched during the multidevice displaying, which will have a very big impact on perception quality. In order to solve this problem, a variety of image retargeting methods have been proposed for the retargeting process. However, how to evaluate the results of different image retargeting is a very critical issue. In various application systems, the subjective evaluation method cannot be applied on a large scale. So we put this problem in the accurate objective-quality evaluation. Currently, most of the image retargeting quality assessment algorithms use simple regression methods as the last step to obtain the evaluation result, which are not corresponding with the perception simulation in the human vision system (HVS). In this paper, a deep quality evaluator for image retargeting based on the segmented stacked AutoEnCoder (SAE) is proposed. Through the help of regularization, the designed deep learning framework can solve the overfitting problem. The main contributions in this framework are to simulate the perception of retargeted images in HVS. Especially, it trains two separated SAE models based on geometrical shape and content matching. Then, the weighting schemes can be used to combine the obtained scores from two models. Experimental results in three well-known databases show that our method can achieve better performance than traditional methods in evaluating different image retargeting results.

Material appearance of rendered objects depends on the underlying BRDF implementation used by rendering software packages. A lack of standards to exchange material parameters and data (between tools) means that artists in digital 3D prototyping and design, manually match the appearance of materials to a reference image. Since their effect on rendered output is often non-uniform and counter intuitive, selecting appropriate parameterisations for BRDF models is far from straightforward. We present a novel BRDF remapping technique, that automatically computes a mapping (BRDF Difference Probe) to match the appearance of a source material model to a target one. Through quantitative analysis, four user studies and psychometric scaling experiments, we validate our remapping framework and demonstrate that it yields a visually faithful remapping among analytical BRDFs. Most notably, our results show that even when the characteristics of the models are substantially different, such as in the case of a phenomenological model and a physically-based one, our remapped renderings are indistinguishable from the original source model.

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

In the field of big data applications, image information is widely used. The value density of information utilization in big data is very low, and how to extract useful information quickly is very important. So we should transform the unstructured image data source into a form that can be analyzed. In this paper, we proposed a fast image retrieval method which designed for big data. First of all, the feature extraction method is necessary and the feature vectors can be obtained for every image. Then, it is the most important step for us to encode the image feature vectors and make them into database, which can optimize the feature structure. Finally, the corresponding similarity matching is used to determined the retrieval results. There are three main contributions for image retrieval in this paper. New feature extraction method, reasonable elements ranking, and appropriate distance metric can improve the algorithm performance. Experiments show that our method has a great improvement in the effective performance of feature extraction and can also get better search matching results.

The majority of current facial age estimation methods are based on appearance based features. However, wrinkle based research has not been widely addressed. In this paper, we propose a novel method based on multi-scale aging patterns (MAP). These directly extract the features from local patches without extensive geometric modelling. First, we locate facial landmarks by using the Face++ detector and then normalize the face by using a linear transformation. We define a face template which consists of ten predefined wrinkle regions. Then, for each region, we detect wrinkles and construct aging patterns by using the MAP. Finally, the age is estimated by implementing the sequential minimal optimization (SMO). The performance of the algorithms is assessed by using mean absolute error (MAE) on the benchmark database - FERET. We observe that MAP produces a lower MAE of 4.87 on FERET compared to the benchmark algorithms. Therefore, we conclude that wrinkle could be used as a feature on face age estimation. Future work would involve improvements of the algorithm by combining other descriptors such as non-wrinkle descriptor and appearance parameters.

Scanning laser ophthalmoscopes (SLOs) can be used for early detection of retinal diseases. With the advent of latest screening technology, the advantage of using SLO is its wide field of view, which can image a large part of the retina for better diagnosis of the retinal diseases. On the other hand, during the imaging process, artefacts such as eyelashes and eyelids are also imaged along with the retinal area. This brings a big challenge on how to exclude these artefacts. In this paper, we propose a novel approach to automatically extract out true retinal area from an SLO image based on image processing and machine learning approaches. To reduce the complexity of image processing tasks and provide a convenient primitive image pattern, we have grouped pixels into different regions based on the regional size and compactness, called superpixels. The framework then calculates image based features reflecting textural and structural information and classifies between retinal area and artefacts. The experimental evaluation results have shown good performance with an overall accuracy of 92%.

The recognition of human activity is a challenging topic for machine learning. We present an analysis of Support Vector Machines (SVM) and Random Forests (RF) in their ability to accurately classify Kinect kinematic activities. Twenty participants were captured using the Microsoft Kinect performing ten physical rehabilitation activities. We extracted the kinematic location, velocity and energy of the skeletal joints at each frame of the activity to form a feature vector. Principle Component Analysis (PCA) was applied as a pre-processing step to reduce dimensionality and identify significant features amongst activity classes. SVM and RF are then trained on the PCA feature space to assess classification performance, we undertook an incremental increase in the dataset size. We analyse the classification accuracy, model training and classification time quantitatively at each incremental increase. The experimental results demonstrate that RF outperformed SVM in classification rate for six out of the ten activities. Although SVM has performance advantages in training time, RF would be more suited to real-time activity classification due to its low classification time and high classification accuracy when using eight to ten participants in the training set.

For model-based 3D human pose estimation, even simple models of the human body lead to high-dimensional state spaces. Where the class of activity is known a priori, low-dimensional activity models learned from training data make possible a thorough and efficient search for the best pose. Conversely, searching for solutions in the full state space places no restriction on the class of motion to be recovered, but is both difficult and expensive. This paper explores a potential middle ground between these approaches, using the hierarchical Gaussian process latent variable model to learn activity at different hierarchical scales within the human skeleton. We show that by training on full-body activity data then descending through the hierarchy in stages and exploring subtrees independently of one another, novel poses may be recovered. Experimental results on motion capture data and monocular video sequences demonstrate the utility of the approach, and comparisons are drawn with existing low-dimensional activity models. © 2009. The copyright of this document resides with its authors.

We present a method for tracking human activity using observations from a moving narrow-baseline stereo camera. Range data are computed from the disparity between stereo image pairs. We propose a novel technique for calculating weighting scores from range data given body configuration hypotheses. We use a modified Annealed Particle Filter to recover the optimal tracking candidate from a low dimensional latent space computed from motion capture data and constrained by an activity model. We evaluate the method on synthetic data and on a walking sequence recorded using a moving hand-held stereo camera. 1

Tracking human pose using observations from less than three cameras is a challenging task due to ambiguity in the available image evidence. This work presents a method for tracking using a pre-trained model of activity to guide sampling within an Annealed Particle Filtering framework. The approach is an example of model-based analysis-by-synthesis and is capable of robust tracking from less than 3 cameras with reduced numbers of samples. We test the scheme on a common dataset containing ground truth motion capture data and compare against quantitative results for standard Annealed Particle Filtering. We find lower absolute and relative error scores for both monocular and 2-camera sequences using 80% fewer particles.

This paper presents an approach to human motion tracking using multiple pre-trained activity models for propagation of particles in Annealed Particle Filtering. Hidden Markov models are trained on dimensionally reduced joint angle data to produce models of activity. Particles are divided between models for propagation by HMM synthesis, before converging on a solution during the annealing process. The approach facilitates multi-view tracking of unknown subjects performing multiple known activities with low particle numbers.

Range images of complex geometry presented by large point data sets almost always yield surface reconstruction imperfections. We propose a novel compact and complete mesh representation for non-uniformly sampled noisy range image data using an adaptive radial basis function network. The network is established using a heuristic learning strategy. Neurons can be inserted, removed or updated iteratively, adapting to the complexity and distribution of the underlying data. This flexibility is particularly suited to highly variable spatial frequencies, and is conducive to data compression with network representations. Experiments confirm the performance advantages of the network when applied to 3D point-cloud surface reconstruction.

We propose a general algorithm for identifying an arbitrary pose of an articulated subject with sparse point features. The algorithm aims to identify a one-to-one correspondence between a model point-set and an observed point-set taken from freeform motion of the articulated subject. We avoid common assumptions such as pose similarity or small motions with respect to the model, and assume no prior knowledge from which to infer an initial or partial correspondence between the two point-sets. The algorithm integrates local segment-based correspondences under a set of affine transformations, and a global hierarchical search strategy. Experimental results, based on synthetic pose and real-world human motion data demonstrate the ability of the algorithm to perform the identification task. Reliability is increasingly compromised with increasing data noise and segmental distortion, but the algorithm can tolerate moderate levels. This work contributes to establishing a crucial self-initializing identification in model-based point-feature tracking for articulated motion.

Feature-based methods in vision analysis often encounter the problem of correspondences between features of two related patterns. The features may be points, lines, curves and surfaces/regions. Point pattern matching (PPM) is a primary and essential approach for establishing a correspondence within two related patterns. Applications exist in wide circumstances. Numerous techniques related to PPM have been studied within a rich and extensive literature, encompassing both theoretical and practical problem domains. In this paper, we provide a review of the PPM techniques and their applications from three aspects: 1) PPM under rigid/affine motion, 2) PPM under non-rigid/elastic motion, and 3) PPM in a dynamic sequence. We also anticipate the trend for the future in adapting existing techniques to novel algorithms for non-rigid PPM.

We describe a general algorithm for identifying an arbitrary pose of an articulated subject with low density feature points. The algorithm aims to establish a one-to-one correspondence between two data point-sets, one representing the model of an observed subject and the other representing the pose taken from freeform motion of the subject. We avoid common assumptions such as pose similarity or small motion with respect to the model, and assume no prior knowledge from which to infer an initial or partial correspondence between the two point-sets. The algorithm integrates local segment-based correspondences under a set of affine transformations, and a global hierarchical search strategy. Experimental results, based on synthetic pose and real-world human motion capture data demonstrate the ability of the algorithm to perform the identification task. Reliability is compromised as noisy data and limited segmental distortion are increased, but the algorithm can tolerate moderate levels. This work therefore contributes to establishing an initial correspondence in point-feature tracking for articulated motion.

Tracking and identifying articulated objects have received growing attention in computer vision in the past decade. In market-based optical motion capture (MoCap) systems, an articulated movement of near-rigid segments is represented via a sequence of moving dots of known 3D coordinates, corresponding to the captured marker positions. We propose a segment-based articulated model-fitting algorithm to address the problem of self-initializing identification and pose estimation utilizing one frame of data in such point-feature tracking systems. It is ultimately crucial for recovering the complete motion sequence. Experimental results, based on synthetic pose and real-world human motion capture data, demonstrate the performance of the algorithm.