Graph-based multi-armed bandit algorithms utilize the relationship between users to select the best item to recommend for maximal reward, which is decided by items’ features and un-known users’ preferences. Therefore, the precise estimation of users’ preferences is fairly important and indispensable for bandit sampling, though it is not the ultimate target. However, existing algorithms generally neglect this crucial point and utilize reward maximization as objective in the first beginning, using inaccurate estimation as input, which deteriorates the performance from a long-term perspective. In this paper, we will propose one hybrid sampling framework for bandit selection, which at first purely focuses on the performance of estimation and then on the performance of reward maximization. Specifically, we propose an ‘unsupervised’ bandit selection objective to minimize expected estimation error, which doesn’t take users’ preferences as input and suppresses an approximate upper-bound of cumulative regret. Then, we design a low-complexity selection algorithm to optimize this formulated problem with simple multiplications between items’ features and users’ graphical relations. Subsequently, for reward maximization, we cascade one graph-based algorithm to find the following bandits on the basis of our proposed warm-starts. Extensive experiments on different graphs indicate that our proposed hybrid framework is substantially better than existing popular methods in terms of recommendation performance.

Wind speed fluctuation generally shortens the lifetime of the insulated gate bipolar transistor (IGBT) module in the doubly fed induction generator (DFIG), which leads to frequent component replacement and cost increase of wind power generation. Conventional power control strategies consider solely the wind turbine output power maximization, and existing junction temperature thermal management strategies focus on IGBT lifetime extension, both of which are not effective under turbulent wind speed conditions and ignore the trade-off between the device reliability and output power for the overall DFIG economic benefit. To bridge the gap between the IGBT reliability and the output power under turbulent wind speed conditions, we propose a novel IGBT thermal management strategy for optimal economic benefit. Specifically, we first smooth the output power through a linear power control (LPC) method with conditions on the wind speed and turbine speed to reduce the IGBT low-frequency junction temperature fluctuation, which extends IGBT lifetime. We then propose a grid encirclement and suppression search (GESS) method that maximizes the economic benefit defined by LPC-smoothed output power and IGBT lifetime. Simulation shows that our proposed LPC-GESS strategy outperforms a competing strategy at economic benefit by a large margin, up to 15.3%. Our proposed strategy can potentially be used in wind power stations for fast and reliable economic benefit-oriented IGBT thermal management.

Unsupervised Domain Adaptive Object Detection (DAOD) could adapt a model trained on a source domain to an unlabeled target domain for object detection. Existing unsupervised DAOD methods usually perform feature alignments from the target to the source. Unidirectional domain transfer would omit information about the target samples and result in suboptimal adaptation when there are large domain shifts. Therefore, we propose a pairwise attentive adversarial network with a Domain Mixup (DomMix) module to mitigate the aforementioned challenges. Specifically, a deep-level mixup is employed to construct an intermediate domain that allows features from both domains to share their differences. Then a pairwise attentive adversarial network is applied with attentive encoding on both image-level and instance-level features at different scales and optimizes domain alignment by adversarial learning. This allows the network to focus on regions with disparate contextual information and learn their similarities between different domains. Extensive experiments are conducted on several benchmark datasets, demonstrating the superiority of our proposed method.

Quality sleep is a basic human need for well-being, yet sleep deprivation has been a long-term global problem. A common type of sleep deprivation is obstrucive sleep apnea, where people repeatedly stop breathing during sleep with subsequent abnormal vital signs, namely, respiration rate and heart rate. While tremendous effort has been made for vital signs monitoring systems during sleep, existing works still lack portability for bulky and intrusive systems and reliability for consumer-level, nonintrusive systems. To bridge the gap between practicability and accuracy and facilitate Internet of Things for smart healthcare, in this article, we propose a vital signs estimation system during sleep via a thermal camera. The system first captures thermal image sequences of a sleeping subject and then processes the facial regions within the thermal images for vital signs signal extraction. Specifically, leveraging on the inherent graph structure among subregions of the facial area, we propose a graph-based, spatial–temporal signal denoising scheme. Experimental results show that the graph-based denoising scheme in our system effectively reduces the noise level introduced by cameras and subjects, and our proposed system outperforms state-of-the-art nonintrusive vital signs monitoring systems. Since the algorithm components in our system have relatively low time complexity and no model training is required, our system can be deployed efficiently at the edge devices in a smart home setting. The extracted vital signs can then be used for sleep abnormality detection and disease screening.

Given a convex and differentiable objective$Q({\mathbf M})$for a real symmetric matrix${\mathbf M}$in the positive definite (PD) cone—used to compute Mahalanobis distances—we propose a fast general metric learning framework that is entirely projection-free. We first assume that${\mathbf M}$resides in a space${\mathcal S}$of generalized graph Laplacian matrices corresponding to balanced signed graphs.${\mathbf M}\in {\mathcal S}$that is also PD is called a graph metric matrix. Unlike low-rank metric matrices common in the literature,${\mathcal S}$includes the important diagonal-only matrices as a special case. The key theorem to circumvent full eigen-decomposition and enable fast metric matrix optimization is Gershgorin disc perfect alignment (GDPA): given${\mathbf M}\in {\mathcal S}$and diagonal matrix${\mathbf S}$, where$S_{ii} = 1/v_i$and${\mathbf v}$is the first eigenvector of${\mathbf M}$, we prove that Gershgorin disc left-ends of similarity transform${\mathbf B}= {\mathbf S}{\mathbf M}{\mathbf S}^{-1}$are perfectly aligned at the smallest eigenvalue$\lambda _{\min }$. Using this theorem, we replace the PD cone constraint in the metric learning problem with tightest possible linear constraints per iteration, so that the alternating optimization of the diagonal / off-diagonal terms in${\mathbf M}$can be solved efficiently as linear programs via the Frank-Wolfe method. We update${\mathbf v}$using Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) with warm start as entries in${\mathbf M}$are optimized successively. Experiments show that our graph metric optimization is significantly faster than cone-projection schemes, and produces competitive binary classification performance.

This paper presents an unobtrusive solution that can automatically identify deep breath when a person is walking past the global depth camera. Existing non-contact breath assessments achieve satisfactory results under restricted conditions when human body stays relatively still. When someone moves forward, the breath signals detected by depth camera are hidden within signals of trunk displacement and deformation, and the signal length is short due to the short stay time, posing great challenges for us to establish models. To over-come these challenges, multiple region of interests (ROIs) based signal extraction and selection method is proposed to automatically obtain the signal informative to breath from depth video. Subsequently, graph signal analysis (GSA) is adopted as a spatial-temporal filter to wipe the components unrelated to breath. Finally, a classifier for identifying deep breath is established based on the selected breath-informative signal. In validation experiments, the proposed approach outperforms the comparative methods with the accuracy, precision, recall and F1 of 75.5%, 76.2%, 75.0% and 75.2%, respectively. This system can be extended to public places to provide timely and ubiquitous help for those who may have or are going through physical or mental trouble.

Conventional video coding schemes employ a hybrid motion prediction / residual transform coding paradigm, which only exploits redundancy in individual pairs of video frames for compression gain. However, rigid geometric structures in 3D space—e.g., a building in a scene’s background—persist across time in a large frame group. Thus if one can extract and encode the geometric structure, then redundancy across the entire frame group can be removed in one shot. In this paper, we extract a best-fitting "semi-regular" geometric structure from a target spatial region in a frame group, which is encoded separately as a unified signal predictor for these frames. By semi-regular, we mean its geometry is simple enough that its shape parameters can be encoded cheaply. This semi-regular structure kernel approximates the 3D shape of an object in the video, on which we project pixels from the frame group to a carefully spaced 2D grid overlaid on the kernel. We encode the projected pixels as an intra-frame using HEVC. The decoded pixels are then back-projected to each frame as the predictor, and the resulting prediction residuals are transform-coded. Experimental results show that employing a semi-regular geometric kernel—a folded 2D plane in our realization—improves coding performance over native HEVC implementation and our previous regular kernel based scheme.

We propose a general projection-free metric learning framework, where the minimization objective ${\min _{{\mathbf{M}} \in \mathcal{S}}}Q({\mathbf{M}})$ is a convex differentiable function of the metric matrix M, and M resides in the set S of generalized graph Laplacian matrices for connected graphs with positive edge weights and node degrees. Unlike low-rank metric matrices common in the literature, S includes the important positivediagonal-only matrices as a special case in the limit. The key idea for fast optimization is to rewrite the positive definite cone constraint in S as signal-adaptive linear constraints via Gershgorin disc alignment, so that the alternating optimization of the diagonal and offdiagonal terms in M can be solved efficiently as linear programs via Frank-Wolfe iterations. We prove that left-ends of the Gershgorin discs can be aligned perfectly using the first eigenvector v of M, which we update iteratively using Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) with warm start as diagonal / off-diagonal terms are optimized. Experiments show that our efficiently computed graph metric matrices outperform metrics learned using competing methods in terms of classification tasks.

To compose a 360° image from a rig with multiple fisheye cameras, a conventional processing pipeline first performs demosaicking on each fisheye camera’s Bayer-patterned grid, then translates demosaicked pixels from the camera grid to a rectified image grid—thus performing two image interpolation steps in sequence. Hence interpolation errors can accumulate, and acquisition noise in the captured pixels can pollute neighbors in two consecutive processing stages. In this paper, we propose a joint processing framework that performs demosaicking and grid-to-grid mapping simultaneously—thus limiting noise pollution to one interpolation. Specifically, we first obtain a reverse mapping function from a regular on-grid location in the rectified image to an irregular off-grid location in the camera’s Bayer-patterned image. For each pair of adjacent pixels in the rectified grid, we estimate its gradient using the pair’s neighboring pixel gradients in three colors in the Bayer-patterned grid. We construct a similarity graph based on the estimated gradients, and interpolate pixels in the rectified grid directly via graph Laplacian regularization (GLR). Experiments show that our joint method outperforms several competing local methods that execute demosaicking and rectification in sequence, by up to 0.52 dB in PSNR and 0.086 in SSIM on the publicly available dataset, and by up to 5.53dB in PSNR and 0.411 in SSIM on the in-house constructed dataset.

Classification of seismic events detected from seismic recordings has been gaining popularity for improved interpretation of subsurface processes, e.g., volcanic systems, earthquake activity, induced seismicity and slope stability, in particular landslides. However, due to the variability of signal representation for different classes in the temporal and spectral space, a large feature space is used to discriminate classes. The consequence is additional complexity on the classifier and overfitting. So far, there has been little attempt to address dimensionality reduction via feature selection. In this paper, we propose an iterative, alternating graph feature and classifier learning method for micro-seismic signals via graph Laplacian regularization and normalized graph Laplacian regularization. Using recorded micro-seismic events from an active landslide, we demonstrate improved classification accuracy with a relatively small feature space compared to the state-of-the-art.

3D point cloud-a new signal representation of volumetric objects-is a discrete collection of triples marking exterior object surface locations in 3D space. Conventional imperfect acquisition processes of 3D point cloud-e.g., stereo-matching from multiple viewpoint images or depth data acquired directly from active light sensors-imply non-negligible noise in the data. In this paper, we extend a previously proposed low-dimensional manifold model for the image patches to surface patches in the point cloud, and seek self-similar patches to denoise them simultaneously using the patch manifold prior. Due to discrete observations of the patches on the manifold, we approximate the manifold dimension computation defined in the continuous domain with a patch-based graph Laplacian regularizer, and propose a new discrete patch distance measure to quantify the similarity between two same-sized surface patches for graph construction that is robust to noise. We show that our graph Laplacian regularizer leads to speedy implementation and has desirable numerical stability properties given its natural graph spectral interpretation. Extensive simulation results show that our proposed denoising scheme outperforms state-of-the-art methods in objective metrics and better preserves visually salient structural features like edges.

Recent advances in image acquisition and analysis have resulted in disruptive innovation in physical rehabilitation systems facilitating cost-effective, portable, video-based gait assessment. While these inexpensive motion capture systems, suitable for home rehabilitation, do not generally provide accurate kinematics measurements on their own, image processing algorithms ensure gait analysis that is accurate enough for rehabilitation programs. This paper proposes high-accuracy classification of gait phases and muscle actions, using readings from low-cost motion capture systems. First, 12 gait parameters, drawn from the medical literature, are defined to characterize gait patterns. These proposed parameters are then used as input to our proposedmulti-channel time-series classificationand gait phase reconstruction methods. Proposed methods fully utilize temporal information of gait parameters, thus improving the final classification accuracy. The validation, conducted using 126 experiments, with 6 healthy volunteers and 9 stroke survivors with manually-labelled gait phases, achieves state-of-art classification accuracy of gait phase with lower computational complexity compared to previous solutions.1

Conventional video coding approaches follow a hybrid motion prediction / residual transform coding paradigm, which limits the discovery of redundancy to individual pairs of video frames. On the other hand, computer vision techniques like structure-from-motion (SfM) have long exploited redundancy across a large group of frames to estimate a rigid 3D object structure. In this paper, leveraging on previous SfM techniques, we construct a rate-distortion (RD) optimized 3D planar model from a target spatial region in a frame group as a unified signal predictor for these frames. The prediction accuracy of the model is optimally traded off with the cost of coding such representation as side information (SI). Specifically, we approximate a roughly flat spatial region in the video as a 2D plane in 3D space, and project pixels from the frame group to a 2D grid on the plane with appropriate density. The boundary of the irregularly shaped pixel body on the plane is first coded using arithmetic edge coding (AEC), and then the body is encapsulated into a tight-fitting rectangular region, which is encoded as an intra-frame using HEVC. The pixels inside the rectangle but outside the pixel body-called don't care region (DCR)-are filled optimally by minimizing an l1-norm of the transform coefficients using linear programming. Experimental results show that the RD-optimized planar model improves coding performance over native HEVC implementation.

Acquired 3D point cloud data, whether from active sensors directly or from stereo-matching algorithms indirectly, typically contain non-negligible noise. To address the point cloud denoising problem, we propose a local graph-based algorithm. Specifically, given a $k$ -nearest-neighbor graph of the 3D points, we first approximate it with a bipartite graph (independent sets of red and blue nodes) using a KL divergence criterion. For each partite of nodes (say red), we first define surface normal of each red node using 3D coordinates of neighboring blue nodes, so that red node normals n can be written as a linear function of red node coordinates p. We then formulate a convex optimization problem, with a quadratic fidelity term $\Vert \mathbf{p}-\mathbf{q}\Vert_{2}^{2}$ given noisy observed red coordinates q and a graph total variation (GTV) regularization term for surface normals of neighboring red nodes. We minimize the resulting $l_{2}-l_{1}$-norm using alternating direction method of multipliers (ADMM) and proximal gradient descent. The two partites of nodes are alternately optimized until convergence. Experimental results show that compared to state-of-the-art schemes with similar complexity, our proposed algorithm achieves the best overall denoising performance objectively and subjectively.

With growing ageing population, acquiring joint measurements with sufficient accuracy for reliable gait assessment is essential. Additionally, the quality of gait analysis relies heavily on accurate feature selection and classification. Sensor-driven and one-camera optical motion capture systems are becoming increasingly popular in the scientific literature due to their portability and cost-efficacy. In this paper, we propose 12 gait parameters to characterise gait patterns and a novel gait-phase classifier, resulting in comparable classification performance with a state-of-the-art multi-sensor optical motion system. Furthermore, a novel multi-channel time series segmentation method is proposed that maximizes the temporal information of gait parameters improving the final classification success rate after gait event reconstruction. The validation, conducted over 126 experiments on 6 healthy volunteers and 9 stroke patients with handlabelled ground truth gait phases, demonstrates high gait classification accuracy.

Localization of a viewer's region of interest (ROI) on eye gaze signal trajectories acquired by eye trackers is a widely used approach in scene analysis, image compression, and quality of experience assessment. In this paper, we propose a novel clustering approach for ROI estimation from potentially noisy raw eye gaze data, based on signal processing on graphs. The clustering approach adapts graph signal processing (GSP)-based classification by first cleverly selecting a starting data sample, and then classifying the remaining samples. Furthermore, Graph Fourier Transform is used to adjust GSP parameters on-the-fly to maximise accuracy. Experimental results show competitive clustering accuracy of our proposed scheme compared to Density-based spatial clustering of applications with noise (DB-SCAN), Distance-Threshold Identification (I-DT), and Mean-Shift on publicly available Shape Dataset and the potential of estimating ROI accurately on true eye tracker data1.

Low-cost depth sensors, such as Microsoft Kinect, have potential for noncontact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suffer from high acquisition noise, and hence, processing them to estimate biometrics is difficult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm; as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depth enhancement/denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules is consistent, improving performance of both restoration and motion tracking. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, bandpass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental results show accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.

Obstructive sleep apnea, characterized by repetitive obstruction in the upper airway during sleep, is a common sleep disorder that could significantly compromise sleep quality and quality of life in general. The obstructive respiratory events can be detected by attended in-laboratory or unattended ambulatory sleep studies. Such studies require many attachments to a patient's body to track respiratory and physiological changes, which can be uncomfortable and compromise the patient's sleep quality. In this paper, we propose to record depth video and audio of a patient using a Microsoft Kinect camera during his/her sleep, and extract relevant features to correlate with obstructive respiratory events scored manually by a scientific officer based on data collected by Philips system Alice6 LDxS that is commonly used in sleep clinics. Specifically, we first propose an alternating-frame H.264 video encoding scheme and bit recovery scheme at the decoder. Next, we perform depth video temporal denoising using a motion vector graph smoothness prior. Then, we build a dual-ellipse model and track a patient's chest and abdominal movements in the denoised videos. Finally, we extract features from both depth video and audio for classifier training and respiratory event detection. Experimental results show 1) that our depth video compression scheme outperforms a competitor that records only the 8 most significant bits, 2) our graph-based temporal denoising scheme reduces the flickering effect without over-smoothing, and 3) our trained classifiers can deduce respiratory events scored manually based on data collected by system Alice6 LDxS with high accuracy.

Depth sensors like Microsoft Kinect can acquire partial geometric information in a 3D scene via captured depth images, with potential application to non-contact health monitoring. However, captured depth videos typically suffer from low bit-depth representation and acquisition noise corruption, and hence using them to deduce health metrics that require tracking subtle 3D structural details is difficult. In this paper, we propose to capture depth video using Kinect 2.0 to estimate the heart rate of a human subject; as blood is pumped to circulate through the head, tiny oscillatory head motion can be detected for periodicity analysis. Specifically, we first perform a joint bit-depth enhancement / denoising procedure to improve the quality of the captured depth images, using a graph-signal smoothness prior for regularization. We then track an automatically detected nose region throughout the depth video to deduce 3D motion vectors. The deduced 3D vectors are then analyzed via principal component analysis to estimate heart rate. Experimental results show improved tracking accuracy using our proposed joint bit-depth enhancement / denoising procedure, and estimated heart rates are close to ground truth.

Optical motion capture systems have been widely adopted for human motion analysis in stroke rehabilitation because of real-time processing and high-accuracy features. However, these systems require a large laboratory space and multiple cameras and thus can be expensive and not transportable. In this paper, we propose a portable, cheap, single-camera motion analysis system to implement upper limb movement analysis. The proposed system consists of video acquisition, camera calibration, marker tracking, autonomous joint angle calculation, visualization, validation and classification. The validation with a state-of-the-art optical motion analysis system using Bland-Altman plot, a typical clinical measure, indicates that the proposed system can accurately capture elbow movement, trunk-tilt, and shoulder movement for diagnosis. Furthermore, the volunteers are explicitly classified into healthy and stroke groups via a support vector machine trained on statistics of the trunk-tilt and shoulder movement. Experimental results show that the proposed system can accurately capture the upper limb movement patterns, automatically classify stroke survivors using ordinal scale classification of upper limb impairment, and offer a convenient and inexpensive solution for upper limb movement analysis.

Quality of sleep greatly affects a person's physiological well-being. Traditional sleep monitoring systems are expensive in cost and intrusive enough that they disturb the natural sleep of clinical patients. In our previous work, we proposed a non-intrusive sleep monitoring system to first record depth video in real-time, then offline analyze recorded depth data to track a patient's chest and abdomen movements over time. Detection of abnormal breathing is then interpreted as episodes of apnoea or hypopnoea. Leveraging on recent advances in graph signal processing (GSP), in this paper we propose two new additions to further improve our sleep monitoring system. First, temporal denoising is performed using a block motion vector smoothness prior expressed in the graph-signal domain, so that unwanted temporal flickering can be removed. Second, a graph-based event classification scheme is proposed, so that detection of apnoea / hypopnoea can be performed accurately and robustly. Experimental results show first that graph-based temporal denoising scheme outperforms an implementation of temporal median filter in terms of flicker removal. Second, we show that our graph-based event classification scheme is noticeably more robust to errors in training data than two conventional implementations of support vector machine (SVM).

Human gait analysis for stroke rehabilitation therapy using video processing tools has become popular in recent years. This paper proposes a single-camera system for capturing gait patterns using a Kalman-Structural-Similarity-based algorithm which tracks multiple markers simultaneously. This algorithm is initialized by obtaining the user-selected blocks in the first frame of each video, and the tracker is implemented by using Structural-Similarity image quality assessment algorithm to detect each marker frame by frame within a search area determined by a discrete Kalman filter. Experimental results show the trajectories of the markers fixed on the joints of a human body. The obtained numerical results are used to generate gait information (e.g., knee joint angle) that is later used for diagnostics. The proposed method aims to explore an alternative and portable way to implement human gait analysis with significantly less cost compared to a state-of-the-art 3D motion capture system.