The ubiquitous presence of surveillance cameras severely compromises the security of private information (e.g. passwords) entered via a conventional keyboard interface in public places. We address this problem by proposing dual modulated QR (DMQR) codes, a novel QR code extension via which users can securely communicate private information in public places using their smartphones and a camera interface. Dual modulated QR codes use the same synchronization patterns and module geometry as conventional monochrome QR codes. Within each module, primary data is embedded using intensity modulation compatible with conventional QR code decoding. Specifically, depending on the bit to be embedded, a module is either left white or an elliptical black dot is placed within it. Additionally, for each module containing an elliptical dot, secondary data is embedded by orientation modulation; that is, by using different orientations for the elliptical dots. Because the orientation of the elliptical dots can only be reliably assessed when the barcodes are captured from a close distance, the secondary data provides "proximal privacy" and can be effectively used to communicate private information securely in public settings. Tests conducted using several alternative parameter settings demonstrate that the proposed DMQR codes are effective in meeting their objective- the secondary data can be accurately decoded for short capture distances (6 in.) but cannot be recovered from images captured over long distances (>12 in.). Furthermore, the proximal privacy can be adapted to application needs by varying the eccentricity of the elliptical dots used.

In combination with current sociological trends, the maturing development of IoT devices is projected to revolutionize healthcare. A network of body-worn sensors, each with a unique ID, can collect health data that is orders-of-magnitude richer than what is available today from sporadic observations in clinical/hospital environments. When databased, analyzed, and compared against information from other individuals using data analytics, HIoT data enables the personalization and modernization of care with radical improvements in outcomes and reductions in cost. In this paper, we survey existing and emerging technologies that can enable this vision for the future of healthcare, particularly in the clinical practice of healthcare. Three main technology areas underlie the development of this field: (a) sensing, where there is an increased drive for miniaturization and power efficiency; (b) communications, where the enabling factors are ubiquitous connectivity, standardized protocols, and the wide availability of cloud infrastructure, and (c) data analytics and inference, where the availability of large amounts of data and computational resources is revolutionizing algorithms for individualizing inference and actions in health management. Throughout the paper, we use a case study to concretely illustrate the impact of these trends. We conclude our paper with a discussion of the emerging directions, open issues, and challenges.

We introduce a dataset for facilitating audio-visual analysis of music performances. The dataset comprises 44 simple multi-instrument classical music pieces assembled from coordinated but separately recorded performances of individual tracks. For each piece, we provide the musical score in MIDI format, the audio recordings of the individual tracks, the audio and video recording of the assembled mixture, and ground-truth annotation files including frame-level and note-level transcriptions. We describe our methodology for the creation of the dataset, particularly highlighting our approaches to address the challenges involved in maintaining synchronization and expressiveness. We demonstrate the high quality of synchronization achieved with our proposed approach by comparing the dataset with existing widely used music audio datasets. We anticipate that the dataset will be useful for the development and evaluation of existing music information retrieval (MIR) tasks, as well as for novel multimodal tasks. We benchmark two existing MIR tasks (multipitch analysis and score-informed source separation) on the dataset and compare them with other existing music audio datasets. In addition, we consider two novel multimodal MIR tasks (visually informed multipitch analysis and polyphonic vibrato analysis) enabled by the dataset and provide evaluation measurements and baseline systems for future comparisons (from our recent work). Finally, we propose several emerging research directions that the dataset enables.

Multi-pitch analysis of polyphonic music requires estimating concurrent pitches (estimation) and organizing them into temporal streams according to their sound sources (streaming). This is challenging for approaches based on audio alone due to the polyphonic nature of the audio signals. Video of the performance, when available, can be useful to alleviate some of the difficulties. In this paper, we propose to detect the play/non-play (P/NP) activities from musical performance videos using optical flow analysis to help with audio-based multi-pitch analysis. Specifically, the detected P/NP activity provides a more accurate estimate of the instantaneous polyphony (i.e., the number of pitches at a time instant), and also helps with assigning pitch estimates to only active sound sources. As the first attempt towards audio-visual multi-pitch analysis of multi-instrument musical performances, we demonstrate the concept on 11 string ensembles. Experiments show a high overall P/NP detection accuracy of 85.3%, and a statistically significant improvement on both the multi-pitch estimation and streaming accuracy, under paired t-tests at a significance level of 0:01 in most cases.

Both audio and visual aspects of a musical performance, especially their association, are important for expressing players' ideas and for engaging the audience. In this paper, we present a framework for combining audio and video analyses of multi-instrument chamber music performances to associate players in the video to the individual separated instrument sources from the audio, in a score-informed fashion. The instrument sources are first separated using a score-informed source separation techniques. The individual sources are then associated with different players in the video by correlating the onset instants of their aligned score tracks with the players' motion detected using optical flow. Experiments on 19 musical pieces with varying polyphony show that the proposed method obtains the correct association for 17 pieces, and an accuracy of 89.2% of the association of all individual tracks. The approach enables novel music enjoyment experiences by allowing users to target an audio source by clicking on the player in the video to separate/enhance it.

As the cost, size and power required by sensor devices decrease, an increasing range of applications are possible. We focus on the application of tracking accelerometer data from body worn sensors over long durations in health monitoring applications. Body worn sensors must be compact for the convenience of the patient and low power to support extended operation, but these benefits can come at the cost of accuracy. To mitigate this loss of accuracy we first examine the ability of calibration to improve accuracy. Next we propose an in-situ method for calibrating the accelerometers using the quiescent acceleration due to gravity as a calibration signal. This in-situ calibration uses only the stored data from the duration the sensor is worn by the patient and does not require any extra procedures or measurements from the physical sensor devices. Compared to a manual three-axis calibration technique, the proposed calibration can be applied to data without the need for specific calibration procedures or even access to the original sensor and provides comparable or better accuracy.

We present a per-channel framework for extending monochrome barcodes to color for display applications, offering increased data rates and capacity. Data is independently encoded into barcodes, incorporated within a color image as the red (R), green (G), and blue (B) channels and decoded from the corresponding R, G, and B channels in the image of the displayed barcode captured with a smartphone. Using a physical model for the display and capture processes, we show that the cross-channel interference in this situation is appropriately modeled by a linear mixing relation. Via experiments done over multiple smartphones and displays, we demonstrate that the impact of the interference is small and can be further mitigated through the use of interference cancellation. Two methods are presented for estimating and canceling the interference: either using pilot blocks in coherent designs where the synchronization marker patterns align between the codes or using expectation maximization. Results are presented for both QR and Aztec color codes in this framework, highlighting its versatility. Overall bit error rates are shown to be below 1.23% and are further reduced by interference cancellation. These low error rates are well within the correction capacity of the error correction codes used for typical color barcodes.