Abstract EMG (electromyography) signals provide critical information on muscle activity and have long been used in prosthetics and health care. With the rise of wearable technologies, EMG is now widely applied in human–computer interaction, rehabilitation and emerging remote control applications such as drone operation and gaming. The primary objective of this study was to accurately identify hand gestures utilizing a limited quantity of user electromyography (EMG) data, thereby improving the applicability and extending the potential use cases of EMG-based systems in real-world scenarios. The proposed method makes use of deep learning methods, namely convolutional neural networks and transfer learning to learn the similarity between samples of EMG signals to predict hand gestures. A pretrained network, fine-tuned in multiple passes by progressive freezing of layers, is applied to extract subject-specific features. Support vector machines is used to learn the similarity of the subject’s features to features in the training set to facilitate the voting for the best hand gesture by means of k-nearest neighbors method. The findings indicate viability of the proposed method even when utilizing a limited dataset collected from the user. Average accuracy obtained by the proposed method on 17 test subjects for 7 hand gestures exceeds the performances of the three approaches proposed in two related works as well as an approach based on recurrent neural network. The proposed method was also evaluated on the NinaPro DB5 Exercise B dataset, where it was compared against four existing methods. Experimental results demonstrate that the proposed approach outperforms all compared methods in terms of classification accuracy.

After a natural disaster, it is very important to be able to make a building damage assessment quickly and accurately. Considering real-world scenarios, one of the most important issues related to the deep learning based computer vision models used in this field is their generalization ability. Ideally, we would like to have a global model which is capable of making damage assessment successfully in all disasters in the future. However, this is a difficult task as images of one disaster varies greatly from images of another due to reasons like different geographical region structure, structure of houses, position of clouds, etc. On the other hand, many damage assessment models are evaluated on an unrealistic in-domain (ID) test set, which contains data from the same disasters in the training and test sets. In this study, we focus on the generalization problem and compare the model performance in an ID test setting with that in an out-of-domain (OOD) test setting, which is more realistic since the test data is formed from disasters not in the training set. We show that the performance of the model is comparatively lower for the OOD test setting and there is a large generalization gap in this case. We conclude that the models should be evaluated by using the OOD test setting. Then, we propose to generate post-disaster images and masks by using generative adversarial network (GAN) models and use the generated data to fine-tune the damage assessment model. It is seen that the generalization ability of the damage assessment model is increased by using this approach.

In this work, we improve the state-of-the-art in crowd counting by further developing a recently proposed multi-scale, and multi-task crowd counting approach. While most of the studies treat density-based architectures, this study proposed a point-based method for crowd analysis. We propose automatic, and optimal weight assignment to constituents of the loss function. This approach, which is applied to each patch, ensures that the weight parameters are updated in each epoch, and added to the optimizer with model parameters rather than remaining constant. For validation of our proposed approach, we use three popular crowd counting datasets, ShanghaiTech A, ShanghaiTech B, and UCF_CC_50. The performance of our approach exceeds the performances of the other studies on the ShanghaiTech dataset, and is highly competitive with the performances of the other studies on the UCF CC 50 dataset.

Determination of the phenological stages of plants is important for the growth of healthy and productive plants. The knowledge of transition times of phenological stages of a plant can provide valuable data for planning, organizing and timely execution of agricultural activities (spraying, irrigation etc.). TARBIL is an agricultural monitoring and information system that is founded and supported by Republic of Turkey Ministry of Food, Agriculture and Livestock. This system has a network of stations located in many parts of Turkey. Stations, that contain many sensors and cameras, periodically collect images and meteorological data from the agricultural fields. Previous works focus on either only about plant identification or only phenological stage recognition using only one texture analysis method. Our approachment to the problem is novel because not only the recognition of the plant type or the recognition of only the phenological stage, but also joint identification of the plant type and the phenological stages are provided with several texture and color feature analysis methods. In this work, a study is conducted to compare the use of several image texture features along with color features extracted from TARBIL field image data for the classification of the plants and their phenological stages. Experimental results show that HOG (Histograms of Oriented Gradients) yields the best performance among the texture features tested.

In this paper, we propose a hybrid 2D-to-3D video conversion system to recover the 3D structure of the scene. Depending on the scene characteristics, geometric or height depth information is adopted to form the initial depth map. This depth map is fused with color-based depth cues to construct the nal depth map of the scene background. The depths of the foreground objects are estimated after their classi cation into human and non-human regions. Speci cally, the depth of a non-human foreground object is directly calculated from the depth of the region behind it in the background. To acquire more accurate depth for the regions containing a human, the estimation of the distance between face landmarks is also taken into account. Finally, the computed depth information of the foreground regions is superimposed on the background depth map to generate the complete depth map of the scene which is the main goal in the process of converting 2D video to 3D.

Frame rate up-conversion (FRUC) increases the quality of a video by increasing its temporal frequency. Motion compensated and non-motion compensated frame rate up conversion techniques make up the two main classes of techniques used in this area. Halo artifacts and jaggy edges cause the quality of video to be reduced both subjectively and objectively in these techniques. In this paper, we introduce a new method of motion compensated FRUC that uses foreground background segmentation to address these problems. It checks up the neighbors of gap pixels and deciding which neighbors should be used to interpolate or extrapolate according to their foreground or background identity. The current work reduces block artifacts by scaling motion vectors of existent frames to motion vectors of interpolated frames by using the foreground/background segmentation information. The proposed method improves the quality of video and obtains better PSNR results than the proposed methods.

Frame rate up-conversion (FRUC) increases the quality of a video by increasing its temporal frequency. Motion compensated and non-motion compensated frame rate up conversion techniques make up the two main classes of techniques used in this area. Halo artifacts and jaggy edges cause the quality of video to be reduced both subjectively and objectively in these techniques. In this paper, we introduce a new method of motion compensated FRUC that uses foreground background segmentation to address these problems. In typical motion compensated FRUC, motion vectors of existent frames are scaled to define the motion vectors of interpolated frames. In the proposed method, those pixels of an interpolated frame with no unidirectionally assigned motion vectors are assigned motion vectors by interpolation based on known motion vectors of their neighbors and their foreground or background identity. The proposed method significantly improves the video quality and yields better PSNR results than the methods of [1] and [2].

Since different regions of a color image generally exhibit different spectral characteristics, the energy compaction of applying a single spectral transform to all regions is largely inefficient from a compression perspective. Thus, it is proposed that different subsets of wavelet coefficients of a color image be subjected to different spectral transforms before the resultant coefficients are coded by an efficient wavelet coefficient coding scheme such as that used in JPEG2000 or color set partitioning in hierarchical trees (CSPIHT). A quadtree represents the spatial partitioning of the set of high frequency coefficients of the color planes into spatially oriented subsets which may be further partitioned into smaller directionally oriented subsets. The partitioning decisions and decisions to employ fixed or signal-dependent bases for each subset are rate-distortion (R-D) optimized by employing a known analytical R-D model for these coefficient coding schemes. A compression system of asymmetric complexity, that integrates the proposed adaptive spectral transform with the CSPIHT coefficient coding scheme yields average coding gains of 0.3 dB and 0.9 dB in the Y component at 1.0 b/p and 2.5 b/p, respectively, and 0.9 dB and 1.35 dB in the U and V components at 1.0 b/p and 2.5 b/p, respectively, over a reference compression system that integrates the single spectral transform derived from the entire image with the CSPIHT coefficient coding scheme.

This paper explains the development of a highly efficient progressive 3-D mesh geometry coder based on the region adaptive transform in the spectral mesh compression method. A hierarchical set partitioning technique, originally proposed for the efficient compression of wavelet transform coefficients in high-performance wavelet-based image coding methods, is proposed for the efficient compression of the coefficients of this transform. Experiments confirm that the proposed coder employing such a region adaptive transform has a high compression performance rarely achieved by other state of the art 3-D mesh geometry compression algorithms. A new, high-performance fixed spectral basis method is also proposed for reducing the computational complexity of the transform. Many-to-one mappings are employed to relate the coded irregular mesh region to a regular mesh whose basis is used. To prevent loss of compression performance due to the low-pass nature of such mappings, transitions are made from transform-based coding to spatial coding on a per region basis at high coding rates. Experimental results show the performance advantage of the newly proposed fixed spectral basis method over the original fixed spectral basis method in the literature that employs one-to-one mappings.

In general, each region of a color image exhibits different spectral characteristics. Therefore, the energy compaction characteristic of a single global spectral transformation is rather weak for compression purposes. In this paper, we propose that different groups of wavelet coefficients of a color image be subjected to different spectral transformations prior to the spectral planes being coded by a wavelet-based image coder such as CSPIHT (Color Set Partitioning in Hierarchical Trees, [Kassim, Lee; 2003]). The decomposition of the color image into such groups is succintly represented with a quadtree structure which is optimized for rate-distortion performance by means of a known analytical rate-distortion model for wavelet-based image codecs. The experiments show that, when integrated with the CSPIHT coder, the proposed region adaptive transformation method yields compression gains of 0.4-0.5 dB, on the average, for rates between 0.5bpp and 2.5bpp over the single global transformation method.

This paper proposes several low complexity algorithmic modifications to the SPIHT (set partitioning in hierarchical trees) image coding method of Said and Pearlman (1996). The modifications exploit universal traits common to the real world images. Approximately 1-2% compression gain (bit rate reduction for a given mean squared error) has been obtained for the images in our test suite by incorporating all of the proposed modifications into SPIHT.

A new rate control method for low delay transmission of H.263 coded video is presented. The method is based on nonlinear MMSE (minimum mean squared error) estimation of the number of coding bits of a macroblock. A class is determined for each macroblock from its coding mode and quantized simple statistics of its data. Each combination of macroblock class and quantization parameter is mapped to a nonlinear estimate which is designed either online or online from empirical data. The quantization parameters of macroblocks of a frame are determined to attain near uniform spatial reconstruction quality as the total estimate of the number of coding bits closely approximates the targeted number of bits for the frame satisfying the delay and encoder buffer size constraints. Simulations demonstrate that the accuracy of the new macroblock layer rate control method is significantly better than the method of TMN5 and very competitive with the method of TMN8.

Constrained storage vector quantization, (CSVQ), introduced by Chan and Gersho (1990, 1991) allows for the stagewise design of balanced tree-structured residual vector quantization codebooks with low encoding and storage complexities. On the other hand, it has been established by Makhoul et al. (1985), Riskin et al. (1991), and by Mahesh et al. (see IEEE Trans. Inform. Theory, vol.41, p.917-30, 1995) that variable-length tree-structured vector quantizer (VLTSVQ) yields better coding performance than a balanced tree-structured vector quantizer and may even outperform a full-search vector quantizer due to the nonuniform distribution of rate among the subsets of its input space. The variable-length constrained storage tree-structured vector quantization (VLCS-TSVQ) algorithm presented in this paper utilizes the codebook sharing by multiple vector sources concept as in CSVQ to greedily grow an unbalanced tree structured residual vector quantizer with constrained storage. It is demonstrated by simulations on test sets from various synthetic one dimensional (1-D) sources and real-world images that the performance of VLCS-TSVQ, whose codebook storage complexity varies linearly with rate, can come very close to the performance of greedy growth VLTSVQ of Riskin et al. and Mahesh et al. The dramatically reduced size of the overall codebook allows the transmission of the code vector probabilities as side information for source adaptive entropy coding.

The optimal joint decoder utilizing NLIVQ (nonlinear interpolative vector quantization) introduced by Gersho [1990] results in vector quantizers which have reduced encoding complexity at the expense of coding performance loss due to the inferiority of their space-filling property. We show a method of improving a high resolution NLIVQ codebook by partitioning its cells in such a way that the resulting lower resolution codebook consists of cells with better space-filling properties. The resolution reduction method is also extended to the case where the quantizer indices are entropy-constrained. From the simulations it is seen that the unconstrained and constrained entropy versions of the proposed vector quantizer have comparable performance to vector quantizers designed by LEG and ECVQ algorithms.

The paper presents two different approaches to image sequence coding which exploit the spatial frequency statistics as well as the spatial and temporal correlation present in the video signal. The first approach is the pyramidal decomposition of the Motion Compensated Frame Difference (MCFD) signal in the frequency domain and the subsequent coding by unbalanced Tree Structured Vector Quantizers (TSVQ) designed to match the statistics of the frequency bands. The type of TSVQ used in this study possess the advantage of low computational complexity with coding performance comparable to full-search vector quantization. The second approach is similar except that the order of motion estimation/compensation and pyramidal decomposition are interchanged.