This paper proposes a deep learning based solution for multi-modal image alignment regarding UAV-taken images. Many recently proposed state-of-the-art alignment techniques rely on using Lucas-Kanade (LK) based solutions for a successful alignment. However, we show that we can achieve state of the art results without using LK-based methods. Our approach carefully utilizes a two-branch based convolutional neural network (CNN) based on feature embedding blocks. We propose two variants of our approach, where in the first variant (ModelA), we directly predict the new coordinates of only the four corners of the image to be aligned; and in the second one (ModelB), we predict the homography matrix directly. Applying alignment on the image corners forces algorithm to match only those four corners as opposed to computing and matching many (key)points, since the latter may cause many outliers, yielding less accurate alignment. We test our proposed approach on four aerial datasets and obtain state of the art results, when compared to the existing recent deep LK-based architectures.

Recent advances in camera equipped drone applications and their widespread use increased the demand on vision based object detection algorithms for aerial images. Object detection process is inherently a challenging task as a generic computer vision problem, however, since the use of object detection algorithms on UAVs (or on drones) is relatively a new area, it remains as a more challenging problem to detect objects in aerial images. There are several reasons for that including: (i) the lack of large drone datasets including large object variance, (ii) the large orientation and scale variance in drone images when compared to the ground images, and (iii) the difference in texture and shape features between the ground and the aerial images. Deep learning based object detection algorithms can be classified under two main categories: (a) single-stage detectors and (b) multi-stage detectors. Both single-stage and multi-stage solutions have their advantages and disadvantages over each other. However, a technique to combine the good sides of each of those solutions could yield even a stronger solution than each of those solutions individually. In this paper, we propose an ensemble network, SyNet, that combines a multi-stage method with a single-stage one with the motivation of decreasing the high false negative rate of multi-stage detectors and increasing the quality of the single-stage detector proposals. As building blocks, CenterNet and Cascade R-CNN with pretrained feature extractors are utilized along with an ensembling strategy. We report the state of the art results obtained by our proposed solution on two different datasets: namely MS-COCO and visDrone with %52.1 mAPIoU=0.75is obtained on MS-COCO val2017 dataset and %26.2 mAPIoU=0.75is obtained on VisDrone test - set. Our code is available at: https://github.com/mertalbaba/SyNet.

There is an increasing demand on utilizing camera equipped drones and their applications in many domains varying from agriculture to entertainment and from sports events to surveillance. In such drone applications, an essential and a common task is tracking an object of interest visually. Drone (or UAV) images have different properties when compared to the ground taken (natural) images and those differences introduce additional complexities to the existing object trackers to be directly applied on drone applications. Some important differences among those complexities include (i) smaller object sizes to be tracked and (ii) different orientations and viewing angles yielding different texture and features to be observed. Therefore, new algorithms trained on drone images are needed for the drone-based applications. In this paper, we introduce a deep reinforcement learning (RL) based single object tracker that tracks an object of interest in drone images by estimating a series of actions to find the location of the object in the next frame. This is the first work introducing a single object tracker using a deep RL-based technique for drone images. Our proposed solution introduces a novel reward function that aims to reduce the total number of actions taken to estimate the object's location in the next frame and also introduces a different backbone network to be used on low resolution images. Additionally, we introduce a set of new actions into the action library to better deal with the above-mentioned complexities. We compare our proposed solutions to a state of the art tracking algorithm from the recent literature and demonstrate up to 3.87 % improvement in precision and 3.6% improvement in IoU values on the VisDrone2019 data set. We also provide additional results on OTB-100 data set and show up to 3.15% improvement in precision on the OTB-100 data set when compared to the same previous state of the art algorithm. Lastly, we analyze the ability to handle some of the challenges faced during tracking, including but not limited to occlusion, deformation, and scale variation for our proposed solutions.

A fundamental challenge in autonomous vehicles is adjusting the steering angle at different road conditions. Recent state-of-the-art solutions addressing this challenge include deep learning techniques as they provide end-to-end solution to predict steering angles directly from the raw input images with higher accuracy. Most of these works ignore the temporal dependencies between the image frames. In this paper, we tackle the problem of utilizing multiple sets of images shared between two autonomous vehicles to improve the accuracy of controlling the steering angle by considering the temporal dependencies between the image frames. This problem has not been studied in the literature widely. We present and study a new deep architecture to predict the steering angle automatically by using Long-Short-Term-Memory (LSTM) in our deep architecture. Our deep architecture is an end-to-end network that utilizes CNN, LSTM and fully connected (FC) layers and it uses both present and futures images (shared by a vehicle ahead via Vehicle-to-Vehicle (V2V) communication) as input to control the steering angle. Our model demonstrates the lowest error when compared to the other existing approaches in the literature.

We present an approach to extend the functionality and the use of kernel machines in image processing applications. We introduce a novel way to design spatial kernel machines with spatial properties and demonstrate how those newly introduced spatial properties enhance the possibilities of the use of kernel machines in image processing applications as a proof of concept. In this paper, we demonstrate four particular extensions: 1) how to model shapes efficiently with spatially computed kernel parameters in a geometrically scalable way; 2) how to visualize the kernel parameters precisely and intuitively on binary 2D shapes; 3) how to construct a one-class classifier from the binary classifier in a straightforward manner without re-training; and 4) how to use the computed kernel parameters for filtering. The existing literature on kernel machines mostly focuses on estimating the optimal kernel parameters via additional cost function(s). In this paper, instead of employing an additional cost function to estimate the kernel-related parameters, we investigate on an analytical solution to predict the actual kernel parameters locally and show how to build a spatial kernel machine with our analytical approach. Classical kernel machines do not perform well on precise shape modeling with a low number of support vectors as demonstrated in this paper. However, we demonstrate and visualize that our analytical approach provides a natural means to relate the kernel parameters to the 2D shapes for sparse shape modeling, where the shape boundary represents the decision boundary. For that, we incorporate the selected kernel function's geometric properties as an additional constraint into the classifier's optimization problem by defining an easy-to-explain and intuitive concept: similarity domains. In our experiments, we study and demonstrate how the resulting new kernel machine enhances the capabilities of the classical kernel machines with applications on shape modeling, (geometrically) scaling the non-linear decision boundary at various scales and precise visualization of the kernel parameters in 2D images.

We provide a deterministic data summarization algorithm that approximates the mean $\bar{p}=\frac{1}{n}\sum_{p\in P} p$ of a set $P$ of $n$ vectors in $\mathbb{R}^d$, by a weighted mean $\tilde{p}$ of a subset of $O(1/\epsilon)$ vectors, i.e., independent of both $n$ and $d$. We prove that the squared Euclidean distance between $\bar{p}$ and $\tilde{p}$ is at most $\epsilon$ multiplied by the variance of $P$. We use this algorithm to maintain an approximated sum of vectors from an unbounded stream, using memory that is independent of $d$, and logarithmic in the $n$ vectors seen so far. Our main application is to extract and represent in a compact way friend groups and activity summaries of users from underlying data exchanges. For example, in the case of mobile networks, we can use GPS traces to identify meetings; in the case of social networks, we can use information exchange to identify friend groups. Our algorithm provably identifies the Heavy Hitter entries in a proximity (adjacency) matrix. The Heavy Hitters can be used to extract and represent in a compact way friend groups and activity summaries of users from underlying data exchanges. We evaluate the algorithm on several large data sets.

For large-scale simulations, the data sets are so massive that it is sometimes not feasible to view the data with basic visualization methods, let alone explore all time steps in detail. Automated tools are necessary for knowledge discovery, i.e., to help sift through the data and isolate specific time steps that can then be further explored. Scientists study patterns and interactions and want to know when and where interesting things happen. Activity detection, the detection of specific interactions of objects which span a limited duration of time, has been an active research area in the computer vision community. In this paper, we introduce activity detection to scientific simulations and show how it can be utilized in scientific visualization. We show how activity detection allows a scientist to model an activity and can then validate their hypothesis on the underlying processes. Three case studies are presented.

In this paper, a method to generalize previously proposed Chebyshev Kernel function is presented for Support Vector Classification in order to obtain more robust and higher classification accuracy. By introducing the generalized Chebyshev polynomials for vector inputs, we increase the performance of this kernel function. The simulation results show that the proposed generalized Chebyshev Kernel has better performance than the previously proposed kernel for Support Vector Classification. Early simulation results show that the proposed kernel function yields the best classification results for a Breast Cancer dataset.