Malware datasets inevitably contain incorrect labels due to the shortage of expertise and experience needed for sample labeling. Previous research demonstrated that a training dataset with incorrectly labeled samples would result in inaccurate model learning. To address this problem, researchers have proposed various noise learning methods to offset the impact of incorrectly labeled samples, and in image recognition and text mining applications, these methods demonstrated great success. In this work, we apply both representative and state-of-the-art noise learning methods to real-world malware classification tasks. We surprisingly observe that none of the existing methods could minimize incorrect labels’ impact. Through a carefully designed experiment, we discover that the inefficacy mainly results from extreme data imbalance and the high percentage of incorrectly labeled data samples. As such, we further propose a new noise learning method and name it after MORSE. Unlike existing methods, MORSE customizes and extends a state-of-the-art semi-supervised learning technique. It takes possibly incorrectly labeled data as unlabeled data and thus avoids their potential negative impact on model learning. In MORSE, we also integrate a sample re-weighting method that balances the training data usage in the model learning and thus handles the data imbalance challenge. We evaluate MORSE on both our synthesized and real-world datasets. We show that MORSE could significantly outperform existing noise learning methods and minimize the impact of incorrectly labeled data.

There is a large body of research on scalable machine learning (ML). Nevertheless, training ML models on large, continuously evolving datasets is still a difficult and costly undertaking for many companies and institutions. We discuss such challenges and derive requirements for an industrial-scale ML platform. Next, we describe the computational model behind Amazon SageMaker, which is designed to meet such challenges. SageMaker is an ML platform provided as part of Amazon Web Services (AWS), and supports incremental training, resumable and elastic learning as well as automatic hyperparameter optimization. We detail how to adapt several popular ML algorithms to its computational model. Finally, we present an experimental evaluation on large datasets, comparing SageMaker to several scalable, JVM-based implementations of ML algorithms, which we significantly outperform with regard to computation time and cost.

${}^{1}$H High-Resolution Magic Angle Spinning (HRMAS) Nuclear Magnetic Resonance (NMR) is a reliable technology used for detecting metabolites in solid tissues. Fast response time enables guiding surgeons in real time, for detecting tumor cells that are left over in the excision cavity. However, severe overlap of spectral resonances in 1D signal often render distinguishing metabolites impossible. In that case, Heteronuclear Single Quantum Coherence Spectroscopy (HSQC) NMR is applied which can distinguish metabolites by generating 2D spectra (${}^{1}$H-${}^{13}$C). Unfortunately, this analysis requires much longer time and prohibits real time analysis. Thus, obtaining 2D spectrum fast has major implications in medicine. In this study, we show that using multiple multivariate regression and statistical total correlation spectroscopy, we can learn the relation between the${}^{1}$H and${}^{13}$C dimensions. Learning is possible with small sample sizes and without the need for performing the HSQC analysis, we can predict the${}^{13}$C dimension by just performing${}^{1}$H HRMAS NMR experiment. We show on a rat model of central nervous system tissues (80 samples, 5 tissues) that our methods achieve 0.971 and 0.957 mean$R^2$values, respectively. Our tests on 15 human brain tumor samples show that we can predict 104 groups of 39 metabolites with 97 percent accuracy. Finally, we show that we can predict the presence of a drug resistant tumor biomarker (creatine) despite obstructed signal in${}^{1}$H dimension. In practice, this information can provide valuable feedback to the surgeon to further resect the cavity to avoid potential recurrence.

Nationality identification unlocks important demographic information, with many applications in biomedical and sociological research. Existing name-based nationality classifiers use name substrings as features and are trained on small, unrepresentative sets of labeled names, typically extracted from Wikipedia. As a result, these methods achieve limited performance and cannot support fine-grained classification.

Most complex tasks on the Internet-both malicious and benign-are collectively carried out by clusters of IP addresses. We demonstrate that it is often possible to discover such clusters by processing data sets and logs collected at various vantage points in the network. Obviously, not all clusters discovered in this way are malicious. Nevertheless, we show that malicious clusters can accurately be distinguished from benign ones by simply using an IP blacklist and without requiring any complex analysis to verify malicious behavior. In this paper, we first propose a novel clustering framework which can be applied on data sets of network interactions to identify IP clusters carrying out a specific task collectively. Then, given such a list of identified clusters of IP addresses, we present a simple procedure to spot the malicious ones using an IP blacklist. We show that by choosing the parameter of the proposed clustering process optimally using a blacklist, hence making it blacklist-aware, we significantly improve our overall ability to detect malicious clusters. Furthermore, we mathematically show that even a blacklist with poor accuracy can be used to detect malicious clusters with high precision and recall. Finally, we demonstrate the efficacy of the proposed scheme using real-world login events captured at the login servers of a large webmail provider with hundreds of millions of active users.

The orthodox paradigm to defend against automated social-engineering attacks in large-scale socio-technical systems is reactive and victim-agnostic. Defenses generally focus on identifying the attacks/attackers (e.g., phishing emails, social-bot infiltrations, malware offered for download). To change the status quo, we propose to identify, even if imperfectly, the vulnerable user population, that is, the users that are likely to fall victim to such attacks. Once identified, information about the vulnerable population can be used in two ways. First, the vulnerable population can be influenced by the defender through several means including: education, specialized user experience, extra protection layers and watchdogs. In the same vein, information about the vulnerable population can ultimately be used to fine-tune and reprioritize defense mechanisms to offer differentiated protection, possibly at the cost of additional friction generated by the defense mechanism. Secondly, information about the user population can be used to identify an attack (or compromised users) based on differences between the general and the vulnerable population. This paper considers the implications of the proposed paradigm on existing defenses in three areas (phishing of user credentials, malware distribution and socialbot infiltration) and discusses how using knowledge of the vulnerable population can enable more robust defenses.

The growth of Smartphones has bridged the telephony/SMS and the IP worlds, and this has resulted in new opportunities for financially motivated attackers. For example, some malicious campaigns in the cellular network aimed at extracting money fraudulently can do so even without any malware. Detecting and mitigating the variety of attacks in cellular network is difficult because they do not necessarily have a fixed 'signature', and new types of campaigns appear frequently. Further complicating matters, detecting a single malicious entity (a domain name, a phone number, or a short code) that is part of a malicious campaign, is usually not very effective, because the attacker simply moves to using another entity in its place. An effective strategy requires detecting all/most elements involved in the campaign at once. In this paper, we describe a system, based on ideas from anomaly detection and clustering, that aims to detect many different families of widespread malicious campaigns in cellular networks. The system reveals an entire campaign as a graph cluster which includes the various entities involved in the campaign and their relationship, such as malware download websites, C&C servers, spammers, etc. Using logs from both SMS and IP portions of the network for millions of users, we detect newly popular entities and cluster them to discover how they are related. By looking for cues of possible malicious behavior from any of the entities in a cluster, we attempt to ascertain whether a detected campaign might be malicious, providing valuable leads to a human analyst. Our system is live and generates daily clusters for human analysts. We provide detailed case studies of real, previously unseen families of malicious campaigns that this system has successfully brought to light.

If an outbound flow is observed at the boundary of a protected network, destined to an IP address within a few addresses of a known malicious IP address, should it be considered a suspicious flow? Conventional blacklisting is not going to cut it in this situation, and the established fact that malicious IP addresses tend to be highly clustered in certain portions of IP address space, should indeed raise suspicions. We present a new approach for perimeter defense that addresses this concern. At the heart of our approach, we attempt to infer internal, hidden boundaries in IP address space, that lie within publicly known boundaries of registered IP netblocks. Our hypothesis is that given a known bad IP address, other IP address in the same internal contiguous block are likely to share similar security properties, and may therefore be vulnerable to being similarly hacked and used by attackers in the future. In this paper, we describe how we infer hidden internal boundaries in IPv4 netblocks, and what effect this has on being able to predict malicious IP addresses.

Short Message Service (SMS) spam is increasingly becoming a problem for many telecommunication service providers. Not only do SMS spam messages use mobile network resources abusively, but also in many cases they represent malware propagation vectors for mobile devices. In this work, we propose a network-based online detection method for SMS spam messages. The proposed scheme uses robust text signatures to identify similar messages that are sent excessively in the SMS platform and is robust against slight modifications in SMS spam messages. Additionally, the method uses a fast online algorithm which can be deployed in large carrier networks to detect spam activities before too many spam messages are delivered. It does not store SMS message contents, therefore it does not compromise the privacy of mobile subscribers.

In this work we show that once a single peer-to-peer (P2P) bot is detected in a network, it may be possible to efficiently identify other members of the same botnet in the same network even before they exhibit any overtly malicious behavior. Detection is based on an analysis of connections made by the hosts in the network. It turns out that if bots select their peers randomly and independently (i.e. unstructured topology), any given pair of P2P bots in a network communicate with at least one mutual peer outside the network with a surprisingly high probability. This, along with the low probability of any other host communicating with this mutual peer, allows us to link local nodes within a P2P botnet together. We propose a simple method to identify potential members of an unstructured P2P botnet in a network starting from a known peer. We formulate the problem as a graph problem and mathematically analyze a solution using an iterative algorithm. The proposed scheme is simple and requires only flow records captured at network borders. We analyze the efficacy of the proposed scheme using real botnet data, including data obtained from both observing and crawling the Nugache botnet.

In this work we propose a Voice over IP (VoIP) call tracking scheme based on robust hashing of VoIP flows. In the proposed scheme the audio content of a possibly encrypted VoIP flow is identified by a short binary string, called the robust hash, using variations on the flow's bitrate over time. These robust hashes are then used to detect pairs of parties communicating with each other. In summary, if two parties are communication with each other then they have a pair of VoIP flows having similar robust hashes with high probability. The basic intuition behind the proposed hash function is twofolds: i)The variable bitrate codec employed in most VoIP applications result in a distinctive bitrate variations over time for each VoIP flow depending on the underlying audio content. ii) Encryption typically doesn't change the bitrate of a VoIP flow. Our experiments show that the proposed scheme is able to identify Skype VoIP flows even under various network impairments such as packet delays, jitter and packet drops.

We present an efficient and robust stepping-stone detection scheme based on succinct packet-timing sketches of network flows. The proposed scheme employs an online algorithm to continuously maintain short sketches of flows from a stream of captured packets at the network boundary. These sketches are then used to identify pairs of network flows with similar packet-timing characteristics, which indicates potential stepping-stones. Succinct flow sketches enable the proposed scheme to compare a given pair of flows in constant time. In addition, flow sketches identify pairs of correlated flows from a given list of flows in sub-quadratic time, thereby allowing a more scalable solution as compared to known schemes. Finally, the proposed scheme is resistant to random delays and chaff, which are often employed by attackers to evade detection. To explore its efficacy, we mathematically analyze the robustness properties of the proposed flow sketch. We also experimentally measure the detection performance of the proposed scheme.

Computational cost of classification is as important as accuracy in on-line classification systems. The computational cost is usually dominated by the cost of computing implicit features of the raw input data. Very few efforts have been made to design classifiers which perform effectively with limited computational power; instead, feature selection is usually employed as a pre-processing step to reduce the cost of running traditional classifiers. We present CoCoST, a novel and effective approach for building classifiers which achieve state-of-the-art classification accuracy, while keeping the expected computational cost of classification low, even without feature selection. CoCost employs a wide range of novel cost-aware decision trees, each of which is tuned to specialize in classifying instances from a subset of the input space, and judiciously consults them depending on the input instance in accordance with a cost-aware meta-classifier. Experimental results on a network flow detection application show that, our approach can achieve better accuracy than classifiers such as SVM and random forests, while achieving 75%-90% reduction in the computational costs.

Relay nodes are a potential threat to networks since they are used in many malicious situations like stepping stone attacks, botnet communication, peer-to-peer streaming etc. Quick and accurate detection of relay nodes in a network can significantly improve security policy enforcement. There has been significant work done and novel solutions proposed for the problem of identifying relay flows active within a node in the network. However, these solutions require quadratic number of comparisons in the number of flows. In this paper, a related problem of identifying relay nodes is investigated where a relay node is defined as a node in the network that has an active relay flow. The problem is formulated as a variance estimation problem and a statistical approach is proposed for the solution. The proposed solution requires linear time and space in the number of flows and therefore can be employed in large scale implementations. It can be used on its own to identify relay nodes or as a first step in a scalable relay flow detection solution that performs known quadratic time analysis techniques for relay flow detection only on nodes that have been detected as relay nodes. Experimental results show that the proposed scheme is able to detect relay nodes even in the presence of intentional inter-packet delays and chaff packets introduced by adversaries in order to defeat timing based detection algorithms.

Powerful digital media editing tools make producing good quality forgeries very easy for almost anyone. Therefore, proving the authenticity and integrity of digital media becomes increasingly important. In this work, we propose a simple method to detect image tampering operations that involve sharpness/blurriness adjustment. Our approach is based on the assumption that if a digital image undergoes a copy-paste type of forgery, average sharpness/blurriness value of the forged region is expected to be different as compared to the non-tampered parts of the image. The method of estimating sharpness/blurriness value of an image is based on the regularity properties of wavelet transform coefficients which involves measuring the decay of wavelet transform coefficients across scales. Our preliminary results show that the estimated sharpness/blurriness scores can be used to identify tampered areas of the image.

Identification and verification of a video clip via its fingerprint find applications in video browsing, database search and security. For this purpose, the video sequence must be collapsed into a short fingerprint using a robust hash function based on signal processing operations. We propose two robust hash algorithms for video based both on the discrete cosine transform (DCT), one on the classical basis set and the other on a novel randomized basis set (RBT). The robustness and randomness properties of the proposed hash functions are investigated in detail. It is found that these hash functions are resistant to signal processing and transmission impairments, and therefore can be instrumental in building database search, broadcast monitoring and watermarking applications for video. The DCT hash is more robust, but lacks security aspect, as it is easy to find different video clips with the same hash value. The RBT based hash, being secret key based, does not allow this and is more secure at the cost of a slight loss in the receiver operating curves