Effective system resource management is key to delivering high performance stream processing. Stream processing engines (SPEs) rely on their host operating system (OS) for managing compute and memory resources, but this is inefficient as the OS is not stream-aware, i.e., the OS does not understand the streaming dataflow or pipeline state in how they relate to the resource requirements of stream processing. Additionally, the lack of stream-awareness inhibits adaptive resource allocation in response to dynamic workload changes. We present Enjima, a modern SPE designed for scale-up on a single machine through adaptive stream-aware management of memory and compute resources. Enjima's eager, cache-aligned, block-based memory management avoids memory allocation on the critical path of system execution while providing efficient data transfer of events between streaming operators. Its variable batching forms event batches based on pending inputs and available output memory, reducing batching delays and memory accesses to enhance system performance. Enjima integrates a stream-aware, state-based operator scheduler that leverages fine-grained operator and pipeline metrics such as operator cost, selectivity, and latency gradient to optimize for both latency and throughput, enabling significant performance gains and rapid adaptation to dynamic workloads. Evaluation against state-of-the-art systems shows that Enjima achieves up to 6.3× higher throughput and up to three orders of magnitude lower latency through integrated stream-aware memory and CPU resource management.

This article addresses the challenges of using call detail record (CDR) data to study migration. Repurposing CDR data for this task have many advantages, including the lower costs of data collection and the potential for contemporaneous analysis. We present a framework for the repurposing and analysis of CDR data. We identify the home location of a subscriber, with corresponding confidence measures, and determine if the subscriber is a definite migrant, likely migrant, likely nonmigrant, or definite nonmigrant. A predictive model then uses mobility and social network features, extracted from the CDR data, to predict the individual decision to migrate. We are the first to address the challenging task of predicting the migration decision at the individual level. We also provide insight into features that can have an impact on the decision to migrate. An in-depth evaluation using CDR data from two provinces in Sri Lanka provides a granular map of migrant inflow and outflow. The success of our prediction model and the insights gained from the evaluation prepare the way for the repurposing of CDR data for social good with a focus on migration.

Every year paddy cultivators lose a significant amount of crop yield due to diseases and pests. Brown Planthopper (BPH) is one of the most common diseases that affect paddy cultivation. Sri Lankan government is struggling to make appropriate estimations regarding Brown Planthopper prevalence due to the absence of accurate and timely data. To solve this issue, a machine learning approach is proposed based on optical and synthetic aperture radar remote sensing data in this study. However, there is no previous effort for detecting Brown Planthopper attacks using machine learning and satellite remote sensing data under field conditions. This study consists of two phases. A time series classification based on SAR imagery is implemented to identify cultivated paddy fields in the first phase. Ratio and standard difference indices derived from optical satellite images are used in the second phase to identify regions affected by BPH attacks in paddy fields. Convolution neural network that is used in the first phase reports an accuracy of 96.20% for identifying cultivated paddy regions. A Support Vector Machine is used to detect areas damaged by BPH attacks in the second phase. The Combined approach of the first and the second phases shows promising results with an accuracy of 96.31% for detecting Brown Planthopper attacks.