Abstract Traditional multimedia classification techniques rely on analyzing either its features or the associated annotated textual information. In this paper, we introduce a technique that leverages deep learning methodologies to extract both low-level visual features and high-level semantic information from the billboard images. We propose a multi-modal hybrid fusion model that integrates text and image features to categorize billboards in video frames into food, sports, and miscellaneous categories. Achieving a 5% accuracy improvement over image-based models and 10% over text-based models, our model demonstrates robust generalization across diverse datasets, benefiting advertising, media, and content creation industries.

Abstract Advert placement in sports broadcasts is a growing strategy to boost sponsor visibility without disrupting live gameplay. Achieving realism, however, requires careful handling of scene geometry and dynamic occlusions from players and the sports ball. In this work, we propose an occlusion-aware and perspective-consistent framework specifically for virtual advert placement in the soccer penalty area. We introduce an automatic procedure to select a geometrically consistent quadrilateral region inside the penalty area from predicted field coordinates, which is then used for homography-based warping. We integrate instance-level occlusion masks with Laplacian Alpha Blending for dynamic occlusion-aware blending so that the virtual advert is correctly placed behind players and the ball. Quantitative evaluations demonstrate that our occlusion-aware advert placement method preserves high visual fidelity with an average SSIM of 0.97 and PSNR of 31.6 dB, while maintaining temporal consistency with flicker index increase of <7%. Furthermore, we analyze occlusion preservation by comparing advert insertion with and without occlusion handling. A decrease in this metric indicates that objects overlapping the advert region become incorrectly hidden after advert insertion. The results highlight the importance of occlusion-aware blending for maintaining scene integrity and visual realism. By effectively managing occlusions, the proposed framework reduces visual artifacts and improves perceptual quality, producing augmented sports footage that is realistic and visually coherent.

Abstract In recent years, computer vision and machine learning have achieved remarkable advancements across health care, autonomous systems, and robotics sectors. However, the educational domain, particularly the automated detection of student engagement in online and offline learning environments, remains rich in research opportunities. Detecting student engagement is inherently complex and requires sophisticated interpretive capabilities. This paper introduces a novel approach to automatically detect and classify student engagement by integrating facial images with behavioral and facial features, providing a comprehensive solution to enhance engagement recognition. This study begins by extracting behavioral features and correlating them with pre-defined engagement labels to perform engagement classification using machine learning techniques. In parallel, deep learning models are trained and validated on both image and behavioral features, offering a complementary approach. Additionally, the relationship between facial action units (AUs) and engagement labels is analyzed using three distinct metrics: conditional activation probability, relative activation ratio, and the statistical discriminant coefficient (SDC). The study utilizes publicly available datasets (WACV and DAiSEE) to perform extensive evaluations. Experimental results demonstrate that integrating action units significantly enhances model performance, with XGBoost achieving the highest accuracy (82.9%) among traditional models and EfficientNet reaching the best performance (47.2%) in deep learning experiments. These findings highlight the potential of multimodal approaches in improving real-time engagement detection, offering valuable insights into educational technologies and pedagogy.

In recent years, Ireland has been experiencing the growing and pervasive threat of NO2pollution, presenting a significant risk to human health. This study emphasizes the importance of utilizing high-resolution, full-coverage satellite data as a crucial resource in the prevention and control of NO2pollution. To enhance the precision of daily near-surface NO2level estimations, we propose a novel estimation architecture, the Nested Vision Transformer-based Network (Nes-ViT-Net), which integrates satellite and ground observations. Nes-ViT-Net operates on the principle of capturing the non-linear relationship between ground-level target NO2and satellite observations to estimate daily near-surface NO2concentrations. Demonstrating superiority over alternative algorithms, Nes-ViT-Net exhibits robust retrieval performance with a 6.96 µg/m3RMSE, surpassing other methods by at least 12%. This success in estimating near-surface NO2pollution from satellite observations is anticipated to contribute significantly to the continuous and dynamic monitoring of regional and global air pollution. The code is available here: https://github.com/Prasanjit-Dey/Nes-ViT-Net.

In this study, we address the key objective of identifying the optimal region within an extensive geographical area for strategic billboard placement, ensuring effective communication with diverse audiences. The challenge lies in finding suitable spaces for billboard placement within a confined geographical area. One potential solution involves extracting urban areas from satellite images to enhance the efficiency of identifying appropriate billboard spaces. To address this, we propose a novel framework named USE-NET, leveraging satellite data to comprehensively cover large geographic regions for the precise identification of optimal billboard locations. Our proposed framework integrates an enhanced UNet model with the Squeeze-and-Excitation (SE) attention method to optimize the identification process based on satellite data. Comparative evaluations against alternative deep learning models, utilizing various metrics, underscore the efficacy of our approach. Experimental results demonstrate a testing accuracy of 87.21%, showcasing a notable improvement of at least 4% compared to Link-Net, MaNet, and UNet models. To enhance the reproducibility of this research, the code of this paper is made available at: https://github.com/sukritidhang/USE-Net_optimalbillboardlocation

Recent advancements in meteorology involve the use of ground-based sky cameras for cloud observation. Analyzing images from these cameras helps in calculating cloud coverage and understanding atmospheric phenomena. Traditionally, cloud image segmentation relied on conventional computer vision techniques. However, with the advent of deep learning, convolutional neural networks (CNNs) are increasingly applied for this purpose. Despite their effectiveness, CNNs often require many epochs to converge, posing challenges for real-time processing in sky camera systems. In this paper, we introduce a residual U-Net with deep supervision for cloud segmentation which provides better accuracy than previous approaches, and with less training consumption. By utilizing residual connection in encoders of UCloudNet, the feature extraction ability is further improved. In the spirit of reproducible research, the model code, dataset, and results of the experiments in this paper are available at: https://github.com/Att100/UCloudNet.

Recent years have seen increased interest in solar as a popular renewable energy source. The adoption of solar energy is directed towards increasing research interest in the incorporation of solar energy estimation and forecasting in the current system. Solar irradiance is the sun’s energy incident on earth, and is dependent on many atmospheric parameters, mainly decided by clouds. The use of a Ground-based sky imaging system is the best for forecasting and estimating for near-real-time temporal spectrum. However, creating efficient solar estimation models for edge devices is still a relatively unexplored field. This paper captures the essence of bringing edge computing on devices for solar estimation on ground-based infrared sky images by proposing a novel knowledge distillation method for enhancing lightweight CNN-regression models. In our study, we leveraged a MobilenetV2-based Teacher model and transferred the knowledge into a simple student model by introducing a sigmoid-based loss in the Knowledge Distillation algorithm. The proposed solution shows an impressive reduction of the Mean Square Error (MSE) of the model from 3015.63 to 2540.67. This research advances the field of solar irradiance estimation by emphasizing the importance of creating efficient and edge device deployable models in this context.

In recent years, global concern for human health has escalated due to the persistent threat of air pollution, resulting in a surge of chronic diseases and premature mortality. Poor air quality not only has adverse effects on human health but also poses negative impacts on vegetation, society, and the economy. Hence, it is imperative to invest more effort in accurately predicting multivariate air pollutants to offer practical and relevant solutions. However, many machine learning (ML) and deep learning (DL) models face significant challenges when dealing with the complexities of multivariate air pollution dynamics and the ill-posed nature of the data. In this letter, we propose a Gaussian-mixture nested factorial variational autoencoder (NF-VAE), specifically designed for multivariate air pollution prediction. To assess the performance of the proposed framework, we conducted experimental validation using air pollution data from six monitoring sites in Chinese cities. Three statistical indicators have been used to evaluate forecasting accuracy. The experimental results demonstrate the satisfactory performance of the NF-VAE model in predicting six pollutants for six different sites. Furthermore, the results indicate that the proposed NF-VAE model can effectively enhance efficiency gains, demonstrating improvements of at least 31% for RMSE, 22% for MAE, and 13% for$R^{2}$compared with popular DL models, namely, long short-term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), and bidirectional GRU (BiGRU).

The introduction of solar power in energy grids is crucial for sustainable progress, but gets restricted due to frequently fluctuating electricity outputs generated by intermittent solar irradiance. The problem requires proper power planning and operations management that requires accurate and reliable solar irradiance forecasts. In this context, this research presents a holistic and lightweight framework for solar irradiance forecasting, the critical component for generating photovoltaic (PV) output. The paper’s holistic approach leverages meteorological variables, historical global horizontal irradiance (GHI) data, ground-based sky imager (GSI) images, satellite-derived cloud masks, and satellite-based clear sky data to improve forecasting accuracy. The proposed framework extends the forecasting horizon to 60 min while covering a continuous 6-h historical context. Innovative feature extraction techniques were implemented to reduce the cloud image dimensions, enabling the development of a lightweight forecasting model. The results demonstrate the effectiveness of this approach, contributing to a more reliable GHI forecasting for efficient energy grid management.

Real-time rendering applications leverage heterogeneous computing to optimize performance. However, software development across multiple devices presents challenges, including data layout inconsistencies, synchronization issues, resource management complexities, and architectural disparities. Additionally, the creation of such systems requires verbose and unsafe programming models. Recent developments in domain-specific and unified shading languages aim to mitigate these issues. Yet, current programming models primarily address data layout consistency, neglecting other persistent challenges.In this paper, we introduce RenderKernel, a programming model designed to simplify the development of real-time rendering systems. Recognizing the need for a high-level approach, RenderKernel addresses the specific challenges of real-time rendering, enabling development on heterogeneous systems as if they were homogeneous. This model allows for early detection and prevention of errors due to system heterogeneity at compile-time. Furthermore, RenderKernel enables the use of common programming patterns from homogeneous environments, freeing developers from the complexities of underlying heterogeneous systems. Developers can focus on coding unique application features, thereby enhancing productivity and reducing the cognitive load associated with real-time rendering system development.

Predicting air pollutant concentrations is an efficient way to prevent incidents by providing early warnings of harmful air pollutants. A precise prediction of air pollutant concentrations is an important factor in controlling and preventing air pollution. In this paper, we develop a bidirectional long-short-term memory and a bidirectional gated recurrent unit (BiLSTM−BiGRU) to predict PM2.5concentrations in a target city for different lead times. The BiLSTM extracts preliminary features, and the BiGRU further extracts deep features from air pollutant and meteorological data. The fully connected (FC) layer receives the output and makes an accurate prediction of the PM2.5concentration. The model is then compared with five other deep learning models in terms of root mean square error (RMSE), mean absolute error (MAE) and correlation (R2) over different lead times. The results indicate that the proposed model has at least 2.2 times lower RMSE than the other models.

Cloud image classification has been extensively studied in the literature, as it has several radio-meteorological and remote sensing applications. Recently, images from ground-based sky imagers (GSIs) are being widely used because of their high temporal and spatial resolution and low infrastructure cost as compared to satellites. To classify sky/cloud images obtained from such GSIs, this paper1examines the application of transfer learning using the standard VGG-16 architecture. The paper further analyzes the importance of adjusting the number of neurons in the top dense layers to improve the performance of the model. The reasons for the same are traced by conducting extensive experiments on multiple datasets exhibiting varied properties.

Over the past few years, photovoltaic systems are replacing traditional sources of energy. These photovoltaic systems are powered by solar energy, which depends on the amount of incident solar irradiance. However, solar irradiance has a large amount of variability due to various spatio-temporal factors including region, cloud cover, humidity, and rainfall. In this paper1, we identify the most relevant spatio-temporal features for estimating solar irradiance. Such an accurate identification of weather variables is useful for developing accurate data-driven solar irradiance estimation models. This leads to an improved forecasting performance, thus creating a stable photovoltaic system. We base our analysis on the dataset of three countries, Ireland, Singapore and India. We identify the cloud type as the most relevant weather variables in all regions. The other relevant features holding differential importance across regions include time, relative humidity, solar zenith angle, and pressure.

The unpredictable nature of solar irradiance presents a significant challenge for integrating photovoltaic (PV) systems into large-scale energy grids. Short-term solar irradiance forecasting, specifically for 2-6 hours ahead, is crucial for optimizing solar energy integration and improving grid stability. Cloud cover monitoring plays a key role in accurate forecasting of solar irradiance. This paper1highlights the utility of satellite-derived cloud mask feature in improving the forcasting accuracy. A lightweight convolutional autoencoder is proposed to encode cloud cover information, enabling nowcasting of solar irradiance values at a 5-minute resolution for a 2-hour lead time. Thus, the approach enhances the accuracy of short-term solar irradiance forecasts with satellite data, facilitating the effective integration of solar energy into energy grids.

Cloud image segmentation plays a pivotal role in fields such as weather prediction, climate modeling, and renewable energy systems. Although ground-based sky imagers are preferred tools for cloud image analysis, their images present unique challenges for segmention due to noise, sun glare, and other factors. Recent success in cloud image segmentation is attributed to the use of deep learning techniques. However, they require large annotated datasets for improved performance and robustness. This paper1introduces a two-step generative framework that simultaneously generates sky/cloud images and their corresponding ground-truth segmentation maps to augment the dataset and demonstrates the performance of the proposed approach over two prominent semantic image segmentation models and sky/cloud patch image datasets.

Land cover classification using satellite imagery plays a crucial role in monitoring changes on the earth’s surface. This paper presents an analysis of the EuroSAT dataset using state-of-the-art deep learning models to benchmark the impact of additional bands on classification accuracy. The dataset consists of 27,000 images across 10 classes captured by the Sentinel-2 satellite, including RGB and multispectral bands. Performance evaluation was conducted using popular convolutional neural network models based on Resnet variants and Vision Transformer (ViT). The results show that the combination of all bands achieved the highest accuracy, with ResNet-152 achieving a validation accuracy of 96.63% on the multispectral dataset. Precision, recall, and F1 scores were also utilized to assess the models’ performance. The findings highlight the significance of incorporating additional bands for improved classification accuracy in satellite image analysis.

We analyse the effectiveness of edge detection algorithms for the purpose of automatically extracting coastlines from satellite images. Four algorithms - Canny, Sobel, Scharr and Prewitt are compared visually and using metrics. With an average SSIM of 0.8, Canny detected edges that were closest to the reference edges. However, the algorithm had difficulty distinguishing noisy edges, e.g. due to development, from coastline edges. In addition, histogram equalization and Gaussian blur were shown to improve the effectiveness of the edge detection algorithms by up to 1.5 and 1.6 times respectively.

Climate data is a valuable resource for understanding past weather patterns, assessing long-term climate trends, and conducting climate-related research. However, most existing knowledge graphs for climate data rely heavily on the standardized (per W3C recommendations) SOSA/SSN ontology, which can help improve general data accessibility, but typically overlooks the analytical applications of multisource climate data. To further enhance the accessibility of heterogeneous data for climate data analytics, this paper extends the CA ontology and implements a virtual knowledge graph for analytical applications. We emphasize the importance of incorporating observation metadata and geospatial representation into analytical applications. Through our study, we demonstrate the applicability of the proposed ontological model in deriving the ETCCDI indices. An example of the formation of the annual maximum daily temperature is given. Furthermore, we showcase the potential of LinkedGeoData in providing a more comprehensive geographical context for accessing climate data within the knowledge graph, leveraging the proposed ontological modeling and linked data principles.

Reducing industrial greenhouse gas (GHG) emissions has become imperative for mitigating the adverse effects of climate change. Accurate measurement and monitoring of industrial smoke plumes, which are a significant source of GHG emissions, are crucial for effective emission control strategies. This paper addresses the prospect of utilizing satellite images to measure industrial smoke plumes and explores the effectiveness of various computer vision (CV) technologies in this context. The study focuses on examining both modern deep learning and traditional machine learning models for detecting and segmenting industrial smoke plumes in satellite images. While deep learning models have shown remarkable performance in various CV tasks, their ability to accurately segment smoke plumes in satellite images remains limited, with an average intersection over union (IOU) of no more than 60%. However, certain deep learning models, such as U-Net and AttU-Net, exhibit promising capabilities in identifying challenging types of noise, including clouds, white building surfaces, and snow, which traditional machine learning models struggle with. Employing deep learning models for industrial smoke plume detection proves advantageous, as all models achieve an approximate detection accuracy and F1-Score of 90%. The findings from this research serve as a valuable foundation for further advancements in developing advanced deep learning models specifically tailored to handle the identified types of noise.

In this paper, we propose SYGNet to strengthen the scene parsing ability of autonomous driving under complicated road conditions. The SYGNet includes feature extraction component and SVD-YOLO GhostNet component. The SVD-YOLO GhostNet component combines Singular Value Decomposition (SVD), You Only Look Once (YOLO) and GhostNet. In the feature extraction component, we propose an algorithm based on VoxelNet to extract point cloud features and image features. In SVD-YOLO GhostNet component, the image data is decomposed by SVD, and we obtain data with stronger spatial and environmental characteristics. YOLOv3 is used to obtain the future map, then convert to GhostNet, which is used to realize the real-time scene parsing. We use KITTI data set to perform our experiments and the results show that the SYGNet is more robust and can further enhance the accuracy of real-time driving scene parsing. The model code, data set, and results of the experiments in this paper are available at: https://github.com/WangHewei16/SYGNet-for-Real-time-Driving-Scene-Parsing.

While intermittent nature of solar irradiance values is primarily attributed to clouds, its equally important to analyze the solar irradiance variance throughout the day in clear sky conditions. Although a vast amount of research has been conducted so far and a numerous clear sky models have been proposed in the literature, none of them holds true temporally$a$nd/or spatially across the earth. Accordingly, this paper proposes a generic framework to identify the most appropriate clear sky model for the location of interest, i.e. Delhi-NCR in this case. The paper further highlights the importance of incorporating seasonal and temporal variations while determining the suitable clear sky model.

Global horizontal irradiance (GHI) plays a significant role in maintaining the earth's ecological balance and generating electricity in photovoltaic systems. While the satellites have more range, they have been shown to over/under-estimate the true values of GHI that are observed at the ground-based stations. Hence, this study aims at analyzing the relationship between these two sources of GHI data in order to better and effectively utilize the reach of satellites for GHI analysis. The paper identifies a near linear relationship between the two and thereby concludes that an approximate mapping from satellite- to ground-based GHI values can be obtained.

An accurate prediction of rainfall is a very important task and has vital effects on human life. The usage of machine learning (ML) in the field of meteorology has provided solutions to improve the rainfall prediction accuracy. In the same direction, this study suggests an efficient methodology for the prediction of rainfall events with the aim of dimensionality reduction. Firstly, we identified most relevant features from the weather dataset which plays a major role in the prediction of a rainfall event using a wrapper-based feature selection (FS) technique. Secondly, principal components analysis (PCA) is integrated with the complete as well as with selected features dataset to reduce the data dimensionality. Finally, a thorough comparative analysis of different ML prediction models is presented with different nature of feature inputs. The performance of classification models improved significantly when using reduced features set. Specially PCA integrated with FS technique provided excellent prediction results.

Today's atmospheric data is generated swiftly as a result of the growth of IoT and sensor technologies and is available via data suppliers' RESTful APIs. However, sensor data mostly consists of live data streams including sensor observations, which are produced in a dispersed manner by several heterogeneous infrastructures, with little or no interoperability. RDF streams incorporating semantic data interoperability have arisen in last years and can be the foundation of intelligent semantic applications (e.g. semantic complex event processing). To enable semantic analysis of live atmospheric data streams, this article proposes a methodology for converting live data streams into Linked Data. The process leverages the most recent technologies for RML semantic mapping, ontology modeling, and Linked Data to extend the semantic usefulness of live atmospheric data, for example, by allowing for easy integration of atmospheric data streams with other live RDF streams.

With the active development of ICT and Internet technologies in climate research, individuals often need to gather different and disparate datasets and preprocess them in preparation for downstream data analysis in order to have a more full understanding of the challenges. This preparatory procedure is often lengthy due to the primary issue that data providers can-not ensure a homogeneous data format for data integration purposes. To overcome this problem, this study proposes enhancing existing relational climate data by layering a virtual knowledge graph on top of the original databases provided by various data vendors. The primary benefit of doing this is that data consumers are able to simply integrate climate data with other data sources using Linked Data principles, and climate data producers do not have to modify their data to conform to standard knowledge graph protocols.

The latest analytical models are becoming frequently used in meteorological science research. For instance, machine learning and deep learning models are being trained for weather forecasting. A solid machine learning model can give trustworthy findings that aid individuals in making weather-related decisions. However, the performance of analytical models is largely determined not only by the design of the model body but also by the input features. We address common issues of modern meteorological studies that take sensor data as the input for various analytical models. In contrast to the traditional practice of combining and preprocessing many fixed sensor data bulks to create an augmented dataset, we tunnel into remote knowledge graphs to fetch and augment the sensor data in a scalable way. As a consequence, we reduce the amount of time and storage space to preprocess diverse data in preparation for analytical models leveraging the high interoperability between knowledge graphs.

Virtual Reality (VR) technology has been shown to achieve remarkable results in multiple fields. Due to the nature of the immersive medium of Virtual Reality it logically follows that it can be used as a high-quality educational tool as it offers potentially a higher bandwidth than other mediums such as text, pictures and videos. This short paper illustrates the development of a climate change educational awareness application for virtual reality to simulate virtual scenes of local scenery and sea level rising until 2100 using prediction data. The paper also reports on the current in progress work of porting the system to Augmented Reality (AR) and future work to evaluate the system.

The reliability of European Remote Sensing 5 (ERA5) satellite-based air temperature data is under investigation in this letter. To evaluate this, the ERA5 data will be compared with land-based data obtained from weather stations on the global historical climatology network (GHCN). Two climate regions are taken into consideration, temperate and tropical. Five years worth of data is collected and compared through box plots, regression models, and statistical metrics. The results show that the satellite temperature performs better in the temperate region than the tropical region. This suggests that the time of year and climate region have an impact on the accuracy of the satellite data as milder temperatures produce better approximations.

Ground-based Whole Sky Imagers (WSIs) are increasingly being used for various remote sensing applications. While the fundamental requirements of a WSI are to make it climate-proof with an ability to capture high resolution images, cost also plays a significant role for wider scale adoption. This paper proposes an extremely low-cost alternative to the existing WSIs. In the designed model, high resolution images are captured with auto adjusting shutter speeds based on the surrounding light intensity. Furthermore, a manual data backup option using a portable memory drive is implemented for remote locations with no internet access.

While cloud/sky image segmentation has extensive real-world applications, a large amount of labelled data is needed to train a highly accurate models to perform the task. Scarcity of such volumes of cloud/sky images with corresponding ground-truth binary maps makes it highly difficult to train such complex image segmentation models. In this paper, we demonstrate the effectiveness of using Generative Adversarial Networks (GANs) to generate data to augment the training set in order to increase the prediction accuracy of image segmentation model. We further present a way to estimate ground-truth binary maps for the GAN-generated images to facilitate their effective use as augmented images. Finally, we validate our work with different statistical techniques.

Air pollution has been a long-existing problem for most of the major metropolitan cities of the world. Several measures including strict climate laws and reduction in the number of vehicles were implemented by several nations. However, in the recent wake of the COVID19 pandemic, there has been a renewed interest in revisiting the problem of low air quality. Several countries implemented strict lockdown measures halting the vehicular traffic and other economic activities, in order to reduce the spread of COVID19. In this paper, we analyze the impact of such COVID19-induced lockdown on the air quality of the atmosphere. Our case study is based in the city of Dublin, Ireland. We analyze the average concentration of common gaseous pollutant majorly responsible for industrial and vehicular pollution, viz. nitrogen dioxide (NO2). These concentrations are obtained from the tropospheric column of the atmosphere collected by Sentinel-5P, which is an earth observation satellite of European Space Agency. We observe that Dublin had a significant drop in the level of NO2concentration, owing to the strict lockdown measures implemented across the nation.

Recently ontologies have been exploited in a wide range of research areas for data modeling and data management. They greatly assists in defining the semantic model of the underlying data combined with domain knowledge. In this paper, we propose the Climate Analysis (CA) Ontology to model climate datasets used by remote sensing analysts. We use the data published by National Oceanic and Atmospheric Administration (NOAA) to further explore how ontology modeling can be used to facilitate the field of climatic data processing. The idea of this work is to convert relational climate data to the Resource Description Framework (RDF) data model, so that it can be stored in a graph database and easily accessed through the Web as Linked Data. Typically, this provides climate researchers, who are interested in datasets such as NOAA, with the potential of enriching and interlinking with other databases. As a result, our approach facilitates data integration and analysis of diverse climatic data sources and allows researchers to interrogate these sources directly on the Web using the standard SPARQL query language.

Large-scale deployment of smart meters has made it possible to collect sufficient and high-resolution data of residential electric demand profiles. Clustering analysis of these profiles is important to further analyze and comment on electricity consumption patterns. Although many clustering techniques have been proposed in the literature over the years, it is often noticed that different techniques fit best for different datasets. To identify the most suitable technique, standard clustering validity indices are often used. These indices focus primarily on the intrinsic characteristics of the clustering results. Moreover, different indices often give conflicting recommendations, which can only be clarified with heuristics about the dataset and/or the expected cluster structures-information that is rarely available in practical situations. This article presents a novel scheme to validate and compare the clustering results objectively. Additionally, the proposed scheme considers all the steps prior to the clustering algorithm, including the preprocessing and dimensionality reduction steps, in order to provide recommendations over the complete framework. Accordingly, the proposed strategy is shown to provide better, unbiased, and uniform recommendations as compared to the standard clustering validity indices.

With the rapid technological advancements, organizations need to rapidly scale up their information technology (IT) infrastructure viz. hardware, software, and services, at a low cost. However, the dynamic growth in the network services and applications creates security vulnerabilities and new risks that can be exploited by various attacks. For example, User to Root (U2R) and Remote to Local (R2L) attack categories can cause a significant damage and paralyze the entire network system. Such attacks are not easy to detect due to the high degree of similarity to normal traffic. While network anomaly detection systems are being widely used to classify and detect malicious traffic, there are many challenges to discover and identify the minority attacks in imbalanced datasets. In this paper, we provide a detailed and systematic analysis of the existing Machine Learning (ML) approaches that can tackle most of these attacks. Furthermore, we propose a Deep Learning (DL) based framework using Long Short Term Memory (LSTM) autoencoder that can accurately detect malicious traffics in network traffic. We perform our experiments in a publicly available dataset of Intrusion Detection Systems (IDSs). We obtain a significant improvement in attack detection, as compared to other benchmarking methods. Hence, our method provides great confidence in securing these networks from malicious traffic.

Software-Defined Networking (SDN) is an emerging paradigm, which evolved in recent years to address the weaknesses in traditional networks. The significant feature of the SDN, which is achieved by disassociating the control plane from the data plane, facilitates network management and allows the network to be efficiently programmable. However, the new architecture can be susceptible to several attacks that lead to resource exhaustion and prevent the SDN controller from supporting legitimate users. One of these attacks, which nowadays is growing significantly, is the Distributed Denial of Service (DDoS) attack. DDoS attack has a high impact on crashing the network resources, making the target servers unable to support the valid users. The current methods deploy Machine Learning (ML) for intrusion detection against DDoS attacks in the SDN network using the standard datasets. However, these methods suffer several drawbacks, and the used datasets do not contain the most recent attack patterns - hence, lacking in attack diversity. In this paper, we propose DDoSNet, an intrusion detection system against DDoS attacks in SDN environments. Our method is based on Deep Learning (DL) technique, combining the Recurrent Neural Network (RNN) with autoencoder. We evaluate our model using the newly released dataset CICDDoS2019, which contains a comprehensive variety of DDoS attacks and addresses the gaps of the existing current datasets. We obtain a significant improvement in attack detection, as compared to other benchmarking methods. Hence, our model provides great confidence in securing these networks.

The rapid increase in the number of online videos provides the marketing and advertising agents ample opportunities to reach out to their audience. One of the most widely used strategies is product placement, or embedded marketing, wherein new advertisements are integrated seamlessly into existing advertisements in videos. Such strategies involve accurately localizing the position of the advert in the image frame, either manually in the video editing phase, or by using machine learning frameworks. However, these machine learning techniques and deep neural networks need a massive amount of data for training. In this paper, we propose and release the first large-scale dataset of advertisement billboards, captured in outdoor scenes. We also benchmark several state-of-the-art semantic segmentation algorithms on our proposed dataset.

We analyze clouds in the earth's atmosphere using ground-based sky cameras. An accurate segmentation of clouds in the captured sky/cloud image is difficult, owing to the fuzzy boundaries of clouds. Several techniques have been proposed, which use color as the discriminatory feature for cloud detection. In the existing literature, however, analysis of daytime and nighttime images is considered separately, mainly because of differences in image characteristics and applications. In this letter, we propose a lightweight deep-learning architecture called CloudSegNet. It is the first that integrates daytime and nighttime (also known as nychthemeron) image segmentation in a single framework and achieves state-of-the-art results on public databases.

In recent years, there has been growing interest in using precipitable water vapor (PWV) derived from global positioning system (GPS) signal delays to predict rainfall. However, the occurrence of rainfall is dependent on a myriad of atmospheric parameters. This paper proposes a systematic approach to analyze various parameters that affect precipitation in the atmosphere. Different ground-based weather features such as Temperature, Relative Humidity, Dew Point, Solar Radiation, PWV along with Seasonal and Diurnal variables are identified, and a detailed feature correlation study is presented. While all features play a significant role in rainfall classification, only a few of them, such as PWV, Solar Radiation, Seasonal, and Diurnal features, stand out for rainfall prediction. Based on these findings, an optimum set of features are used in a data-driven machine learning algorithm for rainfall prediction. The experimental evaluation using a 4-year (2012-2015) database shows a true detection rate of 80.4%, a false alarm rate of 20.3%, and an overall accuracy of 79.6%. Compared to the existing literature, our method significantly reduces the false alarm rates.

Numerous weather parameters affect the occurrence and amount of rainfall. Therefore, it is important to study these parameters and their interdependency. In this paper, different weather and time-related variables - relative humidity, solar radiation, temperature, dew point, day-of-year, and time-of-day are analyzed systematically using Principal Component Analysis (PCA). We found that four principal components explain a cumulative variance of 85%. The first two principal components are applied to distinguish rain and no-rain scenarios as well. We conclude that all 7 variables have similar contribution towards rainfall detection.

Precipitation is dependent on a myriad of atmospheric conditions. In this paper, we study how certain atmospheric parameters impact the occurrence of rainfall. We propose a data-driven, machine-learning based methodology to detect precipitation using various meteorological sensor data. Our approach achieves a true detection rate of 87.4% and a moderately low false alarm rate of 32.2%.

In this paper, the Precipitable Water Vapor (PWV) content of the atmosphere is derived using the Global Positioning System (GPS) signal delays. The PWV values from GPS are calculated at different elevation cut-off angles. It was found that the significant range of elevation cut-off angles is from 5° to 50°. The PWV values calculated from GPS using varying cutoff angles from this range were then compared to the PWV values calculated using the radiosonde data. The correlation coefficient and the Root Mean Square (RMS) error between the GPS and radiosonde derived PWV decreases with the increasing cut-off angle and the distance between the two. The seasonal parameters also effect the relation between the two.

Imaging the atmosphere using ground-based sky cameras is a popular approach to study various atmospheric phenomena. However, it usually focuses on the daytime. Nighttime sky/cloud images are darker and noisier, and thus harder to analyze. An accurate segmentation of sky/cloud images is already challenging because of the clouds' non-rigid structure and size, and the lower and less stable illumination of the night sky increases the difficulty. Nonetheless, nighttime cloud imaging is essential in certain applications, such as continuous weather analysis and satellite communication. In this paper, we propose a superpixel-based method to segment nighttime sky/cloud images. We also release the first nighttime sky/cloud image segmentation database to the research community. The experimental results show the efficacy of our proposed algorithm for nighttime images.

Cloud base height and volume estimation is needed in meteorology and other applications. We have deployed a pair of custom-designed Whole Sky Imagers, which capture stereo pictures of the sky at regular intervals. Using these images, we propose a method to create rough 3D models of the base of clouds, using feature point detection, matching, and triangulation. The novelty of our method lies in the fact that it locates the cloud base in all three dimensions, instead of only estimating cloud base height. For validation, we compare the results with measurements from weather radar.

Color channel selection is essential for accurate segmentation of sky and clouds in images obtained from ground-based sky cameras. Most prior works in cloud segmentation use threshold-based methods on color channels selected in an ad hoc manner. In this letter, we propose the use of rough sets for color channel selection in visible-light images. Our proposed approach assesses color channels with respect to their contribution for segmentation and identifies the most effective ones.

Ground-based whole sky imagers (WSIs) can provide localized images of the sky of high temporal and spatial resolution, which permits fine-grained cloud observation. In this paper, we show how images taken by WSIs can be used to estimate solar radiation. Sky cameras are useful here because they provide additional information about cloud movement and coverage, which are otherwise not available from weather station data. Our setup includes ground-based weather stations at the same location as the imagers. We use their measurements to validate our methods.

Precipitable Water Vapor (PWV) is a good source to monitor precipitation. It is defined by the amount of water vapor present in atmosphere. Traditionally, radiosondes and microwave radiometers were used to derive PWV. However, these devices have poor temporal resolutions and high operational costs. Therefore, GPS signal delay is now widely used for such purposes. The main aim of this paper is to study relationship between GPS derived PWV and precipitation. We present an analysis which shows that PWV increases before any rainfall event, while it decreases after the rainfall event. We also derive a threshold PWV that detects the occurrence of rainfall, once PWV exceeds the threshold value. PWV and rainfall data of June 2010 and 2011 are used for validation.

Ground-based Whole Sky Imagers (WSIs) are now commonly used for cloud observations. Upon deployment, they may not be exactly level or precisely face north. This significantly affects subsequent processing of the images, especially for applications where two or more imagers are required, e.g. 3D volumetric cloud reconstruction. We present a method to remove this mis-alignment using the sun position in images captured over a whole day. Coupled with precise coordinates of the device locations, this method also improves the geo-referencing accuracy of the captured images. We detect the sun in the images and compute the corresponding 3D vectors using the lens calibration function. These vectors are compared to the actual sun direction. The mismatch between the two sets of vectors is then corrected using a 3D rotation matrix. The method can also be applied to other celestial bodies, such as stars or the moon.

We propose a modified texton-based classification approach that integrates both color and texture information for improved classification results. We test our proposed method for the task of cloud classification on SWIMCAT, a large new database of cloud images taken with a ground-based sky imager, with very good results. We perform an extensive evaluation, comparing different color components, filter banks, and other parameters to understand their effect on classification accuracy. Finally, we release the SWIMCAT dataset that was created for the task of cloud categorization.

Sky/cloud images captured by ground-based Whole Sky Imagers (WSIs) are extensively used now-a-days for various applications. In this paper, we learn the semantics of sky/cloud images, which allows an automatic annotation of pixels with different class labels. We model the various labels/classes with a continuous-valued multi-variate distribution. Using a set of training images, the distributions for different labels are learnt, and subsequently used for labeling test images. We also present a method to determine the number of clusters. Our proposed approach is the first for multi-class sky-cloud image annotation and achieves very good results.

Ground-based whole sky imagers are popular for monitoring cloud formations, which is necessary for various applications. We present two new Wide Angle High-Resolution Sky Imaging System (WAHRSIS) models, which were designed especially to withstand the hot and humid climate of Singapore. The first uses a fully sealed casing, whose interior temperature is regulated using a Peltier cooler. The second features a double roof design with ventilation grids on the sides, allowing the outside air to flow through the device. Measurements of temperature inside these two devices show their ability to operate in Singapore weather conditions. Unlike our original WAHRSIS model, neither uses a mechanical sun blocker to prevent the direct sunlight from reaching the camera; instead they rely on high-dynamic-range imaging (HDRI) techniques to reduce the glare from the sun.

Fine scale cloud monitoring using ground-based imagers is becoming popular for a variety of applications and domains. We present a framework for cloud base height estimation using two such imagers; our method is based on stereoscopic scene flow. We demonstrate the feasibility of our approach and use computer-generated images with controlled cloud height to validate the accuracy of our method.

Sky/cloud imaging using ground-based Whole Sky Imagers (WSI) is a cost-effective means to understanding cloud cover and weather patterns. The accurate segmentation of clouds in these images is a challenging task, as clouds do not possess any clear structure. Several algorithms using different color models have been proposed in the literature. This paper presents a systematic approach for the selection of color spaces and components for optimal segmentation of sky/cloud images. Using mainly principal component analysis (PCA) and fuzzy clustering for evaluation, we identify the most suitable color components for this task.