In production environments where applications generate vast data, minimizing downtime is critical. However, large-scale log data can overwhelm storage and computational resources, making system stability challenging to maintain. Poor logging practices worsen this by creating excessive, irrelevant, or unstructured logs that hinder efficient application-crash detection and resolution. This is an enterprise-wide issue often leading to resource exhaustion and prolonged downtime.To address this, we propose Log Sculptor, a tool that leverages GenAI (Large Language Models) to proactively analyze, identify, and improve logging practices in code. It provides recommendations for log statement adjustments (rectifications, additions, removals) and can apply these to produce code with optimized logs. Log Sculptor operates on a prompt-based approach.We benchmark Log Sculptor on nine open-source codebases annotated by industry practitioners for logging practices. It achieves results comparable to human experts and suggests further refinements to enhance logging. We analyze the cost-benefit trade-offs, demonstrating the potential of GenAI in transforming logging practices, improving debugging efficiency, reducing downtime, and increasing reliability in production environments.

Hyperspectral images (HSI) have a huge size, which makes their processing through neural networks cumbersome. We propose novel approximate computing techniques that leverage physical properties of the reflectance spectra to accelerate the processing of HSI images. This makes the images interpretable across various applications. We propose three spectral dimensionality reduction techniques. These techniques use spectral clustering methods that rely on reflectance values to capture inherent characteristics from hyperspectral images across diverse domains. We also evaluate existing spatial dimension reduction techniques and a combination of spatial + spectral dimension reduction techniques. We conduct extensive experiments on three real-world open-source datasets, encompassing urban and rural landscapes. Our techniques reduce the training time by up to 8x and inference time by up to 5x, while reducing the model size by up to 3x. Our techniques have a negligible impact on accuracy. By contrast, PCA and MNF techniques incur 3X higher pre-processing latency overheads than our techniques and also degrade the accuracy. Our techniques are promising for addressing the computational challenges of HSI processing.

With the advent of big data, training deep learning models quickly has gained prime significance. The faster a model is trained, the more relevant are its predictions in a given context. Deep learning is used for non structured data such as images, videos, sounds, text corpus, all of which represent a huge volume of data and also use complex models. Training these workloads can often take days or even weeks, because of various factors such as size of data, complexity of model, network and the underlying hardware infrastructure. The recognized divide and conquer solution to expedite the training process is to distribute either data or the model. Alas, the challenges of a distributed training setup are well known - creating and maintaining a cluster, enabling data or model parallelism along with uninterrupted communication across the cluster nodes.In this paper, we focus on two lightweight libraries for distributed deep learning, RaySGD and Horovod, which aim to alleviate these challenges by providing support for seamless parallellization. We conduct an in-depth benchmarking exercise to evaluate the performance of both libraries for training time(latency) incurred. Our experiments are conducted on a combination of various parameters such as hardware setup (CPU or GPU based), standard and manually coded models, real world and synthetic datasets. We also vary batch sizes of large workloads and number of worker nodes in a distributed setting. The insights obtained from our experiments act as guidelines for data scientists, facilitating the decision making process when conducting distributed training of big data applications on RaySGD or Horovod.

Recommender systems today face major challenges in keeping up with dynamic customer preferences. Disruptions or sudden changes in the environment affect customer preferences drastically and render historical data ineffective for modeling. With businesses relying heavily on Machine Learning(ML) based recommender systems for catering to customer preferences, the accuracy of timely recommendations gains prime significance.