As Artificial Intelligence (AI) is revolutionizing various industries and applications, understanding the hardware requirements and energy consumption of AI pipelines in Big Data (BD) applications has become increasingly essential. This paper presents a comprehensive, scalable framework, designed to systematically measure hardware resources, energy usage, and model performance across two prominent data modalities: tabular data and images. The framework is generalizable, facilitating replicability across the AI research community, and encourages the deployment of AI models with comprehensive metrics beyond traditional accuracy, promoting the optimization of pipelines for real-world scenarios. Through detailed benchmarking, we identify EfficientNet as a standout model for image classification, and XGBoost for tabular data, both excelling in their respective domains. Notably, our findings show that Graphics Processing Units (GPUs) account for approximately 90% of total energy consumption in image-based tasks, while Central Processing Units (CPUs) are responsible for around 50% of energy use in tabular data processing. The merit of our innovative proposed framework combines information theory and probability theory to enhance our understanding of AI model performance in Edge-to-Cloud (E2C) applications that demand efficient Big Data processing in distributed environments. By seamlessly integrating energy efficiency with hardware optimization, it enables realtime monitoring of energy consumption and computing resources in containerized environments, providing precise insights for optimizing AI workloads. This framework facilitates scalable AI deployment on resource-constrained edge devices, reducing energy consumption while enhancing AI model robustness and interpretability, thereby promoting greater trust and transparency in AI-powered decision-making for critical real-world applications. This emphasizes the importance of multi-objective optimization for more sustainable and efficient Big Data AI workflows.

We present a dataset of open source software developed mainly by enterprises rather than volunteers. This can be used to address known generalizability concerns, and, also, to perform research on open source business software development. Based on the premise that an enterprise's employees are likely to contribute to a project developed by their organization using the email account provided by it, we mine domain names associated with enterprises from open data sources as well as through white- and blacklisting, and use them through three heuristics to identify 17 264 enterprise GitHub projects. We provide these as a dataset detailing their provenance and properties. A manual evaluation of a dataset sample shows an identification accuracy of 89%. Through an exploratory data analysis we found that projects are staffed by a plurality of enterprise insiders, who appear to be pulling more than their weight, and that in a small percentage of relatively large projects development happens exclusively through enterprise insiders.

Nowadays, on-line testing is essential for modern microprocessors to detect latent defects that either escape manufacturing testing or appear during system operation. Small memories, such as L1 caches and translation lookaside buffers (TLBs) are not usually equipped with memory built-in self-test (MBIST) hardware. Software-based self-test (SBST) is a flexible and low-cost solution for on-line March test application and error detection in such small memories. Although, L1 caches and TLBs are small components, their reliable operation is crucial for the system performance due to the large penalties caused when L1 cache or TLB misses occur. In this paper, an SBST program development methodology is proposed for on-line testing of small cache memories in microprocessors. To overcome testability challenges that are due to the “hidden” or implicit operation of such memories, the proposed SBST methodology exploits: 1) existing special purpose instructions that modern instruction set architectures implement to access these cache arrays for debug-diagnostic (DD) purposes, termed hereafter direct cache access instructions and 2) performance monitoring and trap handling mechanisms. Besides, the proposed SBST methodology combines features that are crucial for on-line testing: a) compact test validation; b) simplified coding style; c) low invasiveness of the test program; and d) small memory footprint. The methodology is comprehensively demonstrated on the instruction and data L1 cache arrays and the instruction and data TLB arrays of OpenSPARC T1. Experimental results show that the exploitation of such DD instructions has a significant improvement of test time (up to 86% for instruction L1 cache, up to 87% for the data L1 cache, up to 37% for D-TLB, and up to 91% for I-TLB) when compared to SBST solutions that do not utilize these types of instructions.