Tactile perception is essential for embodied agents to understand the physical attributes of objects that cannot be determined through visual inspection alone. While existing methods have made progress in visual and language modalities for physical understanding, they fail to effectively incorporate tactile information that provides crucial haptic feedback for real-world interaction. In this paper, we present VTV-LLM, the first multi-modal large language model that enables universal Visuo-Tactile Video (VTV) understanding, bridging the gap between tactile perception and natural language. To address the challenges of cross-sensor and cross-modal integration, we contribute VTV150K, a comprehensive dataset comprising 150,000 video frames from 100 diverse objects captured across three different tactile sensors (GelSight Mini, DIGIT, and Tac3D), annotated with four fundamental tactile attributes (hardness, protrusion, elasticity, and friction). We develop a novel three-stage training paradigm that includes VTV enhancement for robust visuo-tactile representation, VTV-text alignment for cross-modal correspondence, and text prompt finetuning for natural language generation. Our framework enables sophisticated tactile reasoning capabilities including feature assessment, comparative analysis, and scenario-based decision-making. Extensive experimental evaluations demonstrate that VTV-LLM achieves superior performance in tactile reasoning tasks, establishing a foundation for more intuitive human-machine interaction in tactile domains.

Generating spoken word embeddings that possess semantic information has attracted lots of research interest. Among them, Speech2vec, as one of the most influential works, has reported impressive results of surpassing Word2Vec on word similarity benchmarks. However, since their breakthrough in 2017, this field seems to have stalled. There are no subsequent comparisons, successors, and even successful replications. We think Speech2vec may be overestimated since intrinsic interferences exist between phonetics and semantics, preventing the model from learning effective semantic embeddings. In this study, we first examined the authenticity of Speech2Vec. Proofs on embedding properties and vocabulary compositions suggested that their claimed results may be wrongly produced by a text-based model. In addition, we reproduced the Speech2Vec model and reported the replicable results to set a practical baseline for future developments. Our codes and data are available1.

Leveraging deep learning to explore the associations between voices and faces has attracted extensive research interest. Usually, this research is formalized by cross-modal verification, matching, and retrieval tasks. Most works rely on a single local or global optimization objective for learning, ignoring that testing scenarios may prefer different optimization objectives. For example, local objectives are more helpful for verification and matching tasks, while global objectives contribute more to retrieval. In this study, we proposed a learning framework based on local and global objectives to improve the generalizability of the learned representations. Firstly, we explored two ways of applying supervised contrastive loss (SCL) to learn voice-face representations. Secondly, we designed a contrastive-form global optimization objective, which shows better performance and training efficiency. Experiments on the VoxCeleb dataset demonstrate the effectiveness of our framework.

Learning associations between voices and faces have recently attracted much research interest. Previous studies mainly focused on improving loss function design for learning better representations, neglecting that informative inputs are the prerequisite of association findings and effective representation alignments. Based on this motivation, we proposed an unsupervised transformer-based learning framework, which fuses the knowledge of readily available single-modal expert models to learn better representations. Benefiting from higher-quality inputs, a simple noise contrastive estimation (NCE) loss is used for training. In addition, we proposed a statistical batch construction (SBC) strategy, which obtains high-quality negative samples during the unsupervised learning process. Experiments on the VoxCeleb1 dataset demonstrate the effectiveness of our framework. It yields SOTA results in voice-face verification, matching, and retrieval tasks. In the challenging gender-constrained matching tasks, it achieves over 81% (ACC), which is 5% higher than the previous.

Voice-face association learning (VFAL) aims to tap into the potential connections between voices and faces. Most studies currently address this problem in a supervised manner, which cannot exploit the wealth of unlabeled video data. To solve this problem, we propose an unsupervised learning framework: Self-Lifting (SL), which can use unlabeled video data for learning. This framework includes two iterative steps of "clustering" and "metric learning". In the first step, unlabeled video data is mapped into the feature space by a coarse model. Then unsupervised clustering is leveraged to allocate pseudo-label to each video. In the second step, the pseudo-label is used as supervisory information to guide the metric learning process, which produces the refined model. These two steps are performed alternately to lift the model's performance. Experiments show that our framework can effectively use unlabeled video data for learning. On the VoxCeleb dataset, our approach achieves SOTA results among the unsupervised methods and has competitive performance compared with the supervised competitors. Our code is released on Github.

Numerous crowdsourcing applications have emerged in our daily lives, which enable customers to outsource their complicated tasks to a crowd of workers. However, the information of task tags and worker profiles is explicitly obtained by the crowdsourcing server to recommend tasks effectively, which violates the privacy of both customers and workers. Moreover, the worker's ability to do the task should also be verified in a privacy-preserving way. To address these issues, we propose a privacy-preserving interest-ability based task allocation scheme in crowdsourcing, which protects both task and worker privacy and enables the crowdsourcing server to allocate tasks in a fine-grained way. Specifically, by utilizing attribute-based encryption (ABE) and proxy re-encryption based searchable encryption (PRE-SE) on the task content and task tags respectively, customers are able to enforce fine-grained ability requirements on their tasks, and workers can specify flexible interests to choose their desired tasks. Additionally, ElGamal signature enables workers to prove their abilities to the crowdsourcing server without revealing the task content. Numerical analysis and experiment results demonstrate that our proposed scheme is efficient in terms of computation and storage overhead and is practical to be implemented in crowdsourcing.

Discovery Service (DS), used to trace the movement of objects along supply chains, is an important component of the Internet of Things (IoT). DS is faced with the challenge of huge volume of data as well as large amount of parallel requests of discovery and publish. This paper proposes a new central-indexing DS system using distributed NOSQL database HBase to better support big data and parallel processing. A new storage schema of DS data is designed to optimize the discovery efficiency of DS records. The new storage schema uses object ID as row key, event timestamp as column identifier and event index content as cell value. The typical recursive discovery algorithm, which is needed in DS but often neglected in previous works, is specified to realize the full tracing of object's movement. A prototype of the DS system proposed is implemented. The experiments show that the number of concurrent discoveries the proposed DS can handle per second is about 200 times that of the DS based on RDBMS, and the number of concurrent publishes the proposed DS can process per second is about five times that of the DS based on RDBMS.

Addressing power and energy consumption related issues early in the system design flow ensures good design and minimizes iterations for faster turnaround time. In particular, optimizations at software level, e.g., those supported by compilers, are very important for minimizing energy consumption of embedded applications. Recent research demonstrates that voltage islands provide the flexibility to reduce power by selectively shutting down the different regions of the chip and/or running the select parts of the chip at different voltage/frequency levels. As against most of the prior work on voltage islands that mainly focused on the architecture design and IP placement related issues, this paper studies the necessary software compiler support for voltage islands. Specifically, we focus on an embedded multiprocessor architecture that supports both voltage islands and control domains within these islands, and determine how an optimizing compiler can automatically map an embedded application onto this architecture. Such an automated support is critical since it is unrealistic to expect an application programmer to reach a good mapping correlating multiple factors such as performance and energy at the same time. Our experiments with the proposed compiler support show that our approach is very effective in reducing energy consumption. The experiments also show that the energy savings we achieve are consistent across a wide range of values of our major simulation parameters.

The consumer-oriented technologies categorized as social media and social networking are transforming the way that people communicate and accelerating the spread of information. Small and medium sized enterprises (SMEs) lack the capability of finance and technology, and can not afford to establish a collaborative portal for partners. With social network service increasing rapidly and being used popularly, an enterprise social collaboration platform and system architecture are proposed for improving the communication and cooperation of SMEs. Synchronization method of product catalogue which is extracted from ERP or PDM is described to share product information among partners. Social project collaboration based on social networking is analyzed, and function of social project management is designed. A social collaboration platform (www.yonyou.com) has developed by Ufida Software Company which provides enterprise profile, micro blogging, blog, group, project management, product catalog sharing, document workspace, instant message.

Embedded computing platforms are often resource constrained, requiring great design and implementation attention to memory-power-, and heat-related parameters. An important task for a compiler in such platforms is to simplify the process of developing applications for limited memory devices and resource-constrained clients. Focusing on array-intensive embedded applications to be executed on single CPU-based architectures, this work explores how loop-based compiler optimizations can be used for increasing memory location reuse. Our goal is to transform a given application in such a way that the resulting code has fewer cases (as compared to the original code), where the lifetimes of array elements overlap. The reduction in lifetimes of array elements can then be exploited by reusing memory locations as much as possible. Our experimental results indicate that the proposed strategy reduces data space requirements of 15 resource constrained applications by more than 40%, on average. We also demonstrate how this strategy can be combined with data locality (cache behavior)--enhancing techniques so that a compiler can take advantage of both, that is, reduce data memory requirements and improve data locality at the same time.

The problem attacked in this paper is one of automatically mapping an application onto a Network-on-Chip (NoC) based chip multiprocessor (CMP) architecture in a locality-aware fashion. The proposed compiler approach has four major steps: task scheduling, processor mapping, data mapping, and packet routing. In the first step, the application code is parallelized and the resulting parallel threads are assigned to virtual processors. The second step implements a virtual processor-to-physical processor mapping. The goal of this mapping is to ensure that the threads that are expected to communicate frequently with each other are assigned to neighboring processors as much as possible. In the third step, data elements are mapped to memories attached to CMP nodes. The main objective of this mapping is to place a given data item into a node which is close to the nodes that access it. The last step of our approach determines the paths (between memories and processors) for data to travel in an energy efficient manner. In this paper, we describe the compiler algorithms we implemented in detail and present an experimental evaluation of the framework. In our evaluation, we test our entire framework as well as the impact of omitting some of its steps. This experimental analysis clearly shows that the proposed framework reduces energy consumption of our applications significantly (27.41% on average over a pure performance oriented application mapping strategy) as a result of improved locality of data accesses.

As compared to a large spectrum of performance optimizations, relatively less effort has been dedicated to optimize other aspects of embedded applications such as memory space requirements, power, real-time predictability, and reliability. In particular, many modern embedded systems operate under tight memory space constraints. One way of addressing this constraint is to compress executable code and data as much as possible. While researchers on code compression have studied efficient hardware and software based code compression strategies, many of these techniques do not take application behavior into account; that is, the same compression/decompression strategy is used irrespective of the application being optimized. This article presents an application-sensitive code compression strategy based on control flow graph (CFG) representation of the embedded program. The idea is to start with a memory image wherein all basic blocks of the application are compressed, and decompress only the blocks that are predicted to be needed in the near future. When the current access to a basic block is over, our approach also decides the point at which the block could be compressed. We propose and evaluate several compression and decompression strategies that try to reduce memory requirements without excessively increasing the original instruction cycle counts. Some of our strategies make use of profile data, whereas others are fully automatic. Our experimental evaluation using seven applications from the MediaBench suite and three large embedded applications reveals that the proposed code compression strategy is very successful in practice. Our results also indicate that working at a basic block granularity, as opposed to a procedure granularity, is important for maximizing memory space savings.

The problem attacked in this paper is one of automatically mapping an application onto a Network-on-Chip (NoC) based chip multi-processor architecture in a locality-aware fashion. The proposed compiler approach has four major steps: task scheduling, processor mapping, data mapping, and packet routing. Our experimental result clearly shows that the proposed framework reduces energy consumption of our applications significantly (27.41% on average over a pure performance oriented application mapping strategy) as a result of improved locality of data accesses.

Reducing energy consumption of a Network-on-Chip (NoC) is a critical design goal, especially for power-constrained embedded systems.In response, prior research has proposed several circuit/architectural level mechanisms to reduce NoC power consumption. This paper considers the problem from a different perspective and demonstrates that compiler analysis can be very helpful for enhancing the effectiveness of a hardware-based link power management mechanism by increasing the duration of communication links' idle periods. The proposed profile-based approach achieves its goal by maximizing the communication link reuse through compiler-directed, static message re-routing. That is, it clusters the required data communications into a small set of communication links at any given time, which increases the idle periods for the remaining communication links in the network. This helps hardware shut down more communication links and their corresponding buffers to reduce leakage power. The current experimental evaluation, with twelve data-intensive embedded applications, shows that the proposed profile-driven compiler approach reduces leakage energy by more than 35% (on average) as compared to a pure hardware-based link power management scheme.

Disk subsystem is known to be a major contributor to overall power consumption of high-end parallel systems. Past research proposed several architectural-level techniques to reduce disk power by taking advantage of idle periods experienced by disks. Although such techniques have been known to be effective in certain cases, they share a common drawback: they operate in a reactive manner, i.e., they control disk power by observing past disk activity (for example, idle and active periods) and estimating future ones. Consequently, they can miss opportunities for saving power and incur significant performance penalties due to inaccuracies in predicting idle and active times. Motivated by this observation, this paper proposes and evaluates a compiler-driven approach to reducing disk power consumption of array-based scientific applications executing on parallel architectures. The proposed approach exposes disk layout information to the compiler, allowing it to derive the disk access pattern, i.e., the order in which parallel disks are accessed. This paper demonstrates two uses of this information. First, we can implement proactive disk power management, i.e., we can select the most appropriate power-saving strategy and disk-preactivation strategy based on the compiler-predicted future idle and active periods of parallel disks. Second, we can restructure the application code to increase the length of idle disk periods, which leads to better exploitation of available power-saving capabilities. We implemented both these approaches within an optimizing compiler and tested their effectiveness using a set of benchmark codes from the Spec 2000 suite and a disk power simulator. Our results show that the compiler-driven disk power management is very promising. The experimental results also reveal that, although proactive disk power management is very effective, code restructuring for disk power achieves additional energy savings across all the benchmarks tested, and these savings are very close to optimal savings that can be obtained through an integer linear programming (ILP)-based scheme.

Embedded systems in the form of vehicles and mobile devices such as wireless phones, automatic banking machines and new multi-modal devices operate under tight memory and power constraints. Therefore, their performance demands must be balanced very well against their memory space requirements and power consumption. Automatic tools that can optimize for memory space utilization and performance are expected to be increasingly important in the future as increasingly larger portions of embedded designs are being implemented in software. In this paper, we describe a novel optimization framework that can be used in two different ways: (i) deciding a suitable on-chip memory capacity for a given code, and (ii) restructuring the application code to make better use of the available on-chip memory space. While prior proposals have addressed these two questions, the solutions proposed in this paper are very aggressive in extracting and exploiting all data reuse in the application code, restricted only by inherent data dependences

Memory system presents one of the critical challenges on embedded system design and optimization. This is mainly due to ever-increasing code complexity of embedded applications and exponential increase witnessed in the amount of data they manipulate. As a result, reducing memory space occupancy of embedded applications is very important and is even more important in the next decade. Motivated by this observation, this paper presents and evaluates a compiler-driven approach to data compression for reducing memory space occupancy. Our goal in this paper is to study how automated compiler support can help in deciding the set of data elements to compress/decompress and the points during execution at which these compressions/decompressions should be performed. The proposed compiler support achieves this by analyzing the source code of the application to be optimized and identifying the order in which the different data blocks are accessed. Based on this analysis, the compiler then automatically inserts compression/decompression calls in the application code. The compression calls target the data blocks that are not expected to be used in the near future, whereas the decompression calls target those data blocks with expected reuse but currently in compressed form.

Many of today's data-intensive applications manipulate disk-resident data sets. As a result, their overall behavior is tightly coupled with their disk performance. Unfortunately, most of these applications quickly become disk bound since disk I/O times, the communication latencies, and energy consumption required to transfer disk data to the host machine can be very large. A promising solution to this problem is to embed computational power into the disk storage system. This paper concentrates on such a smart disk based architecture and proposes an automated approach that partitions a given application code between the host machine and the smart disk. The main goal is to perform data filterings, identified at compile time, on the smart disk, thereby reducing the energy spent in communicating disk data to the host unit for processing. To achieve this, the proposed approach uses integer linear programming to identify the code fragments that perform significant data filtering and assigns such fragments to the smart disk for execution. In addition to the communication energy benefits of the proposed approach, we show in this paper that this approach can also help us better exploit the low-power management capabilities provided by the system. Our experiments with four data-intensive applications indicate significant energy savings.

Power consumption of large servers and clusters has recently been a popular research topic, since this issue is important from both technical and environmental viewpoints. The prior research proposed disk power management as one of the important ways of reducing overall power of a large system and considered both hardware-based and software-guided disk power reduction schemes. One of the common characteristics of the previously proposed approaches to disk power reduction is that they work with a given disk access pattern. In comparison, the goal of the approach proposed in this paper is to restructure application code using an optimizing compiler so that disk idle periods are lengthened. This in turn allows the underlying disk power management scheme to be more effective since such schemes usually prefer the long idle periods over the short ones. Our approach targets at large scientific applications that operate on disk-resident arrays using nested loops and exhibit regular data access patterns. To test the effectiveness of the proposed approach, we implemented it within an optimizing compiler and performed experiments with six data-intensive applications that manipulate disk-resident data. Our experimental analysis shows that the proposed approach is very successful in practice and reduces the total disk energy consumption on average by 18.17%, as compared to an execution without any disk power management, and by 11.55%, as compared to an execution that employs disks with low-power capabilities without our code restructuring approach.

This paper proposes compiler-based leakage optimization strategy for on-chip scratch-pad memories (SPMs). The idea is to keep only a small set of SPM regions active at a given time and pre-activate SPM regions based on the compiler-extracted data access pattern. Our strategy, called activity clustering, increases the length of the idle periods of SPM regions by clustering accesses to a small set of regions at a time. It thus allows an SPM to take better advantage of the underlying leakage optimization mechanism

Code and data duplication has been identified as one of the important mechanisms for improving reliability. In a chip multiprocessor-based execution environment, while it is possible to hide the overhead of code duplication through parallelism, hiding the memory space overhead incurred by data duplication is more difficult. This paper presents a compiler-directed memory-conscious data duplication scheme that tries to minimize the extra memory space required by duplicate execution. The proposed approach achieves this goal by using the memory locations that hold dead data to store the duplicates of the actively-used data. In this way, instead of using extra memory storage for duplicate elements, we use the existing memory locations to the extent allowed by usage patterns of data. The results collected from our experiments clearly show that the proposed approach saves significant memory space, as compared to a straightforward approach that implements full duplication

While scalable NoC (Network-on-Chip) based communication architectures have clear advantages over long point-to-point communication channels, their power consumption can be very high. In contrast to most of the existing hardware-based efforts on NoC power optimization, this paper proposes a compiler-directed approach where the compiler decides the appropriate voltage/frequency levels to be used for each communication channel in the NoC. Our approach builds and operates on a novel graph based representation of a parallel program and has been implemented within an optimizing compiler and tested using 12 embedded benchmarks. Our experiments indicate that the proposed approach behaves better - from both performance and power perspectives - than a hardwarebased scheme and the energy savings it achieves are very close to the savings that could be obtained from an optimal, but hypothetical voltage/frequency scaling scheme.

Increasing complexity in the communication patterns of embedded applications parallelized over multiple processing units makes it difficult to continue using the traditional bus-based on-chip communication techniques. The main contribution of this paper is to demonstrate the importance of compiler technology in reducing power consumption of applications designed for emerging multi processor, NoC (Network-on-Chip) based embedded systems. Specifically, we propose and evaluate a compiler-directed approach to NoC power management in the context of array-intensive applications, used frequently in embedded image/video processing. The unique characteristic of the compiler-based approach proposed in this paper is that it increases the idle periods of communication channels by reusing the same set of channels for as many communication messages as possible. The unused channels in this case take better advantage of the underlying power saving mechanism employed by the network architecture. However, this channel reuse optimization should be applied with care as it can hurt performance if two or more simultaneous communications are mapped onto the same set of channels. Therefore, the problem addressed in this paper is one of reducing the number of channels used to implement a set of communications without increasing the communication latency significantly. To test the effectiveness of our approach, we implemented it within an optimizing compiler and performed experiments using twelve application codes and a network simulation environment. Our experiments show that the proposed compiler-based approach is very successful in practice and works well under both hardware based and software based channel turn-off schemes.

MOTIVATION: Biologists frequently align multiple biological sequences to determine consensus sequences and/or search for predominant residues and conserved regions. Particularly, determining conserved regions in an alignment is one of the most important activities. Since protein sequences are often several-hundred residues or longer, it is difficult to distinguish biologically important conserved regions (motifs or domains) from others. The widely used tools, Logos, Al2co, Confind, and the entropy-based method, often fail to highlight such regions. Thus a computational tool that can highlight biologically important regions accurately will be highly desired. RESULTS: This paper presents a new scoring scheme ARCS (Aggregated Related Column Score) for aligned biological sequences. ARCS method considers not only the traditional character similarity measure but also column correlation. In an extensive experimental evaluation using 533 PROSITE patterns, ARCS is able to highlight the motif regions with up to 77.7% accuracy corresponding to the top three peaks. AVAILABILITY: The source code is available on http://bio.informatics.indiana.edu/projects/arcs and http://goldengate.case.edu/projects/arcs

Reconfigurable computing systems are becoming increasingly widespread as they bring the flexibility of programmable systems and approach the performance of ASICs. While the prior research on FPGAs mainly studied issues such as performance, power, and area optimization, reliability related issues have not taken much attention. However, with increasing soft error rates, providing resilience to soft errors in FPGA based embedded platforms is becoming an increasingly important issue. This paper proposes an OS-directed task duplication scheme for increasing reliability by providing resilience against soft errors. The idea is to exploit the unused portions of the FPGA space to schedule duplicates of active tasks. The outputs of the primary and duplicate tasks are compared to check for the existence of soft errors.

Chip multiprocessors are gaining popularity as they are very suitable for data-intensive embedded and high-end processing. In particular, array-intensive embedded image and video applications can benefit a lot from these architectures due to coarse-grain parallelization they offer. However, if not optimized, interprocessor communication can be a major energy consumer. Focusing on a distributed memory chip multiprocessor architecture and array-intensive embedded applications, this paper proposes a compiler-based communication minimization strategy based on data replication. The proposed scheme replicates shared data items across the memories of the processors in a controlled fashion (i.e., under a memory limit), with the goal of eliminating the otherwise necessary interprocessor communication.

Many implementations of Java Virtual Machine (JVM) use optimizing dynamic compilation for improving performance of Java applications. However, optimizing compilation is expensive in terms of both time and memory space complexity. Annotations are frequently used to reduce the dynamic compilation overhead in embedded Java environments where the memory budget and CPU speed are limited. However, some annotations can cause the dynamic compiler to generate code that violates the security policy of the JVM. These annotations present a safety problem if they are not verified before being used by the compiler. In this work, we present annotation schemes for three frequently used optimizations: null pointer check removal, value liveness analysis, and array bound check removal. All our annotations can be verified at low cost during dynamic compilation time to guarantee that they do not cause the compiler to generate code that violates the security policy of the JVM.

Increasing use of parallel computation platforms (both off-chip and on-chip) makes communication analysis and optimization an important target. While there have been numerous studies that target network performance of parallel architectures, the efforts that target network power consumption (in terms of both modeling and optimization) are relatively new. One of the common characteristics of most of the prior approaches to network power management is that they are hardware-based and reactive in the sense that they manage power consumption of the network as a response to observed message traffic. Consequently, they can miss important opportunities for saving power and can incur performance penalties due to inaccuracies in predicting future idle and active times of communication links. Motivated by this observation, this paper proposes a compiler-directed proactive approach to network power management for the class of loop-intensive applications running on small-sized networks used exclusively by a single embedded application at a time.As compared to hardware-based approaches, the proposed compiler-directed approach has two potential benefits. First, based on high-level communication analysis, it determines the points at which a given communication link is idle and can be turned off (i.e., powered down) to save power. Therefore, an idle link can be put in the low-power state without waiting for a certain period of time to make sure that the link has really become idle (as in the case of hardware schemes). Second, since the compiler can also determine the point at which a turned-off link will be needed in the future, it can pre-activate it (i.e., before it is actually needed) to eliminate the turn on (reactivation) performance penalty. Our simulations with seven array-intensive applications and an embedded on-chip network clearly show that the proposed compiler-directed approach is better than a hardware-based scheme from both power and performance perspectives.

Dual-execution/checkpointing based transient error tolerance techniques have been widely used in the high-end mission critical systems. These techniques, however, are not very attractive for cost-sensitive embedded systems because they require extra resources (e.g., large memory, special hardware, etc), and thus increase overall cost of the system. In this paper, we propose a transient error tolerant Java Virtual Machine (JVM) implementation for embedded systems. Our JVM uses dual-execution and checkpointing to detect and recover from transient errors. However, our technique does not require any special hardware support (except for the memory page protection mechanism, which is commonly available in modern embedded processors), and the memory space overhead it incurs is not excessive. Therefore, it is suitable for memory-constrained embedded systems. We implemented our approach and performed experiments with seven embedded Java applications.

Fueled by an unprecedented desire for convenience and self-service, consumers are embracing embedded technology solutions that enhance their mobile lifestyles. Consequently, we witness an unprecedented proliferation of embedded/mobile applications. Most of the environments that execute these applications have severe power, performance, and memory space constraints that need to be accounted for. In particular, memory limitations can present serious challenges to embedded software designers. The current solutions to this problem include sophisticated packaging techniques and code optimizations for effective memory utilization. While the first solution is not scalable, the second one is restricted by intrinsic data dependences in the code that prevent code restructuring. In this paper, we explore an alternate approach for reducing memory space requirements of embedded applications. The idea is to re-compute the result of a code block (potentially multiple times) instead of storing it in memory and performing a memory operation whenever needed. The main benefit of this approach is that it reduces memory space requirements, that is, no memory space is reserved for storing the result of the code block in question.

Excessive power consumption is becoming a major barrier to extracting the maximum performance from high-performance parallel systems. Therefore, techniques oriented towards reducing power consumption of such systems are expected to become increasingly important in the future. Since disk systems of high-performance architectures are known to constitute a large fraction of the overall power budget, they form an important optimization target. Previous work on disk power management focuses primarily on hardware based schemes. However, since disk access pattern, i.e., the order in which disks on a system are accessed, is mainly shaped by the program code access pattern and disk layout of data, software techniques can also play a critical role in disk power management. Motivated by this observation, this paper proposes and evaluates a profile-driven disk layout optimization scheme for reducing energy consumption. The proposed scheme analyzes the array access traces obtained through profiling and determines, for each disk-resident data structure, the start disk from which the data is striped, the number of disks over which the data is striped, and the stripe unit. This paper discusses implementation details of our approach and presents an experimental evaluation of it. Our experiments with the entire suite of Spec95 floating-point benchmarks that are modified to operate on disk-resident data show that the proposed approach is very effective in reducing disk energy consumption. The results also show that the performance degradation caused by our approach is very small. This paper also compares our approach to a code restructuring based optimization mechanism and discusses how the two techniques can be combined for achieving the best results.

Scratch-Pad Memories (SPMs) are a serious alternative to conventional cache memories in embedded computing since they allow software to manage data flowing from and into memory components, resulting in a predictable behavior at runtime. The prior studies considered compiler-directed SPM management using both static and dynamic approaches. One of the assumptions under which most of the proposed approaches to data SPM management operate is that the application code is structured with regular loop nests with little or no control flow within the loops. This assumption, while it makes data SPM management relatively easy to implement, limits the applicability of those approachs to the codes involve conditional execution and complex control flows. To address this problem, this paper proposes a novel data SPM management strategy based on dataflow analysis. This analysis operates on a representation that reflects the conditional execution flow of the application and, consequently, it is applicable to a large class of embedded applications, including those with complex control flows

Disk power management is becoming increasingly important in high-end server and cluster type of environments that execute data-intensive applications. While hardware-only approaches (e.g., low-power modes supported by current disks) are successful to a certain extent, one also needs to consider the software side to achieve further energy savings. This paper first demonstrates that conventional data locality oriented code transformations are not sufficient for minimizing disk power consumption. The reason is that these optimizations do not take into account how disk-resident array data are laid out on the disk system, and consequently, fail to increase idle periods of disks, which is the primary metric using which disk power can be reduced. Instead, we propose a disk layout aware application optimization strategy that uses both code restructuring and data layout optimization. Our experimental evaluation with several benchmark codes reveal that the proposed strategy is very successful in reducing disk energy consumption without performing much worse than a pure data locality oriented scheme, as far as execution cycles are concerned. The experiments also show that the benefits coming from our approach increase with the increased number of disks; i.e., it scales very well

In this paper, we take the idea of application-level processing on disks to one level further, and focus on an architecture, called Cluster of Active Disks (CAD), where the storage system contains a network of parallel "active disks." Each individual active disk (which includes an embedded processor, disk(s), caches, memory, and interconnect) can perform some application level processing; but, more importantly, the active disks can collectively perform parallel Input/Output (I/O) and processing, thereby reducing not just the communication latency but I/O latency and computation time as well. The CAD architecture poses many challenges for the next generation software systems at all levels including programming models, operating and runtime systems, application mapping, compilation, parallelization and performance modeling, and evaluation. In this paper, we focus exclusively on code scheduling support required for clusters of active disks. More specifically, we address the problem of code scheduling with the goal of minimizing the power consumption on the disk system. Our experiments indicate that the proposed scheduling approach is very successful in reducing power and generates better results than three other alternate scheduling schemes tested

There has been a continued growth in the sales of mobile and embedded devices, in spite of the economic recession in many parts of the world. Many of these devices operate under tight memory bounds. Past research dealt with this problem and proposed both hardware and software solutions oriented toward reducing memory space requirements of embedded and mobile applications. One of the common characteristics of most of these prior efforts is that they assume the memory space available to an embedded application is fixed for the entire execution. Unfortunately, this is not a valid assumption in many execution environments since a typical embedded/mobile platform can have multiple applications executing concurrently and sharing a common memory space. As a result, the amount of memory available to a particular application can vary during execution. A particularly interesting scenario is what we call "memory emergency", where the size of the memory available to an application suddenly drops. If the application is not written to cope with this emergency scenario, the result would normally be a premature termination due to insufficient memory. In this paper, we propose compiler-based solutions to this memory emergency problem. The proposed compiler support modifies a given application code assuming a memory emergency model and reduces memory space demand (when necessary) by recomputing data values, thereby performing a tradeoff between memory space reduction and performance overhead. Our goal is to be able to work with the reduced memory space but minimize the performance overhead it brings. We evaluate the proposed approaches using twelve array-based embedded benchmarks. Our experimental analysis shows that the proposed approaches are very successful in responding to many memory emergency scenarios.

Disk subsystem is known to be a major contributor to overall power consumption of high-end parallel systems. Past research proposed several architectural level techniques to reduce disk power by taking advantage of idle periods experienced by disks. While such techniques have been known to be effective in certain cases, they share a common drawback: they operate in a reactive manner; i.e., they control disk power by observing past disk activity (e.g., idle and active periods) and estimating future ones. Consequently, they can miss opportunities for saving power and incur significant performance penalties, due to inaccuracies in predicting idle and active times. Motivated by this observation, this paper proposes and evaluates a compiler-driven approach to reducing disk power consumption of array-based scientific applications executing on parallel architectures. The proposed approach exposes disk layout information to the compiler, allowing it to derive disk access pattern, i.e., the order in which parallel disks are accessed. This paper demonstrates two uses of this information. First, we can do proactive disk power management, i.e., we can select the most appropriate power-saving strategy and disk preactivation strategy based on the compiler-predicted future idle and active periods of parallel disks. Second, we can restructure the application code to increase length of idle periods, which leads to better exploitation of available power-saving capabilities. We implemented both these approaches within an optimizing compiler and tested their effectiveness using a set of benchmark codes from the Spec2000 suite and a disk power simulator. Our results show that the compiler-driven disk power management is very promising. The experimental results also reveal that, while proactive disk power management is very effective, code restructuring for disk power achieves the best energy savings across all the benchmarks tested.

A critical component of a real-time operating system (RTOS) is its process scheduler. While the prior research on process scheduling focuses mostly on meeting hard/soft deadlines, preemption and priory assignment related issues, problems arise from existence of cache memories are largely ignored. Focusing on data accesses and a cache based embedded system, this paper proposes an adaptive locality-conscious process scheduling algorithm. The main goal of the proposed algorithm is to exploit (reuse) the contents of the on-chip cache memory to the highest extent possible. The algorithm tries to achieve its goal by determining the order in which the processes get scheduled such that the successively-executing processes share a large number of data elements. We implemented our scheduler within a customized simulation platform and simulated it using a set of benchmark codes. Our experimental results reveal that the proposed scheduling algorithm is very successful in practice, and reduces process completion times significantly for both rate-monotonic scheduling (RMS) and earliest-deadline-first scheduling (EDF). We also explain how process code transformations can be used for increasing the savings achieved by the locality-conscious scheduler, and show how the proposed approach operates with a base scheduler such as RMS and EDF.

The capabilities of applications executing on embedded and mobile devices are strongly influenced by memory size limitations. In fact, memory limitations are one of the main reasons that applications run slowly or even crash in embedded/mobile devices. While improvements in technology enable the integration of more memory into embedded devices, the amount memory that can be included is also limited by cost, power consumption, and form factor considerations. Consequently, addressing memory limitations will continue to be of importance.Focusing on embedded Java environments, this paper shows how object compression can improve memory space utilization. The main idea is to make use of the observation that a small set of values tend to appear in some fields of the heap-allocated objects much more frequently than other values. Our analysis shows the existence of such frequent field values in the SpecJVM98 benchmark suite. We then propose two object compression schemes that eliminate/reduce the space occupied by the frequent field values. Our extensive experimental evaluation using a set of eight Java benchmarks shows that these schemes can reduce the minimum heap size allowing Java applications to execute without out-of-memory exceptions by up to 24% (14% on an average).

Silicon technology advances have made it possible to pack millions of transistors---switching at high clock speeds---on a single chip. While these advances bring unprecedented performance to electronic products, they also pose difficult power/energy consumption problems. For example, large number of transistors in dense on-chip cache memories consume significant static (leakage) power even if the cache is not used by the current computation. While previous compiler research studied code and data restructuring for improving data cache performance, to our knowledge, there exists no compiler-based study that targets data cache leakage power consumption. In this paper, we present code restructuring techniques for array-based and pointer-intensive applications for reducing data cache leakage energy consumption. The idea is to let the compiler analyze the application code and insert instructions that turn off cache lines that keep variables not used by the current computation. This turning-off does not destroy contents of a cache line and waking up the cache line (when it is accessed later) does not incur much overhead. Due to inherent data locality in applications, we find that, at a given time, only a small portion of the data cache needs to be active; the remaining part can be placed into a leakage-saving mode (state); i.e., they can be turned off. Our experimental results indicate that the proposed compiler-based strategy reduces the cache energy consumption significantly. We also demonstrate how different compiler optimizations can increase the effectiveness of our strategy.

Power consumption and power density for the Translation Look-aside Buffer (TLB) are important considerations not only in its design, but can have a consequence on cache design as well. After pointing out the importance of instruction TLB (iTLB) power optimization, this article embarks on a new philosophy for reducing the number of accesses to this structure. The overall idea is to keep a translation currently being used in a register and avoid going to the iTLB as far as possible---until there is a page change. We propose four different approaches for achieving this, and experimentally demonstrate that one of these schemes that uses a combination of compiler and hardware enhancements can reduce iTLB dynamic power by over 85% in most cases.The proposed approaches can work with different instruction-cache (iL1) lookup mechanisms and achieve significant iTLB power savings without compromising on performance. Their importance grows with higher iL1 miss rates and larger page sizes. They can work very well with large iTLB structures that can possibly consume more power and take longer to lookup, without the iTLB getting into the common case. Further, we also experimentally demonstrate that they can provide performance savings for virtually indexed, virtually tagged iL1 caches, and can even make physically indexed, physically tagged iL1 caches a possible choice for implementation.

Current trends indicate that leakage energy consumption will be an important concern in upcoming process technologies. In this paper, we propose a compiler-based leakage energy optimization strategy for on-chip scratch-pad memories (SPMs). The idea is to divide SPM into banks and use compiler-guided memory-data layout optimization and data migration to maximize SPM bank idleness, thereby increasing the chances of placing banks into a low-power (low-leakage) state. Our experimental results with eight applications show that the proposed compiler-based strategy is very effective in reducing leakage energy of on-chip SPMs.

Scratch-pad memories (SPMs) enable fast access to time-critical data. While prior research studied both static and dynamic SPM management strategies, not being able to keep all hot data (i.e., data with high reuse) in the SPM remains the biggest problem. This paper proposes data compression to increase the number of data blocks that can be kept in the SPM. Our experiments with several embedded applications show that our compression-based SPM management heuristic is very effective and outperforms prior static and dynamic SPM management approaches. We also present an ILP formulation of the problem, and show that the proposed heuristic generates competitive results with those obtained through ILP, while spending much less time in compilation.

Memory constraint presents one of the critical challenges for embedded software writers. While circuit-level solutions based on cramming as many bits as possible into the smallest area possible are certainly important, memory-conscious software can bring much higher benefits. Focusing on an embedded Java-based environment, this paper studies potential benefits and challenges when heap memory is managed at a field granularity instead of object. This paper discusses these benefits and challenges with the help of two field-level analysis techniques. The first of these, called the field-level lifetime analysis, takes advantage of the observation that, for a given object instance, not all the fields have the same lifetime. The field-level lifetime analysis demonstrates the potential benefits of exploiting this information. Our second analysis, referred to as the disjointness analysis, is built upon the fact that, for a given object, some fields have disjoint lifetimes, and therefore, they can potentially share the same memory space. To quantify the impact of these techniques, we performed experiments with several benchmarks, and point out the important characteristics that need to be considered by application writers.

While the machine neutral Java bytecodes are attractive for code distribution in the highly heterogeneous embedded domain, the well-documented and standardized features also make it difficult to protect these codes. In fact, there are several tools to reverse engineer Java bytecodes. The focus of this work is the design of a substitution-based bytecode obfuscation approach that prevents code from being executed on unauthorized devices. Furthermore, we also improve the resilience of this substitution-based approach to frequency-based attacks. Using various Java class files, we show that our approach is 2.5 to 3 times less computationally intensive as compared to a traditional encryption based approach. Our experiments reveal that the protected class files could not execute on unauthorized clients.

Java-enabled wireless devices are preferred for various reasons. For example, users can dynamically download Java applications on demand. The dynamic download capability supports extensibility of the mobile client features and centralizes application maintenance at the server. Also, it enables service providers to customize features for the clients. In this work, we extend this client-server collaboration further by offloading some of the computations (i.e., method execution and dynamic compilation) normally performed by the mobile client to the resource-rich server in order to conserve energy consumed by the client in a wireless Java environment. In the proposed framework, the object serialization feature of Java is used to allow offloading of both method execution and bytecode-to-native code compilation to the server when executing a Java application. Our framework takes into account communication, computation, and compilation energies to decide where to compile and execute a method (locally or remotely), and how to execute it (using interpretation or just-in-time compilation with different levels of optimizations). As both computation and communication energies vary based on external conditions (such as the wireless channel state and user supplied inputs), our decision must be done dynamically when a method is invoked. Our experiments, using a set of Java applications executed on a simulation framework, reveal that the proposed techniques are very effective in conserving the energy of the mobile client.

As datasets processed by embedded processors increase in size and complexity, the management of higher levels of memory hierarchy (e.g., caches) is becoming an important issue. A major limitation of most of the cache locality optimization techniques proposed by previous research is that they handle a single procedure at a time. This prevents compilers from capturing the data access interactions between procedures and may result in poor performance. In this paper, we look at loop and data transformations from a different angle and use them in an interprocedural optimization framework. Employing the call graph representation of a given application, the proposed technique visits each node of this graph twice and uses loop and data transformations in a systematic way for optimizing array layouts whole program wide. Our experimental results show that this interprocedural locality optimization strategy is much more effective than the previous locality-based techniques that handle each procedure in isolation.

We present a compiler-oriented strategy to reduce the memory system energy consumption due to instruction accesses and increase performance by exploiting scratch pad memories. Scratch pad memories (SPMs) are alternatives to conventional cache memories in embedded computing. These small on-chip memories, like caches, provide fast and low-power access to data and instructions; but, they differ from caches in that their contents are managed by software instead of hardware. Our compiler framework keeps the most frequently used instructions in SPM and dynamically changes the contents of the SPM as the (instruction) working set of the application changes.

Silicon technology advances have made it possible to pack millions of transistors --- switching at high clock speeds --- on a single chip. While these advances bring unprecedented performance to electronic products, they pose difficult power/energy consumption problems. For example, large number of transistors in dense on-chip cache memories consume significant static (leakage) power even if the cache is not used by the current computation. While previous compiler research studied code and data restructuring for improving data cache performance, to our knowledge, there is no compiler-based study that targets data cache leakage power consumption. In this paper, we present code restructuring techniques for array-based and pointer-intensive applications for reducing data cache energy consumption. The idea is to let the compiler to analyze the code and insert instructions that turn off cache lines that keep variables not used by the current computation. This turning off does not destroy contents of a cache line, and waking up the cache line incurs very little overhead. Due to data locality, we find that at a given time only a small portion of the data cache needs to be active; the remaining part can be placed into a leakage-saving mode (state); i.e., they can be turned off. Our preliminary results indicate that the proposed strategy reduces the cache energy consumption significantly. We also show that several compiler optimizations increase the effectiveness of our strategy.

Java is becoming the main software platform for consumer and embedded devices such as mobile phones, PDAs, TV set-top boxes, and in-vehicle systems. Since many of these systems are memory constrained, it is extremely important to keep the memory footprint of Java applications under control.The goal of this work is to enable the execution of Java applications using a smaller heap footprint than that possible using current embedded JVMs. We propose a set of memory management strategies to reduce heap footprint of embedded Java applications that execute under severe memory constraints. Our first contribution is a new garbage collector, referred to as the Mark-Compact-Compress (MCC) collector, that allows an application to run with a heap smaller than its footprint. An important characteristic of this collector is that it compresses objects when heap compaction is not sufficient for creating space for the current allocation request. In addition to employing compression, we also consider a heap management strategy and associated garbage collector, called MCL (Mark-Compact-Lazy Allocate), based on lazy allocation of object portions. This new collector operates like the conventional Mark-Compact (MC) collector, but takes advantage of the observation that many Java applications create large objects, of which only a small portion is actually used. In addition, we also combine MCC and MCL, and present MCCL (Mark-Compact-Compress-Lazy Al-locate), which outperforms both MCC and MCL.We have implemented these collectors using KVM, and performed extensive experiments using a set of ten embedded Java applications. We have found our new garbage collection strategies to be useful in two main aspects. First, they reduce the minimum heap size necessary to execute an application without out-of-memory exception. Second, our strategies reduce the heap occupancy. That is, at a given time, they reduce the heap memory requirement of the application being executed. We have also conducted experiments with a more aggressive object compression strategy and discussed its main advantages.

Traditionally, the Java virtual machine (JVM), the cornerstone of Java technology, is tuned for performance, taking into account that the energy consumption requires re-evaluation, and possibly re-design of the virtual machine. This motivates us to tune specific components of the virtual machine for a battery-operated architecture. As embedded JVMs are designed to run for long periods of time on limited-memory embedded systems, creating and managing Java objects is of critical importance. The garbage collector (GC) is an important part of the JVM responsible for the automatic reclamation of unused memory. This paper shows that the GC is not only important for limited-memory systems but also for energy-constrained architectures. In particular, we present a GC-controlled leakage energy optimization technique that shuts off memory banks that do not hold live data. A variety of parameters, such as bank size, the garbage collection frequency, object allocation style, compaction style, and compaction frequency are tuned for energy saving.

While previous compiler research indicates that significant improvements in energy efficiency may be possible if properly optimized code is used, the energy constraints under which a given application code should be optimized may not always be available at compile-time. More importantly, these constraints may change dynamically during the course of execution. In this work, we present a dynamic recompilation/linking framework using which the energy behavior of a given application can be optimized while the application is being executed. Our preliminary experiments indicate that large energy gains are possible through dynamic code recompilation/linking at the expense of a relatively small increase in execution time.

Power consumption and power density for the Translation Lookaside Buffer (TLB) are important considerations not only in its design, but can have a consequence on cache design as well. This paper embarks on a new philosophy for reducing the number of accesses to the instruction TLB (iTLB) for power and performance optimizations. The overall idea is to keep a translation currently being used in a register and avoid going to the iTLB as far as possible - until there is a page change. We propose four different approaches for achieving this, and experimentally demonstrate that one of these schemes that uses a combination of compiler and hardware enhancements can reduce iTLB dynamic power by over 85% in most cases. These mechanisms can work with different instruction-cache (iLl) lookup mechanisms and achieve significant iTLB power savings without compromising on performance. Their importance grows with higher iLl miss rates and larger page sizes. They can work very well with large iTLB structures, that can possibly consume more power and take longer to lookup, without the iTLB getting into the common case. Further, we also experimentally demonstrate that they can provide performance savings for virtually-indexed, virtually-tagged iLl caches, and can even make physically-indexed, physically-tagged iLl caches a possible choice for implementation.

Java has been widely adopted as one of the software platforms for the seamless integration of diverse computing devices. Over the last year, there has been great momentum in adopting Java technology in devices such as cellphones, PDAs, and pagers where optimizing energy consumption is critical. Since, traditionally, the Java virtual machine (JVM), the cornerstone of Java technology, is tuned for performance, taking into account energy consumption requires reevaluation, and possibly redesign of the virtual machine. This motivates us to tune specific components of the virtual machine for a battery-operated architecture. As embedded JVMs are designed to run for long periods of time on limited-memory embedded systems, creating and managing Java objects is of critical importance. The garbage collector (GC) is an important part of the JVM responsible for the automatic reclamation of unused memory. This article shows that the GC is not only important for limited-memory systems but also for energy-constrained architectures.This article focuses on tuning the GC to reduce energy consumption in a multibanked memory architecture. Tuning the GC is important not because it consumes a sizeable portion of overall energy during execution, but because it influences the energy consumed in the memory during application execution. In particular, we present a GC-controlled leakage energy optimization technique that shuts off memory banks that do not hold live data. Using two different commercial GCs and a suite of thirteen mobile applications, we evaluate the effectiveness of the GC-controlled energy optimization technique and study its sensitivity to different parameters such as bank size, the garbage collection frequency, object allocation style, compaction style, and compaction frequency. We observe that the energy consumption of an embedded Java application can be significantly more if the GC parameters are not tuned appropriately. Further, we notice that the object allocation pattern and the number of memory banks available in the underlying architecture are limiting factors on how effectively GC parameters can be used to optimize the memory energy consumption.