The spoken language serves as an accessible and efficient interface, enabling non-experts and disabled users to interact with complex assistant robots. However, accurately grounding language utterances gives a significant challenge due to the acoustic variability in speakers’ voices and environmental noise. In this work, we propose a novel speech-scene graph grounding network (SGGNet2) that robustly grounds spoken utterances by leveraging the acoustic similarity between correctly recognized and misrecognized words obtained from automatic speech recognition (ASR) systems. To incorporate the acoustic similarity, we extend our previous grounding model, the scene-graph-based grounding network (SGGNet), with the ASR model from NVIDIA NeMo. We accomplish this by feeding the latent vector of speech pronunciations into the BERT-based grounding network within SGGNet. We evaluate the effectiveness of using latent vectors of speech commands in grounding through qualitative and quantitative studies. We also demonstrate the capability of SGGNet2 in a speech-based navigation task using a real quadruped robot, RBQ-3, from Rainbow Robotics.

Hardware Transactional Memory (HTM) relies heavily on the on-chip network for intertransaction communication. However, the network bandwidth utilization of transactions has been largely neglected in HTM designs. In this work, we propose a cost model to analyze network bandwidth in transaction execution. The cost model identifies a set of key factors that can be optimized through system design to reduce the communication cost of HTM. Based on the model and network traffic characterization of a representative HTM design, we identify a huge source of superfluous traffic due to failed requests in transaction conflicts. As observed in a spectrum of workloads, 39% of the transactional requests fail due to conflicts, which renders 58% of the transactional network traffic futile. To combat this pathology, a novel in-network filtering mechanism is proposed. The on-chip router is augmented to predict conflicts among transactions and proactively filter out those requests that have a high probability to fail. Experimental results show the proposed mechanism reduces total network traffic by 24% on average for a set of high-contention TM applications, thereby reducing energy consumption by an average of 24%. Meanwhile, the contention in the coherence directory is reduced by 68%, on average. These improvements are achieved with only 5% area added to a conventional on-chip router design.

Hardware Transactional Memory (HTM) promises to simplify parallel programming on shared-memory chip multiprocessors by providing atomic execution of code blocks. Concurrently, Networks-On-Chip (NOCs) have emerged as an efficient on-chip communication infrastructure but have been largely neglected in HTM designs. In this work, we explore the interaction between the HTM paradigm and NOCs. In the process, we find a huge source of unnecessary network traffic incurred by transactional requests that are unsuccessful. This problem is identified as false forwarding that adversely affects network performance and energy efficiency. Surprisingly, 39% (up to 79% for a specific workload) of the transactional requests have incurred false forwarding over a wide spectrum of workloads. To combat this problem, we propose TMNOC, a novel approach that exploits the co-design of HTM and NOCs to mitigate false forwarding. Transactional requests that have a high probability to fail are filtered out in-network as early as possible to save energy and improve concurrency in the memory system. Experimental results show that our design reduces total network traffic by 20% on average (up to 40%) for a set of high-contention benchmarks representative of future TM workloads, thereby reducing energy consumption by an average of 24% (up to 39%). Meanwhile, the contention in the coherence directory is reduced by 66% on average. These improvements are achieved with only 5% area overhead added to a conventional on-chip router design.

The transactional memory (TM) paradigm promises to increase programmer productivity by making it easier to write correct parallel programs. In fulfilling this goal, a TM system should maximize its performance with limited hardware resources. Conflict detection is an essential element for maintaining correctness among concurrent transactions in a TM system. Hardware signatures have been proposed as an area-efficient method for detecting conflicts. However, signatures can degrade TM performance by falsely declaring conflicts. Hence, improving the accuracy of signatures within a given hardware budget is a crucial issue for TM to be adopted as a mainstream programming model. In this paper, we propose a simple and effective signature design, the unified signature. Instead of using separate read- and write-signatures, we implement a single signature to track all read- and write-accesses. By merging read- and write-signatures, a unified signature can effectively enlarge the signature coverage without additional overhead. Within the constraints of a given hardware budget, a TM system with a unified signature outperforms a baseline system with the same-sized traditional signatures by reducing the number of falsely detected conflicts. Even though the unified signature scheme incurs read-read dependencies, we show that these false dependencies do not negate the benefit of unified signatures and can effectively be filtered out. A TM system with a 2-Kbit unified signature with a helper signature scheme achieves speedups of 15 percent over baseline TM with 33 percent less area and 49 percent less power.

In Transactional Memory (TM), a conflict occurs when a memory block is accessed concurrently by two or more transactions and at least one of them is a write access. The management of conflicts significantly impacts TM performance. There are two alternative approaches for managing conflicts: Reactive Contention Management (RCM) [1] and Proactive Contention Management (PCM) [2]. Previous contention management schemes treat all transactions with no weights, and make a decision based on the information provided by the running transaction instance.

Hardware Transactional Memory (HTM) designs must implement conflict detection to guarantee the correctness of transaction execution. A conflict occurs when more than one transaction access the same data and at least one of them attempts to modify the data. The corresponding conflict detection mechanism usually works at a cacheline level that fits naturally into the cache coherence protocol. Thus, the inter-transaction communication for conflict detection is usually mapped onto the coherence communication controlled by the directory-based coherence protocols. In this paper, we identify inefficiency introduced by such mappings. The net effect of such inefficiency is excessive on-chip network traffic that consumes substantial dynamic power as packets are switched over the routers and links. We present TMNOC, a HTM and Network-on-Chip (NoC) co-design to improve network energy efficiency. The on-chip network, instead of a passive communication substrate, proactively filters out transactional requests that waste energy yet having no contribution to the progress of transactions. Experiment results show that TMNOC reduces energy consumption of the on-chip network by 14.5% on average (up to 38%) across a wide range of transaction applications.

Hardware Transactional Memory (HTM) systems implement version management and conflict detection in hardware to guarantee that each transaction is atomic and executes in isolation. In general, HTM implementations fall into two categories, namely, eager systems and lazy systems. Lazy systems have been shown to exploit more concurrency from potentially conflicting transactions. However, lazy systems manage a transaction's entire write set lazily, which gives rise to two main disadvantages: (a) a complex cache protocol and implementation are required to maintain the speculative modifications, and, (b) the latency of committing the entire write set often leads to severe performance degradation of the whole system. It is observed in a wide range of workloads that more than 55% of the transaction aborts are due to conflicts on only three memory blocks. Thus we argue that an eager HTM system can achieve the same level of concurrency as lazy systems by managing only a small portion of a transaction's write set lazily. In this paper, we present Selective-Eager-Lazy HTM (SEL-TM), a new HTM implementation to adopt complementary version management schemes within a transaction whose write set is divided into eagerly- and lazily-managed memory addresses at runtime. An intelligent hardware scheme is designed to select the memory addresses for lazy management as well as determining whether each dynamic instance of a transaction benefits from hybrid management. Experimental results using the STAMP benchmarks show that, on average, SEL-TM improves performance by 14% over an eager system and 22% over a lazy system. The speedup demonstrates that our design is capable of harvesting the concurrency benefit of lazy version management while avoiding some of the performance penalties in lazy HTMs.

Transactional Memory (TM) promises to increase programmer productivity by making it easier to write correct parallel programs. In fulfilling this goal, a TM system should maximize its performance with limited hardware resources. Conflict detection is an essential element for maintaining correctness among concurrent transactions in a TM system. Hardware signatures have been proposed as an area-efficient method for detecting conflicts. However, signatures can degrade TM performance by falsely declaring conflicts. Hence, increasing the quality of signatures within a given hardware budget is a crucial issue for TM to be adopted as a mainstream programming model. In this paper, we propose a simple and effective signature design, unified signature. Instead of using separate read- and write-signatures, as is often done in TM systems, we implement a single signature to track all read- and write-accesses. By merging read- and write-signatures, a unified signature can effectively enlarge the signature size without additional overhead. Within the constraints of a given hardware budget, a TM system with a unified signature outperforms a baseline system with the same hardware budget by reducing the number of falsely detected conflicts. Even though the unified signature scheme incurs read-after-read dependencies, we show that these false dependencies do not negate the benefit of unified signatures for practical signature sizes. A TM system with 2K-bit unified signatures achieves average speedups of 22% over baseline TM systems.

Transactional Memory (TM) has attracted considerable attention because it promises to increase programmer productivity by making it easier to write correct parallel programs. To maintain correctness in the face of concurrency, detecting conflicts among simultaneously running transactions is an essential element. Hardware signatures have been proposed as an area-efficient mechanism for conflict detection. A signature can summarize an unbounded amount of addresses and misses no conflicts, but could falsely declare conflicts even when no true conflict exists (false positives) due to aliasing and occupancy. Previous signature designs assume that false positives are destructive to performance and attempt to reduce the total number of false positives. In this paper, we show that some false positives can be helpful to performance by triggering the early abortion of a transaction which would encounter a true conflict later anyway. Based on this observation, we propose an adaptive grain signature to improve performance by dynamically changing the range of address keys based on the history. With the use of adaptive grain signatures, we can increase the number of performance-friendly false positives as well as decrease the number of performance-destructive false positives.

Hardware signatures for Transactional Memory (TM) systems have been proposed as an efficient mechanism for conflict detection, an essential element in TM for maintaining correctness. A signature misses no conflicts, but could falsely declare conflicts even when no true conflict exists (false positives). In this paper, we show that some false positives can be helpful to the performance by triggering the early abortion of a transaction which would encounter a true conflict later anyway. We propose an adaptive grain signature to improve TM performance by dynamically changing the range of address keys based on the history. With architecture-level simulation and Verilog HDL implementation, we demonstrate that a TM system with our design frequently outperforms baseline TM systems, with marginal area overhead.