Efficient navigation and search in unknown environments for multiple objects is a fundamental challenge in robotics, particularly in applications such as warehouse management, domestic assistance, and search-and-rescue. The Multi-Object Search (MOS) problem involves navigating to a sequence of locations to maximize the likelihood of finding target objects while minimizing travel costs. In this paper, we introduce a novel approach to the MOS problem, called Finder, which leverages vision language models (VLMs) to locate multiple objects across diverse environments. Specifically, our approach introduces multi-channel score maps to track and reason multiple objects simultaneously during navigation, along with a score map technique that combines scene-level and object-level semantic correlations. We validate our approach through extensive experiments in both simulated and real-world environments. The results demonstrate that Finder outperforms existing multi-object search methods using deep reinforcement learning and VLM Additional ablation and scalability studies highlight the importance of our design choices and show the system’s robustness with increasing number of target objects. Website: https://find-all-my-things.github.io/

With ever more complicated functionalities being integrated in modern autonomous systems, traditional design methods may not remain sufficient to deliver trusted and high-performance systems with stringent temporal, safety and cost efficiency requirements. In this paper, we discuss the limitations of the traditional design methods with the above requirements enforced, in which hardware and software design are often considered separately. To tackle these limitations, this paper presents a novel design solution that synthesizes both software-level and hardware-level design. First, we highlight and analyze the interconnections between software-level methods (e.g. priority assignment and task allocation) and hardware design (e.g. cache and memory management), in terms of the resulting system performance, e.g. latency. Second, by applying the identified interconnections, we propose an optimization framework to produce high-quality synthesized solutions of both software and hardware design based on a set of candidate design methods. In addition, we describe potential research directions derived from the work and major challenges that can be investigated jointly by engineers and researchers from embedded systems, system safety and programming languages communities.

Multi-core architectures are increasingly being used in real-time embedded systems. In general, such systems have more processors than the shared memory modules, potentially causing severe interference over memory accesses. This resource contention could lead to substantial variation on memory access latencies, and thus wide fluctuation in the overall system performance, which is highly undesirable especially for the time-critical applications. In this paper, we address resource contention and timing predictability for multi-core architectures with distributed memory interconnects. We focus on the locally arbitrated interconnect constructed by pipelined multiplexing stages with local arbitration, while the globally arbitrated interconnect employing global scheduling to the same architecture potentially suffers synchronisation issue and requires strict coordination. Our contributions are mainly threefold: (i) We analyse the resource contention across the memory access data path, and report the accurate calculational method to bound the worst-case behaviour. (ii) We compare the average-case behaviour of the locally arbitrated and the globally arbitrated architectures with experiments, demonstrating varying memory latencies caused by the resource sharing issue. (iii) We propose an architectural modification to smooth resource sharing. Evaluations on simulators and FPGA implementations with synthetic memory workload show that the latency variation is significantly reduced, contributing towards timing predictability of multi-core systems.

Multicore architectures are widely adopted in the emerging real-time applications, such as autonomous vehicles and robotics, where latency is required to be both bounded in the worst case (i.e., time predictability) and low. With the number of processors growing, the conventional memory interconnects, i.e., shared bus, crossbar, and network-on-chip (NoC), suffer high latency due to the increasing logic size of their centralized arbiter, which is deployed for time predictability. In this article, we introduce a novel distributed multimemory interconnect, Meshed Bluetree, and explain its operation. Constructed by coupling a router network with multiple Bluetree-based memory architectures in parallel, Meshed Bluetree allows simultaneous access to multiple memory modules. We present the analysis for the predictable timing behavior of memory access to bound the worst case. The evaluation of FPGA with synthetic memory workloads and real-world benchmarks demonstrates the effectiveness of our work, i.e., as the number of memory modules increases, the latency is reduced with the same scale. This work reports the first time-predictable distributed multimemory interconnect, significantly contributing to multicore real-time systems.