Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much longer slowing down decoding time affecting the time a user has to wait for an answer. We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings, speeding up the generation time by a large margin. Our method allows for different compression rates, trading off decoding time for answer quality. Compared to earlier methods, COCOM allows for handling multiple contexts more effectively, significantly reducing decoding time for long inputs. Our method demonstrates an inference speed-up of up to 5.69 times while achieving higher performance compared to existing efficient context compression methods

Modern transformer-based information retrieval models achieve state-of-the-art performance across various benchmarks. The self-attention of the transformer models is a powerful mechanism to contextualize terms over the whole input but quickly becomes prohibitively expensive for long input as required in document retrieval. Instead of focusing on the model itself to improve efficiency, this paper explores different bag of words document representations that encode full documents by only a fraction of their characteristic terms, allowing us to control and reduce the input length. We experiment with various models for document retrieval on MS MARCO data, as well as zero-shot document retrieval on Robust04, and show large gains in efficiency while retaining reasonable effectiveness. Inference time efficiency gains are both lowering the time and memory complexity in a controllable way, allowing for further trading off memory footprint and query latency. More generally, this line of research connects traditional IR models with neural “NLP” models and offers novel ways to explore the space between (efficient, but less effective) traditional rankers and (effective, but less efficient) neural rankers elegantly.

Autonomous unmanned aerial vehicles (UAVs) are prime candidates for use in applications like inventory checking of large warehouses. They facilitate access to the tall racks while being agile to maneuver through the confined space of the corridors of the warehouse. They are capable of carrying the sensors needed to perform the application of warehouse inspection and are capable of sustaining flight during extended periods of time. While the environment of an industrial warehouse is well structured, it is also a challenging surrounding for visual odometry due to reflections, moving parts and light sources which might be occurring at the same time. At low altitudes, the movement of the drone is less stable because of turbulence caused by a backflow of air from the drone. The goods in the warehouse are man-placed, which might cause damage to inspection targets or create obstacles in the space. In spite of all the challenges present in this environment, the drone has to perform effortlessly its application of inventory checking. In this paper, we will focus on extending the automated process of inventory checking, presented in previous work, to the real environment of an existing warehouse and provide approaches to problems that arise from practice. We will present our approach to mission planning considering the presented challenges. The system of capturing and processing application data is introduced. The developed solution uses a UAV called AV Discovery built by Airvolute s.r.o. deployed into a real environment of an active warehouse.