Object detection is a key component in computer vision research, allowing a system to determine the location and type of object within any given scene. YOLOv5 is a modern object detection model, which utilises the advantages of the original YOLO implementation while being built from scratch in Python. In this paper, BiFPN-YOLO is proposed, featuring clear improvements over the existing range of YOLOv5 object detection models; these include replacing the traditional Path-Aggregation Network (PANet) with a higher performing Bi-Directional Feature Pyramid Network (BiFPN), requiring complex adaptation from its original implementation to function with YOLOv5, as well as exploring a replacement to the standard Swish activation function by evaluating the performance against a number of other activation functions. The proposed model showcases state-of-the-art performance, benchmarking against well-known datasets such as the German Traffic Sign Detection Benchmark (GTSDB), improving mAP by 3.1 %, and the RoboFEI@Home dataset, where Mean Average Precision (mAP) is improved by 2 % compared to the base YOLOv5 model. Performance was also improved on MSCOCO by 1.1 % and a custom subset of the OpenImagesV6 dataset by 2.4 %.

The task of visuo-tactile object recognition is key in enabling robots to interact with humans and their environment in an efficient and effective manner. The differing statistical properties of visual images and tactile time-series data make visuo-tactile fusion non-trivial and complex. This work investigates the usage of Transformers to perform feature level fusion for visuo-tactile data, utilising the Transformer to generate temporal relationships between the visual and tactile data through its self-attention structure. The proposed pipeline is tested on the PHAC-2 dataset, and a complex ablation experiment is completed across a collection of leading activation functions. The proposed pipeline is demonstrated to achieve state-of-the-art accuracy for visuo-tactile object recognition on the PHAC-2 dataset, achieving a 94.3% accuracy when data from two tactile actions are considered.

In solving many real-world optimization problems, neither mathematical functions nor numerical simulations are available for evaluating the quality of candidate solutions. Instead, surrogate models must be built based on historical data to approximate the objective functions and no new data will be available during the optimization process. Such problems are known as offline data-driven optimization problems. Since the surrogate models solely depend on the given historical data, the optimization algorithm is able to search only in a very limited decision space during offline data-driven optimization. This paper proposes a new offline data-driven evolutionary algorithm to make the full use of the offline data to guide the search. To this end, a surrogate management strategy based on ensemble learning techniques developed in machine learning is adopted, which builds a large number of surrogate models before optimization and adaptively selects a small yet diverse subset of them during the optimization to achieve the best local approximation accuracy and reduce the computational complexity. Our experimental results on the benchmark problems and a transonic airfoil design example show that the proposed algorithm is able to handle offline data-driven optimization problems with up to 100 decision variables.

For multi-scenario airfoil shape optimization problems, an evaluation of a single airfoil is based on its full-scenario drag landscape. To obtain the full-scenario drag landscape, a large number of computational fluid dynamic simulations for different operating conditions must be conducted. Since a single computational fluid dynamic simulation is often time-consuming, evaluations for multi-scenario airfoil shape optimization will be computationally highly intensive. Although surrogate-assisted evolutionary algorithms have been widely applied to expensive optimization problems, existing surrogate-assisted evolutionary algorithms cannot be directly applied to multi-scenario airfoil shape optimization due to the lack of training data. Instead of using surrogate models to directly approximate the multi-scenario evaluations, we employ a hierarchical surrogate model consisting of a K-nearest neighbors classifier and a Kriging model to approximate the full-scenario drag landscape for each candidate design during the optimization. Then, the fitness of the candidate design is evaluated based on the approximated drag landscape to reduce the computational cost. The proposed hierarchical surrogate model is embedded in the covariance matrix adaptation evolution strategy and applied to the RAE2822 airfoil design problem. Our experimental results show that the proposed algorithm is able to obtain an airfoil design with limited computational cost that perform well in different operating conditions.

Many real-world optimization problems involve computationally intensive numerical simulations to accurately evaluate the quality of solutions. Usually, the fidelity of the simulations can be controlled using certain parameters and there is a tradeoff between simulation fidelity and computational cost, i.e., the higher the fidelity, the more complex the simulation will be. To reduce the computational time in simulation-driven optimization, it is a common practice to use multiple fidelity levels in search for the optimal solution. So far, not much work has been done in evolutionary optimization that considers multiple fidelity levels in fitness evaluations. In this paper, we aim to develop test suites that are able to capture some important characteristics in real-world multifidelity optimization, thereby offering a useful benchmark for developing evolutionary algorithms for multifidelity optimization. To demonstrate the usefulness of the proposed test suite, three strategies for adapting the fidelity level of the test problems during optimization are suggested and embedded in a particle swarm optimization (PSO) algorithm. Our simulation results indicate that the use of changing fidelity is able to enhance the performance and reduce the computational cost of the PSO, which is desired in solving expensive optimization problems.

Function evaluations (FEs) of many real-world optimization problems are time or resource consuming, posing a serious challenge to the application of evolutionary algorithms (EAs) to solve these problems. To address this challenge, the research on surrogate-assisted EAs has attracted increasing attention from both academia and industry over the past decades. However, most existing surrogate-assisted EAs (SAEAs) either still require thousands of expensive FEs to obtain acceptable solutions, or are only applied to very low-dimensional problems. In this paper, a novel surrogate-assisted particle swarm optimization (PSO) inspired from committee-based active learning (CAL) is proposed. In the proposed algorithm, a global model management strategy inspired from CAL is developed, which searches for the best and most uncertain solutions according to a surrogate ensemble using a PSO algorithm and evaluates these solutions using the expensive objective function. In addition, a local surrogate model is built around the best solution obtained so far. Then, a PSO algorithm searches on the local surrogate to find its optimum and evaluates it. The evolutionary search using the global model management strategy switches to the local search once no further improvement can be observed, and vice versa. This iterative search process continues until the computational budget is exhausted. Experimental results comparing the proposed algorithm with a few state-of-the-art SAEAs on both benchmark problems up to 30 decision variables as well as an airfoil design problem demonstrate that the proposed algorithm is able to achieve better or competitive solutions with a limited budget of hundreds of exact FEs.

Heterogeneous multicore architectures have the potential to improve energy efficiency by integrating power-efficient wimpy cores with high-performing brawny cores. However, it is an open question as how to deliver energy reduction while ensuring the quality of service (QoS) of latency-sensitive web-services running on such heterogeneous multicores in warehouse-scale computers (WSCs). In this work, we first investigate the implications of heterogeneous multicores in WSCs and show that directly adopting heterogeneous multicores without re-designing the software stack to provide QoS management leads to significant QoS violations. We then present Octopus-Man, a novel QoS-aware task management solution that dynamically maps latency-sensitive tasks to the least power-hungry processing resources that are sufficient to meet the QoS requirements. Using carefully-designed feedback-control mechanisms, Octopus-Man addresses critical challenges that emerge due to uncertainties in workload fluctuations and adaptation dynamics in a real system. Our evaluation using web-search and memcached running on a real-system Intel heterogeneous prototype demonstrates that Octopus-Man improves energy efficiency by up to 41% (CPU power) and up to 15% (system power) over an all-brawny WSC design while adhering to specified QoS targets.

New technologies arise in a number of ways. They may come from advances in scientific research, through new combinations of existing technologies, or by simply imagining what might be possible in the future. This video describes the evolution of Tabletop Telepresence, a system for remote collaboration through desktop videoconferencing combined with a digital desk. Tabletop Telepresence began as a collection of camera, projector, videoconferencing and user interaction technologies. Working together; artists and research scientists combined these technologies into a means of sharing paper documents between remote desktops, interacting with those documents, requesting services (such as translation), and communicating through a videoconference.

Using a surrogate model to evaluate the expensive fitness of candidate solutions in an evolutionary algorithm can significantly reduce the overall computational cost of optimization tasks. In this paper we present a recurrent neural network ensemble that is used as a surrogate for the long-term prediction of computational fluid dynamic simulations. A hybrid multi-objective evolutionary algorithm that trains and optimizes the structure of the recurrent neural networks is introduced. Selection and combination of individual prediction models in the Pareto set of solutions is used to create the ensemble of predictors. Five selection methods are tested on six data sets and the accuracy of the ensembles is compared to the converged computational fluid dynamic data, as well as to the delta change between two flow conditions. Intermediate computational fluid dynamic data is used for training and the method presented can produce accurate and stable results using a third of the intermediate data needed for convergence.

NudgeCam is a mobile application that can help users capture more relevant, higher quality media. To guide users to capture media more relevant to a particular project, third-party template creators can show users media that demonstrates relevant content and can tell users what content should be present in each captured media using tags and other meta-data such as location and camera orientation. To encourage higher quality media capture, NudgeCam provides real time feedback based on standard media capture heuristics, including face positioning, pan speed, audio quality, and many others. We describe an implementation of NudgeCam on the Android platform as well as field deployments of the application.

The Pantheia system enables users to create virtual models by `marking up' the real world with pre-printed markers. The markers have predefined meanings that guide the system as it creates models. Pantheia takes as input user captured images or video of the marked up space. This video illustrates the workings of the system and shows it being used to create three models, one of a cabinet, one of a lab, and one of a conference room. As part of the Pantheia system, we also developed a 3D viewer that spatially integrates a model with images of the model.

In this video demo, we present mTable, a multimedia tabletop system for browsing photo and video collections. We have developed a set of applications for visualizing and exploring photos, a board game for labeling photos, and a 3D cityscape metaphor for browsing videos. The system is suitable for use in a living room or office lounge, and can support multiple displays by visualizing the collections on the tabletop and showing full-size images and videos on another flat panel display in the room.

This video shows the Virtual Physics Circus, a kind of playground for experimenting with simple physical models. The system makes it easy to create worlds with common physical objects such as swings, vehicles, ramps, and walls, and interactively play with those worlds. The system can be used as a creative art medium as well as to gain understanding and intuition about physical systems. The system can be controlled by a number of UI devices such as mouse, keyboard, joystick, and tags which are tracked in 6 degrees of freedom.

A common problem with teleconferences is awkward turn-taking – particularly ‘ collisions,’ whereby multiple parties inadvertently speak over each other due to communication delays. We propose a model for teleconference discussions including the effects of delays, and describe tools that can improve the quality of those interactions. We describe an interface to gently provide latency awareness, and to give advanced notice of ‘ incoming speech’ to help participants avoid collisions. This is possible when codec latencies are significant, or when a low bandwidth side channel or out-of-band signaling is available with lower latency than the primary video channel. We report on results of simulations, and of experiments carried out with transpacific meetings, that demonstrate these tools can improve the quality of teleconference discussions.

ePic is an integrated presentation authoring and playback system that makes it easy to use a wide range of devices installed in one or multiple multimedia venues.

The FXPAL Photo Application is designed to faciliate the organization of digital images from digital cameras and other sources through automated organization and intuitive user interfaces.

We demonstrate the use of detail-on-demand hypervideo in interactive training and video summarization. Detail-on-demand video allows viewers to watch short video segments and to follow hyperlinks to see additional detail. The player for detail-on-demand video displays keyframes indicating what links are available at each point in the video. The Hyper-Hitchcock authoring tool helps users create hypervideo by automatically dividing video into clips that can be combined in a direct manipulation interface. Clips can be grouped into composites and hyperlinks can be placed between clips and composites. A summarization algorithm creates multi-level hypervideo summaries from linear video by automatically selecting clips and placing links between them.

Hitchcock is a semi-automatic video editing system. This video shows users collaboratively authoring a home video.

Hitchcock is a system that allows users to easily create custom videos from raw video shot with a standard video camera.In contrast to other video editing systems, Hitchcock uses automatic analysis to determine the suitability of portions of the raw video.Unsuitable video typically has fast or erratic camera motion.Hitchcock first analyzes video to identify the type and amount of camera motion: fast pan, slow zoom, etc.Based on this analysis, a numerical "unsuitability" score is computed for each frame of the video.Combined with standard editing rules, this score is used to identify clips for inclusion in the final video and to select their start and end points.To create a custom video, the user drags keyframes corresponding to the desired clips into a storyboard.Users can lengthen or shorten the clip without specifying the start and end frames explicitly.Clip lengths are balanced automatically using a spring-based algorithm.