Multi-camera computer vision in industry offers advantages but poses risks to worker privacy and intellectual property through exposure of sensitive contextual information. Existing privacy methods often inadequately protect background details crucial in manufacturing. This issue is prominent in applications like automated ergonomic assessment, where visual data for posture analysis can reveal sensitive workplace information. We propose a system for simultaneous personal privacy and enhanced contextual intellectual property protection, featuring a novel probabilistic obfuscation technique. Our edge-based Generative Adversarial Privacy system employs a modified obfuscator that learns to inject controlled, pixel-wise random noise, particularly into non-critical background regions. This more effectively obscures IP-sensitive environmental details before data transmission for central analysis (e.g., pose estimation). Our approach, validated in a multi-camera ergonomic study, effectively protects worker privacy and contextual IP (metrics-evaluated) and maintains 3D pose accuracy for reliable ergonomic assessment. This work provides a solution for deploying vision systems in sensitive industrial settings by holistically addressing privacy requirements through an advanced, adaptive obfuscation strategy.

Specifying tasks for robotic systems traditionally requires coding expertise, deep domain knowledge, and significant time investment. While learning from demonstration offers a promising alternative, existing methods often struggle with tasks of longer horizons. To address this limitation, we introduce a computationally efficient approach for learning probabilistic deterministic finite automata (PDFA) that capture task structures and expert preferences directly from demonstrations. Our approach infers sub-goals and their temporal dependencies, producing an interpretable task specification that domain experts can easily understand and adjust. We validate our method through experiments involving object manipulation tasks, showcasing how our method enables a robot arm to effectively replicate diverse expert strategies while adapting to changing conditions.

This paper presents an end-to-end, IoT-enabled robotic system for the non-destructive, real-time, and spatially-resolved mapping of grape yield and quality (Brix, Acidity) in vineyards. The system features a comprehensive analytical pipeline that integrates two key modules: a high-performance model for grape bunch detection and weight estimation, and a novel deep learning framework for quality assessment from hyperspectral (HSI) data. A critical barrier to in-field HSI is the “domain shift" caused by variable illumination. To overcome this, our quality assessment is powered by the Light-Invariant Spectral Autoencoder (LISA), a domain-adversarial framework that learns illumination-invariant features from uncalibrated data. We validated the system’s robustness on a purpose-built HSI dataset spanning three distinct illumination domains: controlled artificial lighting (lab), and variable natural sunlight captured in the morning and afternoon. Results show the complete pipeline achieves a recall (0.82) for bunch detection and aR2(0.76) for weight prediction, while the LISA module improves quality prediction generalization by over 20% compared to the baselines. By combining these robust modules, the system successfully generates high-resolution, georeferenced data of both grape yield and quality, providing actionable, data-driven insights for precision viticulture.

Abstract Visual analysis tasks, including crowd management, often require resource-intensive machine learning models, posing challenges for deployment on edge hardware. Consequently, cloud computing emerges as a prevalent solution. To address privacy concerns associated with offloading video data to remote cloud platforms, we present a novel approach using adversarial training to develop a lightweight obfuscator neural network. Our method focuses on pedestrian detection as an example of visual analysis, allowing the transformation of video frames on the camera itself to retain only essential information for pedestrian detection while preserving privacy. Importantly, the obfuscated data remains compatible with publicly available object detectors, requiring no modifications or significant loss in accuracy. Additionally, our technique overcomes the common limitation of relying on labeled sensitive attributes for privacy preservation. By demonstrating the inability of pedestrian attribute recognition models to detect attributes in obfuscated videos, we validate the efficacy of our privacy protection method. Our results suggest that this scalable approach holds promise for enabling camera usage in video analytics while upholding personal privacy.

Abstract Goal recognisers attempt to infer an agent’s intentions from a sequence of observed actions. This is an important component of intelligent systems that aim to assist or thwart actors; however, there are many challenges to overcome. For example, the initial state of the environment could be partially unknown, and agents can act suboptimally and observations could be missing. Approaches that adapt classical planning techniques to goal recognition have previously been proposed, but, generally, they assume the initial world state is accurately defined. In this paper, a state is inaccurate if any fluent’s value is unknown or incorrect. Our aim is to develop a goal recognition approach that is as accurate as the current state-of-the-art algorithms and whose accuracy does not deteriorate when the initial state is inaccurately defined. To cope with this complication, we propose solving goal recognition problems by means of an Action Graph. An Action Graph models the dependencies, i.e. order constraints, between all actions rather than just actions within a plan. Leaf nodes correspond to actions and are connected to their dependencies via operator nodes. After generating an Action Graph, the graph’s nodes are labelled with their distance from each hypothesis goal. This distance is based on the number and type of nodes traversed to reach the node in question from an action node that results in the goal state being reached. For each observation, the goal probabilities are then updated based on either the distance the observed action’s node is from each goal or the change in distance. Our experimental results, for 15 different domains, demonstrate that our approach is robust to inaccuracies within the defined initial state.

This paper presents the initial results of a study exploring whether the perceived rapport with a virtual agent can influence users' decisions on delegating critical tasks to the agent. We hypothesize that users are more likely to delegate to virtual agents that attempt to build rapport with users than to agents that avoid building rapport. The samples we collected so far still need to validate the hypothesis fully. Nevertheless, we found that the perceived rapport with a virtual agent is highly relevant to trust in the agent.

Despite already being commonplace, delegation to robotic virtual agents (VAs) is often considered challenging and error-prone in critical situations by the general public. Theory of mind, the human capacity to take another person's perspective, is deemed an important enabler for human-human cooperation. This study explores the effect of a robotic VA's ability to use theory of mind on users' delegation behavior. To this end, we conducted a between-subjects experiment with participants playing the Colored Trails game with robotic VAs of varying levels of theory of mind. The results invalidate our hypothesis that the ToM level is a reliable indicator of delegation choices. Instead, we found that the participants' performance strongly correlates with their delegatory intentions. Therefore, to facilitate delegation, designers of robots and robotic agents may consider refraining from using ToM-resemblance features and focusing on balancing user performance perception instead to induce the desired delegation behaviors.

Automated anomaly detection in surveillance videos has attracted much interest as it provides a scalable alternative to manual monitoring. Most existing approaches achieve good performance on clean benchmark datasets recorded in well-controlled environments. However, detecting anomalies is much more challenging in the real world. Adverse weather conditions like rain or changing brightness levels cause a significant shift in the input data distribution, which in turn can lead to the detector model incorrectly reporting high anomaly scores. Additionally, surveillance cameras are usually deployed in evolving environments such as a city street of which the appearance changes over time because of seasonal changes or roadworks. The anomaly detection model will need to be updated periodically to deal with these issues. In this paper, we introduce a multi-branch model that is equipped with a trainable preprocessing step and multiple identical branches for detecting anomalies during day and night as well as in sunny and rainy conditions. We experimentally validate our approach on a distorted version of the Avenue dataset and provide qualitative results on real-world surveillance camera data. Experimental results show that our method outperforms the existing methods in terms of detection accuracy while being faster and more robust on scenes with varying visibility.

Abstract Modern deep learning models achieve state-of-the-art results for many tasks in computer vision, such as image classification and segmentation. However, its adoption into high-risk applications, e.g. automated medical diagnosis systems, happens at a slow pace. One of the main reasons for this is that regular neural networks do not capture uncertainty. To assess uncertainty in classification, several techniques have been proposed casting neural network approaches in a Bayesian setting. Amongst these techniques, Monte Carlo dropout is by far the most popular. This particular technique estimates the moments of the output distribution through sampling with different dropout masks. The output uncertainty of a neural network is then approximated as the sample variance. In this paper, we highlight the limitations of such a variance-based uncertainty metric and propose an novel approach. Our approach is based on the overlap between output distributions of different classes. We show that our technique leads to a better approximation of the inter-class output confusion. We illustrate the advantages of our method using benchmark datasets. In addition, we apply our metric to skin lesion classification—a real-world use case—and show that this yields promising results.

Goal recognisers attempt to infer an agent's intentions from a sequence of observations. Approaches that adapt classical planning techniques to goal recognition have previously been proposed but, generally, they assume the initial world state is accurately defined. In this paper, a state is inaccurate if any fluent's value is unknown or incorrect. To cope with this, a cyclic Action Graph, which models the order constraints between actions, is traversed to label each node with their distance from each hypothesis goal. These distances are used to calculate the posterior goal probabilities. Our experimental results, for 15 different domains, demonstrate that our approach is unaffected by an inaccurately defined initial state.

A key aspect of foraging in robot swarms is optimizing the search efficiency when both the environment and target density are unknown.Hence, designing optimal exploration strategies is desirable.This paper proposes a novel approach that extends the individual Lévy walk to a collective one.To achieve this, we adjust the individual motion through applying an artificial potential field method originating from local communication.We demonstrate the effectiveness of the enhanced foraging by confirming that the collective trajectory follows a heavy-tailed distribution over a wide range of swarm sizes.Additionally, we study target search efficiency of the proposed algorithm in comparison with the individual Lévy walk for two different types of target distributions: homogeneous and heterogeneous.Our results highlight the advantages of the proposed approach for both target distributions, while increasing the scalability to large swarm sizes.Finally, we further extend the individual exploration algorithm by adapting the Lévy walk parameter α, altering the motion pattern based on a local estimation of the target density.This adaptive behavior is particularly useful when targets are distributed in patches.

It has been claimed that the deployment of fog and edge computing infrastructure is a necessity to make high-performance cloud-based applications a possibility. However, there are a large number of middle-ground latency-sensitive applications such as online gaming, interactive photo editing and multimedia conferencing that require servers deployed closer to users than in globally centralised clouds but do not necessarily need the extreme low-latency provided by a new infrastructure of micro data centres located at the network edge, e.g., in base stations and ISP Points of Presence. In this paper we analyse a snapshot of today's data centres and the distribution of users around the globe and conclude that existing infrastructure provides a sufficiently distributed platform for middle-ground applications requiring a response time of 20-200 ms. However, while placement and selection of edge servers for extreme low-latency applications is a relatively straightforward matter of choosing the closest, providing a high quality of experience for middle-ground latency applications that use the more widespread distribution of today's data centres, as we advocate in this paper, raises new management challenges to develop algorithms for optimising the placement of and the per-request selection between replicated service instances.

In model-based Reinforcement Learning, an agent aims to learn a transition model between attainable states. Since the agent initially has zero knowledge of the transition model, it needs to resort to random exploration in order to learn the model. In this work, we demonstrate how the Ornstein-Uhlenbeck process can be used as a sampling scheme to generate exploratory Brownian motion in the absence of a transition model. Whereas current approaches rely on knowledge of the transition model to generate the steps of Brownian motion, the Ornstein-Uhlenbeck process does not. Additionally, the Ornstein-Uhlenbeck process naturally includes a drift term originating from a potential function. We show that this potential can be controlled by the agent itself, and allows executing non-equilibrium behavior such as ballistic motion or local trapping.

Upon the advent of Industry 4.0, collaborative robotics and intelligent automation gain more and more traction for enterprises to improve their production processes. In order to adapt to this trend, new programming, learning and collaborative techniques are investigated. Program-bydemonstration is one of the techniques that aim to reduce the burden of manually programming collaborative robots. However, this is often limited to teaching to grasp at a certain position, rather than grasping a certain object. In this paper, we propose a method that learns to grasp an arbitrary object from visual input. While other learning-based approaches for robotic grasping require collecting a large dataset, manually or automatically labeled in a real or simulated world, our approach requires a single demonstration. We present results on grasping various objects with the Franka Panda collaborative robot after capturing a single image from a wrist mounted RGB camera. From this image we learn a robot controller with a convolutional neural network to adapt to changes in the object's position and rotation with less than 5 minutes of training time on a NVIDIA Titan X GPU, achieving over 90% grasp success rate.

Internet applications rely on strong encryption techniques to protect the content of all communications between client and server. These encryption algorithms ensure that third parties are unable to obtain the plain text data but also make it hard for the network administrator to enforce restrictions on the types of traffic that are allowed. In this paper we show that we can train accurate machine learning models which can predict the type of traffic going through an IPsec or TOR tunnel based on features extracted from the encrypted streams. We use small, fast to execute machine learning models that work on small windows of data. This makes it possible to use our approach in real-time, for example as part of a Quality of Service (QoS) system.

Driven by the increasing popularity of the microservice architecture, we see an increase in services with unknown demand pattern located in the edge network. Predeployed instances of such services would be idle most of the time, which is economically infeasible. Also, the finite storage capacity limits the amount of deployed instances we can offer. Instead, we present an on-demand deployment scheme using the Docker platform. In Docker, service images consist of layers, each layer adding specific functionality. This allows different services to reuse layers, avoiding cluttering the storages with redundant replicas. We propose a layer placement method which allows users to connect to a server, retrieve all necessary layers -possibly from multiple locationsand deploy an instance of the requested service within the desired response time. We search for the best layer placement which maximizes the satisfied demand given the storage and delay constraints. We developed an iterative optimization heuristic which is less exhaustive by dividing the global problem in smaller subproblems. Our simulation results show that our heuristic is able to solve the problem with less system resources. Last, we present interesting use-cases to use this approach in real-life scenarios.

Deep reinforcement learning is becoming increasingly popular for robot control algorithms, with the aim for a robot to self-learn useful feature representations from unstructured sensory input leading to the optimal actuation policy. In addition to sensors mounted on the robot, sensors might also be deployed in the environment, although these might need to be accessed via an unreliable wireless connection. In this paper, we demonstrate deep neural network architectures that are able to fuse information generated by multiple sensors and are robust to sensor failures at runtime. We evaluate our method on a search and pick task for a robot both in simulation and the real world.

Contemporary industrial systems are challenged by fast-growing requirements for agile and effective reactivity to rapidly changing market demands. To achieve this set of requirements, new technologies and paradigms like Internet-of-Things (IoT), Big Data Analytics, Internet-of-Services (IoS), and service-oriented architecture (SOA) are being introduced into the industrial environments. These advances result in the confluence of two dissimilar, but complementary domains: the physical operational technologies (OT) and the cyber information technologies (IT) domains. Convergence of these two domains in a cross-layer fashion implies a new set of two major requirements of components and systems: 1) structural connectivity and 2) functional interoperability. However, the wide variety and heterogeneity of industrial systems-especially in the factory floor-entails integration complexity, which is in contrast with the mentioned requirements. In this paper, we focus on how legacy industrial systems can migrate in a cost-effective manner to the new paradigm of integrated IT-OT levels. We propose an interoperability layer requiring no changes on the legacy device that maps field device data into an ISA95-based information model. The performance of our contributed open-source implementation is evaluated in several deployment configurations.

Using deep neural networks on resource constrained devices is a trending topic in neural network research. Various techniques for compressing neural networks have been proposed that allow evaluating a large neural network on a device with limited memory and processing power. These approaches usually generate a single compressed student network based on a larger teacher network. In some cases a more dynamic trade-off may be desired. In this paper we trained a sequence of increasingly large networks where each network is constrained to contain the unmodified features of all smaller networks. The weight matrix of the largest network has submatrices that correspond to the weight matrices of each of the smaller networks. This technique allows us to keep the parameters of several networks in memory while having the same memory footprint as the single largest network. A trade-off between accuracy and speed can be made at runtime. The proposed approach is validated on two image classification tasks running on a real-world Internet-of-Things (IoT) device.

We see a trend to design services as a suite of small service components instead of the typical monolithic nature of classic web services, which led to an increasing amount of long-tail services on the Internet. Deploying instances everywhere to achieve a fast response time results in high costs, especially when these services are used infrequently and remain idle most of the time. One way to avoid needless over-provisioning is to deploy instances on-demand but this requires every component to be available upon request arrival. We propose a placement algorithm to maximize the amount of clients we can serve on-demand using the Docker layered filesystem. Docker facilitates automated deployment of services in lightweight software containers, allowing almost instantaneous deployment. Our algorithm finds the optimal storage location for layers so we can retrieve all service layers, deploy a service instance and provide a first response to a request within the desired time. We solve this problem using integer linear programming (ILP) and present techniques to improve the scalability of ILP while minimizing the performance loss. Results show that our approximation performs better with large scale problems than the classic ILP case.

The amount of services located in the network has drastically increased over the last decade which is why more and more datacenters are located at the network edge, closer to the users. In the current Internet it is up to the client to select a destination using a resolution service (Domain Name System, Content Delivery Networks). In the last few years, research on Information-Centric Networking (ICN) suggests to put this selection responsibility at the network components, routers find the closest copy of a content object using the content name as input. We extend the principle of ICN to services, service routers forward requests to service instances located in datacenters spread across the network edge. To solve this problem, we first present a service selection algorithm based on both server and network metrics. Next, we describe a method to reduce the state required in service routers while minimizing the performance loss caused by this data reduction. Simulation results based on real-life networks show that we are able to find a near-optimal load distribution with only minimal state required in the service routers.

Deep neural networks are the state of the art technique for a wide variety of classification problems. Although deeper networks are able to make more accurate classifications, the value brought by an additional hidden layer diminishes rapidly. Even shallow networks are able to achieve relatively good results on various classification problems. Only for a small subset of the samples do the deeper layers make a significant difference. We describe an architecture in which only the samples that can not be classified with a sufficient confidence by a shallow network have to be processed by the deeper layers. Instead of training a network with one output layer at the end of the network, we train several output layers, one for each hidden layer. When an output layer is sufficiently confident in this result, we stop propagating at this layer and the deeper layers need not be evaluated. The choice of a threshold confidence value allows us to trade-off accuracy and speed.

Today's centralized cloud-computing infrastructures have not been designed with geo-localized, personalized, bandwidth/processing-intensive, real-time applications in mind. High network delay and low throughput can have a significant impact on the user experience. Instead, such services could be deployed in distributed service nodes at the edge of the network, closer to the user. In this paper we focus on composite services of which the components are running in different service nodes. We present a two-layer framework that provides service orchestration and instance selection. We present the orchestration mechanisms to enable the flexible re-use of components across different composite services. For the resolution layer of our framework, we present two modes of operation that combine network and service availability information for efficient per-request instance selection among a multitude of service replicas.

We present a framework for large-scale crowdsourcing of first-person viewpoint videos recorded on mobile devices. Collecting videos at a massive scale poses a number of major issues in terms of network planning. To improve the scalability with regards to the number of users, videos and geographical area and better cope with restrictions on storage, bandwidth and processing power, the framework is distributed and based on the two-layer cloudlet architecture. To mitigate the limited bandwidth in the access network, a set of decision algorithms is constructed and evaluated that are able to filter out irrelevant videos based on their metadata and given selection criteria. To illustrate the crowdsourcing framework, we present Street View Live, an application for presenting videos based on location, similar to the popular Google Street View but with up-to-date videos covering the location instead of possibly outdated images. In order to have an up-to-date view of every location, the video collection is continuously extended and updated by crowdsourcing videos from mobile devices.

We propose a scalable Internet system for continuous collection of crowd-sourced video from devices such as Google Glass. Our hybrid cloud architecture, GigaSight, is effectively a Content Delivery Network (CDN) in reverse. It achieves scalability by decentralizing the collection infrastructure using cloudlets based on virtual machines~(VMs). Based on time, location, and content, privacy sensitive information is automatically removed from the video. This process, which we refer to as denaturing, is executed in a user-specific VM on the cloudlet. Users can perform content-based searches on the total catalog of denatured videos. Our experiments reveal the bottlenecks for video upload, denaturing, indexing, and content-based search. They also provide insight on how parameters such as frame rate and resolution impact scalability.

Having numerous sensors on-board, smartphones have rapidly become a very attractive platform for augmented reality applications. Although the computational resources of mobile devices grow, they still cannot match commonly available desktop hardware, which results in downscaled versions of well known computer vision techniques that sacrifice accuracy for speed. We propose a component-based approach towards mobile augmented reality applications, where components can be configured and distributed at runtime, resulting in a performance increase by offloading CPU intensive tasks to a server in the network. By sharing distributed components between multiple users, collaborative AR applications can easily be developed. In this poster, we present a component-based implementation of the Parallel Tracking And Mapping (PTAM) algorithm, enabling to distribute components to achieve a mobile, distributed version of the original PTAM algorithm, as well as a collaborative scenario.

In thin client computing, applications are executed on a network server instead of on the user terminal. Since the amount of processing at the terminal is reduced, thin clients are potentially energy efficient devices. However, a network connection between client and server is required for the transmission of user input and display updates. The energy needed for this intense network communication might undo or even exceed the power savings achieved by the reduction in client-side processing. In this paper, we present experimental results on power efficiency of the wireless platform on the thin client in case of thin client traffic. The discussion is focused on VNC-RFB, a widespread thin client protocol, over an IEEE 802.11 link in three typical user scenarios. The results indicate that a cross-layer approach between application and wireless link layer could potentially lead to important power savings.

Despite the growing popularity and advantages of thin-client systems, they still have some important shortcomings. Current thin-client systems are ideally suited to be used with classic office-applications but as soon as multimedia and 3D gaming applications are used they require a large amount of bandwidth and processing power. Furthermore, most of these applications heavily rely on the Graphical Processing Unit (GPU). Due to the architectural design of thin-client systems, they cannot profit from the GPU resulting in slow performance and bad image quality. In this paper, we propose a thin-client system which addresses these problems: we introduce a realtime desktopstreamer using a videocodec to stream the graphical output of applications after GPU-processing to a thin-client device, capable of decoding a videostream. We compare this approach to a number of popular classic thin-client systems in terms of bandwidth, delay and image quality. The outcome is an architecture for a hybrid protocol, which can dynamically switch between a classic thin-client protocol and realtime desktopstreaming.