Abstract We address multi-view pedestrian detection in a setting where labeled data is collected using a multi-camera setup different from the one used for testing. While recent multi-view pedestrian detectors perform well on the camera rig used for training, their performance declines when applied to a different setup. To facilitate seamless deployment across varied camera rigs, we propose an unsupervised domain adaptation (UDA) method that adapts the model to new rigs without requiring additional labeled data. Specifically, we leverage the mean teacher self-training framework with a novel pseudo-labeling technique tailored to multi-view pedestrian detection. This method achieves state-of-the-art performance on multiple benchmarks, including MultiviewX $$\rightarrow $$ Wildtrack. Unlike previous methods, our approach eliminates the need for external labeled monocular datasets, thereby reducing reliance on labeled data. Extensive evaluations demonstrate the effectiveness of our method and validate key design choices. By enabling robust adaptation across camera setups, our work enhances the practicality of multi-view pedestrian detectors and establishes a strong UDA baseline for future research.

The effects of personnel scheduling on the quality of care and working conditions for healthcare personnel have been thoroughly documented. However, the ever-present demand and large variation of constraints make healthcare scheduling particularly challenging. This problem has been studied for decades, with limited research aimed at applying Satisfiability Modulo Theories (SMT). SMT has gained momentum within the formal verification community in the last decades, leading to the advancement of SMT solvers that have been shown to outperform standard mathematical programming techniques.In this work, we propose generic constraint formulations that can model a wide range of real-world scheduling constraints. Then, the generic constraints are formulated as SMT and MILP problems and used to compare the respective state-of-the-art solvers, Z3 and Gurobi, on academic and real-world inspired rostering problems. Experimental results show how each solver excels for certain types of problems; the MILP solver generally performs better when the problem is highly constrained or infeasible, while the SMT solver performs better otherwise. On real-world inspired problems containing a more varied set of shifts and personnel, the SMT solver excels. Additionally, it was noted during experimentation that the SMT solver was more sensitive to the way the generic constraints were formulated, requiring careful consideration and experimentation to achieve better performance. We conclude that SMT-based methods present a promising avenue for future research within the domain of personnel scheduling.

The motivation behind this article stems from potential gains to be made by optimizing the movement profile of Automated Guided Vehicles (AGVs) in an industrial setting. By minimizing the energy consumption of an AGV, increased range, larger recharging intervals, and possibly financial benefits can be achieved. Previous research has shown that high acceleration rates can have a negative impact on the average energy consumption of an AGV, while others suggest that using higher speed may lead to energy savings. In this article a test case is built using the Simplex Motion SH 100B BLDC motor on an AGV in the production line. Using two such identical motors, a test rig is built where one motor acts as the driving motor and the other as the brake. Using an Arduino micro controller and a current sensor, power measurements are taken for the development of a power model for this motor. A simulation model presented for the movement and power consumption of an AGV equipped with two such motors, determine the optimal values for the acceleration rate, cruising speed, and deceleration rate, and estimate the potential energy savings.

We consider unsupervised domain adaptation (UDA) for semantic segmentation in which the model is trained on a labeled source dataset and adapted to an unlabeled target dataset. Unfortunately, current self-training methods are susceptible to misclassified pseudo-labels resulting from erroneous predictions. Since certain classes are typically associated with less reliable predictions in UDA, reducing the impact of such pseudo-labels without skewing the training towards some classes is notoriously difficult. To this end, we propose an extensive cut-and-paste strategy (ECAP) to leverage reliable pseudo-labels through data augmentation. Specifically, ECAP maintains a memory bank of pseudo-labeled target samples throughout training and cut-and-pastes the most confident ones onto the current training batch. We implement ECAP on top of the recent method MIC and boost its performance on two synthetic-to-real domain adaptation benchmarks. Notably, MIC+ECAP reaches an unprecedented performance of 69.1 mIoU on the Synthia $\rightarrow$ Cityscapes benchmark. Our code is available at https://github.com/ErikBrorsson/ECAP.

Conventional programming of explicit control code is unsuitable for flexible and collaborative production systems. A model-based approach, which focuses on defining capabilities of a system, instead of specifying how to achieve them, provides an alternative for creating complex, scalable, and reliable systems. This is accomplished through the use of behavior models, and tools such as planning, synthesis, verification, and testing. However, developing such models is not without challenges, as it is possible to overlook or incorrectly specify potential behavior and constraints. This can result in unsolvable planning problems or plans that are invalid for other reasons. When plans are unobtainable, developers receive no feedback, which makes model adjustments a difficult and time-intensive task. This paper recognizes these challenges as crucial barriers for adopting model-based development of intelligent automation systems. To facilitate the development of such systems, an approach for detecting and localizing faults in behavior models is presented. Drawing inspiration from software fault localization techniques, the proposed method involves identifying suspicious resources, variables, and operations. The effectiveness of this approach is illustrated with an example use case.

This paper investigates the problem of reducing the energy use of robot stations with stochastic execution times in production lines in automotive factories. First, real stochastic cycle time data is used to analyze and improve the cycle time. The result shows that the cycle time mean can be decreased with the cost of an increase in cycle time variance. Second, the cycle time data is combined with energy models of real robot stations. A stochastic optimization problem is formulated where the goal is to reduce the energy use of the stations in the production line by lowering the robot velocities, while not affecting the cycle time of the production line. The optimization problem is solved and the resulting energy optimized station is simulated using the improved cycle time. The result shows that up to 23 percent of the energy use can be reduced by only marginally affecting the cycle time variance of the production line.

Fast planning algorithms play a key role in intelligent automation systems where control sequences are constantly calculated. In order to determine which algorithms increase planning performance, we evaluate and compare several high level planning methods on a set of standard benchmarks. We focus on planning as satisfiability as the leading approach for solving difficult planning problems.

This paper investigates the problem of reducing the energy use of the movements of robots in industrial robot stations that have variations in execution times. An online method is presented that repeatedly solves an optimization problem during the execution of the station, that tries to minimize the energy use by finding the optimal execution times of the robot movements while at the same ensuring that the deadline of the station is met with a high enough probability. The method involves reformulating the original optimization problem, which is stochastic and nonlinear, into a convex version that can be solved efficiently. The method is tested on a simulated robot station and the result shows that the method is fast enough to be useable online and reduces the energy use of the station.

We propose an efficient optimization method, which addresses several performance criteria, such as makespan, maximum lateness, and the sum of tardiness for an automated guided vehicle (AGV) system, together with its energy consumption. We show that the most important factors in energy consumption of AGVs are their cruise velocities and traveled distances. We also demonstrate that optimizing the productivity-related performance criteria also reduces energy consumption through less traveled distance. It also allows for the reduction of the cruise velocity, which leads to more energy savings. Our experiments demonstrate that the optimization method outperforms the existing traffic controller with respect to the performance criteria and reduces energy consumption. The proposed method can reduce the energy consumption by around 38%, while the values of makespan, lateness, and tardiness remain better than those obtained from the existing traffic controller. An important advantage of this article is that the evaluations are based on collected data from a real large-scale manufacturing plant. Note to Practitioners-It is commonly believed that reduction of speed, for example, due to safety reasons in critical areas, in automated guided vehicle (AGV) systems leads to lower system efficiency. However, it has been shown that speed management is an effective strategy to reduce the energy consumption of mobile robots and robot stations. If one seeks to utilize the existing slacks in the schedule of the AGVs, it should be possible to reduce energy consumption without affecting system efficiency. It can, furthermore, be combined with better scheduling to even improve performance measures, such as makespan, while reducing energy. In this article, we propose an optimization method that seeks to minimize the number of performance measures, such as makespan, maximum lateness, and sum of tardiness for a real AGV system designed by AGVE, which operates at Volvo Cars, Gothenburg, Sweden. We also show that the optimization method allows for reduction of cruise speed, while the mentioned performance measures are still better than the one obtained from the original traffic controller. We will also show the importance of taking into account the temperature of the drive system of the AGVs when performing energy measurements.

The following topics are dealt with: production engineering computing; learning (artificial intelligence); mobile robots; Internet of Things; control engineering computing; factory automation; protocols; embedded systems; manufacturing systems; condition monitoring.

The development of efficient propositional satisfiability problem solving algorithms (SAT solvers) in the past two decades has made automated planning using SAT-solvers an established AI planning approach. Modern SAT solvers can accommodate a wide variety of planning problems with a large number of variables. However, fast computing of reasonably long plans proves challenging for planning as satisfiability. In order to address this challenge, we present a compositional approach based on abstraction refinement that iteratively generates, solves and composes partial solutions from a parameterized planning problem. We show that this approach decomposes the monolithic planning problem into smaller problems and thus significantly speeds up plan calculation, at least for a class of tested planning problems.

Automation and robotics are two enablers for developing the Smart Factory of the Future, which is based on intelligent machines and collaboration between robots and humans. Especially in final assembly and its material handling, where traditional automation is challenging to use, collaborative robot (cobot) systems may increase the flexibility needed in future production systems. A major obstacle to deploy a truly collaborative application is to design and implement a safe and efficient interaction between humans and robot systems while maintaining industrial requirements such as cost and productivity. Advanced and intelligent control strategies is the enabler when creating this safe, yet efficient, system, but is often hard to design and build.This paper highlights and discusses the challenges in meeting safety requirements according to current safety standards, starting with the mandatory risk assessment and then applying risk reduction measures, when transforming a typical manual final assembly station into an intelligent collaborative station. An important conclusion is that current safety standards and requirements must be updated and improved and the current collaborative modes defined by the standards community should be extended with a new mode, which in this paper is refereed to the deliberative planning and acting mode.

Collaborative and intelligent automation systems need intelligent control systems. Some of this intelligence exist on a per-component basis in the form of vision, sensing, motion, and path planning algorithms. To fully take advantage of this intelligence, also the coordination of subsystems need to exhibit intelligence. While there exist middleware solutions that eases communication, development, and reuse of such subsystems, for example the Robot Operating System (ROS), good coordination also requires knowledge about how control is supposed to be performed, as well as expected behavior of the subsystems. This paper introduces lightweight components that wraps ROS2 nodes into composable control components from which an intelligent control system can be built. The ideas are implemented on a use case involving collaborative robots with on-line path planning, intelligent tools, and human operators.

In order to adapt to stricter system delivery and integration requirements, virtual commissioning (VC) has become a well adopted practice in industry. VC is getting increasingly integrated into the overall engineering process, where the control software is continuously tested with the virtual plant model. At the same time, collaborative and intelligent automation systems are becoming an important part of modern industries. In these complex systems, humans perform operations together with collaborative robots, intelligent machines and smart tools. However, performing VC of such complex, distributed and heterogeneous systems demands new ways of interfacing different hardware and software components. This paper discusses the requirements, process and results of integrated virtual commissioning of an industrial collaborative and intelligent automation system use-case. Moreover, this industrial use-case illustrates challenges and exemplifies the need to use the next generation Robot Operating System (ROS2) due to its robust communication layer as well as easy integration with smart devices and algorithms.

During the last five decades, automation and robotics have transformed the automotive industry by increasing efficiency and improving the product quality. However, future trucks that will be autonomous, electrical and connected will require a completely new type of flexibility and intelligence in the production systems, especially in the final assembly. To handle the increased complexity of the products, production processes and logistic systems, final assembly must be transformed into collaborative and intelligent automation systems. These systems will include collaborative and deliberative robots (cobots), advanced vision-based control, adaptive safety systems, online optimization and learning algorithms and connected and well-informed human operators. But it will be a huge undertaking to transform current trucks industry such that they can design, implement and maintain large scale collaborative and intelligent automation systems. This paper presents the challenges with current planning and preparation processes for final assembly as well as the requirement and possible solutions for the future processes. An industrial use case at Volvo Trucks based on Sequence Planner and ROS2 is used to evaluate the proposed planning and preparation processes.

This paper proposes an optimization model for optimizing the energy use of industrial robots in production systems affected by stochastic disturbances. In the model there are a number of operations that needs to be completed by the robots before a deadline. The operations are of two types, one type that can not be controlled and have stochastic execution times. The other type are robot movements, and by extending their execution times energy can be saved. The goal of the optimization is to find the optimal combination of execution times for the robot movements, while meeting the deadline with a given probability.

Home healthcare (HHC) is a service that dispatches caregivers to people in need of healthcare who live in the home. The task assignment and route generation for caregivers can be formulated as an extension of the well-known vehicle routing problem with time windows (VRPTW). Currently, the most successful exact algorithms for VRPTW are based on a framework combining column generation (CG) and branching. Although such methods could be successful for HHC routing and scheduling problem (HHCRSP) as well, fast approximate algorithms are appealing, especially for large problems. In an early version of this paper, we employed a heuristic distributed gossip algorithm to solve HHCRSP. In this paper, we integrate the gossip algorithm with a local solver based on CG, which makes it an effective algorithm for larger problem instances. As it will be shown with extensive numerical experiments, for large problem instances, gossip-CG performs better than the pure CG.

The ISO 22400 has defined a set of Key Performance Indicators (KPIs) to evaluate the performance of manufacturing operation. Though the calculation formulas and tendency analysis of KPIs are given in the standard, it is important to evaluate how the standard performs when implemented in an industrial setting. Based on analyzing the production process in Volvo car, an aggregation KPI evaluation system is proposed to evaluate the operation performance. Two failure KPIs defined in the ISO 22400, mean operation time between failure (MTBF) and mean time to failure (MTTF), are selected in the system. By calculating the failure KPIs in different levels and analyzing the working state of each station in two operating modes, the applicability and effectiveness of the proposed aggregation KPI evaluation system based on the standard is demonstrated.

Many people in need of care still live in their homes, requiring the caretakers to travel to them. Assigning the people to caretakers and generating their schedules can be formulated as a mixed integer linear programming problem (MILP) that inherits many features of the well-known vehicle routing problem with time windows (VRPTW). Currently, the most successful exact algorithms for VRPTW are based on branch and price framework, which combine column generation (CG) and branching. While these methods could be successful for Home Healthcare Routing and Scheduling Problem (HHCRSP) as well, fast approximate algorithms are appealing, especially for large problems. We recently employed a heuristic distributed gossip algorithm to solve HHCRSP. The method had the potential to provide approximate solutions for relatively large problem instances, but its effectiveness was limited to the performance of its local MILP solver. In this paper, we integrate the gossip algorithm with a local solver based on CG, which makes it an effective algorithm for larger problem instances. We also provide numerical experiments and complexity evaluations of the improved gossip algorithm (gossip-CG) with the standard gossip (gossip-MILP) and CG, and show that gossip-CG outperforms the pure CG in case of large problems.

This paper, as an outcome of the EU project AREUS, heralds an optimization procedure that reduces up to 30% of energy consumption and up to 60% in peak power for the trajectories that have been tested on real industrial robots. We have evaluated a number of cost functions and tested our algorithm for a variety of scenarios such as varying cycle times, payloads, and single/multirobot cases in both ac- and dc-operated robot cells. The significance of our work is not only in the impressive savings, simplicity of implementation, and preserving path and cycle time, but also in the variety of test scenarios that include different kinds of KUKA robots. We have carried out the optimization and experiments in as realistic conditions as possible.

In this paper, a predicate transition model for discrete-event systems is generalized to include continuous dynamics, and the result is a modular hybrid predicate transition model. Based on this model, a hybrid Petri net including explicit differential equations and shared variables is also proposed. It is then shown how this hybrid Petri net model can be optimized based on a simple and robust nonlinear programming formulation. The procedure only assumes that desired sampled paths for a number of interacting moving devices are given, while originally equidistant time instances are adjusted to minimize a given criterion. This optimization of hybrid systems is also applied to a real robot station with interacting devices, which results in about 30% reduction in energy consumption. Moreover, a flexible online and event-based information architecture called the Tweeting Factory is proposed. Simple messages (tweets) from all kinds of equipment are combined into high-level knowledge, and it is demonstrated how this information architecture can be used to support optimization of robot stations.

It is challenging to get an overview and understanding of the activities and their relations at an emergency department (ED). This is due to the complicated relations among activities - called operations in this paper - and the always changing system behavior. Two key enablers to get this overview are the transformation of real-time events into understandable information and an operation-based behavior description. This paper presents an event-based information architecture for healthcare (EVAH) for gathering state changing events in realtime at an ED and how these are transformed into an operation-based representation. EVAH together with the tool Sequence Planner, will be used for visualization, online prognosis and optimization.

The execution behavior of a system or a product can often be specified by a set of operations (sometimes called tasks, actions or activities). The process to specify these operations seems to be a real challenge in various situations, for example when designing automation systems or keeping track of the work at an emergency department. To be flexible during design and development, is about coping with uncertainty. However, in practice, specifying operation behavior is often quite inflexible because every possible execution route is explicitly defined. This approach is sometimes refereed to as point-based engineering, which is characterized by the early selection and approval of a single “best” specific solution. This approach can result in neither a robust nor a flexible design process with many rework iterations. This paper instead suggests a set-based operation specification approach that does not explicitly defines operation routes, where instead the operation behavior is specified using the execution restrictions in transition conditions for each operation. This enables the possibility to create multiple projections of the operation relations to enable better understanding. This is accomplished by creating various Sequences of Operations, sequences of operations (SOP), including a multiplicity of sequences and operation relations. Two example case studies are presented, where the first show the development of an automation system and the second study uses sequence projections in an emergency department at a hospital to handle the complex and flexible operation (task) behavior.

The sequential behavior of a manufacturing system results from several constraints introduced during the product, manufacturing, and control logic development. This paper proposes methods and algorithms for automatically representing and visualizing this behavior from various perspectives throughout the development process. A new sequence planning approach is introduced that uses self-contained operations to model the activities and execution constraints. These operations can be represented and visualized from multiple perspectives using a graphical and formal language called Sequences of Operations (SOPs). The operations in a manufacturing system are related to each other in various ways, due to execution constraints expressed by operation pre- and post-conditions. These operation relations include parallel, sequence, arbitrary order, alternative, and hierarchy relations. Based on the SOP language, these relations are identified and visualized in various SOPs and sequences. A software tool, Sequence Planner, has been developed, for organizing the operations into SOPs that visualize only relevant operations and relations.

In flexible manufacturing systems a large number of operations need to be coordinated and supervised to avoid blocking and deadlock situations. The synthesis of such supervisors soon becomes unmanageable for industrial manufacturing systems, due to state space explosion. In this paper we therefore develop some reduction principles for a recently presented model based on self-contained operations and sequences of operations. First sequential operation behaviors are identified and related operation models are simplified into one model. Then local transitions without interaction with other operation models are removed. This reduction principle is applied to a synthesis of non-blocking operation sequences, where collisions among moving devices are guaranteed to be avoided by a flexible booking process. The number of states in the synthesis procedure and the computation time is reduced dramatically by the suggested reduction principle.

In order to obtain a unified information flow from early product design to final production, an integrated framework for product, process and automation design is presented. The framework is based on sequences of operations and includes a formal relation between product properties and process operations. This relation includes liaisons (interfaces) and precedence relations, where the precedence relations generate preconditions for the related process operations. From this information a set of sequences of operations (SOPs) is generated. A formal graphical language for hierarchical operations and SOPs is then introduced and defined based on automata extended with variables. Since the operations are self-contained they can be grouped and viewed from different angles, e.g., from a product or a resource perspective. These multiple views increase the interoperability between different engineering disciplines. A case study is performed on a car manufacturing cell, where the suggested modeling framework is shown to give comprehensible SOPs.