Future action anticipation aims to infer future actions from the observation of a small set of past video frames. In this paper, we propose a novel Jointly-learnt Action Anticipation Network (J-AAN) via Self-Knowledge Distillation (Self-KD) and cycle consistency for future action anticipation. In contrast to the current state-of-the-art methods which anticipate the future actions either directly or recursively, our proposed J-AAN anticipates the future actions jointly in both direct and recursive ways. However, when dealing with future action anticipation, one important challenge to address is the future’s uncertainty since multiple action sequences may come from or be followed by the same action. Training an action anticipation model with one-hot-encoded hard labels that assign zero probabilities to incorrect yet semantically similar actions may not handle the uncertain future. To address this challenge, we design a Self-KD mechanism to train our J-AAN, where the J-AAN gradually distills its own knowledge during the training to soften the hard labels to model the uncertainty on future action anticipation. Furthermore, we design a forward and backward action anticipation framework with our proposed J-AAN based on a cyclic consistency constraint. The forward J-AAN anticipates the future actions from the observed past actions, and the backward J-AAN verifies the anticipation of the forward J-AAN by anticipating the past actions from the anticipated future actions. The proposed method outperforms all the latest state-of-the-art action anticipation methods on the Breakfast, 50Salads, and EPIC-Kitchens-55 datasets. This project will be publicly available onhttps://github.com/MoniruzzamanMd/J-AAN.

We Introduce a simple yet effective network that embeds a novel Discriminative Feature Pooling (DFP) mechanism and a novel Video Segment Attention Model (VSAM), for video-based human action recognition from both trimmed and untrimmed videos. Our DFP module introduces an attentional pooling mechanism for 3D Convolutional Neural Networks that attentionally pools 3D convolutional feature maps to emphasize the most critical spatial, temporal, and channel-wise features related to the actions within a video segment, while our VSAM ensembles these most critical features from all video segments and learns (1) class-specific attention weights to classify the video segments into the corresponding action categories, and (2) class-agnostic attention weights to rank the video segments based on their relevance to the action class. Our action recognition network can be trained from both trimmed videos in a fully-supervised way and untrimmed videos in a weakly-supervised way. For untrimmed videos with weak labels, our network learns attention weights without the requirement of precise temporal annotations of action occurrences in videos. Evaluated on the untrimmed video datasets of THUMOS14 and ActivityNet1.2, and trimmed video datasets of HMDB51, UCF101, and HOLLYWOOD2, our network achieves promising performance, compared to the latest state-of-the-art method. The implementation code is available athttps://github.com/MoniruzzamanMd/DFP-VSAM-Networks.

The state-of-the-art of fully-supervised methods for temporal action localization from untrimmed videos has achieved impressive results. Yet, it remains unsatisfactory for the weakly-supervised temporal action localization, where only video-level action labels are given without the timestamp annotation on when the actions occur. The main reason comes from that, the weakly-supervised networks only focus on the highly discriminative frames, but there are some ambiguous frames in both background and action classes. The ambiguous frames in background class are very similar to the real actions, which may be treated as target actions and result in false positives. On the other hand, the ambiguous frames in action class which possibly contain action instances, are prone to be false negatives by the weakly-supervised networks and result in a coarse localization. To solve these problems, we introduce a novel weakly-supervised Action Completeness Modeling with Background Aware Networks (ACM-BANets). Our Background Aware Network (BANet) contains a weight-sharing two-branch architecture, with an action guided Background aware Temporal Attention Module (B-TAM) and an asymmetrical training strategy, to suppress both highly discriminative and ambiguous background frames to remove the false positives. Our action completeness modeling contains multiple BANets, and the BANets are forced to discover different but complementary action instances to completely localize the action instances in both highly discriminative and ambiguous action frames. In the i-th iteration, the i-th BANet discovers the discriminative features, which are then erased from the feature map. The partially-erased feature map is fed into the (i+1)-th BANet of the next iteration to force this BANet to discover discriminative features different from the i-th BANet. Evaluated on two challenging untrimmed video datasets, THUMOS14 and ActivityNet1.3, our approach outperforms all the current weakly-supervised methods for temporal action localization.

This study aims at sensing and understanding the worker’s activity in a human-centered intelligent manufacturing system. We propose a novel multi-modal approach for worker activity recognition by leveraging information from different sensors and in different modalities. Specifically, a smart armband and a visual camera are applied to capture Inertial Measurement Unit (IMU) signals and videos, respectively. For the IMU signals, we design two novel feature transform mechanisms, in both frequency and spatial domains, to assemble the captured IMU signals as images, which allow using convolutional neural networks to learn the most discriminative features. Along with the above two modalities, we propose two other modalities for the video data, i.e., at the video frame and video clip levels. Each of the four modalities returns a probability distribution on activity prediction. Then, these probability distributions are fused to output the worker activity classification result. A worker activity dataset is established, which at present contains 6 common activities in assembly tasks, i.e., grab a tool/part, hammer a nail, use a power-screwdriver, rest arms, turn a screwdriver, and use a wrench. The developed multi-modal approach is evaluated on this dataset and achieves recognition accuracies as high as 97% and 100% in the leave-one-out and half-half experiments, respectively.

The cloud manufacturing yields insights of manufacturing services over cyber space based on integration of advanced manufacturing with cloud computing. However, the different communication standards between the different system levels are main challenges for the integration of them without a conflicting communication. Ethernet appears as the best solution to support all levels for industrial manufacturing systems. Since the synchronous integration is fulfilled, the prerequisite for deciding the overall performance of systems is adaptable real-time communication. This motivates us to study the problem of designing a new real-time communication based on Ethernet standard to enable real-time processing for factory networks, termed Real-Time Ethernet (RTEthernet). The objective is to schedule maximum number of machines transmitting data through a controller in a data transmission window (Coordinated Cycle) as well as guarantee the real-time communication. The problem is called the Maximum Number of Real-time Data Transmission (MNRDT) problem. Two relevant solutions are proposed for addressing the MNRDT problem. Simulation results are provided to show the performance of proposed solutions.

The growing number of Additive Manufacturing Web (AMW) services, offered by different providers over the Internet, makes it challenging for consumers to compare these AMW services to select a service of their choice. In addition, it is even more challenging for consumers to compare these AMW services against their personal preferences. This is because, consumers personal preferences on multiple non-functional attributes such as price, material, accuracy and schedule, should be considered for AMW service selection. The decentralized nature of AMW services coupled by the need to consider consumers personal preferences during AMW service selection, requires a system that will serve as a broker between AMW services and consumers. In this paper, we propose a service broker system for AMW services that provides consumers with a single point of access to a large number of AMW services from many additive manufacturing service providers. This broker system also incorporates the first real application of service selection with fuzzy logic based personalized preferences and trade-off. We develop a method to generate fuzzy membership functions for each non-functional attribute. This makes it easy for consumers to specify their fuzzy membership functions. Finally, we present an application case study to demonstrate the feasibility of brokerage in AMW services and also evaluate our method in terms of performance.

The trimming problem for swept volumes — concerning the excision of points ostensibly on the boundary that actually lie in the swept volume interior — is investigated in detail. Building upon several techniques that have appeared in the literature, efficient methods for both local and global trimming of swept volume are developed. These methods are shown to be computationally cost effective when combined with the sweep-envelope differential equation algorithm for the approximate calculation and graphical rendering of swept volumes for quite general objects and sweeps. Examples are presented to demonstrate the efficacy of the trimming strategies.

This paper describes the development of a near-optimal fuzzy controller for maneuvering a car in a parking lot. To generate the rules of the fuzzy controller, a cell mapping method is utilized to systematically generate near-optimal trajectories for all possible initial states in the parking lot. Based on the input-output relations of these trajectories, which represent the states and controls of the corresponding cells, a set of fuzzy rules are generated automatically. In order to result in a small number of fuzzy rules from the large amount of numerical information generated by cell mapping, grouping of trajectories is proposed and each rule applies to the cells in one group. This reduces substantially the number of rules in the fuzzy controller compared with establishing the rules directly using the control data of individual cells.

The kinematic singularities of robot manipulators are studied from the point of view of the theory of singularities. The notion of a 'generic' kinematic map, whose singularities form smooth manifolds of prescribed dimension in the joint space of the manipulator, is examined. For three-joint robots, an equivalent algebraic condition for genericity using the Jacobian determinants is derived. This condition lends itself to symbolic computation and is sufficient for the study of decoupled manipulators. Orientation and translation singularities of manipulators are studied in detail. A complete characterization of orientation singularities of robots with any number of joints is given. The translation singularities of the eight possible topologies of three-joint robots are studied and the conditions on the link parameters for nongenericity are determined.>

Petri net modeling approaches are presented for a flexible workstation for automatic assembly of printed circuit boards. In order to improve the productivity of such a system, the building of mathematical models is a crucial step. Concentrating on the operational aspects, the authors construct ordinary and temporal Petri net models for the AT&T FWS-200 physical flexible workstation. Three outcomes follow from such models. First, designers can have a better understanding of concurrency, synchronization, mutual exclusion, and sequential relations involved in the system control from the graphical representations of Petri nets. Second, the performance analysis of system operations under different settings can be conducted. Thus, the results can be used to help designers choose the best operational setting on a basis of the system parameters. Third, the models can be used as an aid for automatic generation of real-time control programs and construction of Petri-net-based simulation if needed. The approaches suggested can be generalized to many other applications of multirobot assembly systems.>

A cell-mapping method is introduced for planning global trajectories of robotic manipulators in cases where the cell space is composed of combination pairs of plane cells. With the proposed method, optimal trajectory problems in the free field and in the obstacle-constrained field are studied. Two numerical examples are given to show the obtained optimal trajectories and controls.>

The singularities of the differential kinematic map, i.e. of the manipulator Jacobian, are considered. The authors first examine the notion of a generic kinematic map, whose singularities form smooth manifolds of prescribed dimension in the joint space of the manipulator. For three-joint robots, an equivalent condition for genericity using determinants is derived. The condition lends itself to symbolic computation and is sufficient for the study of decoupled manipulators, i.e. manipulators that an be separated into a three-joint translating part and a three-joint orienting part. The results are illustrated by analyzing the singularities of two classes of three-joint positioning robots.>