Biomedical documents contain a wealth of multimodal data, including visual evidence and natural language descriptions. Effectively navigating this vast and expanding data source poses significant challenges, requiring labor-intensive manual curation processes at scale. Multimodal learning methodologies offer a promising approach to mitigate these bottle-necks in domain-specific information retrieval tasks, especially in biomedical sciences. This research investigates the potential of multimodal learning methodologies for various downstream tasks, such as information retrieval and classification based on paired multimodal image-text datasets. The proposed solution employs state-of-the-art transformer-based deep learning techniques and a contrastive learning approach to design a foundational model capable of adapting to biomedical vision-language processing. Through experiments and comparative studies, the effectiveness of our model was validated, demonstrating superior performance compared to existing approaches across diverse standard datasets. This study emphasizes the challenges in domain-specific environments and underscores the critical role of high-quality multimodal data in advancing biomedical AI applications, fostering innovation and progress in healthcare research and practice.

Zero trust networks have emerged as a promising solution to assure comprehensive security in network environments. Different from the traditional perimeter-based security approach, zero trust networks provide a robust and adaptable security framework which addresses the evolving threat landscape and enables organizations to protect their critical assets with a higher assurance of confidence. However, the centralized policy engine employed in current zero trust architectures (ZTA) would introduce bottlenecks and single points of failure (SPoF) for ZTA-based networks, thus hindering the scalability and efficiency as network size increases. This paper introduces a novel decentralized host-to-host authentication schema that enables consistent policy engine decisions in a pair-wise manner. By decentralizing the authentication process, the proposed schema effectively eliminates bottlenecks and single points of failure associated with centralized policy engines. The system incorporates a decentralized authentication ledger and a policy validation protocol to ensure the correct and consistent authentication across all network hosts. Through comprehensive tests and simulations, we compared our proposed novel model with the traditional zero trust network, in terms of the correctness, time complexity, and efficiency. Our findings demonstrate the advantages of our decentralized approach and its potential for enhancing security in zero trust networks.

We consider the problem of resource provisioning for real-time cyber-physical applications in an open system environment where there does not exist a global resource scheduler that has complete knowledge of the real-time performance requirements of each individual application that shares the resources with the other applications. Regularity-based Resource Partition (RRP) model is an effective strategy to hierarchically partition and assign various resource slices among such applications. However, previous work on RRP model only discusses uniform resource environment, where resources are implicitly assumed to be synchronized and clocked at the same frequency. The challenge is that a task utilizing multiple resources may experience unexpected delays in non-uniform environments, where resources are clocked at different frequencies. This paper extends the RRP model to non-uniform multi-resource open system environments to tackle this problem. It first introduces a novel composite resource partition abstraction and then proposes algorithms to construct and reconfigure the composite resource partitions. Specifically, the Acyclic Regular Composite Resource Partition Scheduling (ARCRP-S) algorithm constructs regular composite resource partitions and the Acyclic Regular Composite Resource Partition Dynamic Reconfiguration (ARCRP-DR) algorithm reconfigures the composite resource partitions in the run time upon requests of partition configuration changes. Our experimental results show that compared with state-of-the-art methods, ARCRP-S can prevent unexpected resource supply shortfall and improve the schedulability up to 50%. On the other hand, ARCRP-DR can guarantee the resource supply during the reconfiguration with moderate computational overhead.

As biofilms research grows rapidly, a corpus of bibliographic literature (i.e., documents) is increasing at an incredible rate. Many researchers often need to inspect these large document collections, including (1) text, (2) images, and (3) captions, to understand underlying biological mechanisms and make a critical decision. However, researchers have great difficulty in exploring such ever-growing large datasets in labor-intensive processes. Thus, automation of such tasks is urgently required for the automatic identification or classification of a large volume of document collections. To address this problem, we present a multimodal deep learning-based approach to automatically classify documents for a specialized information retrieval technique based on biofilm images, captions, and texts, which is a major source of information for the classification of documents. Images, captions, and texts from biofilm documents are represented in a large vector space. Then, they are fed into convolutional neural networks (CNNs), to improve similarity matching and relevance. Our extensive experiments and analysis will take captions, texts, or images as unimodal models as inputs and concatenate them all into multimodal models. The trained models for this classification approach in turn help a search engine to precisely identify relevant and domain-specific documents from a large volume of document collections for further research direction in biofilm development.

The onboarding of IoT devices by authorized users constitutes both a challenge and a necessity in a world, where the number of IoT devices and the tampering attacks against them continuously increase. Commonly used onboarding techniques today include the use of QR codes, pin codes, or serial numbers. These techniques typically do not protect against unauthorized device access-a QR code is physically printed on the device, while a pin code may be included in the device packaging. As a result, any entity that has physical access to a device can onboard it onto their network and, potentially, tamper it (e.g., install malware on the device). To address this problem, in this paper, we present a framework, called Deep Learning-based Watermarking for authorized IoT onboarding (DLWIoT), featuring a robust and fully automated image watermarking scheme based on deep neural networks. DLWIoT embeds user credentials into carrier images (e.g., QR codes printed on IoT devices), thus enables IoT onboarding only by authorized users. Our experimental results demonstrate the feasibility of DLWIoT, indicating that authorized users can onboard IoT devices with DLWIoT within 2.5-3sec.

Digital image watermarking is the process of embedding and extracting a watermark covertly on a cover-image. To dynamically adapt image watermarking algorithms, deep learning–based image watermarking schemes have attracted increased attention during recent years. However, existing deep learning–based watermarking methods neither fully apply the fitting ability to learn and automate the embedding and extracting algorithms, nor achieve the properties of robustness and blindness simultaneously. In this paper, a robust and blind image watermarking scheme based on deep learning neural networks is proposed. To minimize the requirement of domain knowledge, the fitting ability of deep neural networks is exploited to learn and generalize an automated image watermarking algorithm. A deep learning architecture is specially designed for image watermarking tasks, which will be trained in an unsupervised manner to avoid human intervention and annotation. To facilitate flexible applications, the robustness of the proposed scheme is achieved without requiring any prior knowledge or adversarial examples of possible attacks. A challenging case of watermark extraction from phone camera–captured images demonstrates the robustness and practicality of the proposal. The experiments, evaluation, and application cases confirm the superiority of the proposed scheme.

Utilizing autonomous robots to perform repetitive and labor-intensive tasks in the construction industry is one of the most promising directions to explore in order to enhance productivity, safety/health, and quality of construction projects. Such robots must have construction-related knowledge and skills in order to generate task plans capable of dealing with the unique and highly dynamic work environment of typical construction sites. However, autonomous and flexible behavior is currently impossible due to the lack of a robotics-compatible construction knowledge base. To overcome this bottleneck, this study proposes the establishment and utilization of such a construction knowledge base for use in generating autonomous behavior in robots. Specifically, this study provides an implementation of a small, mobile, autonomous robotics platform capable of performing fine-grained construction tasks in dynamic environments. Such tasks include painting, drilling screws, and transporting material and equipment. The platform is tested with a simulated robot based on the KUKA youBot tasked with painting walls in a room containing obstacles. In the simulation results, the proposed approach shows promise in being able to achieve autonomous operation of construction robots. Further development of this study will include implementing a more diverse set of skills, expanding the construction knowledge base, and tailoring localization, navigation planning, and task planning algorithms for the characteristics of the construction sites and the hardware tools used.

We consider the problem of resource provisioning for real-time cyber-physical applications in an open system environment where there does not exist a global resource scheduler that has complete knowledge of the real-time performance requirements of each individual application that shares the resources with the other applications. Regularity-based Resource Partition (RRP) model is an effective strategy to hierarchically partition and assign various resource slices among the applications. However, RRP model does not consider changes in resource requests from the applications at run time. To allow for the run time adaptation to change resource requirements, we consider in this paper the issues in online resource partition reconfiguration, including semantics issues that arise in configuration transitions that may cause application failures. Based on the reconfiguration semantics, we study the online resource reconfigurability problem under the RRP model where the availability factors of resource partitions may be reconfigured during run time. We formalize the Dynamic Partition Reconfiguration (DPR) problem and provide a solution to this problem. Extensive experiments have been conducted to evaluate the performance of the proposed approach in different scenarios. We also present a case study using the autonomous F1/10 model car; the controller of the F1/10 car requires resource adaptation to satisfy the computing needs of its PID controller and vision system under different operating conditions. Our implementation demonstrates the effectiveness and benefit of online resource partition reconfiguration using the DPR approach in a real system.

A cyberphysical avatar is a semi-autonomous robot that adjusts to an unstructured environment and performs physical tasks subject to critical timing constraints while under human supervision. This article first realizes a cyberphysical avatar that integrates three key technologies: body-compliant control, neuroevolution, and real-time constraints. Body-compliant control is essential for operator safety, because avatars perform cooperative tasks in close proximity to humans; neuroevolution (NEAT) enables “programming” avatars such that they can be used by non-experts for a large array of tasks, some unforeseen, in an unstructured environment; and real-time constraints are indispensable to provide predictable, bounded-time response in human-avatar interaction. Then, we present a study on the tradeoffs between three design parameters for robotic task systems that must incorporate at least three dimensions: (1) the amount of training effort for robot to perform the task, (2) the time available to complete the task when the command is given, and (3) the quality of the result of the performed task. A tradeoff study in this design space by using the imprecise computation as a framework is to perform a common robotic task, specifically, grasping of unknown objects. The results were validated with a real robot and contribute to the development of a systematic approach for designing robotic task systems that must function in environments like flexible manufacturing systems of the future.

Applying wireless technologies in cyber-physical systems (CPSs) has received significant attention in recent years. In our previous work, a high-speed and flexible real-time wireless communication protocol called RT-WiFi was designed to support a wide range of CPSs, and we presented an implementation with a single access point (AP). To serve the CPS applications with communication nodes geographically distributed over a large area, multicluster RT-WiFi networks with multiple APs need to be deployed. Although effective scheduling algorithms have been designed to schedule tasks in RT-WiFi networks with a single AP, uncoordinated packet transmissions from multicluster RT-WiFi networks may suffer from cochannel interferences that cause performance degradation. The multicluster RT-WiFi network management problem is to resolve the cochannel interference through channel assignment for clusters and through phasing assignment for communication tasks. In this article, we first derive a conjunctive normal form encoding of the problem and design a TScheduler that searches feasible solutions through the SAT solver. A novel LRTree Scheduler is further designed to solve the problem in chain graphs while keeping the number of used channels small and the network management overhead low. A testbed of the multicluster RT-WiFi network is deployed to validate the design of the multicluster RT-WiFi network and evaluate the performance of the proposed scheduling algorithms compared to the contention-based methods in regular WiFi networks. Performance of these scheduling algorithms in large-scale networks is further evaluated through extensive simulations on both static and dynamic multicluster RT-WiFi networks.

Ready-to-assemble furniture is a popular trend for today’s furniture companies such as IKEA due to its relative lower price and easier delivery to customers than assembled furniture. However, assembling furniture from an instruction manual by customers themselves is a tedious task. With the advance in robotics in recent years, having a robot to perform the furniture assembly is a viable idea but the cost of robotic furniture assembly is a barrier, as the overhead of using artificial intelligence techniques such as deep learning can be prohibitive. This paper presents a robotic system with a library of assembly skills that are acquired by machine learning and can be reused for different furniture sets. By applying these skills, the robot can be programmed to automatically perform the assembly task. We describe how to design a robotic task programming system that supports composition of skills and how to specify a complex assembly task to be completed by the robot with its skill set.

Although modern robots in warehousing systems can perform adequately in a goods-to-person model using hand-designed algorithms that are specialized to a particular environment, developing a robotic system that is capable of handling new products at an inexpensive cost remains a challenge. A conspicuous example of this challenge is seen in Amazon's use of autonomous robots to fetch customers' orders in their massive warehouses. To encourage advance in this technology, Amazon organized the competition, Amazon Picking Challenge that asked participants to develop their own hardware and software for the general task of picking a designated set of products from inventory shelves and then placing them at a target location (called a pick-and-place task). Current technology for pick-and-place tasks is still insufficient to meet the demand for low-cost automation. Handling awkward or oddly shaped object must still depend on hand-programming or specialized robotic systems, making manufacturing automation less flexible and expensive. In this paper, we shall present the design and implementation of a software system that is a step in advancing the technology toward full automation at reasonable costs. Our system integrates a set of state-of-the-art techniques in computer vision, deep-learning, trajectory optimization, visual servoing to create a library of skills that can be composed to perform a variety of robotic tasks. We demonstrate the capability of our system for performing autonomous pick-and-place tasks with an implementation using Hoppy, an industrial robotic arm in an environment similar to the Amazon Picking Challenge.

In open systems, no global scheduler has knowledge of the complete resource requirements from all the applications. Each application has its own task group and can generate tasks on demand at run time. Regularity-based Resource Partition (RRP) model is an effective strategy to hierarchically allocate resource in such environments. However, when applying the RRP model to multi-resource environments, end-to-end tasks could experience unexpected delay and miss the deadlines. The tasks might arrive at non-resource-slice boundaries because the resource slice sizes of different physical resource may vary in such non-uniform environments. This paper extends the RRP model to non-uniform multi-resource open systems. It introduces a novel composite resource partition abstraction, identifies the feasible conditions for hierarchical regular composite resource partitioning and proposes an acyclic regular composite resource partition scheduling (ARCRPS) algorithm. Simulation results show that compared with the state-of-the-art approach, ARCRPS improves the acceptance ratio by 20% and 25% in uniform and non-uniform multi-resource environments, respectively. A multi-resource scheduling framework jointly considering the CPU and network resources is also designed and implemented to evaluate the feasibility of this theoretical model in practice.

We present a study on the tradeoffs between three design parameters for robotic task systems that function in partially unknown and unstructured environments, and under timing constraints. The design space of these robotic tasks must incorporate at least three dimensions: (1) the amount of training effort to teach the robot to perform the task, (2) the time available to complete the task from the point when the command is given to perform the task, and (3) the quality of the result from performing the task. This paper presents a tradeoff study in this design space for a common robotic task, specifically, grasping of unknown objects in unstructured environments. The imprecise computation model is used to provide a framework for this study. The results were validated with a real robot and contribute to the development of a systematic approach for designing robotic task systems that must function in environments like flexible manufacturing systems of the future.

Robotic grasping of a target object without advance knowledge of its three-dimensional model is a challenging problem. Many studies indicate that robot learning from demonstration (LfD) is a promising way to improve grasping performance, but complete automation of the grasping task in unforeseen circumstances remains difficult. As an alternative to LfD, this paper leverages limited human supervision to achieve robotic grasping of unknown objects in unforeseen circumstances. The technical question is what form of human supervision best minimizes the effort of the human supervisor. The approach here applies a human-supplied bounding box to focus the robot's visual processing on the target object, thereby lessening the dimensionality of the robot's computer vision processing. After the human supervisor defines the bounding box through the man-machine interface, the rest of the grasping task is automated through a vision-based feature-extraction approach where the dexterous hand learns to grasp objects without relying on pre-computed object models through the NEAT neuroevolution algorithm. Given only low-level sensing data from a commercial depth sensor Kinect, our approach evolves neural networks to identify appropriate hand positions and orientations for grasping novel objects. Further, the machine learning results from simulation have been validated by transferring the training results to a physical robot called Dreamer made by the Meka Robotics company. The results demonstrate that grasping novel objects through exploiting neuroevolution from simulation to reality is possible.

Localization in wireless sensor networks is an important functionality that is required for tracking personnel and assets in industrial environments, especially for emergency response. Current commercial localization systems such as GPS suffer from the limitations of either high cost or low availability in many situations (e.g., indoor environments that exclude direct line-of-sight signal reception). The development of industrial wireless sensor networks such as WirelessHART provides an alternative. In this article, we present the design and implementation of ColLoc: a collaborative location and tracking system on WirelessHART as an industrially viable solution. This solution is built upon several technological advances. First, ColLoc adds the roaming functionality to WirelessHART and thus provides a means for keeping mobile WirelessHART devices connected to the network. Second, ColLoc employs a collaborative framework to integrate different types of distance measurements into the location estimation algorithm by weighing them according to their precision levels. ColLoc adopts several novel techniques to improve distance estimation accuracy and decreases the RSSI presurvey cost. These techniques include introducing distance error range constraints to the measurements, judiciously selecting the initial point in location estimation and online updating the signal propagation models in the anchor nodes, integrating Extended Kalman Filter (EKF) with trilateration to track moving objects. Our implementation of ColLoc can be applied to any WirelessHART-conforming network because no modification is needed on the WirelessHART field devices. We have implemented a complete ColLoc system to validate both the design and the effectiveness of our localization algorithm. Our experiments show that the mobile device never drops out of the WirelessHART network while moving around; with the help of even one dependable anchor, using RSSI can yield at least 75% of distance errors below 5 meters, which is quite acceptable for many typical industrial automation applications.

Localization in wireless sensor networks is an important functionality that is required for tracking personnel and assets in industrial environments, especially for emergency response. Current commercial localization systems such as GPS suffer from the limitations of either high cost or low availability in many situations (e.g., in-door environments that exclude direct line-of-sight signal reception). The development of industrial wireless sensor networks such as Wireless Hart provides an alternative. In this paper, we present the design and implementation of Roaming Hart: a collaborative localization system on Wireless Hart as an industrially viable solution. This solution is built upon several technological advances. First, Roaming Hart adds the roaming functionality to Wireless Hart and thus provides a means for keeping mobile Wireless Hart devices connected to the network. Second, Roaming Hart employs a collaborative framework to integrate different types of distance measurements into the location estimation algorithm by weighing them according to their precision levels. Roaming Hart adopts several novel techniques to improve distance estimation accuracy and decreases the RSSI pre-survey cost. These techniques include introducing distance error range constraints to the measurements, judiciously selecting the initial point in location estimation and on line updating the signal propagation models in the anchor nodes. Our implementation of Roaming Hart can be applied to any Wireless Hart-conforming network because no modification is needed on the Wireless Hart field devices. We have implemented a complete Roaming Hart system to validate both the design and the effectiveness of our localization algorithm. Our experiments show that the mobile device never drops out of the Wireless Hart network while moving around, with the help of even one dependable anchor, using RSSI can yield at least 75% of distance errors below 5 meters, which is quite acceptable for many typical industrial automation applications.

The traditional sampling method in process control systems is based on a periodic task model. This is because controllers are executed in a strictly periodic manner. Sensors sample the process data and send it periodically to the appropriate controllers through a communication system such as the field bus. Since the field bus is shared by multiple sensors, there is some delay (control loop latency)between the sampling and control actions. In order to minimize the control loop latency, a higher than necessary sampling frequency is typically adopted, which results in unnecessary waste of energy. In this paper, we propose Min Max: a sampling interval control algorithm for tackling this problem. In Min Max, sampling tasks are not periodic but have both maximum and minimum distance constraints. This sampling model has advantages that are especially important in the domain of wireless control for industrial automation. We shall then discuss the jitter property of sampling schemes under this model and propose algorithms for controlling the sampling intervals of sensors in terms of the Min Max problem (UMin Max) which we shall introduce. Though this problem is NP-hard in general, even for special case of unit-time tasks, we show how to reduce Min Max to well-studied scheduling models such as Liu and Layland-type periodic models and pinwheel models, at the expense of some loss of schedulability. These reductions allow us to derive efficient schedulability tests that can be used to solve the sampling interval control problem in practice. Simulations are used to compare the performance of different UMin Max schedulers in two key figures of merit: the acceptance ratio and the jitter ratio. Simulation of a process control system model also shows that UMin Max can reduce about 40% of the traffic load on the communication system which is especially important for energy-aware wireless process control applications.

In this paper, we report on the design and implementation of MBStar, a higher-frequency, real-time, reliable, secure protocol for wireless body area networks (WBAN). As in most proposals for body sensor networks, MBStar adopts the star topology for communication, and is designed to support a message rate as high as 400 Hz, which to the best of our knowledge, is the highest among low-power wireless communication protocols implemented at the present time. The physical layer of MBStar utilizes 802.15.4 DSSS compatible radio for which a higher-frequency, reliable, TDMA MAC layer is built. There is a simple application layer designed for security on top of it. MBStar utilizes public/private key encryption for provisioning devices and does not involve any human configuration before device join. Considering the resource limit of most embedded systems, the TDMA requirement of computing a shared global communication schedule presents a practical problem since it may not be feasible for all the devices to communicate in a long hyper-period while the communication schedule between devices is being created or modified as devices depart and rejoin. We solve this problem by keeping only the global hyper-period schedule on the gateway side, with each device being configured with a shorter, local period. Then, retransmission is employed to resolve any conflicts between the devices. Our strategy has the property that, given any fixed task set, the minimal average number of retransmissions is independent of any communication scheduling algorithm, and the EDF (Earliest Deadline First) is optimal for our communication architecture. Finally, we present experimental results that demonstrate that MBStar is an effective protocol for wireless body area networks.

Peak power consumption of buildings in large facilities like hospitals and universities becomes a big issue because peak prices are much higher than normal rates. During a power demand surge an automated power controller of a building may need to schedule ON and OFF different environment actuators such as heaters and air quality control while maintaining the state variables such as temperature or air quality of any room within comfortable ranges. The green scheduling problem asks whether a scheduling policy is possible for a system and what is the necessary and sufficient condition for systems to be feasible. In this paper we study the feasibility of the green scheduling problem for HVAC(Heating, Ventilating, and Air Conditioning) systems which are approximated by a discrete-time model with constant increasing and decreasing rates of the state variables. We first investigate the systems consisting of two tasks and find the analytical form of the necessary and sufficient conditions for such systems to be feasible under certain assumptions. Then we present our algorithmic solution for general systems of more than 2 tasks. Given the increasing and decreasing rates of the tasks, our algorithm returns a subset of the state space such that the system is feasible if and only if the initial state is in this subset. With the knowledge of that subset, a scheduling policy can be computed on the fly as the system runs, with the flexibility to add power-saving, priority-based or fair sub-policies.

Real-time disk scheduling is an important research topic for time-critical multimedia applications. Some well-known research results, such as SCAN-earliest deadline first (EDF) and DM-SCAN, applied the SCAN scheme to reschedule service sequence of input tasks and reduce their service time. In this paper, we prove that the general disk-scheduling problem with linear cost-function is NP hard. We also propose the shortest-task-first-DM, a new real-time disk-scheduling algorithm using the concept of the shortest-task-first and the deadline modification. As shown in the experimental results, our approach can schedule more tasks to meet their deadlines.

Data broadcast is an efficient information delivery model that can deliver information to a large population simultaneously. In this paper, we propose two efficient algorithms to broadcast real-time and non-real-time data together. The goal of our algorithms is to reduce the average response time of non-real-time data under the constraint that all real-time data must meet their deadlines. The experimental results show that our proposed algorithms can reduce the average response time while guaranteeing the timing constraints.