Trigger-action programming (TAP) is an Internet of Things (IoT) paradigm that enables nonprofessional end-users to automate smart devices by formulating rules, such as “IF you leave home, THEN turn off lights.” As the number of possible rules increases, manually browsing these rules becomes increasingly time-consuming for users. Recently, graph-based recommendation systems have shown promise in automatically suggesting rules, yet they face two issues. First, these studies struggle to identify and differentiate users’ demands (e.g., turning off lights) and intentions (e.g., energy saving). Second, they overlook the issue of sparse user-rule interactions. In this article, we propose LLM4TAP, a large language model (LLM) enhanced TAP rule recommendation framework, to address these issues. Prior to LLM4TAP, a user-rule graph is constructed to represent the interactions between users and rules. Within LLM4TAP, singular value decomposition is first employed to generate an augmented graph, strengthening global structural relationships between users and rules. Next, the reasoning capabilities of LLMs are utilized to infer users’ demands and intentions from the textual descriptions of rules and user-rule interactions, producing representations of these inferred demands and intentions. Finally, a dual representation alignment method is introduced, integrating user demands and intentions derived from LLMs with the global structural information from the augmentation graph within a contrastive learning framework to enhance representation performance. Extensive experiments demonstrate the effectiveness of LLM4TAP, achieving the maximum improvements of 8.96% and 4.72% over the strongest compared methods on the IFTTT and Wyze datasets, respectively.

The increased utilization of Trigger-Action Programming (TAP) rules in smart homes has raised concerns regarding potential security threats in the interactions between smart digital devices/online services (DD/OS) and the physical environment. To ensure the secure use of intelligent and convenient infrastructure for users, we introduce an approach aimed at detecting potential security threats. In this paper, we propose IoT security threat models and categorize the threats into Risky DD/OS with Physical Security, Contradictory Operation of DD/OS and Environmental Impact Conflict. To effectively detect security threats, we construct an Internet of Things-Heterogeneous Information Networks (IoT-HIN) and enhance it with a knowledge base tool, transforming it into a knowledge-based IoT-HIN. We establish meta-paths to conduct analysis of events triggered by rules, and a threat detection algorithm is proposed to identify potential security threats and determine the rules leading to these threats. The proposed approach is validated using a real-world dataset, and the experimental results demonstrate its efficiency and practicality. Furthermore, a comparative analysis with similar works is conducted to highlight the superiority of our proposed approach.

Nowadays, users can personalize Internet of Things (IoT) devices/web services via trigger-action programming (TAP). As the number of connected entities grows, the relations of triggers and actions become progressively complex (i.e., the heterogeneity of TAP), which becomes a challenge for existing models to completely preserve the heterogeneous data and semantic information in trigger and action. To address this issue, in this article, we propose IoT nonnegative matrix factorization (IoT-NMF), a NMF-based heterogeneous graph embedding method for TAP. Prior to using IoT-NMF, we map triggers and actions to an IoT heterogeneous information network, from which we can extract three structures that preserve heterogeneous relations in triggers and actions. IoT-NMF can factorize the structures simultaneously for getting low-dimensional representation vectors of the triggers and actions, which can be further utilized in Artificial Intelligence of Things applications (e.g., TAP rule recommendation). Finally, we demonstrate the proposed approach using an if this then that (IFTTT) dataset. The result shows that IoT-NMF outperforms the state-of-the-art approaches.

The advent of fifth generation (5G) enables the Internet of Things (IoT) to connect a massive number of things. The interaction and communication among these things generate an enormous amount of context-aware data that is semantically diverse. Traditional data representation approaches, such as semantic annotation, ontology, and semantic Web technology are rule based, which lack flexibility and adaptability when applied to IoT. To address the challenge, this article mainly focuses on the problem of semantic representation, which is essential for processing and fusion of IoT data. To serve as a bridge, we propose a high-level framework, namely, Things2Vec, which aims to produce the latent semantic representations from the interaction of things through the graph embedding technique. These semantic representations benefit various IoT semantic analysis tasks, such as the IoT service recommendation and automation of things. In Things2Vec, we utilize the graph to model the function sequence relationships that are generated by the interaction of things, which is called the IoT context graph. Since these function sequence relationships are heterogeneous in terms of semantics, it causes general graph embeddings to fail to learn complete information. Thus, we propose a biased random walk procedure, which is tailored to capture the neighborhoods of nodes with different types of semantic relationships. Extensive experiments are carried out, and our results show that the proposed method can effectively capture the semantic relationships among context-aware data in IoT.

Landuse classification is an important problem in the remote sensing field. It can be used in a wide range of applications. In this paper, we propose a hybrid method fusing edges and regions information for the landuse classification of multispectral images. It mainly includes the steps of image pre-processing, initial segmentation and region merging. Especially, a novel spatial mean shift procedure is proposed so that some information can be extracted and used in the successive steps. Aiming at the multispectral images processing, we also design a band weighting strategy that give a proper weight to each band adoptively according to the region to be processed. Experimental results on the Landsat TM and ETM+ images validate the performance of the proposed method.

A dike detection method based on active contour model is introduced in this paper for our flood management project. Experimental result on DEM (Digital Elevation Model) of Jingjiang River of China shows that a continuous and closed dike description could be gotten by this method.