To address growing concerns about data privacy on mobile devices, the federated learning (FL) paradigm enables clients to collaboratively train models while sharing only local model updates. However, privacy risks remain in FL, as adversaries can still infer sensitive information from these updates. To enhance secure aggregation in FL, various protection mechanisms combining encryption and multi-party computation (MPC) have been proposed. These approaches, however, often introduce substantial communication and computational overhead, making secure aggregation impractical on resource-constrained devices, e.g., smart phones. To tackle these efficiency challenges, we are among the first to propose the integration of differential privacy (DP) with encryption and MPC for secure aggregation. Our proposed protocol, Federated Learning with Noise-based Secure Aggregation (FedNSA), injects noise through DP to obfuscate individual model updates. Encryption is employed to correlate the noise across different clients, while MPC ensures perfect noise cancellation at the server side. Finally, we theoretically analyze its advantages and conduct extensive experiments on public datasets to demonstrate the superiority of our approach across multiple dimensions in comparison with the state-of-the-art baselines.

Federated Learning (FL) has emerged as a pivotal paradigm for multi-client collaborative learning, primarily due to its inherent capability to safeguard privacy. Nonetheless, the heterogeneity among FL clients, characterized by their disparate resource capabilities and not independent and identical (NonIID) local datasets, presents a significant challenge. Specifically, low-resource clients, such as edge devices, grapple with the inadequacy to accommodate the entire model parameter set for training purposes. To mitigate this issue, preceding research has ventured into devising methodologies that entail extracting sub-models from the overarching global model, tailored to the specific communication, computational, and memory constraints of individual clients. Despite these advancements, prevailing sub-model extraction techniques, which predominantly hinge on pre-established rules, overlook a crucial factor: the impact of NonIID local data on the trajectory of neuron update dynamics. This oversight can amplify discrepancies between the practical local updates inferred by the sub-model and those anticipated via the entire model, thereby undermining overall performance. In this paper, we introduceFedGSE, an innovative Gradient-based Neuron Selection methodology designed explicitly for FL environments. This methodology aims to curate sub-models that significantly reduce discrepancies in local updates, enhancing alignment with the global model's learning trajectory. Central to theFedGSEapproach is a sophisticated algorithm that handpicks critical neurons for sub-model construction. These neurons are identified through their pronounced gradient magnitudes, resulting from the training of the global model on a dataset mirroring the client's data distribution. Consequently, the sub-model's induced local gradient updates closely emulate those derived from directly training the client's data on the full global model, fostering enhanced alignment and performance. Extensive experiments over diverse datasets and tasks demonstrate the superiority ofFedGSEover existing baselines.

The scarcity data of medical field brings the collaborative training in medical vision-language pre-training (VLP) cross different clients. Therefore, the collaborative training in medical VLP faces two challenges: First, the medical data requires privacy, thus can not directly shared across different clients. Second, medical data distribution across institutes is typically heterogeneous, hindering local model alignment and representation capabilities. To simultaneously overcome these two challenges, we propose the framework called personalized model selector with fused multimodal information (PMS-FM). The contribution of PMS-FM is two-fold: 1) PMS-FM uses embeddings to represent information in different formats, allowing for the fusion of multimodal data. 2) PMS-FM adapts to personalized data distributions by training multiple models. A model selector then identifies and selects the best-performing model for each individual client. Extensive experiments with multiple real-world medical datasets demonstrate the superb performance of PMS-FM over existing federated learning methods on different zero-shot classification tasks.

Optimization and sampling based algorithms are two branches of methods in machine learning, while existing safe reinforcement learning (RL) algorithms are mainly based on optimization, it is still unclear whether sampling based methods can lead to desirable performance with safe policy. This paper formulates the Langevin policy for safe RL, and proposes Langevin Actor-Critic (LAC) to accelerate the process of policy inference. Concretely, instead of parametric policy, the proposed Langevin policy provides a stochastic process that directly infers actions, which is the numerical solver to the Langevin dynamic of actions on the continuous time. Furthermore, to make Langevin policy practical on RL tasks, the proposed LAC accumulates the transitions induced by Langevin policy and reproduces them with a generator. Finally, extensive empirical results show the effectiveness and superiority of LAC on the MuJoCo-based and Safety Gym tasks.

The segmentation of 3D shapes is a critical aspect of shape analysis. However, most existing methods for 3D shape segmentation treat each face of the original mesh model with equal importance. This uniform approach becomes problematic in areas where the faces are smaller but denser, especially around the junctions of different segments. In such regions, greater importance should be assigned compared to the flatter areas. To address this issue, this paper proposes a novel 3D shape segmentation method that incorporates attentive nonuniform sampling into the segmentation pipeline. By leveraging a transformer-based mechanism, our method adaptively identifies the intricate details of 3D shapes, calculating varying degrees of attention to each face. Consequently, the mesh model is downsampled by eliminating faces with lower attention, thereby optimizing the segmentation process. Our approach outperforms most state-of-the-art methods on multiple public datasets, making it a promising avenue for future research.

Piecewise Linear Approximation (PLA) is one of the most widely used approaches for representing a time series with a set of approximated line segments. With this compressed form of representation, many large complicated time series can be efficiently stored, transmitted and analyzed. In this article, with the introduced concept of "semi-connection" that allowing two representation lines to be connected at a point between two consecutive time stamps, we propose a new optimal linear-time PLA algorithm SemiOptConnAlg for generating the least number of semi-connected line segments with guaranteed maximum error bound. With extended experimental tests, we demonstrate that the proposed algorithm is very efficient in execution and achieves better performances than the state-of-art solutions.

3-D human pose estimation or human tracking has always been the focus of research in the human–computer interaction community. As the calibration step of human pose estimation, subject-specific modeling is crucially important to the subsequent pose estimation process. It not only provides a priori knowledge but also clearly defines the tracking target. This article presents a fully automatic subject modeling framework to reconstruct human pose, shape, as well as the body texture in a challenging optimization scenario. By integrating powerful differentiable rendering into the subject-specific modeling pipeline, the proposed method transforms the texture reconstruction problem into analysis by synthesis minimization and solves it efficiently by a gradient-based method. Furthermore, a novel covariance matrix adaptation annealing algorithm is proposed to attack the high-dimensional multimodal optimization problem in an adaptive manner. The domain knowledge of hierarchical human anatomy is seamlessly injected to the annealing optimization process by using a soft covariance matrix mask. All together contributes to the novel algorithm robust to the temptation of local minima. Experiments on the Human3.6 M dataset and the People-Snapshot dataset demonstrate the competitive results to the state of the art both qualitatively and quantitatively.

Summary Nowadays, people use multiple devices to meet the growing requirement for computing. With the application of multicard computing, fault tolerance, load balance, and resource sharing have been the hot issues and the checkpoint/restart (CPR) mechanism is critical in a preemptive system. This article proposes a CPR framework including the automatic compiler (CRAC) to achieve a feasible CPR system, especially for graphics processing unit applications on heterogeneous devices in OpenCL programs. By offering the positions of the CPR in source code, CRAC inserts primitives into programs and invokes the runtime support modules for final results. A comprehensive example and experiments have demonstrated the feasibility and effectiveness of proposed framework.

Abstract As a fundamental part of indoor scene understanding, the research of indoor room layout estimation has attracted much attention recently. The task is to predict the structure of a room from a single image. In this paper, we illustrate that this task can be well solved even without sophisticated post-processing program, by adopting Feature Pyramid Networks (FPN) to solve this problem with adaptive changes. The proposed model employs two strategies to deliver quality output. First, it can predicts the coarse positions of key points correctly by preserving the order of these key points in the data augmentation stage. Then the coordinate of each corner point is refined by moving each corner point to its nearest image boundary as output. Our method has demonstrated great performance on the benchmark LSUN dataset on both processing efficiency and accuracy. Compared with the state-of-the-art end-to-end method, our method is two times faster at processing speed (32 ms) than its speed (86 ms), with 0.71% lower key point error and 0.2% higher pixel error respectively. Besides, the advanced two-step method is only 0.02% better than our result on key point error. Both the high efficiency and accuracy make our method a good choice for some real-time room layout estimation tasks.

Piecewise Linear Approximation (PLA) is one of the most widely used approaches for representing a time series with a set of approximated line segments. With this compressed form of representation, many large complicated time series can be efficiently stored, transmitted and analyzed. In this article, with the introduced concept of “semi-connection” that allowing two representation lines to be connected at a point between two consecutive time stamps, we propose a new optimal linear-time PLA algorithm SemiOptConnAlg for generating the least number of semi-connected line segments with guaranteed maximum error bound. With extended experimental tests, we demonstrate that the proposed algorithm is very efficient in execution time and achieves better performances than the state-of-art solutions.

Detecting points of interest on 3D shapes is a fundamental research problem in geometry processing. Due to the complicated relationship between points of interest and their geometric features, detecting points of interest on any given 3D shape remains challenging. Due to the lack of training data, previous data-driven methods for detecting 3D points of interest mainly focus on utilizing hand-crafted geometric features to predict the probabilities of each point being a POI, which greatly limits detection performance. In this paper, we propose a novel algorithm for detecting 3D points of interest by using projective neural networks. Our method first projects the labeled training 3D shapes into multiple 2D views and then learns the required features from the 2D views in an end-to-end fashion. The points of interest on test 3D shapes are then automatically detected by applying the learned neural network and our improved density peak clustering. Our method relies neither on hand-crafted feature descriptors nor a large quantity of expensive 3D training data to obtain satisfactory results. Experimental results show significantly superior detection performance of our method over the state-of-the-art methods.

Abstract Facial expression recognition (FER) is a significant research task in the computer vision field. In this paper, we present a novel network FaceCaps for facial expression recognition with the following novel characteristics: an embedding structure based on a Capsule network which encodes relative spatial relationships between features; incorporates the feature polymerization property of FaceNet, thus offering a more efficient approach to discriminate complex facial expressions; a target reconstruction loss as a better regularization term for Capsule networks. Experimental results on both lab‐controlled datasets (CK+) and real‐world databases (RAF‐DB and SFEW 2.0) demonstrate that the method significantly outperforms the state‐of‐the‐art.

The surge of medical and e-commerce applications has generated tremendous amount of data, which brings people to a so-called “Big Data” era. Different from traditional large data sets, the term “Big Data” not only means the large size of data volume but also indicates the high velocity of data generation. However, current data mining and analytical techniques are facing the challenge of dealing with large volume data in a short period of time. This paper explores the efficiency of utilizing the Normal Distribution (ND) method for splitting and processing large volume medical data in cloud environment, which can provide representative information in the split data sets. The ND-based new model consists of two stages. The first stage adopts the ND method for large data sets splitting and processing, which can reduce the volume of data sets. The second stage implements the ND-based model in a cloud computing infrastructure for allocating the split data sets. The experimental results show substantial efficiency gains of the proposed method over the conventional methods without splitting data into small partitions. The ND-based method can generate representative data sets, which can offer efficient solution for large data processing. The split data sets can be processed in parallel in Cloud computing environment.

Checkpoint/restart is an important mechanism to achieve fault tolerance, load balancing and resources sharing in a preemptive system. As Graphics Processing Unit (GPU) becomes quite popular in high performance computing as well as OpenCL programs are portable across various CPUs and GPUs, checkpoint/restart of OpenCL programs on GPUs is in demand. However, due to the intricacy of computation states inside GPUs, there is no effective checkpoint/restart scheme for heterogeneous devices now. This paper proposes a feasible system, CRState, to achieve checkpoint/restart in GPU kernels. With the assistant of a pre-compiler, the primitives are inserted into programs. In run-time, the computation state existing in the underlying hardware is concretized and reconstructed at application level and is ported to heterogeneous devices. Comprehensive experiments have been conducted to demonstrate CRState's feasibility and effectiveness. The experimental results also indicate that CRState has the potential to reschedule resources and balance workload across heterogeneous devices.

Complex social network analysis methods have been applied extensively in various domains including online social media, biological complex networks, etc. Complex social networks are facing the challenge of information overload. The demands for efficient complex network analysis methods have been rising in recent years, particularly the extensive use of online social applications, such as Flickr, Facebook and LinkedIn. This paper aims to simplify the network complexity through partitioning a large complex network into a set of less complex networks. Existing social network analysis methods are mainly based on complex network theory and data mining techniques. These methods are facing the challenges while dealing with extreme large social network data sets. Particularly, the difficulties of maintaining the statistical characteristics of partitioned sub-networks have been increasing dramatically. The proposed Normal Distribution (ND) based method can balance the distribution of the partitioned sub-networks according to the original complex network. Therefore, each subnetwork can have its degree distribution similar to that of the original network. This can be very beneficial for analyzing sub-divided networks and potentially reducing the complexity in dynamic online social environment.

The study of urban networks reveals that the accessibility of important city objects for the vehicle traffic and pedestrians is significantly correlated to the popularity, micro-criminality, micro-economic vitality, and social liveability of the city, and is always the chief factor in regulating the growth and expansion of the city. The accessibility between different components of an urban structure are frequently measured along the streets and routes considered as edges of a planar graph, while the traffic ultimate destination points and street junctions are treated as vertices. For estimation of the accessibility of destination vertex$j$from vertex$i$through urban networks, in particular, the random walks are used to calculate the expected distance a random walker starting from$i$makes before$j$is visited (known asaccess time). The state-of-the-art of access time computation is costly in large planar graphs since it involves matrix operation over entire graph. The time complexity is$O(n^{2.376})$where$n$is the number of vertices in the planar graph. To enable efficient access time query answering in large planar graphs, this work proposes the first access time oracle which is based on the proposed access time decomposition and reconstruction scheme. The oracle is a hierarchical data structure with deliberate design on the relationships between different hierarchical levels. The storage requirement of the proposed oracle is$O(n^{\frac{4}{3}}\log \log n)$and the access time query response time is$O(n^{\frac{2}{3}})$. The extensive tests on a number of large real-world road networks (with up to about 2 million vertices) have verified the superiority of the proposed oracle.

This paper presents a semi-supervised method for categorizing human actions using multiple visual features. The proposed algorithm simultaneously learns multiple features from a small number of labeled videos, and automatically utilizes data distributions between labeled and unlabeled data to boost the recognition performance. Shared structural analysis is applied in our approach to discover a common subspace shared by each type of feature. In the subspace, the proposed algorithm is able to characterize more discriminative information of each feature type. Additionally, data distribution information of each type of feature has been preserved. The aforementioned attributes make our algorithm robust for action recognition, especially when only limited labeled training samples are provided. Extensive experiments have been conducted on both the choreographed and the realistic video datasets, including KTH, Youtube action and UCF50. Experimental results show that our method outperforms several state-of-the-art algorithms. Most notably, much better performances have been achieved when there are only a few labeled training samples.

Online human gesture recognition has a wide range of applications in computer vision, especially in human-computer interaction applications. The recent introduction of cost-effective depth cameras brings a new trend of research on body-movement gesture recognition. However, there are two major challenges: (i) how to continuously detect gestures from unsegmented streams, and (ii) how to differentiate different styles of the same gesture from other types of gestures. In this article, we solve these two problems with a new effective and efficient feature extraction method—Structured Streaming Skeleton (SSS)—which uses a dynamic matching approach to construct a feature vector for each frame. Our comprehensive experiments on MSRC-12 Kinect Gesture, Huawei/3DLife-2013, and MSR-Action3D datasets have demonstrated superior performances than the state-of-the-art approaches. We also demonstrate model selection based on the proposed SSS feature, where the classifier of squared loss regression with l 2,1 norm regularization is a recommended classifier for best performance.

State-of-the-art studies on cyberbullying detection, using text classification, predominantly take it for granted that streaming text can be completely labelled. However, the rapid growth of unlabelled data generated in real time from online content

This paper addresses the challenges in detecting the potential correlation between numerical data streams, which facilitates the research of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The uncertainty on the correlation occurrence and the time delay make it difficult to monitor the correlation online. Furthermore, the conventional correlation measure lacks the ability of reflecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Transform to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity measure into the framework, which refines the results by representative trend patterns to enhance the significance of linearity. The similarity of proposed linear representations can quickly estimate the correlation, and the window sliding strategy in segment level improves the efficiency for online detection. The empirical study demonstrates the accuracy of our detection approach, as well as more than $30\%$ improvement of efficiency.

Online human gesture recognition has a wide range of applications in computer vision, especially in human-computer interaction applications. Recent introduction of cost-effective depth cameras brings on a new trend of research on body-movement gesture recognition. However, there are two major challenges: i) how to continuously recognize gestures from unsegmented streams, and ii) how to differentiate different styles of a same gesture from other types of gestures. In this paper, we solve these two problems with a new effective and efficient feature extraction method that uses a dynamic matching approach to construct a feature vector for each frame and improves sensitivity to the features of different gestures and decreases sensitivity to the features of gestures within the same class. Our comprehensive experiments on MSRC-12 Kinect Gesture and MSR-Action3D datasets have demonstrated a superior performance than the stat-of-the-art approaches.

Piecewise Linear Representation (PLR) has been a widely used method for approximating data streams in the form of compact line segments. The buffer-based approach to PLR enables a semi-global approximation which relies on the aggregated processing of batches of streamed data so that to adjust and improve the approximation results. However, one challenge towards applying the buffer-based approach is allocating the necessary memory resources for stream buffering. This challenge is further complicated in a multi-stream environment where multiple data streams are competing for the available memory resources, especially in resource-constrained systems such as sensors and mobile devices.

Human action recognition in videos draws strong research interest in computer vision because of its promising applications for video surveillance, video annotation, interactive gaming, etc. However, the amount of video data containing human actions is increasing exponentially, which makes the management of these resources a challenging task. Given a database with huge volumes of unlabeled videos, it is prohibitive to manually assign specific action types to these videos. Considering that it is much easier to obtain a small number of labeled videos, a practical solution for organizing them is to build a mechanism which is able to conduct action annotation automatically by leveraging the limited labeled videos. Motivated by this intuition, we propose an automatic video annotation algorithm by integrating semi-supervised learning and shared structure analysis into a joint framework for human action recognition. We apply our algorithm on both synthetic and realistic video datasets, including KTH [20], CareMedia dataset [1], Youtube action [12] and its extended version, UCF50 [2]. Extensive experiments demonstrate that the proposed algorithm outperforms the compared algorithms for action recognition. Most notably, our method has a very distinct advantage over other compared algorithms when we have only a few labeled samples.

Online video steam data is surging to an unprecedented level. Massive video publishing and sharing impose heavy demands on continuous video near-duplicate detection for many novel video applications. This paper presents an accurate and accelerated system for video near-duplicate detection over continuous video streams. We propose to transform a high-dimensional video stream into a one-dimensional Video Trend Stream (VTS) to monitor the continuous luminance changes of consecutive frames, based on which video similarity is derived. In order to do fast comparison and effective early pruning, a compact auxiliary signature named CutSig is proposed to approximate the video structure. CutSig explores cut distribution feature of the video structure and contributes to filter candidates quickly. To scan along a video stream in a rapid way, shot cuts with local maximum AI (average information) in a query video are used as reference cuts, and a skipping approach based on reference cut alignment is embedded for efficient acceleration. Extensive experimental results on detecting diverse near-duplicates in real video streams show the effectiveness and efficiency of our method.

Tree patterns represent important fragments of XPath. In this paper, we show that some classes of tree patterns exhibit such a property that, given a finite number of tree patterns P1, ..., Pn, there exists another pattern P (tree pattern or DAG-pattern) such that P1, ..., Pn, are all contained in P, and for any tree pattern Q belonging to a given class C, P1, ..., Pn, are contained in Q implies P is contained in Q.

Constructing Haar wavelet synopses under a given approximation error has many real world applications. In this paper, we take a novel approach towards constructing unrestricted Haar wavelet synopses under an error bound on uniform norm (L∞). We provide two approximation algorithms which both have linear time complexity and a (log N)-approximation ratio. The space complexities of these two algorithms are O (log N) and O (N) respectively. These two algorithms have the advantage of being both simple in structure and naturally adaptable for stream data processing. Unlike traditional approaches for synopses construction that rely heavily on examining wavelet coefficients and their summations, the proposed construction methods solely depend on examining the original data and are extendable to other findings. Extensive experiments indicate that these techniques are highly practical and surpass related ones in both efficiency and effectiveness.

Abstract Scientific workflow offers a framework for cooperation between remote and shared resources on a grid computing environment (GCE) for scientific discovery. One major function of scientific workflow is to schedule a collection of computational subtasks in well‐defined orders for efficient outputs by estimating task duration at runtime. In this paper, we propose a novel time computation model based on algorithm complexity (termed as TCMAC model) for high‐level data intensive scientific workflow design. The proposed model schedules the subtasks based on their durations and the complexities of participant algorithms. Characterized by utilization of task duration computation function for time efficiency, the TCMAC model has three features for a full‐aspect scientific workflow including both dataflow and control‐flow: (1) provides flexible and reusable task duration functions in GCE; (2) facilitates better parallelism in iteration structures for providing more precise task durations; and (3) accommodates dynamic task durations for rescheduling in selective structures of control flow. We will also present theories and examples in scientific workflows to show the efficiency of the TCMAC model, especially for control‐flow. Copyright © 2009 John Wiley & Sons, Ltd.

In this paper, we study the maintenance of role-based access control (RBAC) models in database environments using transitive closure relations. In particular, the algorithms that express and remove redundancy from a component, a RBAC state, and from conflict constraints. The transitive closure relations on a RBAC state specify the reachability among user groups, roles and from user groups to roles. These relations can assist the process of authorization and make some queries easier to answer. Paper [17] shows that the transitive closure relations on a RBAC model can be used to manage and maintain the model's dynamic changes in a simple and efficient way. In this paper, we firstly show that the transitive closure relations are natural byproducts when formulating RBAC components. We then adapt the conventional RBAC model to accord the inherent reachability of a RBAC model. We show that the use of transitive closure relations as the auxiliary relations for the maintenance of a RBAC state alleviates the process of query evaluation, removing redundancy and the description of hierarchies. Thirdly, in the presence of conflict constraints, we explain how conflicts can be expressed, checked and evaluated under the existence of TC relations, in addition to the removal of conflicts redundancy and finding inferred conflicts. Lastly, we briefly discuss the first-order maintenance operations.All the algorithms for the maintenance are first-order algorithms with simple structures and can be implemented in SQL.

Given a database, the view maintenance problem is concerned with the efficient computation of the new contents of a given view when updates to the database happen. We consider the view maintenance problem for the situation when the database contains a weighted graph and the view is either the transitive closure or the answer to the all-pairs shortest-distance problem ( APSD ). We give incremental algorithms for APSD , which support both edge insertions and deletions. For transitive closure, the algorithm is applicable to a more general class of graphs than those previously explored. Our algorithms use first-order queries, along with addition (+) and less-than (<) operations ( FO (+,<)); they store O ( n 2 ) number of tuples, where n is the number of vertices, and have AC 0 data complexity for integer weights. Since FO (+,<) is a sublanguage of SQL and is supported by almost all current database systems, our maintenance algorithms are more appropriate for database applications than nondatabase query types of maintenance algorithms.