The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized augmentation was recently proposed in ELECTRA and achieved state-of-the-art performance by introducing an auxiliary generation network (generator) to produce contextualized data augmentation for the training of a main discrimination network (discriminator). This design, however, introduces extra computation cost of the generator and a need to adjust the relative capability between the generator and the discriminator. In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs. Essentially, this strategy eliminates a separate generator and uses the single network to jointly conduct two pre-training tasks with MLM (Masked Language Modeling) and RTD (Replaced Token Detection) heads. It avoids the challenge to search for an appropriate size of the generator, which is critical to the performance as evidenced in ELECTRA and its subsequent variant models. In addition, SAS is a general strategy that can be seamlessly combined with many new techniques emerging recently or in the future, such as the disentangled attention mechanism from DeBERTa. Our experiments show that SAS is able to outperform ELECTRA and other state-of-the-art models in the GLUE tasks with similar or less computation cost.

We study the Bayesian model averaging approach to learning Bayesian network structures (DAGs) from data. We develop new algorithms including the first algorithm that is able to efficiently sample DAGs of a moderate size (with up to about 25 variables) according to the exact structure posterior. The DAG samples can then be used to construct estimators for the posterior of any feature. We theoretically prove good properties of our estimators and empirically show that our estimators considerably outperform the estimators from the previous state- of-the-art methods.

We perform session analysis for our domain of people search within a professional social network. We find that the content‐based method is appropriate to serve as a basis for the session identification in our domain. However, there remain some problems reported in previous research which degrade the identification performance (such as accuracy) of the content‐based method. Therefore, in this article, we propose two important refinements to address these problems. We describe the underlying rationale of our refinements and then empirically show that the content‐based method equipped with our refinements is able to achieve an excellent identification performance in our domain (such as 99.820% accuracy and 99.707% F‐measure in our experiments). Next, because the time‐based method has extremely low computation costs, which makes it suitable for many real‐world applications, we investigate the feasibility of the time‐based method in our domain by evaluating its identification performance based on our refined content‐based method. Our experiments demonstrate that the performance of the time‐based method is potentially acceptable to many real applications in our domain. Finally, we analyze several features of the identified sessions in our domain and compare them with the corresponding ones in general web search. The results illustrate the profession‐oriented characteristics of our domain.

We study the problem of statistical model checking of probabilistic systems for PCTL unbounded until property PJoinp(Æ1UÆ2) (where Join |X| {<, d, >, e}) using the computation of P d 0(Æ1UÆ2). The approach is first proposed by Sen et al. in CAV'05 but their approach suffers from two drawbacks. Firstly, the computation of Pd0Æ1UÆ2) requires for its validity, a user-specified input parameter ´2 which the user is unlikely to correctly provide. Secondly, the validity of computation of Pd0Æ1UÆ2) is limited only to probabilistic models that do not contain loops. We present a new technique which addresses both problems described above. Essentially our technique transforms the hypothesis test for the unbounded until property in the original model into a new equivalent hypothesis test for bounded until property in our modified model. We empirically show the effectiveness of our technique and compare our results with those using the method proposed by Sen et al.

This paper proposes a positioning routing algorithm for UWB ad hoc networks. The algorithm is deployed in two main scenarios: Multi-Coordinate System (MCS) and Unique Coordinate System (UCS). A Packet Forwarding Zone (PFZ) is proposed to investigate the performance where a limited forwarding region is used. New ideas have been included in the routing algorithm, such as the use of the Davidon-Fletcher-Powell (DFP) algorithm to estimate the position of the locally available beacon nodes, and the Reference Group (RG) concept as formed by suitable nodes with the Reference Group Master (RGM) chosen as elected by the nodes in the RG. The RG group characteristics particularly apply in the UCS case. In addition, the performance of Direct Method (DM) and DFP position estimation algorithms are also analyzed.