The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized augmentation was recently proposed in ELECTRA and achieved state-of-the-art performance by introducing an auxiliary generation network (generator) to produce contextualized data augmentation for the training of a main discrimination network (discriminator). This design, however, introduces extra computation cost of the generator and a need to adjust the relative capability between the generator and the discriminator. In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs. Essentially, this strategy eliminates a separate generator and uses the single network to jointly conduct two pre-training tasks with MLM (Masked Language Modeling) and RTD (Replaced Token Detection) heads. It avoids the challenge to search for an appropriate size of the generator, which is critical to the performance as evidenced in ELECTRA and its subsequent variant models. In addition, SAS is a general strategy that can be seamlessly combined with many new techniques emerging recently or in the future, such as the disentangled attention mechanism from DeBERTa. Our experiments show that SAS is able to outperform ELECTRA and other state-of-the-art models in the GLUE tasks with similar or less computation cost.

Following SimCSE, contrastive learning based methods have achieved the state-of-the-art (SOTA) performance in learning sentence embeddings. However, the unsupervised contrastive learning methods still lag far behind the supervised counterparts. We attribute this to the quality of positive and negative samples, and aim to improve both. Specifically, for positive samples, we propose switch-case augmentation to flip the case of the first letter of randomly selected words in a sentence. This is to counteract the intrinsic bias of pre-trained token embeddings to frequency, word cases and subwords. For negative samples, we sample hard negatives from the whole dataset based on a pre-trained language model. Combining the above two methods with SimCSE, our proposed Contrastive learning with Augmented and Retrieved Data for Sentence embedding (CARDS) method significantly surpasses the current SOTA on STS benchmarks in the unsupervised setting.

Most existing sign language recognition works are based on videos, which have the characteristics of weak robustness and high computational complexity. In this paper, we propose a new continuous sign language recognition (CSLR) framework to translate sign language represented by multi-part skeleton data into text. In the framework, the Multi-Stream Spatial-Temporal Graph Convolutional Networks (Multi-Stream ST-GCNs) is designed to model dynamic multi-part skeletons with graph structure. Meanwhile, we utilize adaptive fusion to weighted fuse the features of different parts, which can highlight the critical parts and weaken the auxiliary parts. The Attention-based Encoder-Decoder Network is applied to learn the mapping between the fused feature sequence and target sentence. In addition, we build the first skeleton-based CSLR dataset using Leap Motion, named LMSLR. Experimental results demonstrate that our approach achieves advanced performance in word error rate (WER).

Glomerulus is the filtering unit of the kidney. In the computer aided diagnosis system designed for kidney disease, glomerulus extraction is an important step for analyzing kidney-tissue image. Against the disadvantages of traditional methods, this paper proposes a glomerulus extraction method using genetic algorithm for edge patching. Firstly, Canny edge detector is applied to get discontinuous edges of glomerulus. After labeling to remove the noises, genetic algorithm is used to search for optimal patching segments to join those edges together. Lastly, the edges and the patching segments with high fitness would be able to form the whole edge of the glomerulus. Experiments and comparisons indicate the proposed method can extract the glomerulus from kidney-tissue image both fast and accurately.

In this paper, we propose a new algorithm, named JACC-G, for large scale optimization problems. The motivation is to improve our previous work on grouping and adaptive weighting based cooperative coevolution algorithm, DECC-G [1], which uses random grouping strategy to divide the objective vector into subcomponents, and solve each of them in a cyclical fashion. The adaptive weighting mechanism is used to adjust all the subcomponents together at the end of each cycle. In the new JACC-G algorithm: (1) A most recent and efficient Differential Evolution (DE) variant, JADE [2], is employed as the subcomponent optimizer to seek for a better performance; (2) The adaptive weighting is time-consuming and expected to work only in the first few cycles, so a detection module is added to prevent applying it arbitrarily; (3) JADE is also used to optimize the weight vector in adaptive weighting process instead of using a basic DE in previous DECC-G. The efficacy of the proposed JACC-G algorithm is evaluated on two sets of widely used benchmark functions up to 1000 dimensions.

A new differential evolution (DE) algorithm, JADE, is proposed to improve optimization performance by implementing a new mutation strategy ldquoDE/current-to-pbestrdquo with optional external archive and updating control parameters in an adaptive manner. The DE/current-to-pbest is a generalization of the classic ldquoDE/current-to-best,rdquo while the optional archive operation utilizes historical data to provide information of progress direction. Both operations diversify the population and improve the convergence performance. The parameter adaptation automatically updates the control parameters to appropriate values and avoids a user's prior knowledge of the relationship between the parameter settings and the characteristics of optimization problems. It is thus helpful to improve the robustness of the algorithm. Simulation results show that JADE is better than, or at least comparable to, other classic or adaptive DE algorithms, the canonical particle swarm optimization, and other evolutionary algorithms from the literature in terms of convergence performance for a set of 20 benchmark problems. JADE with an external archive shows promising results for relatively high dimensional problems. In addition, it clearly shows that there is no fixed control parameter setting suitable for various problems or even at different optimization stages of a single problem.

Differential evolution (DE) mutates solution vectors by the weighted difference of other vectors using arithmetic operations. As these operations cannot be directly extended to discrete combinatorial space, DE algorithms have been traditionally applied to optimization problems where the search space is continuous. In this paper, we use JADE, a self-adaptive DE algorithm, for winner determination in Combinatorial Auctions (CAs) where users place bids on combina tions of items. To adapt JADE to discrete optimization, we use a rank-based representation schema that produces only feasible solu tions and a regeneration operation that constricts the problem search space. It is shown that JADE compares favorably to a local stochastic search algorithm, Casanova, and a genetic algorithm based approach, SGA.

We propose a new self-adaptive differential evolution algorithm for multi-objective optimization problems. To address the challenges in multi-objective optimization, we introduce an archive to store recently explored inferior solutions whose difference with the current population is utilized as direction information about the optimum, and also consider a fairness measure in calculating crowding distances to prefer the solutions whose distances to nearest neighbors are large and close to be uniform. As a result, the obtained solutions can spread well over the computed non-dominated front and the front can be moved fast toward the Pareto-optimal front. In addition, the control parameters of the algorithm are adjusted in a self-adaptive manner, avoiding parameter tuning for prob lems of different characteristics. The proposed algorithm, named JADE2, achieves better or at least competitive results compared to NSGA-II and GDE3 for a set of twenty-two benchmark problems.

In this paper, we derive closed form upper bounds on the error probability of low-density parity-check (LDPC) coded modulation schemes operating on quasi-static fading channels. The bounds are obtained from the so-called Fano-Gallager's tight bounding techniques, and can be readily calculated when the distance spectrum of the code is available. In deriving the bounds for multiple-input multiple-output (MIMO) systems, we assume the LDPC code is concatenated with the orthogonal space-time block code as an inner code. We obtain an equivalent single-input single-output (SISO) channel model for this concatenated coded-modulation system. The upper bounds derived here indicate good matches with simulation results of a complete transceiver system over Rayleigh and Rician MIMO fading channels in which the iterative detection and decoding algorithm is employed at the receiver.

An analytical method is proposed to study the evo lutionary stochastic properties of the population in Differential Evolution (DE) for a spherical function model. Properties of mutation and selection are developed, based on which a Gaussian approximate model of DE is introduced to facilitate mathematical derivations. The evolutionary dynamics and the convergence behavior of DE are investigated based on the derived analytical formulae and their appropriateness is verified by experimental results. It is shown that the lower limit of mutation factor should be as high as 0.68 to avoid premature convergence if the initial population is isotropically normally distributed and infinitely far from the optimum (i.e., the function landscape becomes a by per-plane). The lower limit, however, may be decreased if the population becomes closer to the optimum and an accordingly smaller mutation factor is beneficial to speed up the convergence. This motivates future research to improve DE by dynamically adapting control parameters as evolution search proceeds.

A new differential evolution algorithm, JADE, is proposed to improve the rate and the reliability of convergence performance by implementing a new mutation strategy ‘DE/current-top-best’ and controlling the parameters in a self-adaptive manner. The °DE/current-top-best’ is a generaliza tion of ‘DE/current-to-best’. It diversifies the population but still inherits the fast convergence property. Self-adaptation is benefi cial for performance improvement. Also, it avoids the require ment of prior knowledge about parameter settings and thus works well without user interaction. Compared to other self-adaptive DE algorithms, JADE converges faster and reliably in at least 10 out of a set of 13 benchmark problems and shows competitive results in other cases as well. Simulations results also clearly show that there is no single parameter value suitable for various problems or even at different optimization stages of a single problem.

A new differential evolution algorithm, DE-AEC, is proposed based on adaptive evolution control utilizing the information provided by a surrogate model. The algorithm is useful for optimization problems with expensive function evaluations, because it can significantly reduce the number of true function evaluations. Specifically, DE-AEC generates multiple offspring for each parent and chooses the promising one based on the accuracy and the predicted function value of the current surrogate model. The model’s accuracy is also used as an indicator of potential false convergence and special measures are taken to improve the convergence reliability. Simulation results on a set of fifteen test functions show that, compared to an already improved DE algorithm, DE-AEC reduces the number of true function evaluations by 30% - 80% for fourteen functions in the achievement of either low-level (10-2) or high-level (10-8) accuracy.

An assumption of random block codes is made for the closed-form derivation of union upper bounds. The random code works as the outer code for concatenation with the inner space-time modulation over the multi-input multi-output (MIMO) fading channels. From the union bound expression, we further obtain the closed-form expressions of error exponent and cut-off rate for MIMO channels. The error exponent is compared with Gallager's classical error exponent which is generally evaluated in the Monte Carlo method. The obtained bounds can be useful to serve as space-time code design tools or as performance prediction measures for the MIMO systems which employ turbo-like block codes as the outer code to drive the inner space-time code.

A performance evaluation framework based on the maximum-likelihood (ML) union bound is proposed for the concatenated coding scheme where the outer linear block code, low-density parity-check (LDPC) or Turbo code, drives a sequence of inner space-time codes. The outer code can be partitioned in a combinatorial manner such that all codewords in one subset/subcode are associated with an identical pairwise error probability. The union bound is therefore the summation of all distinct pairwise error probabilities, each of which is weighted by the cardinality of the corresponding subset. The cardinality is computable by making use of the statistical property of an ensemble of codes, and an approach of polynomial expansion is proposed for efficient evaluation of the closed-form union bound. Numerical results are presented for different channel scenarios and modulation schemes, which indicate that the ML union bounds are useful to benchmark the performance of practical iterative detection and decoding algorithms.

Under the assumption of random coding, we derive a closed-form union bound on the error performance of MIMO systems. The distance spectrum of random block codes, instead of that of LDPC or turbo codes, is used in this paper for the purpose of closed-form derivation. As compared with the simulation results, the bound is useful as a performance benchmark for a concatenated coding scheme over MIMO systems in which the LDPC or turbo code serves as the outer code to drive an inner space-time modulation block. In addition, the derived closed-form error exponent and cutoff rate are consistent with Gallager's random coding exponent and the information-theoretic channel capacity in a variety of modulation and channel scenarios

Closed-form union bounds are derived for the error performance of low-density parity-check (LDPC) coded Q-ary modulation systems over fast fading multi-input multi-output (MIMO) channels. The LDPC codewords are mapped onto space-time modulation blocks, referred to as associated space-time (AST) codewords, in one-to-one correspondence. Based on this relationship, the distance spectrum of the AST code is identified in a combinatorial manner, and each AST codeword possessing the same distance metric renders the identical pairwise error probability. The union bound is obtained in a closed form and can be efficiently evaluated by the method of polynomial expansion. In addition, the analysis is extended to the system with random coding which contributes to a concise expression of the error exponent. Numerical results are presented for different channel scenarios and modulation schemes, which indicate that the bounds are useful to benchmark the practical performance of iterative detection and decoding algorithms

In this paper, we derive closed-form upper bounds on the error probability of coded modulation schemes used for multi-input multi-output (MIMO) quasi-static fading channels. The bounds are obtained from new tight union-bound techniques, and can be readily calculated when the distance spectrum is given. In deriving the bounds for the MIMO system, we assume the use of orthogonal space-time block code (OSTBC) as the inner code. The outer code is the block code. We obtain an equivalent single-input single-output (SISO) channel model for this concatenated coded-modulation system. The upper bounds derived here indicate good match with the simulation results of the complete transceiver the system in which the turbo-iterative decoding algorithm is employed at the receiver.

In this paper, we propose a new MAC protocol for ad-hoc networks that uses multiple channels and a dynamic channel selection scheme. The proposed protocol, DCC-MAC (dual control channel based MAC), deploys two control channels (CH/sub R/C/ and CH/sub ACK/ and N data channels, where N is independent of the number of nodes in the network. Through RTS/CTS dialogues on CH/sub R/C/ senders and receivers negotiate the most robust channel for data transmission with maximum-CIR selection criterion. ACK transmission on CH/sub ACK/ contributes to the unidirectional transmission on data channels and therefore the decrease of the co-channel reuse distance. Performance evaluation has been carried out and the simulation results demonstrate the effectiveness of the proposed protocol.