Job-shop scheduling is an important but difficult combinatorial optimization problem for low-volume and high-variety manufacturing, with solutions required to be obtained quickly at the beginning of each shift. In view of the increasing demand for customized products, problem sizes are growing. A promising direction is to take advantage of Machine Learning (ML). Direct learning to predict solutions for job-shop scheduling, however, suffers from major difficulties when problem scales are large. In this paper, a Deep Neural Network (DNN) is synergistically integrated within the decomposition and coordination framework of Surrogate Lagrangian Relaxation (SLR) to predict good-enough solutions for subproblems. Since a subproblem is associated with a single part, learning difficulties caused by large scales are overcome. Nevertheless, the learning still presents challenges. Because of the high-variety nature of parts, the DNN is desired to be able to generalize to solve all possible parts. To this end, our idea is to establish “surrogate” part subproblems that are easier to learn, develop a DNN based on Pointer Network to learn to predict their solutions, and calculate the solutions of the original part subproblems based on the predictions. Moreover, a masking mechanism is developed such that all the predictions are feasible. Numerical results demonstrate that good-enough subproblem solutions are predicted in many iterations, and high-quality solutions of the overall problem are obtained in a computationally efficient manner. The performance of the method is further improved through continuous learning.Note to Practitioners—Scheduling is important for the planning and operation of job shops, and high-quality schedules need to be obtained quickly at the beginning of each shift. To take advantage of ML, in this paper, a DNN is integrated within our recent decomposition and coordination approach to learn to predict “good-enough” solutions to part subproblems. To be able to predict solutions for parts of various characteristics“, surrogate” part subproblems that are easier to learn are established, and a generic “pointer network” is developed to learn to predict their solutions. To satisfy the constraints of the surrogate part subproblems, the pointer network is enhanced with a novel “masking mechanism” such that all the predictions are feasible. The solutions to the original part subproblems are calculated based on the predictions. Testing results demonstrate that subproblem solutions are efficiently obtained based on predictions, and the high-quality solutions of the overall problem are thus efficiently obtained. Through continuous learning, the performance of the method is further improved. Python codes and datasets are submitted together with the paper.

Syntactic parsing is a highly linguistic processing task whose parser requires training on treebanks from the expensive human annotation. As it is unlikely to obtain a treebank for every human language, in this work, we propose an effective cross-lingual UD parsing framework for transferring parser from only one source monolingual treebank to any other target languages without treebank available. To reach satisfactory parsing accuracy among quite different languages, we introduce two language modeling tasks into the training process of dependency parsing as multi-tasking. Assuming only unlabeled data from target languages plus the source treebank can be exploited together, we adopt a self-training strategy for further performance improvement in terms of our multi-task framework. Our proposed cross-lingual parsers are implemented for English, Chinese, and 29 UD treebanks. The empirical study shows that our cross-lingual parsers yield promising results for all target languages, approaching the parser performance which is trained in its own target treebank.

Discontinuous constituency parsing is still kept developing for its efficiency and accuracy are far behind its continuous counterparts.Motivated by the observation that a discontinuous constituent tree can be simply transformed into a pseudo-continuous one by artificially reordering words in the sentence, we propose a novel reordering method, thereby construct fast and accurate discontinuous constituency parsing systems working in continuous way.Specifically, we model the relative position changes of words as a list of actions.By parsing and performing this actions, the corresponding pseudocontinuous sequence is derived.Discontinuous parse tree can be further inferred via integrating a high-performance pseudo-continuous constituency parser.Our systems are evaluated on three classical discontinuous constituency treebanks, achieving new state-of-the-art on two treebanks and showing a distinct advantage in speed.

The pre-trained language model (PrLM) demonstrates domination in downstream natural language processing tasks, in which multilingual PrLM takes advantage of language universality to alleviate the issue of limited resources for low-resource languages. Despite its successes, the performance of multilingual PrLM is still unsatisfactory, when multilingual PrLMs only focus on plain text and ignore obvious universal linguistic structure clues. Existing PrLMs have shown that monolingual linguistic structure knowledge may bring about better performance. Thus we propose a novel multilingual PrLM that supports both explicit universal dependency parsing and implicit language modeling. Syntax in terms of universal dependency parse serves as not only pre-training objective but also learned representation in our model, which brings unprecedented PrLM interpretability and convenience in downstream task use. Our model outperforms two popular multilingual PrLM, multilingual-BERT and XLM-R, on cross-lingual natural language understanding (NLU) benchmarks and linguistic structure parsing datasets, demonstrating the effectiveness and stronger cross-lingual modeling capabilities of our approach.