We present a version of Earley's general parsing algorithm which uses a precomputed table. Our algorithm generates a set based representation of sentence derivations, precomputed components of which are also held in the table. We give experimental results for Java and ANSI C showing that the data structures produced are considerably smaller than the corresponding Earley data structures, and that the algorithm runs faster. The algorithm retains the simplicity of Earley's approach and, without explanatory discussion, takes only about a page to fully specify. This paper contains both motivational discussion, describing a recogniser version of the algorithm first and then its extension to a parser, and a concise, bare, but complete parser specification.

Fully general parsers permit the syntax specification of formal languages to be unrestricted, allowing language designers to use a syntax specification that supports semantics specification, but also permitting ambiguity. Language workbenches that support fully general grammars need tools which can generate robust default behaviour but also allow experienced designers to specify particular choices when ambiguity is encountered. The standard longest match approach to ambiguity resolution is not robust when a grammar contains cycles. Although cycles can be removed from a grammar, this disrupts the syntax specification. We present an algorithm that safely removes cycles from the shared packed parse forests generated by general parsers and explore the application of the algorithm to the 1997 SML Definition. The algorithm results in a sub-forest to which further designer-specified or default disambiguation rules can be applied.

The Generalised-LL (GLL) context-free parsing algorithm was introduced at the 2009 LDTA workshop, and since then a series of variant algorithms and implementations have been described. There is a wide variety of optimisations that may be applied to GLL, some of which were already present in the originally published form.

This article introduces two new approaches in the areas of lexical analysis and context-free parsing. We present an extension, MGLL, of generalised parsing which allows multiple input strings to be parsed together efficiently, and we present an enhanced approach to lexical analysis which exploits this multiple parsing capability. The work provides new power to formal language specification and disambiguation, and brings new techniques into the historically well-studied areas of lexical and syntax analysis. It encompasses character-level parsing at one extreme and the classical LEX/YACC style division at the other, allowing the advantages of both approaches.

We consider the possibility of making the lexicalisation phase of compilation more powerful by avoiding the need for the lexer to return a single token string from the input character string. This has the potential to empower language design by softening the boundaries between lexical and phrase level specification. The large number of lexicalisations makes it impractical to parse each one individually, but it is possible to share the parsing of common subparts, reducing the number of tokens parsed from the product of the token numbers associated with the components to their sum. We report total numbers of lexicalisations of example Java strings, and the impact on these numbers of various lexical disambiguation strategies, and we introduce a new generalised parsing technique that can efficiently parse multiple lexicalisations of character string simultaneously. We then use this technique on Java, reporting on the number of lexicalisations that correspond to syntactically correct Java strings and the degree to which the standard Java lexer is safe in the sense that it does not remove all the syntactically correct lexicalisations of an input character string. Our multi-lexer parser is an alternative to scannerless parsing of a character level grammar, retaining the separation between grammar terminals and the corresponding lexical tokens. This has the advantages of allowing the parser to use terminal level lookahead and keeping lexical level disambiguation separate from the context free grammar.

At SLE in 2014, Ridge presented the P3 combinator library with which parsers can be developed for left-recursive, non-deterministic and ambiguous grammars. A combinator expression in P3 yields a binarised grammar reflecting the expression's structure. The grammar is given to an underlying, generalised parsing procedure computing all derivations.

The right nulled generalized LR parsing algorithm is a new generalization of LR parsing which provides an elegant correction to, and extension of, Tomita's GLR methods whereby we extend the notion of a reduction in a shift-reduce parser to include right nulled items. The result is a parsing technique which runs in linear time on LR(1) grammars and whose performance degrades gracefully to a polynomial bound in the presence of nonLR(1) rules. Compared to other GLR-based techniques, our algorithm is simpler and faster.

We describe a generalized bottom up parser in which non-embedded recursive rules are handled directly by the underlying automaton, thus limiting stack activity to the activation of rules displaying embedded recursion. Our strategy is motivated by Aycock and Horspool's approach, but uses a different automaton construction and leads to parsers that are correct for all context-free grammars, including those with hidden left recursion. The automaton features edges which directly connnect states containing reduction actions with their associated goto state: hence we call the approach reduction incorporated generalized LR parsing. Our parser constructs shared packed parse forests in a style similar to that of Tomita parsers. We give formal proofs of the correctness of our algorithm, and compare it with Tomita's algorithm in terms of the space and time requirements of the running parsers and the size of the parsers' tables. Experimental results are given for standard grammars for ANSI-C, ISO-Pascal; for a non-deterministic grammar for IBM VS-COBOL, and for a small grammar that triggers asymptotic worst case behaviour in our parser.

We discuss generalisations of bottom up parsing, emphasising the relative costs for real programming languages. Our goal is to provide a roadmap of the available approaches in terms of their space and time performance for programming language applications, focusing mainly on GLR style algorithms. It is well known that the original Tomita GLR algorithm fails to terminate on hidden left recursion: here we analyse two approaches to correct GLR parsing (i) the modification due to Farshi that is incorporated into Visser’s work and (ii) our own right-nullable GLR (RNGLR) algorithm, showing that Farshi’s approach can be expensive. We also present results from our new Binary RNGLR algorithm which is asymptotically the fastest parser in this family and show that the recently reported reduction incorporated parsers can require automata that are too large to be practical on current machines. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Digital signal processors (DSPs) are special purpose microprocessors optimised for embedded applications that require high arithmetic rates. These devices are often difficult to compile for; compared to modern general purpose processors, DSPs often have very small address spaces. In addition they contain unusual hardware features and they require correct scheduling of operands against pipeline registers to extract the highest levels of available performance. As a result, high level language compilers for these devices generate poor quality code, and are rarely used in practice. Recently, new generation processors have been launched that are very hard to program by hand in assembler because of the complexity of their internal pipelines and arithmetic structures. DSP users are therefore having to migrate to using high level language compilers since this is the only practical development environment. However, there exist large quantities of legacy code written in assembler which represent a significant investment to the user who would like to be able to deploy core algorithms on the new processors without having to re-code from scratch. The article discusses the development and use of a tool to automatically reverse-compile assembler source for the ADSP-21xx family of DSPs to ANSI-C. We include a discussion of the architectural features of the ADSP-21xx processors and the ways in which they assist the translation process. We also identify a series of translation challenges which, in the limit, can only be handled with manual intervention and give some statistics for the frequency with which these pathological cases appear in real applications.

This paper presents a construct for mapping arbitrary non-left recursive context-free grammars into recursive descent parsers that: handle ambiguous grammars correctly; perform with LL(1) efficiency on LL(1) grammars; allow straightforward implementation of both inherited and synthesized attributes; and allow semantic actions to be added at any point in the grammar. We describe both the basic algorithm and a tool, GRDP, which generates parsers which use this technique. Modifications of the basic algorithm to improve efficiency lead to a discussion of follow-determinism , a fundamental property that gives insights into the behaviour of both LL and LR parsers. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.