Abstract A lot has happened since OpenAI released ChatGPT to the public in November 2022. We review how things unfolded over the course of the year, tracking significant events and announcements from the tech giants leading the generative AI race and from other players of note; along the way we note the wider impacts of the technology’s progress.

Abstract The technical and mainstream media’s headline coverage of AI invariably centers around the often astounding abilities demonstrated by large language models. That’s hardly surprising, since to all intents and purposes that’s where the newsworthy magic of generative AI lies. But it takes a village to raise a child: behind the scenes, there’s an entire ecosystem that supports the development and deployment of these models and the applications that are built on top of them. Some parts of that ecosystem are dominated by the Big Tech incumbents, but there are also many niches where start-ups are aiming to gain a foothold. We take a look at some components of that ecosystem, with a particular focus on ideas that have led to investment in start-ups over the last year or so.

Abstract It’s less than a year since OpenAI’s board voted to fire Sam Altman as CEO, in a palace coup that lasted just a weekend before Altman was reinstated. That weekend and subsequent events in OpenAI’s storyline provide all the ingredients for a soap opera. So, just in case Netflix is interested, here’s a stab at a synopsis of what might be just the first of many seasons of ‘The Generative AI Wars’.

Abstract It’s no secret that the commercial application of NLP technologies has exploded in recent years. From chatbots and virtual assistants to machine translation and sentiment analysis, NLP technologies are now being used in a wide variety of applications across a range of industries. With the increasing demand for technologies that can process human language, investors have been eager to get a piece of the action. In this article, we look at NLP startup funding over the past year, identifying the applications and domains that have received investment.

Abstract Since the release of ChatGPT at the end of November 2022, generative AI has been talked about endlessly in both the technical press and the mainstream media. Large language model technology has been heralded as many things: the disruption of the search engine, the end of the student essay, the bringer of disinformation … but what does it mean for commercial providers of earlier iterations of natural language generation technology? We look at how the major players in the space are responding, and where things might go in the future.

Abstract Funding for AI start-ups in general is booming, and natural language processing as a subfield has not missed out. We take a closer look at early-stage funding over the last year—just over US$1B in total—for companies that offer solutions that are based on or make significant use of NLP, providing a picture of what funders think is innovative and bankable in this space, and we make some observations on notable trends and developments.

Abstract In the past few years, high-quality automated text-to-speech synthesis has effectively become a commodity, with easy access to cloud-based APIs provided by a number of major players. At the same time, developments in deep learning have broadened the scope of voice synthesis functionalities that can be delivered, leading to a growth in the range of commercially viable use cases. We take a look at the technology features and use cases that have attracted attention and investment in the past few years, identifying the major players and recent start-ups in the space.

Abstract GPT-3 made the mainstream media headlines this year, generating far more interest than we’d normally expect of a technical advance in NLP. People are fascinated by its ability to produce apparently novel text that reads as if it was written by a human. But what kind of practical applications can we expect to see, and can they be trusted?

Abstract Automated writing assistance – a category that encompasses a variety of computer-based tools that help with writing – has been around in one form or another for 60 years, although it’s always been a relatively minor part of the NLP landscape. But the category has been given a substantial boost from recent advances in deep learning. We review some history, look at where things stand today, and consider where things might be going.

Abstract The end of the calendar year always seems like a good time to pause for breath and reflect on what’s been happening over the last 12 months, and that’s as true in the world of commercial NLP as it is in any other domain. In particular, 2019 has been a busy year for voice assistance, thanks to the focus placed on this area by all the major technology players. So, we take this opportunity to review a number of key themes that have defined recent developments in the commercialization of voice technology.

Abstract It took a while, but natural language generation is now an established commercial software category. It’s commented upon frequently in both industry media and the mainstream press, and businesses are willing to pay hard cash to take advantage of the technology. We look at who’s active in the space, the nature of the technology that’s available today and where things might go in the future.

Abstract The law has language at its heart, so it’s not surprising that software that operates on natural language has played a role in some areas of the legal profession for a long time. But the last few years have seen an increased interest in applying modern techniques to a wider range of problems, so I look here at how natural language processing is being used in the legal sector today.

Abstract The Journal of Natural Language Engineering is now in its 25th year. The editorial preface to the first issue emphasised that the focus of the journal was to be on the practical application of natural language processing (NLP) technologies: the time was ripe for a serious publication that helped encourage research ideas to find their way into real products. The commercialisation of NLP technologies had already started by that point, but things have advanced tremendously over the last quarter-century. So, to celebrate the journal’s anniversary, we look at how commercial NLP products have developed over the last 25 years.

Abstract It’s now remarkably easy to release to the world a cloud-based application programming interface (API) that provides some software function as a service. As a consequence, the cloud API space has become very densely populated, so that even if a particular API offers a service whose potential value is considerable, there are many other factors that play a role in determining whether or not that API will be commercially successful. If you’re thinking about entering the API marketplace with your latest and greatest idea, this post offers some entirely subjective advice on how you might increase the chances of your offering not being lost in all the noise.

Abstract If you’re in the market for an off-the-shelf text analytics API, you have a lot of options. You can choose to go with a major player in the software world, for whom each AI-related service is just another entry in their vast catalogues of tools, or you can go for a smaller provider that focusses on text analytics as their core business. In this first of two related posts, we look at what the most prominent software giants have to offer today.

Abstract It seems like there’s yet another cloud-based text analytics Application Programming Interface (API) on the market every few weeks. If you’re interested in building an application using these kinds of services, how do you decide which API to go for? In the previous Industry Watch post, we looked at the text analytics APIs from the behemoths in the cloud software world: Amazon, Google, IBM and Microsoft. In this post, we survey sixteen APIs offered by smaller players in the market.

Abstract We live in a post-truth world. It now matters more whether people think something is true than whether something really is true. This is dangerous, and technology is at least partly to blame. So, as technologists, how can we help to fix this?

Abstract The commercialisation of natural language processing began over 35 years ago, but it’s only in the last year or two that it’s become substantially more visible, largely because of the intense popular interest in artificial intelligence. So what’s the state of commercial NLP today? We survey the main industry categories of relevance, and offer comment on where the action is today.

Abstract Vastly improved speech recognition, backed by a more slowly improving ability to make sense of the recognized speech, has brought state-of-the-art NLP into our homes in the form of smart speakers and other devices that listen. There’s no doubt these devices can be incredibly useful, but they also may also support incursions into our privacy. We look at where we are today, consider what might be coming, and express just a tad of caution.

Abstract Machine Translation research suffered a major blow in the 1960s, but it came back with a vengeance. From a commercial point of view, it’s now a mature technology that many Internet users take for granted. We look at where we are now, and consider the scope for new entrants into the market.

Abstract Ten years ago, Microsoft Word's grammar checker was really the only game in town. The software world, and the world of natural language processing, have changed a lot in that time, so what does the grammar checker marketplace have to offer today?

Abstract By all accounts, 2016 is the year of the chatbot. Some commentators take the view that chatbot technology will be so disruptive that it will eliminate the need for websites and apps. But chatbots have a long history. So what's new, and what's different this time? And is there an opportunity here to improve how our industry does technology transfer?

Abstract In almost every science fiction movie you’ll see people conversing with machines. Of course, the rise of intelligent personal assistants means you probably do this yourself already. This posting asks: what’s the difference? Also, recent news on Facebook acquisitions, spoken language translation, and sentiment analysis.

Abstract With NLP services now widely available via cloud APIs, tasks like named entity recognition and sentiment analysis are virtually commodities. We look at what's on offer, and make some suggestions for how to get rich.

The Big Australian Speech Corpus project incorporates the strategic goals of 30 Chief Investigators from various speech science areas. Speech from 1000 geographically and socially diverse speakers is being recorded using a uniform and automated protocol plus standardized hardware and software to produce a widely applicable and extensible database – AusTalk. Here we describe the project’s major components and organization; share the lessons learnt from difficulties and challenges; and present the results achieved so far.

You are reading the last issue of Computational Linguistics that will appear in printed hardcopy form.The beginning of 2009 heralds a new era for the journal in at least two major respects: As of the first issue of volume 35, Computational Linguistics will be published only electronically, and it will be open access

A year ago in this column I wondered aloud whether 2007 was to be the year in which question-answering (QA) really took off in the commercial space. I was provoked to ask that question by the increasing number of Web-based QA systems that were portraying themselves as the Next Thing in search for the masses. There was, in particular, a lot of buzz around the $12.5 million funding deal announced by Powerset. The San Francisco-based company had gained exclusive access to parsing technology from PARC, but hadn't at that point displayed any of its wares to the general public. However, the company was inviting people to sign up for access to Powerset Labs, where, we were told, we would get the opportunity to be the first to play with the technology and to provide feedback to make it better. Since then, there have been occasional screen images of the application seen in blog posts and other news items, and a small number of claimed sightings by bloggers who were granted privileged access. But Powerset Labs was finally launched at TechCrunch 40 in mid-September. The company's Web site says that they've begun to let people have access to the technology, and that they'll be ‘letting in the next wave of users as soon as possible’. My surf board is ready, but I'm not holding my breath. I signed up in June 2007, and haven't heard a thing since. Comments posted on the Powerset site suggest that there might be quite a few in the line before me.

Here's a round-up of notable events in the commercial language technology space in the last quarter of 2007, organized by broad application category. A common thread that pops up throughout many of these is the integration of language technology into social networking applications and other related Web 2.0 themes. I'd put my money on this being a hot direction in 2008.

Back at the beginning of 2000, I was a member of a working group tasked to come up with some guidelines for revamping my University's website.During one of our meetings, someone made the suggestion thatdecisions about howto structure and present information on the website should be driven by the kinds of questions that users come to the site with.Suddenly a light went on, and there appeared an idea for data gathering that might provide us withsome useful information. To find out what people really wanted to know whenthey visited the website, we would replace the University's searchengine by a page that invited the user to type in his or her query as a full natural language question. Appropriately chosen examples would be given todemonstrate that using real questions delivered better pages as a result. The data gathered would tell us what people were really looking for, more than could be gleaned from conventional search queries, and would therefore help usto better structure the information available on the website.

“Powerset Hype to Boiling Point”, said a February headline on TechCrunch. In the last installment of this column, I asked whether 2007 would be the year of question-answering. My query was occasioned by a number of new attempts at natural language question-answering that were being promoted in the marketplace as the next advance upon search, and particularly by the buzz around the stealth-mode natural language search company Powerset. That buzz continued with a major news item in the first quarter of this year: in February, Xerox PARC and PowerSet struck a much-anticipated deal whereby PowerSet won exclusive rights to use PARC's natural language technology, as announced in a VentureBeat posting. Following the scoop, other news sources drew the battle lines with titles like “Can natural language search bring down Google?”, “Xerox vs. Google?”, and “Powerset and Xerox PARC team up to beat Google”. An April posting on Barron's Online noted that an analyst at Global Equities Research had cited Powerset in his downgrading of Google from Buy to Neutral. And, all this on the basis of a product which, at the time of writing, very few people have actually seen. Indications are that the search engine is expected to go live by the end of the year, so we have a few more months to wait to see whether this really is a Google-killer. Meanwhile, another question remaining unanswered is what happened to the Powerset engineer who seemed less sure about the technology's capabilities: see the segment at the end of D7TV's PartyCrasher video from the Powerset launch party. For a more confident appraisal of natural language search, check out the podcast of Barney Pell, CEO of Powerset, giving a lecture at the University of California–Berkeley.

I've just come back from the 45th Annual Meeting of the Association for Computational Linguistics (ACL) in Prague; this was the biggest ever ACL conference, with more than 1,000 people attending for the first time. Attendance at ACL conferences has been growing year on year, and that is a sign of a healthy field. Another sign of health is industry sponsorship. For this year's conference, the Gold Sponsor was Google, and Microsoft and Yahoo! were Silver Sponsors, along with a few companies we have not seen as ACL sponsors before: −textkernel, News Tin, and – a name that seems now to pop up regularly in this column – Powerset. There are all sorts of reasons why companies sponsor conferences like this, but clearly a major purpose is to make themselves visible to potential employees. And, if companies are hiring, that's good news across the board: it gives us a way of attracting more students into the field, and more generally, it speaks to the industrial and commercial relevance of what we do. There is nothing like external validation – especially commercial validation – to wash away those niggling self-doubts about the utility of your research endeavours. I remember attending MT Summit VII in Singapore in 1999, when Jo Lernout, then of Lernout & Hauspie, gave an invited talk in which (I'm sure I'm remembering this correctly) he said his ambition was to hire everyone in the hall. There were around 250 attendees – big for an NLP conference at the time – so that created quite a buzz. Jo wanted to hire everyone, not just cherry pick those with the near-to-product big ideas; in his vision of the future, every teensy-weensy tightly-focused research contribution had a role to play. For just a moment, everybody felt wanted.

The interpretation of temporal expressions in text is an important constituent task for many practical natural language processing tasks, including question-answering, information extraction and text summarisation. Although temporal expressions have long been studied in the research literature, it is only more recently, with the impetus provided by exercises like the ACE Program, that attention has been directed to broad-coverage, implemented systems. In this paper, we describe our approach to intermediate semantic representations in the interpretation of temporal expressions.

This workshop addresses the issue of evaluating multimodal dialogue systems, and in particular the characteristics and interaction styles that are particularly effective for human-machine collaborative task performance.

The last six months have seen a number of interesting announcements and products in the commercial language technology arena. Here are some that caught my eye, and which you might find interesting too.

No abstract available.

A spanned cell in a table is a single, complete unit that physically occupies multiple columns and/or multiple rows. Spanned cells are common in tables, and they are a significant cause of error in the extraction of tables from free text documents. In this paper, we present a model for the detection and merging of vertically spanned cells for tables presented in plain text documents. Our model and algorithm are based purely on the layout features of the tables, and they require no semantic understanding of the documents. When tested on the 98 tables appearing in 40 randomly selected documents from a corpus of company announcements from the Australian Stock Exchange (ASX), our algorithm achieves an accuracy of 86.79% in detecting and merging vertically spanned cells.

One way to keep in touch with what is happening in the commercial speech and language technology world is to pay occasional visits to the websites of HLT Central (at www.hltcentral.org ) and LT World (at www.lt-world.org ). Both sites provide links to news stories and press releases from companies and other organizations active in the area. The people who run these sites trawl the web for news stories of relevance, saving you the trouble of doing that yourself.

Suppose you're a corporate vice president at a well-known international software company, and you want to check on the visibility of one of your leading researchers in the outside world. You're sitting at your desk, so the most obvious thing to do is to enter their name into a search engine. If the well-known international software company happened to be Microsoft, and if the leading researcher happened to be Microsoft's Susan Dumais, and if the search engine you decided to use happened to be Google, you might be surprised to find that the sponsored link that comes atop the search results is actually from Google itself, exhorting you to ‘Work on NLP at Google’, and alerting you to the fact that ‘Google is hiring experts in statistical language processing’.

If you wanted to make some money out of natural language processing, an appropriate strategy might be to identify an area of technology that was relatively mature—one where the more fundamental technical problems had been resolved through a significant amount of research activity—and then identify potential applications for that technology.

Here's something I read the other day: IVR [interactive voice response] technology is at a point now where consumers almost cannot tell the difference between talking to a person and talking to a computer.

A principal aim of this journal is to bridge the gap between traditional computational linguistics research and the implementation of practical applications with potential real-world use. This new column aims to help with that bridge-building, by providing a window on to what is happening with speech and language technologies in the industrial and commercial worlds, and a discussion forum for topics of interest in the technology transfer space.

Providing timely and high-quality feedback has been identified as one of the critical factors in learning. In courses with large class sizes the amount of time required for critiquing makes it prohibitive to give feedback to students on more than two or three assignments per semester. As a result, students have limited opportunities to learn from their mistakes, and it is difficult for lecturers to monitor student progress. The alternative to manual critiquing is to automate the process. The authors expect automated program critiquing to deliver two major benefits to students: improved feedback and more useful access to lecturers. This paper introduces their project and the expected benefits.

We discuss a task requiring the coherent presentation of heterogeneous information about objects recorded in electronic catalogues. We consider the advantages of combining hypermedia delivery with natural language generation technology, so as to allow us to view a session with such a system as a coherent conversation or dialogue. We describe two prototype systems we have built which make use of these combined techniques, and focus on those aspects of the systems which attempt to provide coherence. Although the techniques themselves are not novel, their combination is relatively recent, and promises to help forge useful tools for accomplishing our specific information retrieval task.

In this article, we give an overview of Natural Language Generation (NLG) from an applied system-building perspective. The article includes a discussion of when NLG techniques should be used; suggestions for carrying out requirements analyses; and a description of the basic NLG tasks of content determination, discourse planning, sentence aggregation, lexicalization, referring expression generation, and linguistic realisation. Throughout, the emphasis is on established techniques that can be used to build simple but practical working systems now. We also provide pointers to techniques in the literature that are appropriate for more complicated scenarios.

Comparisons are typically employed to distinguish similar entities, or to illustrate a property of an entity by referring to another com-• monly known entity which shares that property.Based on an analysis of a corpus of encyclopaedia texts, we define three types of comparisons and outline some strategies for applying these in the generation of entity descriptions.We describe how these comparison strategies are used within the PEBA-II hypertext generation system to generate descriptions of animals.

This paper describes the referring expression generation mechanisms used in EPICURE, a computer program which produces natural language descriptions of cookery recipes. Major features of the system include: an underlying ontology which permits the representation of non-singular entities; a notion of discriminatory power, to determine what properties should be used in a description; and a PATR-like unification grammar to produce surface linguistic strings.