The "multispecies" coalescent (MSC) model that underlies many genomic species-delimitation approaches is problematic because it does not distinguish between genetic structure associated with species versus that of populations within species. Consequently, as both the genomic and spatial resolution of data increases, a proliferation of artifactual species results as within-species population lineages, detected due to restrictions in gene flow, are identified as distinct species. The toll of this extends beyond systematic studies, getting magnified across the many disciplines that rely upon an accurate framework of identified species. Here we present the first of a new class of approaches that addresses this issue by incorporating an extended speciation process for species delimitation. We model the formation of population lineages and their subsequent development into independent species as separate processes and provide for a way to incorporate current understanding of the species boundaries in the system through specification of species identities of a subset of population lineages. As a result, species boundaries and within-species lineages boundaries can be discriminated across the entire system, and species identities can be assigned to the remaining lineages of unknown affinities with quantified probabilities. In addition to the identification of species units in nature, the primary goal of species delimitation, the incorporation of a speciation model also allows us insights into the links between population and species-level processes. By explicitly accounting for restrictions in gene flow not only between, but also within, species, we also address the limits of genetic data for delimiting species. Specifically, while genetic data alone is not sufficient for accurate delimitation, when considered in conjunction with other information we are able to not only learn about species boundaries, but also about the tempo of the speciation process itself.

MOTIVATION: As the density of sampled population increases, especially as studies incorporate aspects of the spatial landscape to study evolutionary processes, efficient simulation of genetic data under the coalescent becomes a primary challenge. Beyond the computational demands, coalescence-based simulation strategies have to be reconsidered because traditional assumptions about the dynamics of coalescing lineages within local populations may be violated (e.g. more than two daughter lineages may coalesce to a parent at low population densities). Specifically, to efficiently assign n lineages to m parents, the order relation between n and m strongly affects the relevant algorithm for the coalescent simulator (e.g. only when n<2m, it is reasonable to assume that two lineages, at most, can be assigned to the same parent). Controlling the details of the simulation model as a function of n and m is then crucial to represent accurately and efficiently the assignment process, but current implementations make it difficult to switch between different types of lineage mergers at run-time or even compile-time. RESULTS: With the described occupancy spectrum and algorithm that generates the support of the joint probability distribution of the occupancy spectrum; computation is much faster than realizing the whole assignment process under the coalescent. Using general definitions of lineage merges, which also makes the codebase reusable, we implement several variants of coalescent mergers, including an approximation where low probability spectrums are discarded. Comparison of runtimes and performance of the different C++ highly reusable coalescence mergers (binary, multiple, hybrids) are given, and we illustrate their potential utility with example applications. AVAILABILITY AND IMPLEMENTATION: All components are integrated into Quetzal, an open-source C++ library for coalescence available at https://becheler.github.io/pages/quetzal.html. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

UNLABELLED: STEM is a software package written in the C language to obtain maximum likelihood (ML) estimates for phylogenetic species trees given a sample of gene trees under the coalescent model. It includes options to compute the ML species tree, search the space of all species trees for the k trees of highest likelihood and compute ML branch lengths for a user-input species tree. AVAILABILITY: The STEM package, including source code, is freely available at http://www.stat.osu.edu/~lkubatko/software/STEM/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.