.. _introduction: Introduction ============ ``sfsutils`` is a Python toolkit for deriving site-frequency spectra (SFS) from raw variant data. It provides a configurable variant-to-SFS parser together with the annotations and filters commonly required to prepare population-genetic data for downstream analysis. The package is object-oriented and thoroughly documented, and because it applies the same polarisation, filtering, and projection to a common sample size across datasets, the resulting spectra are directly comparable. Motivation ---------- The SFS condenses population genetic variation by quantifying the number of alleles observed at each frequency, and is the input to a wide range of population-genetic methods. Obtaining a spectrum from real data is rarely a single step: sites must be polarised against an ancestral state, stratified into meaningful categories (for example by degeneracy or synonymy), and filtered to remove sites that violate modelling assumptions. Because comparisons across species, populations, or genomic regions are only meaningful when the spectra are derived in a consistent manner, ``sfsutils`` collects these operations behind a single, reproducible parsing interface. How it works ------------ ``sfsutils`` reads variants from VCF files, VCF-Zarr stores, or tskit tree sequences (ARGs) through a single streamed site interface, and counts derived-allele frequencies into one or more spectra. Rather than depending on pre-annotated input, it can derive the required site-level information itself. Ancestral alleles may be taken from an existing ``AA`` info tag, or inferred from one or more outgroups either by maximum parsimony or under a maximum-likelihood substitution model; site degeneracy and synonymy can be computed directly from FASTA and GFF references. The same annotations let the parser stratify sites as it counts them, producing a separate spectrum per category, for example a synonymous and a non-synonymous spectrum, or one spectrum per degeneracy class. A collection of filters excludes sites that violate downstream modelling assumptions, including non-biallelic and non-SNV sites, sites outside coding sequences, CpG sites, and sites affected by GC-biased gene conversion or by deviant outgroups. Beyond the ordinary one-dimensional spectrum, the same parser can also derive the joint SFS across several populations and the two-site SFS of linked pairs of sites. All of these spectra support the operations needed to inspect and reshape them, including folding, polarising, marginalising, resampling, and visualisation. Parsing, filtering, and annotating are also available from the command line (see the :doc:`CLI reference `). Features -------- ``sfsutils`` offers a range of features for preparing site-frequency spectra. **Parsing**: streamlining the extraction of spectra from raw variant data: - Built-in variant-to-SFS parser, with support for versatile stratification - Joint site-frequency spectra across several populations and two-site spectra of linked pairs of sites - Stratification by degeneracy, synonymy, base transition or transversion type, ancestral base, genomic context, contig, and more - Utilities to determine the number of mutational target sites when monomorphic sites are not present in the provided input - Serialization of spectra and parser configurations - A command-line interface for parsing, filtering, and annotating - Object-oriented and customizable design **Annotation**: adding the site-level information required to build meaningful spectra: - Site-degeneracy annotation from FASTA and GFF references - Ancestral-allele annotation, either by maximum parsimony or by a maximum-likelihood model using one or more outgroups - Synonymy annotation of coding variants **Filtration**: removing sites that violate downstream modelling assumptions: - Filtering of non-biallelic and non-SNV sites - Restriction to coding sequences - Removal of sites affected by GC-biased gene conversion or by deviant outgroups **Visualization**: plotting utilities for inspecting spectra: - Visualization of individual and stratified spectra Contents -------- .. toctree:: :caption: Python Reference reference/Python/installation reference/Python/quickstart reference/Python/parser reference/Python/input_output reference/Python/annotations reference/Python/filtrations reference/Python/spectra reference/Python/joint reference/Python/two_sfs .. toctree:: :caption: R Reference reference/R/installation reference/R/quickstart reference/R/parser reference/R/input_output reference/R/annotations reference/R/filtrations reference/R/spectra reference/R/joint reference/R/two_sfs .. toctree:: :caption: CLI Reference reference/CLI/usage .. toctree:: :caption: API Reference :maxdepth: 1 modules/spectrum modules/parser modules/stratification modules/annotation modules/filtration modules/io modules/settings .. toctree:: :caption: Miscellaneous :maxdepth: 1 modules/citing modules/changelog References ---------- .. bibliography:: :style: plain