Input and output

Input and output#

The Parser, Filterer and Annotator read variants through a single streamed site interface and write them back in the format implied by the output file’s extension. Any input backend (VCF, VCF-Zarr store, or tskit tree sequence) is exposed to downstream code as a Site, and any output format is written through a VariantWriter.

`Site`	Structural interface for a single streamed variant site: the abstraction layer over the input backends.
`NoTypeException`	Exception thrown when no type can be determined.
`Variant`	Minimal concrete implementation of the `Site` interface: a duck-typed stand-in for a `cyvcf2.Variant` exposing the subset of its interface that the parser, filtrations, annotations and stratifications rely on: `CHROM`, `POS`, `REF`, `ALT`, `INFO`, the `is_*` type flags and the per-sample `gt_bases`, alongside the numeric `allele_indices` these backends hold natively.
`VariantReader`	Common streaming interface over a variant source.
`TskitVariantReader`	Stream variants from a tskit tree sequence (e.g. an inferred ARG or an msprime simulation).
`ZarrVariantReader`	Stream variants from a VCF-Zarr store in the vcf2zarr (VCZ) layout.
`VariantWriter`	Abstract writer mirroring `VariantReader`: it consumes the same streamed `Variant` interface and writes it to a concrete on-disk format.
`VCFVariantWriter`	Write variants to a VCF (optionally gzipped) via cyvcf2, copying the header from the input VCF.
`ZarrVariantWriter`	Write variants to a VCF-Zarr store in the vcf2zarr (VCZ) layout read back by `ZarrVariantReader`, so the output can come from any input (VCF, tree sequence or another VCF-Zarr store).
`TskitVariantWriter`	Write a site-subset of an input tree sequence to a `.trees` file.
`FileHandler`	Base class for the handlers that read a local or remote file.
`VCFHandler`	Base class for the handlers that read a variant source.
`FASTAHandler`	Handler for a FASTA reference.
`GFFHandler`	Handler for a GFF/GTF annotation.

Site#

class Site(*args, **kwargs)[source]#

Bases: Protocol

Structural interface for a single streamed variant site: the abstraction layer over the input backends. Both cyvcf2.Variant (the VCF backend) and the concrete Variant emitted by the tree-sequence and VCF-Zarr backends satisfy it structurally, so the parser, filtrations, annotations and stratifications are typed against Site alone rather than a union of the concrete backend types.

Methods:

__init__

CHROM: str#: The contig.

POS: int#: The 1-based position.

REF: str#: The reference allele.

ALT: List[str]#: The alternate alleles.

INFO: Dict[str, object]#: The INFO field.

gt_bases: ndarray#: The per-sample genotype strings (e.g. "A/T"), as for cyvcf2.Variant.gt_bases.

allele_indices: ndarray | None#: The per-haplotype allele indices into [REF] + ALT, of shape (n_samples, ploidy) and integer dtype, with -1 for a missing call. Optional: it is None (or absent) on backends that carry the genotypes as strings only, in which case consumers read gt_bases instead.

is_snp: bool#: Whether the site is an SNP.

is_mnp: bool#: Whether the site is an MNP.

is_indel: bool#: Whether the site is an indel.

is_deletion: bool#: Whether the site is a deletion.

is_sv: bool#: Whether the site is a structural variant.

__init__(*args, **kwargs)#

Variant#

Bases: object

Minimal concrete implementation of the Site interface: a duck-typed stand-in for a cyvcf2.Variant exposing the subset of its interface that the parser, filtrations, annotations and stratifications rely on: CHROM, POS, REF, ALT, INFO, the is_* type flags and the per-sample gt_bases, alongside the numeric allele_indices these backends hold natively. Non-VCF backends (tree sequences, VCF-Zarr stores) emit these objects.

Methods:

__init__

Initialize the variant.

is_indel: bool = False#: Whether the variant is an indel

is_deletion: bool = False#: Whether the variant is a deletion

is_sv: bool = False#: Whether the variant is a structural variant

Initialize the variant.

Parameters:

ref (str) – The reference allele.
pos (int) – The position.
chrom (str) – The contig.
gt_bases (Union[Sequence[str], ndarray, None]) – The per-sample genotype strings (e.g. "A/T"), as for cyvcf2.Variant.gt_bases.
alt (Optional[Sequence[str]]) – The alternate alleles.
is_snp (bool) – Whether the site is a single-nucleotide polymorphism.
is_mnp (bool) – Whether the site is a multi-nucleotide polymorphism.
info (Optional[Dict[str, object]]) – The INFO field.
allele_indices (ndarray | None) – The per-haplotype allele indices into [ref] + alt, of shape (n_samples, ploidy), with -1 for a missing call. Where gt_bases is omitted the genotype strings are assembled from these on demand.
phased (Union[bool, Sequence[bool], ndarray, None]) – Whether each sample’s genotype is phased, either per sample or for the site as a whole, which decides the separator of the assembled genotype strings.

REF: str#: The reference allele

POS: int#: The position

CHROM: str#: The contig

ALT: List[str]#: The alternate alleles

allele_indices: ndarray | None = None#: The per-haplotype allele indices, absent unless the backend supplies them

is_snp: bool = False#: Whether the site is an SNP

is_mnp: bool = False#: Whether the site is an MNP

INFO: Dict[str, object]#: Info field

property gt_bases: ndarray#

The per-sample genotype strings, assembled from the allele indices. Most consumers read the indices instead, so a backend that holds the calls numerically supplies those alone and the strings are built only where something actually asks for them.

Returns:: The genotype strings, one per sample.

VariantReader#

class VariantReader[source]#

Bases: Iterable, ABC

Common streaming interface over a variant source. Concrete readers wrap a VCF-Zarr store or a tskit tree sequence and yield Variant objects in file order, so the parser can consume any input format through a single for variant in reader loop. Readers are re-iterable: each iter(reader) starts a fresh pass over the source.

Methods:

`add_info_to_header`	Declare an INFO field, registering it with the underlying VCF header.
`count_sites`	Count the number of sites in the source.
`close`	Release any resources held by the reader.

abstract property samples: List[str]#

The sample names, in genotype-column order.

Returns:: The sample names.

abstract property seqnames: List[str]#

The contig (sequence) names present in the source, matching cyvcf2.VCF.seqnames.

Returns:: The contig names.

property sequence_length: float | None#

The total length (bp) of the source region, when the source knows it (as a tree sequence does). Used to estimate the site density when extrapolating monomorphic sites from a target-site count. Returns None when no reliable length is available (e.g. a VCF-Zarr store), in which case the observed variant span is used instead.

Returns:: The sequence length, or None.

property contig_lengths: Dict[str, int] | None#

The declared length (bp) of each contig of the source, where the source declares them. Only the contigs whose length is known are present. Used to size the region a spectrum extrapolates over, which the span of the observed variants understates on a sparsely covered contig.

Returns:: The per-contig lengths, or None where the source declares none.

add_info_to_header(data: dict)[source]#

Declare an INFO field, registering it with the underlying VCF header. Does nothing for non-VCF sources, which have no header.

Parameters:: data (dict) – The INFO field definition.

count_sites()[source]#

Count the number of sites in the source.

Return type:: int
Returns:: The number of sites.

close()[source]#: Release any resources held by the reader.

TskitVariantReader#

class TskitVariantReader(ts: tskit.TreeSequence, contig: str = '1')[source]#

Bases: VariantReader

Stream variants from a tskit tree sequence (e.g. an inferred ARG or an msprime simulation). Sample haplotype nodes are grouped into diploid (or higher-ploidy) samples by individual, exactly as tskit.TreeSequence.write_vcf() does, so parsing a .trees file yields the same spectrum as parsing the VCF written from it. tskit stores the ancestral state as allele 0, which becomes the reference allele, so the parser recovers the correct polarisation with skip_non_polarized=False.

Methods:

`__init__`	Initialize the reader.
`count_sites`	The number of sites in the tree sequence.
`add_info_to_header`	Declare an INFO field, registering it with the underlying VCF header.
`close`	Release any resources held by the reader.

__init__(ts: tskit.TreeSequence, contig: str = '1')[source]#

Initialize the reader.

Parameters:

ts (tskit.TreeSequence) – The tree sequence.
contig (str) – The contig name to report (tree sequences have no contig concept).

property samples: List[str]#

The sample names.

Returns:: The sample names.

property seqnames: List[str]#

The contig name (tree sequences have a single synthetic contig).

Returns:: The contig names.

property tree_sequence: tskit.TreeSequence#

The underlying tree sequence, used by TskitVariantWriter to write a site-subset .trees.

Returns:: The tree sequence.

property sequence_length: float#

The length of the tree-sequence genome, i.e. the region over which sites are distributed. This is the true extent even when the observed (polymorphic) sites do not reach the ends.

Returns:: The sequence length.

property contig_lengths: Dict[str, int]#

The length of the single synthetic contig the tree sequence is reported under.

Returns:: The per-contig lengths.

count_sites()[source]#

The number of sites in the tree sequence.

Return type:: int
Returns:: The number of sites.

add_info_to_header(data: dict)#

Declare an INFO field, registering it with the underlying VCF header. Does nothing for non-VCF sources, which have no header.

Parameters:: data (dict) – The INFO field definition.

close()#: Release any resources held by the reader.

ZarrVariantReader#

class ZarrVariantReader(path: str, info_ancestral: str = 'AA', chunk_size: int | None = None)[source]#

Bases: VariantReader

Stream variants from a VCF-Zarr store in the vcf2zarr (VCZ) layout. Genotypes are read in variant chunks to bound memory. An ancestral-allele INFO field, if encoded as a variant_<tag> array, is surfaced under INFO so the usual polarisation logic applies.

Methods:

`__init__`	Initialize the reader.
`count_sites`	The number of sites in the store.
`add_info_to_header`	Declare an INFO field, registering it with the underlying VCF header.
`close`	Release any resources held by the reader.

__init__(path: str, info_ancestral: str = 'AA', chunk_size: int | None = None)[source]#

Initialize the reader.

Parameters:

path (str) – The path to the VCF-Zarr store.
info_ancestral (str) – The INFO tag holding the ancestral allele.
chunk_size (Optional[int]) – The number of variants read per batch, or None to follow the store’s own chunking along the variants axis.

property samples: List[str]#

The sample names.

Returns:: The sample names.

property seqnames: List[str]#

The contig names declared in the store.

Returns:: The contig names.

property contig_lengths: Dict[str, int] | None#

The contig lengths the store declares in its contig_length array, which vcf2zarr fills from the ##contig headers of the VCF it converts. A contig whose length the store leaves unset is absent from the mapping. A store written by ZarrVariantWriter from a source declaring no lengths carries the last position written on each contig instead, so a length read here is a lower bound on the contig rather than a guarantee.

Returns:: The per-contig lengths, or None where the store carries no lengths at all.

count_sites()[source]#

The number of sites in the store.

Return type:: int
Returns:: The number of sites.

add_info_to_header(data: dict)#

Declare an INFO field, registering it with the underlying VCF header. Does nothing for non-VCF sources, which have no header.

Parameters:: data (dict) – The INFO field definition.

close()#: Release any resources held by the reader.

property sequence_length: float | None#

Returns:: The sequence length, or None.

VariantWriter#

class VariantWriter[source]#

Bases: ABC

Abstract writer mirroring VariantReader: it consumes the same streamed Variant interface and writes it to a concrete on-disk format. The output format follows the output file’s extension (see VariantWriter.open()).

Methods:

`open`	Open the variant writer matching the output file's extension: a VCF-Zarr store for `.vcz`/`.zarr` (from any input), a tskit tree sequence for `.trees` (only when the input is itself a tree sequence, since a genealogy cannot be reconstructed from genotype data), and a VCF otherwise.
`write`	Write a single variant.
`close`	Flush and release any resources held by the writer.

static open(output: str, reader: cyvcf2.VCF | VariantReader, info_ancestral: str = 'AA')[source]#

Open the variant writer matching the output file’s extension: a VCF-Zarr store for .vcz/.zarr (from any input), a tskit tree sequence for .trees (only when the input is itself a tree sequence, since a genealogy cannot be reconstructed from genotype data), and a VCF otherwise.

Parameters:

output (str) – The output path; its extension selects the format.
reader (Union[cyvcf2.VCF, VariantReader]) – The open input reader (a cyvcf2 VCF or a VariantReader).
info_ancestral (str) – The INFO tag holding the ancestral allele, for the VCF-Zarr writer.

Return type:

VariantWriter

Returns:

The writer.

Raises:

ValueError – If a .trees output is requested from a non-tree-sequence input.

write(variant: Site)[source]#

Write a single variant.

Parameters:: variant (Site) – The variant to write.
Return type:: None

close()[source]#

Flush and release any resources held by the writer.

Return type:: None

VCFVariantWriter#

class VCFVariantWriter(output: str, template: cyvcf2.VCF)[source]#

Bases: VariantWriter

Write variants to a VCF (optionally gzipped) via cyvcf2, copying the header from the input VCF. Because it reuses the input’s cyvcf2 header, this writer requires a VCF input; writing VCF output from a tree sequence or VCF-Zarr store is not supported (use a .vcz/.zarr output instead).

Methods:

`__init__`	Open the writer.
`write`	Write a single record.
`close`	Close the underlying writer.
`open`	Open the variant writer matching the output file's extension: a VCF-Zarr store for `.vcz`/`.zarr` (from any input), a tskit tree sequence for `.trees` (only when the input is itself a tree sequence, since a genealogy cannot be reconstructed from genotype data), and a VCF otherwise.

__init__(output: str, template: cyvcf2.VCF)[source]#

Open the writer.

Parameters:

output (str) – The output VCF path.
template (cyvcf2.VCF) – The input cyvcf2 VCF whose header is copied.

Raises:

ValueError – If the input is not a VCF (no cyvcf2 header to copy).

write(variant: cyvcf2.Variant)[source]#

Write a single record.

Parameters:: variant (cyvcf2.Variant) – The cyvcf2 variant to write.
Return type:: None

close()[source]#

Close the underlying writer.

Return type:: None

static open(output: str, reader: cyvcf2.VCF | VariantReader, info_ancestral: str = 'AA')#

Parameters:

output (str) – The output path; its extension selects the format.
reader (Union[cyvcf2.VCF, VariantReader]) – The open input reader (a cyvcf2 VCF or a VariantReader).
info_ancestral (str) – The INFO tag holding the ancestral allele, for the VCF-Zarr writer.

Return type:

VariantWriter

Returns:

The writer.

Raises:

ValueError – If a .trees output is requested from a non-tree-sequence input.

ZarrVariantWriter#

class ZarrVariantWriter(output: str, samples: List[str], seqnames: List[str], info_ancestral: str = 'AA', contig_lengths: Dict[str, int] | None = None)[source]#

Bases: VariantWriter

Write variants to a VCF-Zarr store in the vcf2zarr (VCZ) layout read back by ZarrVariantReader, so the output can come from any input (VCF, tree sequence or another VCF-Zarr store). Any INFO fields present on the variants (for example an annotated ancestral allele) are persisted as variant_<tag> arrays.

Positions, genotypes, alleles and INFO values accumulate in chunk-sized buffers and each complete chunk is flushed to the store, so the memory held is one chunk rather than the whole input. The two ragged axes (the ploidy and the allele width) are only known once every variant has been seen, so the arrays are created from the first chunk and rebuilt on the rare site that widens them. The same holds of the type of an INFO field, which is taken from the chunk it first appears in and rewritten on the rare chunk that widens it (an integer field first seen without a value, a numeric one that turns out to carry a string).

Methods:

`__init__`	Open the writer.
`write`	Buffer a single variant, flushing the current chunk to the store once it is full.
`close`	Write the remaining variants and finalise the store.
`open`	Open the variant writer matching the output file's extension: a VCF-Zarr store for `.vcz`/`.zarr` (from any input), a tskit tree sequence for `.trees` (only when the input is itself a tree sequence, since a genealogy cannot be reconstructed from genotype data), and a VCF otherwise.

__init__(output: str, samples: List[str], seqnames: List[str], info_ancestral: str = 'AA', contig_lengths: Dict[str, int] | None = None)[source]#

Open the writer.

Parameters:

output (str) – The output store path (.vcz or .zarr).
samples (List[str]) – The sample names.
seqnames (List[str]) – The contig names.
info_ancestral (str) – The INFO tag holding the ancestral allele (written as variant_<tag>).
contig_lengths (Optional[Dict[str, int]]) – The lengths the source declares for its contigs, written to contig_length so the store reports the region rather than the span of the variants it holds. A contig absent from the mapping falls back to the last position written on it.

write(variant: Site)[source]#

Buffer a single variant, flushing the current chunk to the store once it is full.

Parameters:: variant (Site) – The variant to write.
Return type:: None

close()[source]#

Write the remaining variants and finalise the store.

Return type:: None

static open(output: str, reader: cyvcf2.VCF | VariantReader, info_ancestral: str = 'AA')#

Parameters:

output (str) – The output path; its extension selects the format.
reader (Union[cyvcf2.VCF, VariantReader]) – The open input reader (a cyvcf2 VCF or a VariantReader).
info_ancestral (str) – The INFO tag holding the ancestral allele, for the VCF-Zarr writer.

Return type:

VariantWriter

Returns:

The writer.

Raises:

ValueError – If a .trees output is requested from a non-tree-sequence input.

TskitVariantWriter#

class TskitVariantWriter(ts: tskit.TreeSequence, output: str)[source]#

Bases: VariantWriter

Write a site-subset of an input tree sequence to a .trees file. Only the sites whose variants are written (i.e. survive filtering) are kept, via tskit.TreeSequence.delete_sites(), leaving the genealogy untouched. This is the only well-defined .trees output: a genealogy cannot be reconstructed from genotype data, so a tree-sequence output requires a tree-sequence input. INFO fields added by annotations are attached to the kept sites as JSON metadata on a best-effort basis.

Methods:

`__init__`	Open the writer.
`open`	Open the variant writer matching the output file's extension: a VCF-Zarr store for `.vcz`/`.zarr` (from any input), a tskit tree sequence for `.trees` (only when the input is itself a tree sequence, since a genealogy cannot be reconstructed from genotype data), and a VCF otherwise.
`write`	Mark the variant's site as kept.
`close`	Delete the dropped sites and dump the resulting tree sequence.

__init__(ts: tskit.TreeSequence, output: str)[source]#

Open the writer.

Parameters:

ts (tskit.TreeSequence) – The source tree sequence.
output (str) – The output .trees path.

static open(output: str, reader: cyvcf2.VCF | VariantReader, info_ancestral: str = 'AA')#

Parameters:

output (str) – The output path; its extension selects the format.
reader (Union[cyvcf2.VCF, VariantReader]) – The open input reader (a cyvcf2 VCF or a VariantReader).
info_ancestral (str) – The INFO tag holding the ancestral allele, for the VCF-Zarr writer.

Return type:

VariantWriter

Returns:

The writer.

Raises:

ValueError – If a .trees output is requested from a non-tree-sequence input.

write(variant: Site)[source]#

Mark the variant’s site as kept.

Parameters:: variant (Site) – The variant to keep.
Return type:: None

close()[source]#

Delete the dropped sites and dump the resulting tree sequence.

Return type:: None

NoTypeException#

class NoTypeException[source]#

Bases: BaseException

Exception thrown when no type can be determined.

Methods:

`__init__`
`add_note`	Exception.add_note(note) -- add a note to the exception
`with_traceback`	Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.

__init__(*args, **kwargs)#

add_note()#: Exception.add_note(note) – add a note to the exception

args#

with_traceback()#: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

FileHandler#

class FileHandler(cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Bases: object

Base class for the handlers that read a local or remote file. Provides the shared plumbing: resolving a URL or a gzipped path to a readable local file, optional caching of downloads, and per-contig name aliasing.

Methods:

`__init__`	Create a new FileHandler instance.
`is_url`	Check if the given path is a URL.
`download_if_url`	Download the file if it is a URL.
`unzip_if_zipped`	If the given file is gzipped, unzip it and return the path to the unzipped file.
`get_filename`	Return the file extension of a URL.
`hash`	Return a truncated SHA1 hash of a string.
`download_file`	Download a file from a URL.
`get_aliases`	Get all aliases for the given contig alias including the primary alias.

__init__(cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Create a new FileHandler instance.

Parameters:

cache (bool) – Whether to cache files that are downloaded from URLs
aliases (Dict[str, List[str]]) – The contig aliases.

cache: bool#: Whether to cache files that are downloaded from URLs

aliases#: The contig mappings

static is_url(path: str)[source]#

Check if the given path is a URL.

Parameters:: path (str) – The path to check.
Return type:: bool
Returns:: True if the path is a URL, False otherwise.

download_if_url(path: str)[source]#

Download the file if it is a URL.

Parameters:: path (str) – The path to the file.
Return type:: str
Returns:: The path to the downloaded file or the original path.

static unzip_if_zipped(file: str)[source]#

If the given file is gzipped, unzip it and return the path to the unzipped file. If the file is not gzipped, return the path to the original file.

Parameters:: file (str) – The path to the file.
Returns:: The path to the unzipped file, or the original file if it was not gzipped.

static get_filename(url: str)[source]#

Return the file extension of a URL.

Parameters:: url (str) – The URL to get the file extension from.
Returns:: The file extension.

static hash(s: str)[source]#

Return a truncated SHA1 hash of a string.

Parameters:: s (str) – The string to hash.
Return type:: str
Returns:: The SHA1 hash.

classmethod download_file(url: str, cache: bool = True, desc: str = 'Downloading file')[source]#

Download a file from a URL.

Parameters:

cache (bool) – Whether to cache the file.
url (str) – The URL to download the file from.
desc (str) – Description for the progress bar

Return type:

str

Returns:

The path to the downloaded file.

get_aliases(contig: str)[source]#

Get all aliases for the given contig alias including the primary alias.

Parameters:: contig (str) – The contig.
Return type:: List[str]
Returns:: The aliases.

VCFHandler#

class VCFHandler(vcf, info_ancestral='AA', max_sites=inf, seed=0, cache=True, aliases={})[source]#

Bases: FileHandler

Base class for the handlers that read a variant source. Opens the source (a VCF file, a VCF-Zarr store or a tskit tree sequence) as a streamed VariantReader and exposes the sample names, contig names and site count the parser and filters need.

Methods:

`__init__`	Create a new variant handler instance.
`load_vcf`	Open a VCF file for streaming.
`count_sites`	Count the number of sites in the source.
`get_pbar`	Return a progress bar for the number of sites.
`download_file`	Download a file from a URL.
`download_if_url`	Download the file if it is a URL.
`get_aliases`	Get all aliases for the given contig alias including the primary alias.
`get_filename`	Return the file extension of a URL.
`hash`	Return a truncated SHA1 hash of a string.
`is_url`	Check if the given path is a URL.
`unzip_if_zipped`	If the given file is gzipped, unzip it and return the path to the unzipped file.

__init__(vcf, info_ancestral='AA', max_sites=inf, seed=0, cache=True, aliases={})[source]#

Create a new variant handler instance.

Parameters:

vcf – The variant source: a VCF path (gzipped or a URL), a VCF-Zarr store (.vcz/.zarr), or a tskit tree sequence (a .trees path or a TreeSequence object)
info_ancestral – The tag in the INFO field that contains the ancestral allele
max_sites – Maximum number of sites to consider
seed – Seed for the random number generator. Use None for no seed.
cache – Whether to cache files that are downloaded from URLs
aliases – The contig aliases.

vcf#: The variant source (a path or a tskit TreeSequence object)

info_ancestral: str#: The tag in the INFO field that contains the ancestral allele

max_sites: int#: Maximum number of sites to consider

seed: int | None#: Seed for the random number generator

rng#: Random generator instance

load_vcf()[source]#

Open a VCF file for streaming.

Return type:: cyvcf2.VCF
Returns:: The VCF reader.

property contig_lengths: Dict[str, int] | None#

The declared length (bp) of each contig of the source: the ##contig headers of a VCF, the contig_length array of a VCF-Zarr store, or the length of the genome of a tree sequence. Only the contigs whose length is known are present, and a source declaring none gives None. The span of the observed variants understates the region on a sparsely covered contig, so this is what a spectrum extrapolating monomorphic sites should be sized against.

Returns:: The per-contig lengths, or None where the source declares none.

property n_sites: int#

Get the number of sites in the input.

Returns:: Number of sites

count_sites()[source]#

Count the number of sites in the source.

Return type:: int
Returns:: Number of sites

get_pbar(desc: str = 'Processing sites', total: int | None = 0)[source]#

Return a progress bar for the number of sites.

Parameters:

desc (str) – Description for the progress bar
total (int | None) – Total number of items

Return type:

tqdm

Returns:

tqdm

classmethod download_file(url: str, cache: bool = True, desc: str = 'Downloading file')#

Download a file from a URL.

Parameters:

cache (bool) – Whether to cache the file.
url (str) – The URL to download the file from.
desc (str) – Description for the progress bar

Return type:

str

Returns:

The path to the downloaded file.

download_if_url(path: str)#

Download the file if it is a URL.

Parameters:: path (str) – The path to the file.
Return type:: str
Returns:: The path to the downloaded file or the original path.

get_aliases(contig: str)#

Get all aliases for the given contig alias including the primary alias.

Parameters:: contig (str) – The contig.
Return type:: List[str]
Returns:: The aliases.

static get_filename(url: str)#

Return the file extension of a URL.

Parameters:: url (str) – The URL to get the file extension from.
Returns:: The file extension.

static hash(s: str)#

Return a truncated SHA1 hash of a string.

Parameters:: s (str) – The string to hash.
Return type:: str
Returns:: The SHA1 hash.

static is_url(path: str)#

Check if the given path is a URL.

Parameters:: path (str) – The path to check.
Return type:: bool
Returns:: True if the path is a URL, False otherwise.

static unzip_if_zipped(file: str)#

If the given file is gzipped, unzip it and return the path to the unzipped file. If the file is not gzipped, return the path to the original file.

Parameters:: file (str) – The path to the file.
Returns:: The path to the unzipped file, or the original file if it was not gzipped.

cache: bool#: Whether to cache files that are downloaded from URLs

aliases#: The contig mappings

FASTAHandler#

class FASTAHandler(fasta: str | None, cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Bases: FileHandler

Handler for a FASTA reference. Reads the reference sequence used by the degeneracy, synonymy and CpG logic, indexed by record offset so a contig can be fetched without scanning the whole file, and resolving contig name aliases against the variant source.

Methods:

`__init__`	Create a new FASTAHandler instance.
`load_fasta`	Load a FASTA file into a dictionary.
`get_contig`	Get the contig from the FASTA file.
`get_contig_names`	Get the names of the contigs in the FASTA file.
`download_file`	Download a file from a URL.
`download_if_url`	Download the file if it is a URL.
`get_aliases`	Get all aliases for the given contig alias including the primary alias.
`get_filename`	Return the file extension of a URL.
`hash`	Return a truncated SHA1 hash of a string.
`is_url`	Check if the given path is a URL.
`unzip_if_zipped`	If the given file is gzipped, unzip it and return the path to the unzipped file.

__init__(fasta: str | None, cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Create a new FASTAHandler instance.

Parameters:

fasta (str | None) – The path to the FASTA file.
cache (bool) – Whether to cache files that are downloaded from URLs
aliases (Dict[str, List[str]]) – The contig aliases.

fasta: str#: The path to the FASTA file.

load_fasta(file: str)[source]#

Load a FASTA file into a dictionary.

Parameters:: file (str) – The path to The FASTA file path, possibly gzipped or a URL
Return type:: FastaIterator
Returns:: Iterator over the sequences.

get_contig(aliases, rewind: bool = True, notify: bool = True)[source]#

Get the contig from the FASTA file. The contig is looked up in the header index and read by seeking to it, so the sites may visit the contigs in any order at the same cost.

Note that pyfaidx would be more efficient here, but there were problems when running it in parallel.

Parameters:

aliases – The contig aliases.
rewind (bool) – Unused, the lookup does not depend on the position of a cursor.
notify (bool) – Unused, the lookup does not depend on the position of a cursor.

Return type:

SeqRecord

Returns:

The contig.

Raises:

LookupError – Where the FASTA carries none of the aliases.

get_contig_names()[source]#

Get the names of the contigs in the FASTA file.

Return type:: List[str]
Returns:: The contig names.

classmethod download_file(url: str, cache: bool = True, desc: str = 'Downloading file')#

Download a file from a URL.

Parameters:

cache (bool) – Whether to cache the file.
url (str) – The URL to download the file from.
desc (str) – Description for the progress bar

Return type:

str

Returns:

The path to the downloaded file.

download_if_url(path: str)#

Download the file if it is a URL.

Parameters:: path (str) – The path to the file.
Return type:: str
Returns:: The path to the downloaded file or the original path.

get_aliases(contig: str)#

Get all aliases for the given contig alias including the primary alias.

Parameters:: contig (str) – The contig.
Return type:: List[str]
Returns:: The aliases.

static get_filename(url: str)#

Return the file extension of a URL.

Parameters:: url (str) – The URL to get the file extension from.
Returns:: The file extension.

static hash(s: str)#

Return a truncated SHA1 hash of a string.

Parameters:: s (str) – The string to hash.
Return type:: str
Returns:: The SHA1 hash.

static is_url(path: str)#

Check if the given path is a URL.

Parameters:: path (str) – The path to check.
Return type:: bool
Returns:: True if the path is a URL, False otherwise.

static unzip_if_zipped(file: str)#

If the given file is gzipped, unzip it and return the path to the unzipped file. If the file is not gzipped, return the path to the original file.

Parameters:: file (str) – The path to the file.
Returns:: The path to the unzipped file, or the original file if it was not gzipped.

cache: bool#: Whether to cache files that are downloaded from URLs

aliases#: The contig mappings

GFFHandler#

class GFFHandler(gff: str | None, cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Bases: FileHandler

Handler for a GFF/GTF annotation. Loads the coding-sequence records the degeneracy and synonymy stratifications and the coding-sequence filtration rely on, keyed per contig and per transcript.

Methods:

`__init__`	Constructor.
`remove_overlaps`	Remove overlapping coding sequences.
`download_file`	Download a file from a URL.
`download_if_url`	Download the file if it is a URL.
`get_aliases`	Get all aliases for the given contig alias including the primary alias.
`get_filename`	Return the file extension of a URL.
`hash`	Return a truncated SHA1 hash of a string.
`is_url`	Check if the given path is a URL.
`unzip_if_zipped`	If the given file is gzipped, unzip it and return the path to the unzipped file.

__init__(gff: str | None, cache: bool = True, aliases: Dict[str, List[str]] = {})[source]#

Constructor.

Parameters:

gff (str | None) – The path to the GFF file.
cache (bool) – Whether to cache the file.
aliases (Dict[str, List[str]]) – The contig aliases.

gff#: The GFF file path

static remove_overlaps(df: DataFrame)[source]#

Remove overlapping coding sequences.

Parameters:: df (DataFrame) – The coding sequences.
Return type:: DataFrame
Returns:: The coding sequences without overlaps.

classmethod download_file(url: str, cache: bool = True, desc: str = 'Downloading file')#

Download a file from a URL.

Parameters:

cache (bool) – Whether to cache the file.
url (str) – The URL to download the file from.
desc (str) – Description for the progress bar

Return type:

str

Returns:

The path to the downloaded file.

download_if_url(path: str)#

Download the file if it is a URL.

Parameters:: path (str) – The path to the file.
Return type:: str
Returns:: The path to the downloaded file or the original path.

get_aliases(contig: str)#

Get all aliases for the given contig alias including the primary alias.

Parameters:: contig (str) – The contig.
Return type:: List[str]
Returns:: The aliases.

static get_filename(url: str)#

Return the file extension of a URL.

Parameters:: url (str) – The URL to get the file extension from.
Returns:: The file extension.

static hash(s: str)#

Return a truncated SHA1 hash of a string.

Parameters:: s (str) – The string to hash.
Return type:: str
Returns:: The SHA1 hash.

static is_url(path: str)#

Check if the given path is a URL.

Parameters:: path (str) – The path to check.
Return type:: bool
Returns:: True if the path is a URL, False otherwise.

static unzip_if_zipped(file: str)#

If the given file is gzipped, unzip it and return the path to the unzipped file. If the file is not gzipped, return the path to the original file.

Parameters:: file (str) – The path to the file.
Returns:: The path to the unzipped file, or the original file if it was not gzipped.

cache: bool#: Whether to cache files that are downloaded from URLs

aliases#: The contig mappings

Input and output

Contents

Input and output#

Site#

Variant#

VariantReader#

TskitVariantReader#

ZarrVariantReader#

VariantWriter#

VCFVariantWriter#

ZarrVariantWriter#

TskitVariantWriter#

NoTypeException#

FileHandler#

VCFHandler#

FASTAHandler#

GFFHandler#