API Reference¶

This section provides the API reference for the raiiaf package, including its public classes, functions, and modules.

Package Index¶

The following modules are available. Each entry links to a page with detailed API documentation.

`raiiaf`	RAIIAF public API.
`raiiaf.handlers.file_handler`	RAIIAF file handler for encoding and decoding .raiiaf artifacts.
`raiiaf.chunks.latent`	Latent chunk utilities for RAIIAF.
`raiiaf.chunks.image`	Image chunk utilities for RAIIAF.
`raiiaf.chunks.env`	Environment chunk utilities for RAIIAF.
`raiiaf.chunks.metadata`	Metadata chunk utilities for RAIIAF.
`raiiaf.core.constants`
`raiiaf.core.exceptions`	Exception hierarchy for RAIIAF operations.
`raiiaf.core.header`	Header utilities for RAIIAF files.

Top-level Package¶

RAIIAF public API.

Exposes the main file handler class for encoding/decoding RAIIAF artifacts.

class raiiaf.raiiafFileHandler(max_file_size: int | None = None, max_chunk_size: int | None = None)[source]¶

Bases: object

High-level API for reading and writing RAIIAF files.

static bytes_to_png(img_bytes: bytes) → Image[source]¶

Convert PNG bytes to a PIL Image.

Parameters:: img_bytes (bytes) – PNG image bytes.
Returns:: Loaded image instance.
Return type:: PIL.Image.Image
Raises:: raiiafImageError – If the bytes cannot be decoded as an image.

file_decoder(filename: str)[source]¶

Decode a RAIIAF file.

Reads the header, chunk table (metadata), and iteratively decodes the latent, image, and environment chunks.

Parameters:

filename (str) – Path to the input .raiiaf file.

Returns:

A dictionary with keys:

header (dict): Parsed header fields.
chunks (dict): Parsed chunks: ‘latent’ (list), ‘image’ (bytes), ‘env’ (EnvChunk).
metadata (dict): Parsed metadata manifest.

Return type:

dict

Raises:

raiiafCorruptHeader – If the header fails validation.
raiiafChunkError – If a chunk is truncated or corrupt.
raiiafEnvChunkError – If the environment chunk cannot be parsed when uncompressed.

file_encoder(filename: str, latent: Dict[str, ndarray], chunk_records: list, model_name: str, model_version: str, prompt: str, tags: list, img_binary: bytes, should_compress: bool = True, convert_float16: bool = True, generation_settings: dict | None = None, hardware_info: dict | None = None, extra_image: Dict[str, Any] | None = None)[source]¶

Encode a RAIIAF file.

Orchestrates packing latent, image, environment, and metadata chunks, builds the header, and writes the final file.

Parameters:

filename (str) – Output .raiiaf filename. The .raiiaf extension is required.
latent (Dict[str, np.ndarray]) – Mapping of latent keys to arrays.
chunk_records (list) – Mutable list that will be appended with chunk records.
model_name (str) – Name of the model.
model_version (str) – Version of the model.
prompt (str) – Prompt used for generation.
tags (list) – Tags associated with the generation.
img_binary (bytes) – PNG image bytes to embed.
should_compress (bool) – Whether to compress chunks. Defaults to True.
convert_float16 (bool) – Convert latents to float16 for storage. Defaults to True.
generation_settings (Optional[dict]) – Generation configuration to include in metadata.
hardware_info (Optional[dict]) – Hardware information to include in metadata.
extra_image (Optional[Dict[str, Any]]) – Extra fields for the image chunk record.

Returns:

A dictionary containing header bytes and chunk bytes with keys:

header (bytes)
latent_chunks (bytes)
metadata_chunk (bytes)
image_chunk (Optional[bytes])

Return type:

dict

static png_to_bytes(png_path: str) → bytes[source]¶

Convert a PNG image to bytes.

Preserves transparency by converting to RGBA.

Parameters:: png_path (str) – Path to the PNG image.
Returns:: PNG image data in bytes.
Return type:: bytes
Raises:: raiiafImageError – If the image cannot be read or encoded.

validate_chunk_count(count: int) → bool[source]¶

Validate the number of chunks.

Parameters:: count (int) – Number of chunks.
Returns:: True if the count is within limits.
Return type:: bool
Raises:: raiiafDecodeError – If count exceeds configured limits.

validate_file_size(size: int, context: str = 'file') → bool[source]¶

Validate a file or chunk size.

Parameters:

size (int) – Size in bytes to validate.
context (str) – Validation context; either “file” or “chunk”.

Returns:

True if the size is within limits.

Return type:

bool

Raises:

raiiafDecodeError – If size exceeds configured limits.

Handlers¶

RAIIAF file handler for encoding and decoding .raiiaf artifacts.

This module orchestrates packing/unpacking of latent, image, environment, and metadata chunks, builds the header, and provides a high-level API for reading and writing RAIIAF files.

class raiiaf.handlers.file_handler.raiiafFileHandler(max_file_size: int | None = None, max_chunk_size: int | None = None)[source]¶

Bases: object

High-level API for reading and writing RAIIAF files.

static bytes_to_png(img_bytes: bytes) → Image[source]¶

Convert PNG bytes to a PIL Image.

Parameters:: img_bytes (bytes) – PNG image bytes.
Returns:: Loaded image instance.
Return type:: PIL.Image.Image
Raises:: raiiafImageError – If the bytes cannot be decoded as an image.

file_decoder(filename: str)[source]¶

Decode a RAIIAF file.

Reads the header, chunk table (metadata), and iteratively decodes the latent, image, and environment chunks.

Parameters:

filename (str) – Path to the input .raiiaf file.

Returns:

A dictionary with keys:

header (dict): Parsed header fields.
chunks (dict): Parsed chunks: ‘latent’ (list), ‘image’ (bytes), ‘env’ (EnvChunk).
metadata (dict): Parsed metadata manifest.

Return type:

dict

Raises:

raiiafCorruptHeader – If the header fails validation.
raiiafChunkError – If a chunk is truncated or corrupt.
raiiafEnvChunkError – If the environment chunk cannot be parsed when uncompressed.

file_encoder(filename: str, latent: Dict[str, ndarray], chunk_records: list, model_name: str, model_version: str, prompt: str, tags: list, img_binary: bytes, should_compress: bool = True, convert_float16: bool = True, generation_settings: dict | None = None, hardware_info: dict | None = None, extra_image: Dict[str, Any] | None = None)[source]¶

Encode a RAIIAF file.

Orchestrates packing latent, image, environment, and metadata chunks, builds the header, and writes the final file.

Parameters:

filename (str) – Output .raiiaf filename. The .raiiaf extension is required.
latent (Dict[str, np.ndarray]) – Mapping of latent keys to arrays.
chunk_records (list) – Mutable list that will be appended with chunk records.
model_name (str) – Name of the model.
model_version (str) – Version of the model.
prompt (str) – Prompt used for generation.
tags (list) – Tags associated with the generation.
img_binary (bytes) – PNG image bytes to embed.
should_compress (bool) – Whether to compress chunks. Defaults to True.
convert_float16 (bool) – Convert latents to float16 for storage. Defaults to True.
generation_settings (Optional[dict]) – Generation configuration to include in metadata.
hardware_info (Optional[dict]) – Hardware information to include in metadata.
extra_image (Optional[Dict[str, Any]]) – Extra fields for the image chunk record.

Returns:

A dictionary containing header bytes and chunk bytes with keys:

header (bytes)
latent_chunks (bytes)
metadata_chunk (bytes)
image_chunk (Optional[bytes])

Return type:

dict

static png_to_bytes(png_path: str) → bytes[source]¶

Convert a PNG image to bytes.

Preserves transparency by converting to RGBA.

Parameters:: png_path (str) – Path to the PNG image.
Returns:: PNG image data in bytes.
Return type:: bytes
Raises:: raiiafImageError – If the image cannot be read or encoded.

validate_chunk_count(count: int) → bool[source]¶

Validate the number of chunks.

Parameters:: count (int) – Number of chunks.
Returns:: True if the count is within limits.
Return type:: bool
Raises:: raiiafDecodeError – If count exceeds configured limits.

validate_file_size(size: int, context: str = 'file') → bool[source]¶

Validate a file or chunk size.

Parameters:

size (int) – Size in bytes to validate.
context (str) – Validation context; either “file” or “chunk”.

Returns:

True if the size is within limits.

Return type:

bool

Raises:

raiiafDecodeError – If size exceeds configured limits.

Chunks¶

Latent chunk utilities for RAIIAF.

This module provides functionality to validate, pack, compress, and parse latent arrays as RAIIAF LATN chunks. It supports compressed (zstd + zfpy) and uncompressed representations and exposes helpers for lazy loading.

class raiiaf.chunks.latent.raiiafLatent[source]¶

Bases: object

Operations for RAIIAF latent (LATN) chunks.

dtype_from_flags(flags: bytes) → dtype[source]¶

Map 4-byte chunk flags to a NumPy dtype.

Handles null-padding and validates known types.

Parameters:: flags (bytes) – 4-byte flag field from the chunk header (e.g., b”F16").
Returns:: Corresponding dtype (float16 or float32).
Return type:: numpy.dtype
Raises:: ValueError – If the flag does not correspond to a known dtype.

iter_lazy_latents(filename: str, chunk_records: list)[source]¶

Yield callables for each latent chunk to enable lazy loading.

Parameters:

filename (str) – Path to the RAIIAF file.
chunk_records (list) – List of manifest chunk records.

Yields:

Callable[[], np.ndarray] – A function that loads and returns each latent array.

latent_compressor(chunk_type: bytes, chunk_flags: bytes, data: ndarray, tolerance: float = 0.0) → dict[source]¶

Compress a latent array into a LATN chunk.

The header is compressed with Zstd and payload with ZFPY (optionally lossy).

Parameters:

chunk_type (bytes) – 4-byte chunk identifier (b”LATN”).
chunk_flags (bytes) – 4-byte dtype flags (b”F16" or b”F32").
data (np.ndarray) – NumPy array to compress (float16/float32).
tolerance (float) – zfpy lossy tolerance; 0.0 for lossless. Defaults to 0.0.

Returns:

Dictionary with keys ‘chunk’, ‘len_header’, ‘len_data’.

Return type:

dict

latent_dtype_validator(latent_array: ndarray) → bool[source]¶

Validate latent data type and values.

Parameters:: latent_array (np.ndarray) – Latent array to validate.
Returns:: True if valid.
Return type:: bool
Raises:: raiiafLatentError – If dtype or values are invalid (NaN/Inf or unsupported dtype).

latent_packer(latent: Dict[str, ndarray], file_offset: int = 0, chunk_records=None, should_compress: bool = True, convert_float16: bool = True) → list[source]¶

Pack latent arrays into chunk bytes and append manifest records.

Parameters:

latent (Dict[str, np.ndarray]) – Mapping of latent key to array.
file_offset (int) – Current file offset where the first latent chunk will be placed.
chunk_records (Optional[list]) – Mutable list to append chunk records to.
should_compress (bool) – Whether to compress using zstd+zfpy. Defaults to True.
convert_float16 (bool) – Convert arrays to float16 before storage. Defaults to True.

Returns:

List of chunk byte strings (to be concatenated externally).

Return type:

list[bytes]

Raises:

raiiafLatentError – If inputs are invalid or unsupported dtypes are used.

latent_parser(chunk: bytes | dict, shape: tuple, compressed: bool = True)[source]¶

Parse a latent chunk and return the latent array.

Parameters:

chunk (Union[bytes, dict]) – Raw bytes for uncompressed chunks or dict with keys ‘chunk’, ‘len_header’, ‘len_data’ for compressed chunks.
shape (tuple) – Expected array shape.
compressed (bool) – Whether the chunk is compressed. Defaults to True.

Returns:

Parsed latent array with the expected dtype and shape.

Return type:

np.ndarray

Raises:

raiiafLatentError – If the chunk is malformed or decompression fails.

latent_shape_validator(latent_array: ndarray, expected_dims: int = 4, max_dimension_size: int = 8192) → bool[source]¶

Validate latent array shape and dimensions.

Parameters:

latent_array (np.ndarray) – Latent array to validate.
expected_dims (int) – Expected number of dimensions (default: 4 for NCHW).
max_dimension_size (int) – Maximum allowed size for any single dimension.

Returns:

True if valid.

Return type:

bool

Raises:

raiiafLatentError – If validation fails.

make_lazy_latent_loader(filename: str, chunk_record: dict)[source]¶

Create a callable that lazily loads the latent array on demand.

Parameters:

filename (str) – Path to the RAIIAF file.
chunk_record (dict) – Manifest record describing the latent chunk.

Returns:

A function that loads and returns the latent array.

Return type:

Callable[[], np.ndarray]

Raises:

raiiafLatentError – If the chunk cannot be loaded from the file.

Image chunk utilities for RAIIAF.

Provides validation, packing (compression) and parsing for image data chunks (DATA). Uses zstd for compression.

class raiiaf.chunks.image.raiiafImage[source]¶

Bases: object

Operations for RAIIAF image (DATA) chunks.

image_bytes_validator(image_bytes: bytes)[source]¶

Validate image bytes.

Parameters:: image_bytes (bytes) – Image bytes to validate.
Returns:: True if the image bytes form a valid image.
Return type:: bool
Raises:: raiiafImageError – If validation fails.

image_data_chunk_builder(image_binary: bytes)[source]¶

Build a compressed image DATA chunk from raw image bytes.

Parameters:: image_binary (bytes) – Raw PNG image bytes to store.
Returns:: Compressed chunk bytes suitable for writing to the RAIIAF file.
Return type:: bytes
Raises:: raiiafImageError – If the chunk cannot be constructed.

image_data_chunk_parser(compressed_chunk)[source]¶

Parse a compressed image DATA chunk.

Parameters:: compressed_chunk (bytes) – Compressed image data chunk.
Returns:: Parsed info with keys ‘chunk_type’, ‘chunk_flags’, ‘chunk_size’, ‘image_data’.
Return type:: dict
Raises:: raiiafImageError – If decompression or parsing fails.

Environment chunk utilities for RAIIAF.

Defines dataclasses for environment components and provides helpers to populate, build (compress), and parse environment (ENVC) chunks.

class raiiaf.chunks.env.EnvChunk(env_version: int, components: List[EnvComponent])[source]¶

Bases: object

Structured environment chunk content.

env_version¶

Environment chunk version number.

Type:: int

components¶

List of environment components.

Type:: List[EnvComponent]

components: List[EnvComponent]¶

env_version: int¶

class raiiaf.chunks.env.EnvComponent(component_id: str, cononical_str: str, component_sha256_digest: bytes)[source]¶

Bases: object

Single environment component captured in the ENVC chunk.

component_id¶

Identifier (e.g., ‘torch’, ‘numpy’, ‘python’, ‘cuda’, ‘os’, ‘gpu’).

Type:: str

cononical_str¶

Canonical string describing the component and version.

Type:: str

component_sha256_digest¶

SHA-256 digest for the canonical string.

Type:: bytes

component_id: str¶

component_sha256_digest: bytes¶

cononical_str: str¶

class raiiaf.chunks.env.raiiafEnv[source]¶

Bases: object

Operations for RAIIAF environment (ENVC) chunks.

env_chunk_builder(env_chunk: EnvChunk)[source]¶

Build a compressed ENVC chunk from an EnvChunk object.

Parameters:: env_chunk (EnvChunk) – Populated environment chunk object.
Returns:: Compressed chunk bytes and the raw env JSON bytes.
Return type:: Tuple[bytes, bytes]
Raises:: raiiafEnvChunkError – If building or compression fails.

env_chunk_parser(compressed_chunk)[source]¶

Parse a compressed ENVC chunk.

Parameters:: compressed_chunk (bytes) – Compressed environment chunk.
Returns:: Parsed environment with keys ‘chunk_type’, ‘chunk_flags’, ‘chunk_size’, ‘env_chunk’.
Return type:: dict

env_chunk_populator()[source]¶

Populate the environment chunk with component hashes.

Returns:: Populated environment chunk object with common components.
Return type:: EnvChunk

Metadata chunk utilities for RAIIAF.

Provides validation against a JSON schema, compression/decompression helpers for the metadata chunk (META), and manifest construction helpers.

class raiiaf.chunks.metadata.raiiafMetadata[source]¶

Bases: object

Operations for RAIIAF metadata (META) chunks.

build_manifest(version_major: int, version_minor: int, model_name: str, model_version: str, prompt: str, tags: list, chunk_records: list, generation_settings: dict | None = None, hardware_info: dict | None = None)[source]¶

Build a manifest dictionary from inputs and chunk records.

chunk_records must be a list of dicts, with each having:

{
    "type": str,
    "flags": str,
    "offset": int,
    "compressed_size": int,
    "uncompressed_size": int,
    "hash": str,
    "extra": dict
}

Parameters:

version_major (int) – Major version number.
version_minor (int) – Minor version number.
model_name (str) – Name of the model.
model_version (str) – Version of the model.
prompt (str) – Prompt used for generation.
tags (list) – List of tags.
chunk_records (list) – List of chunk record dictionaries.
generation_settings (Optional[dict]) – Generation settings. Defaults to predefined structure if not provided.
hardware_info (Optional[dict]) – Hardware information. Defaults to predefined structure if not provided.

Returns:

Manifest dictionary.

Return type:

dict

metadata_compressor(manifest)[source]¶

Compress a manifest into a META chunk.

Parameters:: manifest (dict) – Manifest dictionary to serialize and compress.
Returns:: Compressed META chunk bytes.
Return type:: bytes

metadata_parser(compressed_chunk: bytes) → dict[source]¶

Parse and decompress a META chunk.

Parameters:: compressed_chunk (bytes) – Compressed metadata chunk.
Returns:: Parsed metadata manifest.
Return type:: dict

metadata_validator(manifest) → bool[source]¶

Validate a metadata manifest using the JSON Schema.

Parameters:: manifest (dict) – Metadata manifest.
Returns:: True if it is valid.
Return type:: bool
Raises:: raiiafMetadataError – If the metadata is invalid.

Core¶

Exception hierarchy for RAIIAF operations.

Defines specialized exceptions for decode errors, corrupt headers, metadata issues, chunk-level errors, and specific latent/image/environment chunk problems.

exception raiiaf.core.exceptions.raiiafChunkError(message: str)[source]¶

Bases: raiiafDecodeError

Raised for generic chunk-level issues (truncation, bounds, etc.).

exception raiiaf.core.exceptions.raiiafCorruptHeader(message: str)[source]¶

Bases: raiiafDecodeError

Raised when the file header is detected as corrupt or invalid.

exception raiiaf.core.exceptions.raiiafDecodeError(message: str)[source]¶

Bases: Exception

Base error for RAIIAF decoding/validation failures.

exception raiiaf.core.exceptions.raiiafEnvChunkError(message: str)[source]¶

Bases: raiiafChunkError

Raised for environment (ENVC) chunk parsing/validation issues.

exception raiiaf.core.exceptions.raiiafImageError(message: str)[source]¶

Bases: raiiafChunkError

Raised for image (DATA) chunk issues or image conversion errors.

exception raiiaf.core.exceptions.raiiafLatentError(message: str)[source]¶

Bases: raiiafChunkError

Raised for latent (LATN) chunk issues.

exception raiiaf.core.exceptions.raiiafMetadataError(message: str)[source]¶

Bases: raiiafDecodeError

Raised when the metadata (manifest) is invalid or cannot be parsed.

This module defines functions to initialize, parse, and validate the RAIIAF file header. The header layout is defined by raiiaf.core.constants.HEADER_FORMAT and has the following fields:

magic (6s): ASCII bytes b”raiiaf”
version_major (B)
version_minor (B)
flags (B)
chunk_table_offset (I)
chunk_table_size (I)
chunk_count (I)
file_size (I)
reserved (I)

raiiaf.core.header.header_init(version_major=1, version_minor=0, flags=0, chunk_table_offset=0, chunk_table_size=0, chunk_count=0, file_size=0, reserved=0)[source]¶

Initialize and pack a RAIIAF file header.

Parameters:

version_major (int) – Major version number.
version_minor (int) – Minor version number.
flags (int) – Header flags.
chunk_table_offset (int) – Offset (in bytes) to the start of the chunk table.
chunk_table_size (int) – Size (in bytes) of the chunk table.
chunk_count (int) – Number of chunks referenced in the table.
file_size (int) – Total file size in bytes.
reserved (int) – Reserved for future use.

Returns:

Packed header bytes conforming to HEADER_FORMAT.

Return type:

bytes

raiiaf.core.header.header_parse(buf: bytes) → dict[source]¶

Parse a RAIIAF header from raw bytes.

Parameters:

buf (bytes) – Raw file bytes containing the header at the beginning.

Returns:

Parsed header fields with keys: magic, version_major, version_minor,: flags, chunk_table_offset, chunk_table_size, chunk_count, file_size, reserved.

Return type:

dict

Raises:

raiiafCorruptHeader – If the provided buffer is too small.

raiiaf.core.header.header_validate(header: dict) → bool[source]¶

Validate a parsed RAIIAF header.

Parameters:: header (dict) – Parsed header fields as returned by header_parse.
Returns:: True if the header is valid, False otherwise.
Return type:: bool

API Reference¶

Package Index¶

Top-level Package¶

Handlers¶

Chunks¶

Core¶

RAIIAF

Navigation

Related Topics