API Reference

Auto-generated API reference for the llm package. Pages are built from the source docstrings by mkdocstrings — if something looks stale or missing, the canonical source is the Python file, not this page.

Layout

The package is organised by responsibility; each subpackage owns one slice of the framework.

Subpackage	Responsibility	Page
llm.core	Building blocks (attention, MLP, norms, KV cache, embeddings)	core.md
llm.core.peft	Parameter-efficient fine-tuning (LoRA, QLoRA, adapters, etc.)	peft.md
llm.runtime	Plugin registries + model factory	runtime.md
llm.models	High-level model definitions (decoder, regression MLP)	models.md
llm.generation	Sampling, generation backends, registry	generation.md
llm.serving	FastAPI app, routers, batched engine, metrics	serving.md
llm.training	Trainer, configs, callbacks, RLHF	training-detail.md
llm.evaluation	Metrics, offline tasks, lm-eval-harness adapter (optional)	evaluation.md
llm.data	Datasets, DataModules, streaming sources, dataset presets	data.md
llm.quantization	GPTQ, PTQ, mixed-precision quantization	quantization.md
llm.export	ONNX / TorchScript / GGUF export backends	export.md
llm.compat	HuggingFace checkpoint loading/publishing	compat.md
llm.tokenization	Tokenizer implementations (BPE, character-level)	tokenization.md
llm.cli	CLI tools (migrate-ckpt, quantize)	cli.md

Stability

The public surface is whatever is exported through llm/__init__.py and the per-subpackage __init__.py files. Anything reachable through from llm.X import Y is part of the supported API; under-the-hood helpers prefixed with _ are not.