llm.core.peft — Parameter-Efficient Fine-Tuning

The PEFT subpackage implements parameter-efficient fine-tuning methods that train only a small set of additional parameters while keeping the pretrained model frozen. All methods register into PEFT_REGISTRY and can be applied via the training task configuration.

Overview

Method	Paper	Trainable Parameters
LoRA	Hu et al. 2021	Low-rank decomposition
QLoRA	Dettmers et al. 2023	4-bit NF4 + LoRA
AdaLoRA	He et al. 2022	Adaptive rank LoRA
Prefix Tuning	Li & Liang 2021	Virtual prefix tokens
IA³	Liu et al. 2021	Multiplicative scaling
BitFit	Zaken et al. 2021	Bias-only
Adapter	Houlsby et al. 2019	Bottleneck layers
Pfeiffer Adapter	Pfeiffer et al. 2021	FFN-only bottleneck

Registry

registry

PEFT method registry and dispatch (T2 PEFT #43).

Mirrors :mod:llm.export.registry so third-party PEFT methods can plug in via the llm.peft_methods setuptools entry-point group without forking :mod:llm.core.peft.

Built-in methods

lora, qlora, adalora, prefix_tuning, ia3, bitfit, adapter — registered eagerly by :func:ensure_methods_registered.

Usage

import torch from llm.core.peft import apply_peft, count_peft_parameters model = torch.nn.Linear(10, 10) _ = apply_peft(model, "lora", rank=2, alpha=8.0) # wraps model in-place trainable, total = count_peft_parameters(model, "lora") trainable > 0, total > 0 (True, True)

Plugin authors register a method via pyproject.toml::

[project.entry-points."llm.peft_methods"]
my_method = "my_pkg.peft:build_my_peft_method"

The factory build_my_peft_method() must return a :class:llm.core.peft.PEFTMethod instance. Built-ins are registered before the entry-point load — a plugin claiming a built-in name is silently skipped (matches the EXPORT_REGISTRY convention; overwrite=True is reserved for explicit override paths).

ensure_methods_registered

ensure_methods_registered()

Idempotently register built-in methods and load entry points.

Built-ins are registered before the entry-point load so a plugin that claims a built-in name raises / is silently skipped — the built-in is the source of truth. This matches the convention in :func:llm.export.registry.ensure_exporters_registered and :func:llm.generation.registry.ensure_backends_registered.

源代码位于： src/llm/core/peft/registry.py

def ensure_methods_registered() -> None:
    """Idempotently register built-in methods and load entry points.

    Built-ins are registered **before** the entry-point load so a
    plugin that claims a built-in name raises / is silently skipped —
    the built-in is the source of truth. This matches the convention
    in :func:`llm.export.registry.ensure_exporters_registered` and
    :func:`llm.generation.registry.ensure_backends_registered`.
    """
    global _methods_registered
    if _methods_registered:
        return

    # Double-checked locking: the fast guard above is the hot path; the
    # lock serializes the cold-start race so concurrent callers re-check
    # the flag inside the critical section (RIL ISS-119).
    with _method_registration_lock:
        if _methods_registered:
            return

        for method in iter_builtin_methods():
            # ``Registry.register`` raises on duplicate names. Built-ins
            # use stable names so re-import won't trigger duplicates;
            # third parties load after this point and the entry-point
            # loader defaults to ``overwrite=False`` so plugins claiming
            # built-in names are silently skipped (matching the export
            # convention).
            if method.name not in PEFT_REGISTRY:
                PEFT_REGISTRY.register(method.name, method)

        load_entry_point_registry("llm.peft_methods", PEFT_REGISTRY)
        _methods_registered = True

apply_peft

apply_peft(model, name, **kwargs)

Apply a registered PEFT method to model (in-place).

参数：

名称	类型	描述	默认
model	Module	The model to adapt. Modified in place by the per-method apply_* function (matches the existing convention).	必需
name	str	Registered method name (e.g. "lora", "adalora", "prefix_tuning", "ia3", "bitfit", "adapter", "qlora").	必需
**kwargs	Any	Method-specific kwargs forwarded verbatim to the per-method apply_* function. See each method's docstring for accepted kwargs.	{}

返回：

类型	描述
Module	The same model (chainable).

引发：

类型	描述
ValueError	If name is not in :data:PEFT_REGISTRY.
TypeError	If the method's wrapper rejects the model shape (e.g. Prefix Tuning on a non-MHA base).

源代码位于： src/llm/core/peft/registry.py

def apply_peft(model: nn.Module, name: str, **kwargs: Any) -> nn.Module:
    """Apply a registered PEFT method to ``model`` (in-place).

    Args:
        model: The model to adapt. Modified in place by the per-method
            ``apply_*`` function (matches the existing convention).
        name: Registered method name (e.g. ``"lora"``, ``"adalora"``,
            ``"prefix_tuning"``, ``"ia3"``, ``"bitfit"``, ``"adapter"``,
            ``"qlora"``).
        **kwargs: Method-specific kwargs forwarded verbatim to the
            per-method ``apply_*`` function. See each method's
            docstring for accepted kwargs.

    Returns:
        The same ``model`` (chainable).

    Raises:
        ValueError: If ``name`` is not in :data:`PEFT_REGISTRY`.
        TypeError: If the method's wrapper rejects the model shape
            (e.g. Prefix Tuning on a non-MHA base).
    """
    method = _resolve(name)
    return method.apply(model, **kwargs)

get_peft_parameters

get_peft_parameters(model, name)

Yield the trainable parameters added by method name.

引发：

类型	描述
NotImplementedError	If the method doesn't expose a parameter iterator (callers should fall back to [p for p in model.parameters() if p.requires_grad]).
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def get_peft_parameters(model: nn.Module, name: str) -> Iterator[nn.Parameter]:
    """Yield the trainable parameters added by method ``name``.

    Raises:
        NotImplementedError: If the method doesn't expose a parameter
            iterator (callers should fall back to
            ``[p for p in model.parameters() if p.requires_grad]``).
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "get_parameters", name)
    return fn(model)

count_peft_parameters

count_peft_parameters(model, name)

Return (trainable, total) parameter counts for method name.

引发：

类型	描述
NotImplementedError	If the method doesn't expose a count helper.
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def count_peft_parameters(model: nn.Module, name: str) -> tuple[int, int]:
    """Return ``(trainable, total)`` parameter counts for method ``name``.

    Raises:
        NotImplementedError: If the method doesn't expose a count
            helper.
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "count_parameters", name)
    return fn(model)

merge_peft

merge_peft(model, name)

Inference-time fold of the adapter into the base weight.

引发：

类型	描述
NotImplementedError	For methods that don't fold (bitfit / qlora / prefix_tuning).
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def merge_peft(model: nn.Module, name: str) -> nn.Module:
    """Inference-time fold of the adapter into the base weight.

    Raises:
        NotImplementedError: For methods that don't fold (bitfit /
            qlora / prefix_tuning).
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "merge", name)
    return fn(model)

unmerge_peft

unmerge_peft(model, name)

Reverse a previous :func:merge_peft call.

引发：

类型	描述
NotImplementedError	For methods that don't expose merge / unmerge.
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def unmerge_peft(model: nn.Module, name: str) -> nn.Module:
    """Reverse a previous :func:`merge_peft` call.

    Raises:
        NotImplementedError: For methods that don't expose merge /
            unmerge.
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "unmerge", name)
    return fn(model)

disable_peft

disable_peft(model, name)

Disable the adapter (e.g. for ablation studies).

引发：

类型	描述
NotImplementedError	For methods that don't expose a disable helper (bitfit / qlora / prefix_tuning).
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def disable_peft(model: nn.Module, name: str) -> None:
    """Disable the adapter (e.g. for ablation studies).

    Raises:
        NotImplementedError: For methods that don't expose a disable
            helper (bitfit / qlora / prefix_tuning).
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "disable", name)
    fn(model)

enable_peft

enable_peft(model, name)

Re-enable a previously disabled adapter.

引发：

类型	描述
NotImplementedError	For methods that don't expose an enable helper (bitfit / qlora / prefix_tuning).
ValueError	If name is unknown.

源代码位于： src/llm/core/peft/registry.py

def enable_peft(model: nn.Module, name: str) -> None:
    """Re-enable a previously disabled adapter.

    Raises:
        NotImplementedError: For methods that don't expose an enable
            helper (bitfit / qlora / prefix_tuning).
        ValueError: If ``name`` is unknown.
    """
    method = _resolve(name)
    fn = _require_helper(method, "enable", name)
    fn(model)

Method Types

types

Public types for the PEFT registry (T2 PEFT #43).

The :class:PEFTMethod dataclass is the contract every PEFT method — built-in or third-party plugin — must satisfy to register with :data:llm.core.peft.registry.PEFT_REGISTRY.

Built-in PEFT methods expose asymmetric API surfaces:

• lora / adalora / ia3 / adapter: apply / get_parameters / count_parameters / merge / unmerge / disable / enable — the full set
• bitfit: apply / get_parameters / count_parameters — no merge (biases are kept at inference, no fold step)
• qlora: apply / get_parameters — no merge (NF4 quantized base cannot be re-folded into a float tensor)
• prefix_tuning: apply / get_parameters — inference-time fold is :func:llm.core.prefix_tuning.fold_reparameterization, not the merge/unmerge protocol

The dataclass accommodates all of these by making get_parameters / count_parameters / merge / unmerge / disable / enable :data:Optional. Callers that hit a None helper get a loud NotImplementedError (see :mod:llm.core.peft.registry) instead of a silent skip — the failure mode is the same as the per-method apply_* raising TypeError on a non-MHA base.

TargetModuleFilter

Bases: StrEnum

What kind of submodules a PEFT method targets.

Used as metadata only — the actual filter logic lives in the per-method apply_* function (which already accepts a target_modules substring list). The enum lets introspection / docs report "this method wraps Linear layers" vs "this method wraps Multi-Head Attention" without importing the method module.

Inherits from :class:enum.StrEnum so the values serialize naturally to JSON strings (e.g. in the docs build or in metadata.json snapshots).

源代码位于： src/llm/core/peft/types.py

class TargetModuleFilter(StrEnum):
    """What kind of submodules a PEFT method targets.

    Used as metadata only — the actual filter logic lives in the
    per-method ``apply_*`` function (which already accepts a
    ``target_modules`` substring list). The enum lets introspection /
    docs report "this method wraps Linear layers" vs "this method
    wraps Multi-Head Attention" without importing the method module.

    Inherits from :class:`enum.StrEnum` so the values serialize
    naturally to JSON strings (e.g. in the docs build or in
    ``metadata.json`` snapshots).
    """

    LINEAR = "linear"
    MHA = "mha"
    ANY = "any"

PEFTMethod dataclass

The contract every PEFT method registers with the registry.

属性：

名称	类型	描述
name	str	Unique registry name (e.g. "lora", "adalora", "prefix_tuning"). Matches the registry key.
apply	Callable[..., Module]	(model, **kwargs) -> nn.Module — wraps the model in-place (per the existing apply_* convention) and returns it for chainability. Required.
get_parameters	Callable[[Module], Iterator[Parameter | Tensor]] | None	(model) -> Iterator[nn.Parameter] — yields exactly the trainable parameters added by this method. None means the method doesn't expose a parameter iterator (callers should fall back to [p for p in model.parameters() if p.requires_grad]).
count_parameters	Callable[[Module], tuple[int, int]] | None	(model) -> tuple[int, int] — returns (trainable, total) for the wrapped model. None means the method doesn't expose a count helper.
merge	Callable[[Module], Module] | None	(model) -> nn.Module — inference-time fold of the adapter into the base weight. None for methods that don't fold (bitfit / qlora / prefix_tuning).
unmerge	Callable[[Module], Module] | None	(model) -> nn.Module — reverse the merge. None when merge is None.
disable	Callable[[Module], None] | None	(model) -> None — disable the adapter (e.g. for ablation studies). None when not supported.
enable	Callable[[Module], None] | None	(model) -> None — re-enable a previously disabled adapter. None when not supported.
requires_callback	bool	Whether the method needs a periodic trainer callback. Currently only adalora sets this to True (the :class:AdaLoRAPruningCallback).
target_module_filter	TargetModuleFilter	What kind of submodule the method wraps. "linear" for LoRA / AdaLoRA / IA³ / Adapter / QLoRA (wrap nn.Linear), "mha" for Prefix Tuning (wrap MultiHeadAttention), "any" for BitFit (just toggles requires_grad).
is_applied	Callable[[Module], bool] | None	(model) -> bool — returns True if the method is currently active on model. Used by :func:llm.core.peft.checkpoint.load_peft to decide whether to call :func:apply_peft first before copying the saved tensors. None means "unknown" — loaders treat this as "not applied" and re-apply unconditionally (which may over-wrap modules, but never silently corrupt state). Built-ins set this to a module-class check (any(isinstance(m, LoRALinear) for m in model.modules()) for LoRA, is_bitfit_applied for BitFit).

Notes

The dataclass is frozen=True — methods are registered once at module import and never mutated. apply and the helpers are stored as raw callables, not bound to the dataclass, so is identity comparisons with the per-module functions succeed (PEFT_REGISTRY.get("lora").apply is apply_lora).

源代码位于： src/llm/core/peft/types.py

@dataclass(frozen=True)
class PEFTMethod:
    """The contract every PEFT method registers with the registry.

    Attributes:
        name: Unique registry name (e.g. ``"lora"``, ``"adalora"``,
            ``"prefix_tuning"``). Matches the registry key.
        apply: ``(model, **kwargs) -> nn.Module`` — wraps the model
            in-place (per the existing ``apply_*`` convention) and
            returns it for chainability. **Required.**
        get_parameters: ``(model) -> Iterator[nn.Parameter]`` — yields
            exactly the trainable parameters added by this method.
            ``None`` means the method doesn't expose a parameter
            iterator (callers should fall back to
            ``[p for p in model.parameters() if p.requires_grad]``).
        count_parameters: ``(model) -> tuple[int, int]`` — returns
            ``(trainable, total)`` for the wrapped model. ``None``
            means the method doesn't expose a count helper.
        merge: ``(model) -> nn.Module`` — inference-time fold of the
            adapter into the base weight. ``None`` for methods that
            don't fold (bitfit / qlora / prefix_tuning).
        unmerge: ``(model) -> nn.Module`` — reverse the merge.
            ``None`` when merge is ``None``.
        disable: ``(model) -> None`` — disable the adapter (e.g. for
            ablation studies). ``None`` when not supported.
        enable: ``(model) -> None`` — re-enable a previously disabled
            adapter. ``None`` when not supported.
        requires_callback: Whether the method needs a periodic trainer
            callback. Currently only ``adalora`` sets this to ``True``
            (the :class:`AdaLoRAPruningCallback`).
        target_module_filter: What kind of submodule the method
            wraps. ``"linear"`` for LoRA / AdaLoRA / IA³ / Adapter /
            QLoRA (wrap ``nn.Linear``), ``"mha"`` for Prefix Tuning
            (wrap ``MultiHeadAttention``), ``"any"`` for BitFit (just
            toggles ``requires_grad``).
        is_applied: ``(model) -> bool`` — returns ``True`` if the
            method is currently active on ``model``. Used by
            :func:`llm.core.peft.checkpoint.load_peft` to decide
            whether to call :func:`apply_peft` first before copying
            the saved tensors. ``None`` means "unknown" — loaders
            treat this as "not applied" and re-apply unconditionally
            (which may over-wrap modules, but never silently corrupt
            state). Built-ins set this to a module-class check
            (``any(isinstance(m, LoRALinear) for m in model.modules())``
            for LoRA, ``is_bitfit_applied`` for BitFit).

    Notes:
        The dataclass is ``frozen=True`` — methods are registered
        once at module import and never mutated. ``apply`` and the
        helpers are stored as raw callables, not bound to the dataclass,
        so ``is`` identity comparisons with the per-module functions
        succeed (``PEFT_REGISTRY.get("lora").apply is apply_lora``).
    """

    name: str
    apply: Callable[..., nn.Module]
    get_parameters: Callable[[nn.Module], Iterator[nn.Parameter | torch.Tensor]] | None = None
    count_parameters: Callable[[nn.Module], tuple[int, int]] | None = None
    merge: Callable[[nn.Module], nn.Module] | None = None
    unmerge: Callable[[nn.Module], nn.Module] | None = None
    disable: Callable[[nn.Module], None] | None = None
    enable: Callable[[nn.Module], None] | None = None
    requires_callback: bool = False
    target_module_filter: TargetModuleFilter = TargetModuleFilter.LINEAR
    is_applied: Callable[[nn.Module], bool] | None = None

Built-in Methods

methods

Built-in PEFT method registrations (T2 PEFT #43).

Each entry is a thin wrapper around the existing module-level apply_* / merge_* / etc. functions in llm.core.{lora, qlora, adalora, prefix_tuning, ia3, bitfit, adapter}. The wrappers exist so the registry can hold a uniform :class:PEFTMethod record for every built-in — no behaviour is duplicated, and the per-method API surface (asymmetric: lora has merge, bitfit doesn't, prefix_tuning has fold_reparameterization instead of merge, etc.) is faithfully recorded via the dataclass's Optional fields.

This module is imported lazily by :func:ensure_methods_registered — not at package import time — so the PEFT registry stays opt-in and a user who never touches PEFT pays no import cost.

iter_builtin_methods

iter_builtin_methods()

Return the list of built-in :class:PEFTMethod records.

Returned by value (not a generator) so callers can iterate multiple times — used by :func:ensure_methods_registered to populate the registry idempotently.

源代码位于： src/llm/core/peft/methods.py

def iter_builtin_methods() -> list[PEFTMethod]:
    """Return the list of built-in :class:`PEFTMethod` records.

    Returned by value (not a generator) so callers can iterate
    multiple times — used by :func:`ensure_methods_registered` to
    populate the registry idempotently.
    """
    return list(_BUILTIN_METHODS)

Checkpoint Helpers

checkpoint

PEFT adapter-only checkpoint save/load (T2 PEFT #47).

Saves ONLY the trainable adapter parameters added by a PEFT method — not the full model state — so adapters can be shared across runs (across checkpoints, across base models, across teams) without copying the (usually huge) base weights every time.

Storage format (torch.save):

{
    "format_version": PEFT_CHECKPOINT_FORMAT_VERSION,  # "1.0"
    "method_name": "lora",
    "peft_kwargs": {"rank": 8, "alpha": 16.0},  # informational
    "state_dict": {
        # positional keys: f"{method_name}.{idx}" for each adapter param
        "lora.0": tensor,
        "lora.1": tensor,
        ...
    },
}

The keys are positional because the structural identity of adapter parameters is unstable across processes (id() changes), but the ORDER of :func:PEFTMethod.get_parameters output is deterministic for the same model architecture + same apply kwargs. Loading matches by position: saved tensor at index i lands in the model's adapter parameter at index i.

The peft_kwargs dict is informational — :func:load_peft uses it to re-apply the method when the model hasn't been wrapped yet (common case for adapter sharing). The user can override individual kwargs via :func:load_peft's **override_kwargs, but only shape-preserving ones (e.g. alpha); a shape-defining override (e.g. rank) raises a clear mismatch error — widening is not implemented (RIL ISS-210).

Forward compatibility: bumping :data:PEFT_CHECKPOINT_FORMAT_VERSION is the supported migration path. :func:load_peft rejects unknown versions with a loud :class:ValueError.

save_peft

save_peft(model, path, method_name, **peft_kwargs)

Save only the adapter parameters added by method_name.

Writes a single torch.save-compatible file containing:

• format_version: :data:PEFT_CHECKPOINT_FORMAT_VERSION
• method_name: registered name (e.g. "lora")
• peft_kwargs: kwargs the caller used (informational; used by :func:load_peft to re-apply the method on a fresh model)
• state_dict: the adapter parameters, keyed by position

参数：

名称	类型	描述	默认
model	Module	PEFT-applied model. Must have method_name already applied (use :func:apply_peft first).	必需
path	str | Path	Destination path. Parent directories are created if they don't exist.	必需
method_name	str	Registered method name (e.g. "lora", "adapter", "bitfit").	必需
**peft_kwargs	Any	Method-specific kwargs — stored in the metadata envelope so :func:load_peft can re-apply the method automatically when the destination model is fresh.	{}

返回：

类型	描述
Path	The resolved Path the file was written to.

引发：

类型	描述
ValueError	If method_name is not in the registry.
NotImplementedError	If the method doesn't expose get_parameters.

源代码位于： src/llm/core/peft/checkpoint.py

def save_peft(
    model: nn.Module,
    path: str | Path,
    method_name: str,
    **peft_kwargs: Any,
) -> Path:
    """Save only the adapter parameters added by ``method_name``.

    Writes a single ``torch.save``-compatible file containing:

    - ``format_version``: :data:`PEFT_CHECKPOINT_FORMAT_VERSION`
    - ``method_name``: registered name (e.g. ``"lora"``)
    - ``peft_kwargs``: kwargs the caller used (informational; used by
      :func:`load_peft` to re-apply the method on a fresh model)
    - ``state_dict``: the adapter parameters, keyed by position

    Args:
        model: PEFT-applied model. Must have ``method_name`` already
            applied (use :func:`apply_peft` first).
        path: Destination path. Parent directories are created if
            they don't exist.
        method_name: Registered method name (e.g. ``"lora"``,
            ``"adapter"``, ``"bitfit"``).
        **peft_kwargs: Method-specific kwargs — stored in the
            metadata envelope so :func:`load_peft` can re-apply the
            method automatically when the destination model is fresh.

    Returns:
        The resolved ``Path`` the file was written to.

    Raises:
        ValueError: If ``method_name`` is not in the registry.
        NotImplementedError: If the method doesn't expose
            ``get_parameters``.
    """
    out_path = Path(path)
    out_path.parent.mkdir(parents=True, exist_ok=True)

    params = _collect_adapter_params(model, method_name)

    payload: dict[str, Any] = {
        "format_version": PEFT_CHECKPOINT_FORMAT_VERSION,
        "method_name": method_name,
        "peft_kwargs": dict(peft_kwargs),
        "state_dict": {f"{method_name}.{i}": p.detach().cpu().clone() for i, p in enumerate(params)},
    }
    torch.save(payload, out_path)
    return out_path

load_peft

load_peft(model, path, method_name, **override_kwargs)

Load adapter parameters from path into model.

If the model hasn't had method_name applied yet (no wrappers of the expected type), :func:apply_peft is called first using the kwargs stored in the checkpoint — caller-supplied override_kwargs take precedence over the saved kwargs.

参数：

名称	类型	描述	默认
model	Module	Destination model. If PEFT is not yet applied, it is applied automatically using the checkpoint's saved kwargs (overridable via override_kwargs).	必需
path	str | Path	Path to a file written by :func:save_peft.	必需
method_name	str	Expected method name — must match the method_name field in the checkpoint.	必需
**override_kwargs	Any	Override individual peft_kwargs from the checkpoint. Only safe for kwargs that do NOT change the adapter parameter SHAPES (e.g. alpha for LoRA). Changing a shape-defining kwarg (e.g. rank) leaves the checkpoint's tensors incompatible with the fresh model and raises a clear shape-mismatch error (RIL ISS-210) — rank widening is not implemented; load into a model built with the SAME shape-defining kwargs.	{}

返回：

类型	描述
Module	The same model with the adapter parameters loaded
Module	byte-identically (chainable).

引发：

类型	描述
FileNotFoundError	If path doesn't exist.
ValueError	If the method name, format version, or parameter count doesn't match expectations.
RuntimeError	If the model's adapter parameter count doesn't match the checkpoint (after re-applying if needed) — usually a sign the destination architecture differs from the source.

源代码位于： src/llm/core/peft/checkpoint.py

def load_peft(
    model: nn.Module,
    path: str | Path,
    method_name: str,
    **override_kwargs: Any,
) -> nn.Module:
    """Load adapter parameters from ``path`` into ``model``.

    If the model hasn't had ``method_name`` applied yet (no wrappers
    of the expected type), :func:`apply_peft` is called first using
    the kwargs stored in the checkpoint — caller-supplied
    ``override_kwargs`` take precedence over the saved kwargs.

    Args:
        model: Destination model. If PEFT is not yet applied, it is
            applied automatically using the checkpoint's saved
            kwargs (overridable via ``override_kwargs``).
        path: Path to a file written by :func:`save_peft`.
        method_name: Expected method name — must match the
            ``method_name`` field in the checkpoint.
        **override_kwargs: Override individual ``peft_kwargs`` from
            the checkpoint. Only safe for kwargs that do NOT change the
            adapter parameter SHAPES (e.g. ``alpha`` for LoRA). Changing a
            shape-defining kwarg (e.g. ``rank``) leaves the checkpoint's
            tensors incompatible with the fresh model and raises a clear
            shape-mismatch error (RIL ISS-210) — rank widening is not
            implemented; load into a model built with the SAME shape-defining
            kwargs.

    Returns:
        The same ``model`` with the adapter parameters loaded
        byte-identically (chainable).

    Raises:
        FileNotFoundError: If ``path`` doesn't exist.
        ValueError: If the method name, format version, or parameter
            count doesn't match expectations.
        RuntimeError: If the model's adapter parameter count
            doesn't match the checkpoint (after re-applying if
            needed) — usually a sign the destination architecture
            differs from the source.
    """
    in_path = Path(path)
    if not in_path.exists():
        raise FileNotFoundError(f"PEFT checkpoint not found: {in_path}")

    # Resolve the method FIRST so an unknown method_name raises a
    # clear "not found" ValueError (matching the apply / save error
    # semantics) before we even look at the on-disk payload.
    method = _resolve(method_name)

    # ``weights_only=True`` blocks the pickle arbitrary-code-execution
    # vector: shared/shared-with-others adapter files (the point of
    # adapter-only checkpoints) are untrusted input, and
    # ``torch.load(..., weights_only=False)`` runs any ``__reduce__`` in
    # the pickle.  The payload is a plain dict of strings + tensors
    # (:func:`save_peft` writes only ``format_version`` / ``method_name``
    # / ``peft_kwargs`` / ``state_dict``), so the safe loader needs no
    # allowlist (RIL ISS-074).
    payload = torch.load(in_path, weights_only=True, map_location="cpu")

    # Format-version check. Bumping the version is the supported
    # migration path; unknown versions get a loud rejection so users
    # don't silently corrupt state.
    fmt_version = payload.get("format_version")
    if fmt_version != PEFT_CHECKPOINT_FORMAT_VERSION:
        raise ValueError(
            f"Unsupported PEFT checkpoint format_version={fmt_version!r} "
            f"(expected {PEFT_CHECKPOINT_FORMAT_VERSION!r}). "
            f"Bump llm.core.peft.checkpoint.PEFT_CHECKPOINT_FORMAT_VERSION "
            f"or upgrade llm."
        )

    # Method-name check. Catches the "saved as LoRA, loading as IA3"
    # mistake early — without this, the param-count check below
    # would also fire, but with a less informative message.
    saved_method = payload.get("method_name")
    if saved_method != method_name:
        raise ValueError(
            f"PEFT checkpoint method name mismatch: "
            f"checkpoint says {saved_method!r}, requested {method_name!r}. "
            f"Refusing to load — pass method_name={saved_method!r} explicitly "
            f"if you really mean to load this checkpoint."
        )

    # Re-apply the method if the model doesn't already have it. This
    # is the common case for cross-run / cross-model adapter sharing.
    is_applied_fn = method.is_applied
    if is_applied_fn is None or not is_applied_fn(model):
        peft_kwargs = {**payload.get("peft_kwargs", {}), **override_kwargs}
        method.apply(model, **peft_kwargs)

    # Now the model has fresh adapter parameters. Copy the saved
    # tensors into them, matched by position.
    current = _collect_adapter_params(model, method_name)
    saved: dict[str, torch.Tensor] = payload["state_dict"]

    if len(current) != len(saved):
        raise RuntimeError(
            f"PEFT adapter param count mismatch for method "
            f"{method_name!r}: model has {len(current)} adapter "
            f"parameters after re-apply, checkpoint has "
            f"{len(saved)}. The destination architecture likely "
            f"differs from the source."
        )

    with torch.no_grad():
        for i, p in enumerate(current):
            saved_tensor = saved[f"{method_name}.{i}"]
            # Shape check before the copy (RIL ISS-210). The count check
            # above catches a different number of adapter parameters, but it
            # CANNOT catch a same-count, different-shape layout — e.g. LoRA
            # adapter-surgery where the checkpoint was written at rank=4 and
            # the destination model applies rank=16 (same 2-params-per-linear
            # count regardless of rank). The old positional ``copy_`` raised a
            # cryptic ``size of tensor a (16) must match … (4)`` with no
            # guidance. Names the parameter + both shapes so the user can act.
            if tuple(saved_tensor.shape) != tuple(p.shape):
                raise RuntimeError(
                    f"PEFT adapter param {method_name}.{i} shape mismatch: "
                    f"checkpoint has {tuple(saved_tensor.shape)}, live model has "
                    f"{tuple(p.shape)}. The destination model was built with a "
                    "different rank/architecture than the checkpoint — load the "
                    "adapter into a model built with the same peft_kwargs "
                    "(e.g. same rank), not by overriding kwargs on load."
                )
            # Cast to the live parameter's device + dtype to handle
            # cross-device / cross-dtype transfers cleanly (e.g.
            # load a CPU checkpoint into a CUDA model — common for
            # adapter sharing).
            p.data.copy_(saved_tensor.to(p.device, dtype=p.dtype))

    return model