vllm.v1.hisparse.types

Classes:

• SparseKVOffloadCommand –

  Opaque scheduler-to-worker command for one model step.

• SparseKVPageTransfer –

  Copy one logical KV page between cache-manager and worker-owned tiers.

• SparseKVResidencyUpdate –

  GPU block ids of some of a request's resident pages, per resident group.

• SparseKVRowMirror –

  Mirror one contiguous resident-row span into the host cache.

SparseKVOffloadCommand dataclass

Opaque scheduler-to-worker command for one model step.

Source code in vllm/v1/hisparse/types.py

@dataclass
class SparseKVOffloadCommand:
    """Opaque scheduler-to-worker command for one model step."""

    page_transfers: list[SparseKVPageTransfer]

SparseKVPageTransfer dataclass

Copy one logical KV page between cache-manager and worker-owned tiers.

Source code in vllm/v1/hisparse/types.py

@dataclass(frozen=True)
class SparseKVPageTransfer:
    """Copy one logical KV page between cache-manager and worker-owned tiers."""

    transfer_id: int
    host_block_id: int
    resident_block_ids: tuple[int, ...]
    after_forward: bool
    restore: bool = False

SparseKVResidencyUpdate dataclass

GPU block ids of some of a request's resident pages, per resident group. A null block id means the page is read from the host.

Source code in vllm/v1/hisparse/types.py

@dataclass(frozen=True)
class SparseKVResidencyUpdate:
    """GPU block ids of some of a request's resident pages, per resident group.
    A null block id means the page is read from the host."""

    pages: list[int]
    block_ids: tuple[list[int], ...]

SparseKVRowMirror dataclass

Mirror one contiguous resident-row span into the host cache.

Source code in vllm/v1/hisparse/types.py

@dataclass(frozen=True)
class SparseKVRowMirror:
    """Mirror one contiguous resident-row span into the host cache."""

    source_starts: tuple[int, ...]
    destination_start: int
    num_rows: int