Model Releases
b10242
CUDA: Add backend sampler for penalties sampler (#25262) sampling: enhance penalty handling in common_sampler_init Set default value for penalty_last_n based on model context if not specified. Ensure
CUDA: Add backend sampler for penalties sampler (#25262) sampling: enhance penalty handling in common_sampler_init Set default value for penalty_last_n based on model context if not specified. Ensure penalty_last_n and n_prev are non-negative. Update llama_sampler_penalties structure to inherit from llama_sampler_backend and add backend input handling for penalties. Implement backend initialization and application logic for penalties, including frequency and presence adjustments. tests: add backend penalties sampling tests and utility functions Introduced accept_prompt and unique_prompt_tokens functions to handle prompt acceptance and token uniqueness. Implemented compare_penalties_logits to compare logits from backend and CPU samplers with penalties. Added test_backend_penalties_sampling to validate backend penalties with various configurations. Enhanced the test suite for better coverage of penalty handling in sampling. sampling: add support for top-k penalties in backend sampling sampling: add fix to ensure stable numerical results. Preserve masked logits as -Inf and no longer generate NaN. sampling: enhance penalty comparison tests with masking penalties logic add comments on padding sampling: add comments on modifications add the unit test to cover masked-out token as -INF validate repeat penalty to ensure it is finite and greater than 0; add tests for invalid values refactor: test functions to share logic and be less verbose add test to cover case where previously penalized token is not part of candidates remove comments remove redundant penalty_last_n initialization and validation in common_sampler_init add support for penalties in sampler chain with configurable positions add validation for penalty parameters and enhance tests for non-finite values add context parameter to common_sampler_init and set default for penalty_last_n add llama_n_ctx parameter to common_sampler_init for improved sampler initialization replace penalty_last_n x n_candidates comparison matrix with a vocabulary-sized count tensor add tests for backend penalties sampling without filler entries , token_count.size() == n_active == n_max == 64 add test for backend penalties sampling after top-p with large history window remove as unused add is_disabled method, tensor logits reshape, add rest review suggestions clarify comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.2) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (HIP) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI
Source: llama.cpp Releases | 2026-08-03