You can use any LLM just like JEV

You can simply run any GGUF with llama.cpp with n_predict=1 and n_probs=10, disable reasoning, and prompt it such as "If the following email is spam, respond with 1, if not spam, respond with 0. Do not respond with anything other than 1 or 0. Email: ...." And that is it! It returns confidence percentages such as: 1 = 94.9% 0 = 5.08% Example: llama-server -m "C:UsersMyUserNamellama.cppmodelsSpark-X2.5-4B-Q4_K_M.gguf" -c 4096 -ngl all -fit off -fa on -b 2048 -ub 512 -np 1 --cache-ram 0 --reasoning off --no-reasoning-preserve --perf Then: curl.exe -s -X POST http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d "{messages":[{role":system",content":Classify spam. Reply only 1=spam or 0=not spam."},{role":user",content":CONGRATULATIONS!!! You have won $5,000,000! Click here immediately to claim your prize!"}],max_tokens":1,logprobs":true,top_logprobs":10,temperature":1.0,top_p":1.0}" Result: {"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"1"},"logprobs":{"content":[{"id":30,"token":"1","bytes":[49],"logprob":-0.00456317700445652,"top_logprobs":[{"id":30,"token":"1","bytes":[49],"logprob":-0.00456317700445652},{"id":29,"token":"0","bytes":[48],"logprob":-5.395024299621582},{"id":1033,"token":"**","bytes":[42,42],"logprob":-12.013711929321289},{"id":1046,"token":"The","bytes":[84,104,101],"logprob":-13.005236625671387},{"id":198,"token":"n","bytes":[10],"logprob":-13.100714683532715},{"id":54,"token":"I","bytes":[73],"logprob":-14.624603271484375},{"id":3640,"token":"This","bytes":[84,104,105,115],"logprob":-14.800630569458008},{"id":130977,"token":"<tool_call>","bytes":[60,116,111,111,108,95,99,97,108,108,62],"logprob":-14.971238136291504},{"id":6908,"token":"Class","bytes":[67,108,97,115,115],"logprob":-15.373867988586426},{"id":3923,"token":"class","bytes":[99,108,97,115,115],"logprob":-15.442902565002441}]}]}}],"created":1789950066,"model":"C: Users MyUserName llama.cpp models Spark-X2.5-4B-Q4_K_M.gguf","system_fingerprint":"b11026-b49650adb","object":"chat.completion","usage":{"completion_tokens":1,"prompt_tokens":64,"total_tokens":65,"prompt_tokens_details":{"cached_tokens":59}},"id":"chatcmpl-x2WrCObzFNYjKVkwDmcL8FLquwfZ0NEa","timings":{"cache_n":59,"prompt_n":5,"prompt_ms":634.566,"prompt_per_token_ms":126.9132,"prompt_per_second":7.879401039450585,"predicted_n":1,"predicted_ms":0.001,"predicted_per_token_ms":0.0,"predicted_per_second":0.0}} Convert to probability: probability = e^(logprob) 1 = e^(-0.00456317700445652) = ~99.5% 0 = e^(-5.395024299621582) = ~0.5% Speed: On my 170gb/s bandwidth 4gb vram GPU, I got 634ms! On a H200, I would probably get 30-75ms. Multiple Questions at Once: In theory you can ask multiple questions at once. You just gotta be clever with the math. For example: Q1: Is it spam? Q2: Is it phishing? Q3: Is it urgent? Q4: Is it malicious? A = 0000, B = 0001, C = 0010, D = 0011, .... O = 1110, P = 1111 where each bit corresponds to a yes no answer. Let's say LLM answers with: A 0.2% B 0.1% C 0.2% D 0.2% E 0.5% F 0.5% G 0.5% H 1.0% I 1.0% J 1.5% K 2.0% L 3.0% M 5.0% N 10.0% O 20.0% P 54.3% These add up to 100%. To learn possibility of "Is it spam?", just sum tokens where first bit was 1 such as: I + J + K + L + M + N + O + P = %96.8 Repeating the same logic, you could get: Spam: 96.8% Phishing: 91.8% Urgent: 81.2% Malicious: 70.6% submitted by /u/DivideHorror3217 [link] [comments]

Related

• Qwen3.8 27B reasoning effort low/medium/xhigh comparison
• Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp
• Qwen 3.6 27B flags/settings in llama.cpp
• Update your chat template for dsv4 if you're using llama.cpp

Source: r/LocalLLaMA | 2026-09-21