OpenCode with Qwen3.8-27B for Small Games or Browsing the Web With 16GB VRAM

In the past, I have use llama.cpp, but I read that the exl3 quantization format should give better precision, so I have tried exllamav3/tabbyAPI. It was able to write the shown simple HTML game without interaction after asking some questions. The following was tested on a laptop with a NVIDIA RTX A5000 laptop (16 GB) GPU. With the 3 bpw model and 6 bit/5 bit KV cache, the maximum context length is around 110k tokens with MTP. This gives around 55 tokens/s decode speed for code and around 10 tokens/s for content where MTP doesn't help (e.g. complicated calculations). Without MTP, one could try the 3.5 or 4 bpw model or a longer context length. Install tabbyAPI/exllamav3 Install the latest Nvidia drivers Install Git (e.g. sudo apt install git or on Windows with winget install -e --id Git.Git) Install the uv Python package manager: https://docs.astral.sh/uv/getting-started/installation/ (e.g. curl -LsSf https://astral.sh/uv/install.sh | sh or winget install --id=astral-sh.uv -e) Make somewhere a folder and install tabbyAPI: bash git clone https://github.com/theroyallab/tabbyAPI cd tabbyAPI uv venv --python 3.13 .venv uv pip install -e ".[cu13]" Test if CUDA works (on Linux, use .venv/bin/python) bash .venv/Scripts/python -c "import torch; print(torch.version, torch.cuda.is_available(), torch.cuda.get_device_name(0))" Create somewhere where you have enough space a "models" folder, download the model turboderp/Qwen3.8-27B-exl3: bash mkdir models uvx hf download turboderp/Qwen3.8-27B-exl3 --revision SC_3.00bpw_H4 --local-dir models/qwen3.8-27b Replace the chat_template.jinja with the latest version from froggeric/Qwen-Fixed-Chat-Templates Go back to the clone tabbyAPI folder and create a config.yml file like this (see the config_sample.yml file as example): yaml network: disable_auth: true model: model_dir: e:/models # path to the models folder model_name: qwen3.8-27b # download folder name cache_mode: 6,5 # K and V cache quantization, number of bits from 2-8 cache_size: 109824 # must be divisible by 256, so use e.g. .venv/Scripts/python -c 'print(110000//256*256)' to get the next lower max_batch_size: 1 # allow only 1 parallel request to save VRAM tool_format: qwen3_coder vision: true draft_model: # can be removed to save VRAM draft_mode: mtp draft_cache_mode: Q8 # can be 'FP16', 'Q8', 'Q6', 'Q4' draft_num_tokens: 5 # usuallly a value of 2-6 gives best results memory: sysmem_recurrent_cache: 8192 # Max size of recurrent cache in system memory, in MB (default: 4096), lower it to save normal memory sysmem_kv_cache: 8192 # Size of system memory second-tier K/V cache, in MB (default: 0), remove it to save system memory Start tabbyAPI: .venv/Scripts/python main.py To measure the performance, create the Python script speed.py and run it with .venv/Scripts/python speed.py: python import json import time import requests MODEL = "qwen3.8-27b" API_URL = "http://127.0.0.1:5000" PROMPT = """Write a complete Python implementation of a production-quality LRU cache. Requirements: Use type hints throughout. Include detailed docstrings. Support: get(key) put(key, value) remove(key) clear() len() Use a doubly linked list and hash map. Include custom exceptions. Include a comprehensive unittest test suite with at least 20 test cases. Follow PEP8 conventions. Return only Python code. """ payload = { "model": MODEL, "messages": [{"role": "user", "content": PROMPT}], "max_tokens": 10000, "stream": True, "chat_template_kwargs": {"enable_thinking": False} } start_time = time.perf_counter() first_token_time = None stream_end_time = None full_response_content = "" with requests.post(API_URL + "/v1/chat/completions", json=payload, timeout=120, stream=True) as response: response.raise_for_status() print("Response:") for line in response.iter_lines(): # Iterate over Server-Sent Events (SSE) if line.startswith(b"data:"): # Strip the "data: " prefix data = line[6:] # Stop if we hit the stream termination message if data.strip() == b"[DONE]": break try: chunk = json.loads(data) if 'choices' in chunk and chunk['choices'] and (chunk['choices'][0]['delta'].get('content') or chunk['choices'][0]['delta'].get('reasoning')): if first_token_time is None: # First token received first_token_time = time.perf_counter() if chunk['choices'][0]['delta'].get('content'): # Get content and count tokens token_text = chunk['choices'][0]['delta']['content'] else: token_text = chunk['choices'][0]['delta']['reasoning'] full_response_content += token_text print(token_text, end="", flush=True) except json.JSONDecodeError: pass stream_end_time = time.perf_counter() print("n" + "-"*20) Calculate and print metrics ttft = first_token_time - start_time stream_duration = stream_end_time - first_token_time total_output_tokens = requests.post(API_URL + "/v1/token/encode", json={"add_bos_token": False, "text": full_response_content}).json()["length"] if stream_duration > 0: tokens_per_second = total_output_tokens / stream_duration else: tokens_per_second = float('inf') print(f"Time to first token (TTFT): {ttft:.2f}s") print(f"Completion tokens: {total_output_tokens}") print(f"Stream duration (first to last token): {stream_duration:.2f}s") print(f"Tokens per second (T/s): {tokens_per_second:.2f}") I got 56.3 tokens/s. Install OpenCode OpenCode works usually better on Linux, so I install it in WSL when working with Windows, but it can also be used directly as a Windows application. For OpenCode, I recommended to install Node.js first (e.g. apt install npm or winget install -e --id OpenJS.NodeJS on Windows). Because we don't have so much context length, I recommend to install a better compactation plugin than the integrated one, e.g. magic-compact I use this OpenCode config (~/.config/opencode/opencode.jsonc) json { "$schema": "https://opencode.ai/config.json", "plugin": [ "opencode-anthropic-auth@latest", "opencode-copilot-auth@latest", "magic-compact" ], "share": "disabled", "provider": { "local": { "npm": "@ai-sdk/openai-compatible", "name": "local (OpenAI Compatible)", "options": { "baseURL": "http://127.0.0.1:5000/v1", "apiKey": "1234" }, "models": { "qwen3.8-27b": { "name": "Qwen3.8 27B", "interleaved": { "field": "reasoning_content" }, "limit": { "context": 109824, "output": 32000 }, "temperature": true, "reasoning": true, "attachment": false, "tool_call": true, "modalities": { "input": [ "text", "image" ], "output": [ "text" ] }, "cost": { "input": 0, "output": 0, "cache_read": 0, "cache_write": 0 }, "variants": { "xhigh": { "reasoningEffort": "xhigh" }, "medium": { "reasoningEffort": "medium" }, "low": { "reasoningEffort": "low" } } } } } }, "agent": { "plan": { "model": "local/qwen3.8-27b" } }, "model": "local/qwen3.8-27b", "small_model": "local/qwen3.8-27b", "mcp": { "playwright": { "type": "local", "command": [ "npx", "@playwright/mcp@latest", "--caps", "vision,pdf,devtools", "--browser=firefox" ], "enabled": true } } } I would recommend to use the reasoning effort (Ctrl-t) "medium" because "xhigh" could produce to much output tokens. For Playwright, we have to install a browser first: npx @playwright/mcp install-browser --with-deps firefox Now the following should work: bash opencode --prompt "Can you check for me on www.meteoschweiz.ch the weather for Zurich?" To create the small HTML game from above, I have entered in plan mode (press Tab to change mode) the following: "I want to build a simple HTML game where you can drive a car with the keyboard arrow keys (similar like old versions of Mario Kart, but just one car driving without opponents is enough)." After some time, it has asked me some question. Then, I switched to the "Build" mode and started it with "Start the implementation". Without any other interaction, it finished the the small game. submitted by /u/Due-Project-7507 [link] [comments]

Source: r/LocalLLaMA | 2026-08-26