Search: “ui”

arXiv:2609.32036v1 Announce Type: new Abstract: We introduce ScreenHaystack, a dynamic needle-in-a-haystack benchmark for evaluating spatial reliability in GUI grounding. Instead of testing each targe

Stop Making TUIs Thomas Ptacek advocates for building real native user interfaces for even the smallest of personal tools, because coding agents have reduced the cost of getting a usable-enough GUI up

arXiv:2607.24112v1 Announce Type: new Abstract: We introduce State Transition Pretraining (STP) as a new scaling axis for GUI agents. During the STP stage, we continually pretrain a unified multimodal

arXiv:2603.26266v3 Announce Type: replace Abstract: Large vision-language models have endowed GUI agents with strong general capabilities for interface understanding and interaction. However, due to i

arXiv:2605.12549v1 Announce Type: new Abstract: Existing training-free approaches for GUI grounding often rely on multiple inference runs, such as iterative cropping or candidate aggregation, to ident

arXiv:2604.27974v1 Announce Type: new Abstract: Despite the rapid progress of large vision-language models (LVLMs), fine-grained, state-conditioned GUI interaction remains challenging. Current evaluat

arXiv:2604.21268v1 Announce Type: cross Abstract: Graphical User Interface (GUI) grounding requires mapping natural language instructions to precise pixel coordinates. However, due to visually homogen

arXiv:2604.20279v1 Announce Type: cross Abstract: Mobile GUI agents can automate smartphone tasks by interacting directly with app interfaces, but how they should communicate with users during executi

arXiv:2506.09373v3 Announce Type: replace-cross Abstract: The advent of autonomous agents is transforming interactions with Graphical User Interfaces (GUIs) by employing natural language as a powerful

A community-built framework that layers on top of Ollama to transform any spare PC into a fully private, locally-run AI assistant inspired by Iron Man's JARVIS, featuring voice interaction, 29 built-i

arXiv:2602.07153v2 Announce Type: replace Abstract: End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data, yet collecting human demonstrations is e

arXiv:2604.09581v1 Announce Type: new Abstract: Evaluating web usability typically requires time-consuming user studies and expert reviews, which often limits iteration speed during product developmen

arXiv:2512.19396v3 Announce Type: replace Abstract: Contemporary GUI agents, while increasingly capable due to advances in Large Vision-Language Models (VLMs), often operate with a critical limitation

arXiv:2604.06240v1 Announce Type: cross Abstract: Verifying the success of computer use agent (CUA) trajectories is a critical challenge: without reliable verification, neither evaluation nor training

arXiv:2604.06182v1 Announce Type: cross Abstract: Existing online benchmarks for mobile GUI agents remain largely app-centric and task-homogeneous, failing to reflect the diversity and instability of

Agents need a better interface. In my own agent orchestrator, I often render more actionable, valuable outputs as small interfaces. @openuidev makes this easy for any AI app. It is an open-source, mod

arXiv:2610.19100v1 Announce Type: cross Abstract: Constructing robust GUI agents relies on multimodal visual grounding: the ability to precisely map natural language intents to actionable on-screen co

arXiv:2610.17792v1 Announce Type: new Abstract: GUI grounding enables web agents to translate natural-language instructions into actionable interface locations. However, dense layouts and small, visua

I started Flama in 2018 to take a trained model to a production API with as little ceremony as possible. With 2.0 LLMs became first-class alongside classic models, and together with Miguel Durán-Olive

Hi all, a bunch of performance improvements have been landed in audio.cpp. The biggest highlight is Higgs Audio TTS, which now runs with around 6 GB VRAM, a 48% reduction in peak memory usage compared

arXiv:2610.07972v1 Announce Type: new Abstract: Mobile GUI agents increasingly operate on interfaces influenced by users' histories and preferences, but their reliability across different users remain

arXiv:2610.05861v1 Announce Type: new Abstract: Graphical User Interface (GUI) agents have emerged as a promising paradigm for automating complex digital workflows across diverse applications. However

Hey r/LocalLLaMA, Like many anime fans, I've always been frustrated by traditional MT engines (like Google Translate or base DeepL) when dealing with raw Japanese anime: - They completely butcher Japa

Whenever I feed codebases into local models (Qwen, DeepSeek R1) or API models, the biggest annoyance is prompt pollution: - Lockfiles (`Cargo.lock`, `package-lock.json`) burning 30,000+ tokens for zer

Learn to build a custom harness, folks. Mods in Claude Code, DeepSeek Harness, and now Pi Durable. What do these all have in common? Increased extensibility. Builders need coding agents and harnesses

arXiv:2609.39547v1 Announce Type: new Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GUI

arXiv:2609.36923v1 Announce Type: new Abstract: Existing reactive Graphical User Interface (GUI) agents often fail in long-horizon, dynamic scenarios, where unexpected disturbances trigger attention-d

arXiv:2602.09007v3 Announce Type: replace-cross Abstract: An image generation model used as a graphical user interface (GUI) environment must produce not only a plausible screen but also the correct r

arXiv:2609.34321v1 Announce Type: new Abstract: GUI agents are deployed with frozen weights and discard everything they experience on the job. Existing ways to update an agent's weights assume somethi

full disclosure, i am the developer of locally uncensored :) i use ollama myself and honestly ollama was never really the problem. the problem was everything around it. one app for local chat, another

Hi everyone, Inspired by tools like Chessvision.ai, I wanted to take a different architectural approach and build a browser extension (ChessInsights AI) that performs chessboard detection and piece re

arXiv:2609.12491v1 Announce Type: new Abstract: Skeleton-based action recognition can support automated tracking of physiotherapy exercises, particularly in remote rehabilitation settings where contin

Repo here- https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder Alrighty, back again with an update to my (manual, no LLM connected) Fantastic MiniMax H3 Prompt Builder. Yeah it's

6 months ago, I started building an internal customer insights tool at Together AI. It pulls in our sales calls, slack channels, & support tickets and turns them into searchable customer insights. Tod

— Available to Google AI Pro and Ultra subscribers across the @GeminiApp, AI Mode in Google Search, and Google Sheets — Build in the Gemini API via @GoogleAIStudio and @AndroidStudio, explore agent-fi

arXiv:2609.00048v1 Announce Type: cross Abstract: GUI world models are increasingly evaluated as one-step next-screen predictors, yet their intended use is often as multi-step environments for GUI age

A research agent is only as good as the search layer behind it! I built a Deep Research Agent using LangChain Deep Agents with a search-first architecture designed around grounded, source-backed resea

arXiv:2509.23263v3 Announce Type: replace Abstract: Long-horizon GUI automation remains challenging due to error accumulation over extended interaction sequences. Process Reward Models (PRMs) provide

Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built) under my supervision - not vibe-coded. Its free, no telemetry, and open-source (AGPL). Works on Windows,

arXiv:2603.00188v2 Announce Type: replace-cross Abstract: Training-free KV cache compression is essential for deploying vision-language GUI agents under memory and latency constraints, yet existing me

arXiv:2608.22847v1 Announce Type: new Abstract: Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottle

Now, some of you might not believe me from the get go, and it's completely fine, But for those who are willing to give Tap a chance, a whole new world of voice dictation is going to unlock for you. Ta

Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo

arXiv:2607.18046v2 Announce Type: replace Abstract: Graphical User Interface (GUI) agents powered by vision-language models hold promise for automating real-world mobile tasks. However, progress is li

The article describes an end‑to‑end DIY project that builds an AI text detector from scratch, including dataset construction, model training, local deployment, and a reinforcement learning verifier (R

🧩 DeepSeek Harness v0.1 is now available in Developer Preview! 🔹 We’re opening it up to developers building agent harnesses worldwide and open-sourcing the codebase in MIT license. 🔹 Powered by the Co