Instead of predicting text tokens, @si_pbc learns to use a computer from raw screen data, predicting the next mouse movement, click, and key…
Instead of predicting text tokens, @si_pbc learns to use a computer from raw screen data, predicting the next mouse movement, click, and keystroke from the pixels in front of it. This is the @Tesla FS