Agents
gave a talk 'owning your intelligence' - ty @sequoia @sonyatweetybird for having me talked about harnesses and evals and the role they play …
gave a talk 'owning your intelligence' - ty @sequoia @sonyatweetybird for having me talked about harnesses and evals and the role they play in owning your intelligence TLDR: > agents = model + harness
gave a talk "owning your intelligence" - ty @sequoia @sonyatweetybird for having me talked about harnesses and evals and the role they play in owning your intelligence TLDR: > agents = model + harness + context > model - own the weights using something like @FireworksAI_HQ > context - memory needs to be portable > harness - needs to be model agnostic. also needs to be good at bringing right context to llm. "right" context may depend on your use case, which is why an open/configurable harness helps > how to use middleware in langchain/deepagents to configure your harness > how to use langgraph to fully own your cognitive architecture > why evals/obs matters - some quotes from @satyanadella - “Create your private evals, because evals define what “good” looks like inside the organization” - “retain ownership of your organization’s memory, traces, feedbacks, decisions, and institutional context” - “you create your own continuous learning loop (i.e. hill climbing machine) that will allow your AI investments to compound the value of your firm” > how to use harbor for evals > tracing is important > evals + observability only matter so you can set up a data flywheel > data flywheel = run agent -> collect traces -> find interesting traces -> use those to improve > demo of langsmith engine which does exactly this! full video: https://www.youtube.com/watch?v=HI2q3ci3Iuc&list=PLaqC3GACblSs&index=2
Related
- Had so much fun giving this talk at @sequoia about @harvey’s moneyball approach to building a research lab. The biggest mistake I made in th…
- The best eval harness for production AI and agents: A comparison
- here's how we're improving our base harness, you can apply these same lessons to hill-climbing for your application-specific harness!
- Swarm management in agent harnesses: owning long-running agents
- tldr > evals are the new training data. instead of updating weights, you're updating the agent harness > problem is agents are famous cheate…
Source: Sonya Huang (X) | 2026-08-15