AI has a secondary intent problem. I was negotiating a rent renewal (I live in NYC and they raised rent 10% because I guess they were bored)…

AI has a secondary intent problem. I was negotiating a rent renewal (I live in NYC and they raised rent 10% because I guess they were bored). The property manager said she would "do everything she could" to get me a better rate. Claude celebrated and assumed a meaningful discount was coming. I had to tell it she was saying that to save face and was probably not going to move much (reader: I was right). Primary intent = the words. Secondary intent = the subtext, what the words actually mean in context. I honestly think it's one of the reasons that AI is a mediocre TV and movie writer. We don't want to train AI that all humans are manipulative. At worst, we would have models that assume the worst and distrust everything; at best, they would become less helpful. But we do need it to understand that polite phrasing, face-saving language, and soft no's are real categories of human speech and might be deceptive. And missing language, emotion, and context cues has real-world consequences. Big impact zone: literally everywhere AI interacts with people. Customer support, sales, BD, any system helping leaders make decisions or calls about human behavior.

Source: Allie K. Miller (X) | 2026-05-12