Model Releases
This week, three hikers were rescued from a mountain after following AI's advice. On the same day, OpenAI announced that we have reached the era of AGI. The two stories together got me thinking.
On September 1st, three young men from California used Gemini to plan a hiking route on Mount Shasta. The AI told them to bring far less water and food than was actually needed for a hike of that di
On September 1st, three young men from California used Gemini to plan a hiking route on Mount Shasta. The AI told them to bring far less water and food than was actually needed for a hike of that difficulty.They reached the summit four hours behind schedule. They descended after dark. One of them injured his knee. They spent the night trapped in a ravine until they were rescued the next day. Google says it cannot reproduce the erroneous advice Gemini gave. Perhaps the prompts were poorly written, or perhaps the model was overly optimistic. But that isn't the point.The point is that three people treated an AI's response as if it were advice from an expert mountain guide. And no one told them not to.Two days later, OpenAI launched GPT-6 Astra and announced that we have entered the era of AGI. The benchmarks are impressive. Independent evaluators are somewhat more cautious,ranking Anthropic’s Fable 5.1 ahead in the overall index,but the leap in capability is real. And here is what I can't get out of my head. As models become more capable, people use them in more critical, high-stakes situations. Yet, knowing when to trust them,and when not to,doesn't improve simply because the model improves. If anything, it gets worse, because the answers sound more convincing. I’ve been working with companies on AI adoption for some time, and the most common problem isn't that people are too distrustful; it’s that they don't know when to stop trusting. Do you think more powerful models make that problem easier or harder to manage? submitted by /u/Dapper-Tale-4021 [link] [comments]
Related
Source: r/ChatGPT | 2026-09-07