Be Wary of Sycophantic Assistants -- The Next Platform Commoditization Breaker
All chat platforms and LLM tools have become commodified tasked with creating as much lock-in as possible for users. In general, most GenAI power users prefrer to flip their models as some are margianly better at some tasks than another – but on average, the capabilities are reaching parody.
As advertising and MCPs are designed to keep a user hooked in, using a single model that will benefit its parent company financially, more and more needs arise to create this lock-in, in order to build its user base and collect more training model data–thus, letting the training and experiences expand.
On the downside, a good number of users will be utilizing these tools for not only research, productivity, and automation tasks, but create the sycophancy illusion that these models are “alive” and show “emotion” to keep the non-tech enthused user also hooked on what a model has to offer. In this category, we have Mico – an assistant built into Microsoft’s CoPilot that’s designed to be its 2025 equivalent of Clippy that we all know and love from the 90s.
Mico is part of a Microsoft program dedicated to the idea that “technology should work in service of people,” Microsoft wrote. The company insists this effort is “not [about] chasing engagement or optimizing for screen time. We’re building AI that gets you back to your life. That deepens human connection.” - Ars Technica
LLMs are tools, and I worry about stories that some consumers are mis-utilizing these parasocial relationships giving them emotion, reason, and the illusion of relationships – whether that be true friendships or sometimes more.
[Microsoft] says it’s also working to evolve Copilot’s personality and tone, with the introduction of a new mode called “Real Talk.” This will allow the AI to mirror the user’s conversational style but won’t be as sycophantic as other AI assistants have been. Instead, Microsoft says that it will feel like something that’s “grounded in its own perspective,” and will push back and challenge your ideas, which could encourage you to see things from a different point of view. - Ars Technica
This attempt to break the parasocial bounds is something that must be welcomed, especially with a business forward facing product such as CoPilot. Microsoft’s goal should always be increasing productivity, a task that they’ve excelled at (no pun intended) over the past many decades with its Office and 365 incarnations.
With differentiation, it is important that each of these models and platforms do try to target different audiences with their needs. As Windows has its aim of becoming more agentic in nature moving forward, it seems more likely that Mico will be more integrated on that consumer level, between the OS, consumer portions of Microsoft 365, Xbox and other ecosystem products.
Even with the best of intentions, The Ars Technica article ends with the following salient point, one that I’ve been raising issue with here:
But adding a friendly, Pixar-like face to Copilot’s voice mode may make it much easier to be sucked into feeling like Copilot isn’t just a neural network but a real, caring personality—one you might even start thinking of the same way you’d think of the real loved ones in your life.
In the end, our attention is the commodity. Just as it was with social media and the app ecosystems, particular use cases of GenAI seem to be attempting to replace that. We have to be careful when interacting with these tools and try not to take our eye off the ball when it comes to creating productivity enhancements, not false relationships.