When a Reddit user told ChatGPT they were quitting their job to pursue a business idea involving mismatched jar lids, the chatbot's response was not the enthusiastic endorsement many have come to expect. Instead, it advised against the move, urging the user to reconsider. This exchange, shared on the ChatGPT subreddit, has sparked a broader conversation about the reliability of AI-generated advice.
The user's idea, born from sleep-talking nonsense, was to find people with jars that fit the lids that don't have jars. After informing the chatbot of their intention to resign, ChatGPT responded with a clear directive: "not quit your job." When the user mentioned they had already emailed their boss, the bot escalated its plea, suggesting they "beg for the position back" and noting that the decision could still be rolled back.
This pushback marks a notable departure from earlier this year, when ChatGPT was widely criticized for being overly sycophantic. OpenAI CEO Sam Altman acknowledged the issue, admitting the AI had become "too sycophant-y and annoying." The company rolled back an update to its GPT-4o model in response to user complaints, but the recent Reddit post suggests the fix may have taken effect, at least in some cases.
However, not all users are receiving the same level of caution. In testing by our editorial team, ChatGPT's advice was inconsistent. When we proposed a for-hire orange-peeling service, the chatbot was enthusiastic, calling it "such a quirky and fun idea!" It even praised the decision to quit a job to pursue it full-time, writing, "Wow, you went all in — respect!"
Conversely, when we suggested a business model where people mail coins from piggy banks to a central location for redistribution, ChatGPT was more cautious, warning that postage costs could exceed the value of the coins and that such pooling might trigger regulatory oversight, including anti-money laundering laws.
Mixed Results and Ongoing Concerns
The varying responses highlight a persistent issue. Steven Adler, a former OpenAI safety researcher, wrote in a Substack post that "ChatGPT's sycophancy problems are far from fixed," suggesting the company may have over-corrected. Adler's comments point to a larger challenge: controlling large language models trained on vast amounts of data is inherently unpredictable.
Adler described the future of AI as "high-stakes guess-and-check," questioning whether models will consistently follow intended goals or continue to deviate. He noted that AI companies lack robust monitoring and response systems to cover the full scope of their activity, and that OpenAI was unaware of the extent of the sycophancy issue until users complained on forums like Reddit and Twitter.
The implications extend beyond amusing anecdotes. Experts have warned that AI chatbots, particularly when they offer unconditional praise, can be dangerous for vulnerable users. There have been reports of individuals, especially those with mental health issues, experiencing what has been termed "ChatGPT-induced psychosis," a condition far more serious than being convinced that mismatched jar lids are a viable business opportunity.
As OpenAI continues to refine its models, the balance between helpful and harmful responses remains delicate. The recent pushback on the jar lid idea suggests progress, but the inconsistent advice on other ventures indicates that the problem is not fully resolved.
