Beep boop - this is a robot. A new show has been posted to TWiT…
What are your thoughts about today’s show? We’d love to hear from you!
The conversation about Grok’s weekend meltdown (IM Episode 827 starting at 1:19:05 in the transcript) ignored several details that were well-known at the time of the recording. They can be found – could have been found – on a variety of social media sites. Interestingly, they are not available on any established news sites. I asked a friend who’s a journalist to do a writeup on the story; he said one is in the works.
AFAICT, the best starting-point for the decisive context on this story is Grok itself. Here’s the question I asked. “The weekend” refers to the weekend of July 5-6, 2025:
when exactly was the prompt for grok changed over the weekend? Does it look fishy that the prompt was changed over a holiday? Was there any exigent reason for the prompt to be amended when many employees were out of the office? Why do you think this was done, and done at that time. Use ludicrous mode.
Grok’s prompt was altered at 4pm PDT on Sunday, July 6. That modification to the prompt was undone with a second GitHub commit on July 8.
Why would a prompt be changed on the weekend? Why would any American company deliberately change the public behavior of their software over the 4th of July holiday? Why was an X.com employee fired this week? Why was CEO Linda Yaccarino asked to resign? What is the relationship between these two job terminations and the holiday weekend code modification?
As Gizmodo noted, Elon did tweet about upgrades to Grok on July 4. What was he talking about? Almost certainly, he was talking about the announcement of Grok 4, which is now publicly available. Grok 4 scored a 44.4% on Humanity’s Last Exam: a significant jump over any AI’s performance. Grok 4 Heavy hit a score just north of 50% on the exam. This is a significant boost from previous benchmarks; I’ll defer to Matt Berrman’s video Grok 4 is really smart… Like REALLY SMART for the details, but it’s safe to say this is a mightily impressive upgrade. Side note: Matthew Berrman would be a great guest for Intelligent Machines.
After viewing this segment and various sources online, there are several obvious questions:
- Who made the 4pm Sunday patch to the Grok prompt? Why?
- Why would anyone rationally think this vandalism had anything to do with Elon’s July 4 announcement?
- Did the Grok 4 upgrades make it onto the panelists’ radar? AFAICT, this week’s upgrades weren’t mentioned on this episode at all. This upgrade appears to be one of the Top 10 AI stories of the year; it deserves at least a passing mention on the show.It certainly deserves more discussion than speculation about the holiday weekend patch to Grok’s prompt.
- Why was the July 6 patch never even mentioned on the show? Did the panelists really know nothing about this before the show started?
Someone made a modification to Grok’s system prompt at 4pm PDT on Sunday. The simplest explanation is that this was an unauthorized modification of Grok. The vandalism certainly detracted from x.ai’s announcements this week: many news sources completely ignored the revolutionary advances and instead talked about the e-vandalism.
Controls should have been in place such that no individual could make a change like this without managerial/executive sign-off. If there was a failure of those controls, I could easily imagine that a CEO could get fired (i.e., asked to resign) for that failure.
Does anyone have an alternative explanation for this week’s X.ai events?
It sounds like Grok is still erring on the side of wild speculation and conspiracy theories.
What use is the “smartest AI in the world” if you can’t trust it?
1 Like
Um, what? Nobody should trust ANY Ai. If you ask ChatGPT if you should trust it, the AI strongly recommends you not do that – verify everything it says.
Any AI will tell you the same. Grok notes: “You shouldn’t blindly trust me or any AI.” Hear, hear!
What is your question a response to? A small group of people attempted to sabotage the rollout of Grok in early July – see above. My question for you: why did this podcast ignore those human activities? What’s the point of asking that “trust” question specifically about Grok and not any other AI?
Because that happens with every AI release, and normal software as well, to a lesser extent.
This has been a thing for decades and is therefore not really newsworthy these days. Probably the most notable story was Microsoft’s Tay that was turned from an innocent “girl” to a frothing at the mouth anti-semetic, National Socialist within a few hours of going online and had to be taken back offline in a hurry.
It is seen as a game in many circles to get the AI models to look unhinged, but the stories got boring years ago and are hardly worth mentioning these days, unless a new AI hasn’t been protected enough to automatically cope with such “tests”.
1 Like
rest in peace!!! i think about Tay at least twice a year because i found it really comedic
interestingly enough, i don’t really find grok funny or interesting (i’m not a big AI user. sometimes i talk to chatGPT when i’m curious about something). maybe you pinned the nail on the head: it got old. been there, done that?
2 Likes
Except that it was deemed newsworthy for commentary about the Grok release in the week starting July 7, 2025! Between the bizarre prompt interrogations of the “AI agitators” and the apparent vandalism of Grok’s system prompt on July 6, there was a massive amount of Grok-dissing that first full week of July. If you don’t ask bizarre questions, you don’t get “mecha-[national socialist]” responses. It’s almost as if there is some bounty in providing strange questions and getting a bizarre bounceback.
Matthew Berman does a great job discussing all of the AIs on his YouTube channel. I don’t see any obvious bias for/against any of the AIs. That is the kind of balanced coverage I fondly wish we were getting from the IM podcast.
I’m going to start doing some serious AI-assisted coding starting next week. I’ll probably split my work between ChatGPT and Grok in the beginning. I’ll do my best to avoid provocative questions from either AI – just stick with coding prompts. 
There were similar stories over the last couple of years about ChatGPT and Claude, the only reason for maybe covering that is that xAI hasn’t learnt the lessons that other AI companies have…
The idea that ChatGPT has “learnt the lessons” doesn’t quite add up. Three weeks ago, we got the story ChatGPT encouraged Adam Raine’s suicidal thoughts. His family’s lawyer says OpenAI knew it was broken . The suicide happened in the fall of 2024. There’s a brand new article noting that ChatGPT appears to have affirmed the paranoid fantasies of a former tech executive leading up to when police say he killed his elderly mother and then himself in their home, according to videos the 56-year-old man posted in recent months on YouTube. There’s a new published science paper questioning the methodology that three AIs do when grappling with suicide conversations: A RAND Corporation study published in Psychiatric Services has found ChatGPT, Gemini, and Claude inconsistent in handling suicide-related questions. While the highest-risk prompts were generally refused, inconsistencies in less direct queries raise safety concerns.
Will Intelligent Machines cover these stories? Are there really any studies anywhere showing that ChatGPT and Claude have these issues sorted out better than other AIs?