Personal experience, for what I’ve used them for so far, they have been very hit and miss. About 70% usable answers, but so many errors that I call a lot of the answers into question.
I do use them some of the time, I’m not a skeptic, but I do feel they are so error prone that they shouldn’t be sold as finished products.
And, yes, they return links, but if I have to read the summary, then check the links to ensure the summary is correct, I can save time by just reading the links… For the queries where I know roughly what the answer should be, I can use the summaries most of the time, but it is the wildly and obviously wrong answers that then have me going back and double checking the answers I thought were right.
In other words, anecdotes. You didn’t answer my question: how do you reconcile your personal experience against the experience of our esteemed MBW panelists?
Maybe you’re asking the wrong questions. Or maybe you’re asking them in the wrong way. Did you ask anyone experienced to see if this lookup operation would be a good thing to use with AIs? Did you ask the AIs themselves?
The Wolfram Notebook Assistant punctures your conjecture. It’s definitely a finished product. Based on discussions here, Leo has signed up for a year using Mathematica and the Wolfram Notebook Assistant. To risk repeating myself, why do you think our panelists would be recommending AIs if they themselved didn’t think AIs were productive tools for them – and for the audience?
Fine. For this discussion I’ll stipulate that you did your due diligence, found an appropriate AI, and learned how to prompt your requests in a way that [should] produce results. What should we learn from your anecdote? Why would a skeptic extrapolate from your one example to ALL applications of AIs? All [a skeptic] needs to show that AIs are revolutionary is ONE example. Alex is deeply immersed interacting with 2 different AIs; he definitely thinks they’re revolutionary.
I went through Wolfram U’s one-week course What Is ChatGPT Doing … and Why Does It Work? One of the more interesting things in the course was the things that LLMs do poorly: computationally irreducible problems like cellular automatons (something Stephen has studied extensively), factoring large composite numbers, etc. There’s a lot of ignorance about LLMs; Wolfram Research is doing an excellent job educating anyone curious about this world-changing tech. Stephen also has this available as a book – with a free version on his website. I’m glad he and his company are on the case; I highly recommend his teachings on this topic.
He doesn’t need to reconcile his personal experience with anyone else. Because it’s his personal experience. No validation is necessary because - whatever others may have experienced, this was his experience. How about letting him have his point of view, especially as he’s not trying to invalidate anyone else’s perspective.
