Brilliant quote from Stephen Wolfram (@0:07:20):
The analogy that I’ve found useful, that sort of comes out of some science I’ve done in this area is let’s imagine that your task is to build a wall. Well, one way you can do that is you make these very precisely engineered bricks and you set them up in this very kind of organized way and you build this wall and you can keep building it, and you can keep building it very tall. Okay, that’s plan A. Plan B is you see a bunch of rocks lying around on the ground and as you build your wall you find a rock that’s roughly the right shape. You stick that one in. You keep building that way.
That second thing is pretty much what machine learning is doing. When you train a neural network, what it’s doing is it’s finding these kind of lumps of computation that happen to fit into what the training looks like and so on, and so it sort of puts that rock into the wall and keeps building. And it’s something where you can build the wall to a certain height, just with these randomly shaped rocks. But it’s not something where you’re going to be able to sort of systematically build it up very tall.
I think that’s perhaps a way to think about what’s going on. But in the end, it’s that kind of, you know, machine learning is getting things roughly right and that’s a big achievement In many domains. You know, getting it roughly right, writing that essay that makes sense is great. You can’t say that essay is precisely the right essay. It’s just oh, it’s an essay that makes sense. You know it’s a distinction between what needs to be precise and really built up many, many steps, and what needs to be happen, sort of roughly right.
[It’s amusing that the TWiT transcription AI completely hallucinated a line or two of text in this part of the transcript. I had to manually edit to fix it – but that’s far easier than transcribing by hand.]
This is the kind of a metaphor a native Englishman would concoct. Anyone who has seen the PBS show All Creatures Great and Small has witnessed the fantastic drystone walls in Yorkshire Dales:
Stephen is noting there’s a limit to the “height” of a wall when asking an LLM a question. With the Wolfram Language, you’re asking their Notebook Assistant to produce a tiny amount of code. They worked very hard on their LLM Prompt; it generates very good Wolfram Language code with minimal odds at hallucinations. You can instantly run that code in a “Notebook” interpreter. If it doesn’t work as desired, you can ask again or hand-edit the code yourself. At any instant, you’re never building the rock wall very high. This iterating model is a nice way to plow through projects.
I think Wolfram’s AI proposition is quite elegant. Wolfram has been pounding on the Wolfram Language API for 37 years. Notebooks that were coded on Day 1 can still be run seamlessly today. It’s a big API, and it runs flawlessly and is meticulously documented. ChatGPT has assimilated the entire documentation-set, so it has a rather tremendous number of rocks to make the drywalls with.
Make no mistake: Stephen Wolfram’s presentation is designed to promote the strength of their AI offering. At the same time, it’s a compelling argument and he’s genuinely enthusiastic about what they’ve done. They have been grinding away for ~40 years to produce their API an the interpretation system. AFAIK, nobody has a platform that is remotely comparable to that.
Stephen writes entirely in Wolfram Notebooks. I bet everybody on staff does the same. Stephen’s What is ChatGPT Doing… And Why Does it Work is a straight computational essay. He writes with text and interjects bits of Wolfram Language code whenever appropriate. It is a tremendous way to make a presentation and a tremendous way for a company to work together.
IMHO, the only terrible thing that Wolfram Research produces is their home-rolled version of a Discord server: https://community.wolfram.com . It feels like a 20th Century bbs. It scales poorly; a discussion with >300 messages is almost unreadable. Inserting an emoji in your text will crash your front end when you try to post the message. I have asked repeately why they don’t port their discussions to Discord; I’ve never gotten an answer. I believe asking about that is the Third Rail inside of the company. Wolfram dogfoods their software like nobody in the industry; this is one place where dogfooding is a bad idea. At least they don’t try to roll their own web browser or operating system.
This interview might get some traction on YouTube. @Leo, did the production team edit this interview down and put it up?