I don't know AI, this is just a general, non-technical observation. LLM/GPT is really amazing, it "understands" humans in a sophisticated way that is truly unprecedented.
But philosophically, it is still far from the ultimate artificial intelligence that we can imagine.
The current status quo, after all, is that it is still a garbage-in, garbage-out learning system. It can be treated as a text-centric information-retrieval system, or as David Irvine would like to put it, a compression engine for free-form human knowledge.
In a way, it is a bullshit artist, but a very, very good one, probably the best there ever is. I'm not saying it often produces bullshit responses, on the contrary, most of the time, we find the answers highly relevant and even useful. But that's the issue - there is no verification capability involved, anything from basic fact checking (well, it can include the source of the info, at least sometimes), to rigorous testing or even formal proof.
Some examples:
- There was this news story where a lawyer was caught using GPT to generate fake cases that never happened (but great for fiction-writing).
- Fang Zhouzi asked GPT about himself, and found lots of misinformation, again, garbage in, garbage out. The Chinese sector of the training corpus is both lacking in quantity and full of lies.
Indeed, people love GPT so much, largely because it tends to answer more than what we ask about, as if it has first elaborated our question to a great yet appropriate extent, before giving its response. But this is also a double-edged sword, its nuance, inference, and "creativity". That's why people find it highly valuable for generative tasks, the more the merrier.
But here comes the limitation. If what we aim for is correctness, reliability, or even certainty, then there is no such thing in offer. GPT places the following disclaimer instead:
ChatGPT can make mistakes. Check important info.
For instance, we once asked it about a simple plugin/script for modifying the default behavior of a system. It gave a number of code snippets that was supposed to achieve that purpose. But not a single one of them worked, some of them wasn't even syntactically valid, others produces runtime errors.
In general, if the task at hand is something that needs to be executed according to a plan (a program in the general sense of the word), then GPT/LLM is helpless.
In an introductory AI course in college, there was this notion about what people want to achieve with AI, and there were two schools: one wants AI to be more intelligent, and the other wants it to be more like human. Now this can be interpreted in two distinctive ways:
- The classic rivalry between human expert knowledge-based algorithms, and learning from scratch, and this argument has been decisively settled, by AlphaZero.
- What I'm saying is from a different perspective: in a sense, GPT is just like a real human (super-human to be fair), in that it inherits all the good parts, and bad, from human-generated texts, and that's why it's so good at "understanding" us, namely, giving us the most relevant and interesting response below the prompt. However, for the same reason, it cannot go beyond human wisdom, to critically analyze, verify or disprove the generated content.
Hence, when it's "important", you are responsible for "checking".
And that means, we're still far from that singularity today. Isn't that good news, that we're still ultimately the decision maker, i.e., to accept or reject what AI tells us?
And in practice, in spite of the admirable advancement of GPT, we still need to manually verify the info, in a domain-specific manner. For instance, in programming, we still need a well-engineered system to analyze, debug, and test the code generated by the AI in order to actually get the "copilot" business to the next level.
But if it's manual, or domain-specific, then it's not really AI, right? They're like AI-mate, something as the quality assurance mechanism to make sure the charming bullshit artist is not bullshitting when things matter.
Now from the futuristic perspective, general-purpose, automatic verification of information is the next step of AI towards Singularity. But again, when that day comes, there is, I'm afraid, not much meaningful stuff left for us to do, because all we do will be to query the oracle, and then follow the order, and any deviation from that (critical thinking, creativity) would be a screw up by us. That is literally death and hell. Or heaven, if you're the lazy kind.
We humans have been imagining a god (many versions of it in fact), but in a way, it is equivalent to the notion of singularity, and the horrifying state of perfection. As a species, we started as a flawed, but heroic spirit of adventure, but if the end is a techno-god that reduces all of us as a bunch of flaw balls, then have we committed spiritual suicide?
Or, it will be like the perfect happy ending of a movie. But it is an ending, regardless.