Look, it's impressive yes or yes but is it that surprising that a system like an LLM does well on exams?
What is an exam? It's an attempt to test that you have consumed and understood large amounts of information and can apply it to novel (ish) situations, but in a very 'sandboxed' way that is just text-in-text-out. That's literally what these systems are specialists in doing.
Is it maybe a bit like saying a driverless car can outperform most human drivers in a time trial on a race track? Not a trivial feat, to be sure, but also not that surprising perhaps once the basics are dialled in.
It’s impressive and surprising. Imagine going back to the year 2021 and telling the good people of HN that in 2023 AI would be as advanced like it is now. Literally no one would have believed you in 2021 if you did.
The knee-jerk contrarianism seems to be in vogue right now. Reminds me of some colleagues in 2007 lamenting about the iPhone, "It's not that great; it doesn't even have copy & paste!" Fast forward a few years, and no blackberries in sight.
Oh God, when the iPhone came out people were raging "But it doesn't have a keyboard!"
It's so hard for people, including myself, to look at the future potential of a technology. My attitude is to simply keep an open mind nowadays instead of holding very strong opinions about the trajectory of a given technology.
Not everybody happy living in the future. I'm annoyed most of the time I had to use on-screen keyboard on iPhone - it is slow, error prone and auto-correction only makes it worse when I use uncommon terms. I want hardware keyboard like it was on Nokia with Symbian or Blackberry back.
I think it’s reasonable to critique anything and everything. People have, historically, been over eager to believe any AI hype. Hence the numerous AI winters.
> No one called this, and certainly no one called it would be so easy to access.
Thanks for capturing a point that I couldn't quite articulate myself as to why ChatGPT feels different - the ease of use.
Literally a year ago I was planning to take a python programming course for ML, thinking that a deep understanding of code would be needed to make things work well.
With GPT it's like...I just ask it things in English and it'll do it?
The ease of access to LMMs is groundbreaking the way the simplicity of IOS launched the modern smartphone era. Even babies could use an Iphone.
And now any child old enough to type sentences can use ChatGPT.
> It's an attempt to test that you have consumed and understood large amounts of information and can apply it to novel (ish) situations,
So.. that's basically all we need. So what's left? It can't drink a beer?
Economically useful intelligence is not much more than storing and retrieving large amount of contextually relevant information. Sure, humans can do more, like interpretive dancing for example. But that's not really what we are looking for in a desk job.
If we all earned our money by driving time-trials on race tracks I'd be worried yes.
What is an exam? It's an attempt to test that you have consumed and understood large amounts of information and can apply it to novel (ish) situations, but in a very 'sandboxed' way that is just text-in-text-out. That's literally what these systems are specialists in doing.
Is it maybe a bit like saying a driverless car can outperform most human drivers in a time trial on a race track? Not a trivial feat, to be sure, but also not that surprising perhaps once the basics are dialled in.