Aren’t LLM already pretty much out of (past) training data? Like, they’ve already chewed through Reddit/Facebook etc and are now caught up to current posts. Of course people will continue talking online and they’ll continue to use it to train AI. But if devouring decades of human data, basically everything online, resulted in models that hallucinate, lie to us, and regurgitate troll posts, how can it reach the exponential improvement they promise us!? It already has all the data, has been trained on it, and the average person still sees no value in it…
Your mistake is in thinking AI is giving incorrect responses. What if we simply change our definition of correctness and apply the rubric that whatever AI creates must be superior to human work product? What if we just assume AI is right and work backwards from there?
Then AI is actually perfect and the best thing to feed AI as training data is more AI output. That’s the next logical step forward. Eventually, we won’t even need humans. We’ll just have perfect machines perfectly executing perfection.
It already has all the data, has been trained on it, and the average person still sees no value in it…
And that data that it has been trained on is mostly “pre-GPT”. They’re going to have to spend another untold fortune in tagging and labeling data before training the newer ones because if they train on AI-generated content they will be prone to rot.
Aren’t LLM already pretty much out of (past) training data? Like, they’ve already chewed through Reddit/Facebook etc and are now caught up to current posts. Of course people will continue talking online and they’ll continue to use it to train AI. But if devouring decades of human data, basically everything online, resulted in models that hallucinate, lie to us, and regurgitate troll posts, how can it reach the exponential improvement they promise us!? It already has all the data, has been trained on it, and the average person still sees no value in it…
Your mistake is in thinking AI is giving incorrect responses. What if we simply change our definition of correctness and apply the rubric that whatever AI creates must be superior to human work product? What if we just assume AI is right and work backwards from there?
Then AI is actually perfect and the best thing to feed AI as training data is more AI output. That’s the next logical step forward. Eventually, we won’t even need humans. We’ll just have perfect machines perfectly executing perfection.
What if we invent inbred robots?
That’s pretty much what neural networks are
And that data that it has been trained on is mostly “pre-GPT”. They’re going to have to spend another untold fortune in tagging and labeling data before training the newer ones because if they train on AI-generated content they will be prone to rot.
Me too.
