This has always been true of anything anyone has ever googled or looked up on stackoverflow
I copy paste code from stackoverflow all the time. I used to agonize over making sure I fully understand every line it's copying. Now I have the discretion of making that decision: sometimes it does really matter, sometimes all you need to know is that it produces the right result for your limited use & test case of it. (it's no different than relying on a 3rd party library in that way)
I think we need to apply the same discretion to LLM output. The answer "it depends". Sometimes using its output blindly leads to disaster. Sometimes using it without fully understanding all the details is a great way to make progress.
I copy paste code from stackoverflow all the time. I used to agonize over making sure I fully understand every line it's copying. Now I have the discretion of making that decision: sometimes it does really matter, sometimes all you need to know is that it produces the right result for your limited use & test case of it. (it's no different than relying on a 3rd party library in that way)
I think we need to apply the same discretion to LLM output. The answer "it depends". Sometimes using its output blindly leads to disaster. Sometimes using it without fully understanding all the details is a great way to make progress.