Consider the implications if ChatGPT started saying “I don’t know” to even 30% of queries – a conservative estimate based on the paper’s analysis of factual uncertainty in training data. Users accustomed to receiving confident answers to virtually any question would likely abandon such systems rapidly.
I think we would just be more careful with how we used the technology. E.g. don’t autocomplete code if the threshold is not met for reasonable certainty.
I would argue that it’s more useful having a system that says it doesn’t know half the time than a system that’s confidently wrong half the time
Depends on the product. From an original AI research point of view this is what you want, a model that can realize it is missing information and deviates from giving a result. But once profit became involved, marketing requires a fully confident output to get everyone to buy in. So we get what we get and not something that’s more reliable.
I think we would just be more careful with how we used the technology. E.g. don’t autocomplete code if the threshold is not met for reasonable certainty.
I would argue that it’s more useful having a system that says it doesn’t know half the time than a system that’s confidently wrong half the time
Obviously. But more useful ≠ more money. So the fascocapitalists will ofc not implement that.
Depends on the product. From an original AI research point of view this is what you want, a model that can realize it is missing information and deviates from giving a result. But once profit became involved, marketing requires a fully confident output to get everyone to buy in. So we get what we get and not something that’s more reliable.