It may help to distinguish between the “true” probability of an event and the observer’s internal probability for that event. If the observer’s probability is 0 or 1 then you’re right, it can never change. This is why your prior should never be 0 or 1 for anything.
A better example of this is “how sure are you that 2+2=4 ?” It makes sense to assign a prior probability of 1 to such mathematical certainties, because they don’t depend on our uncertain world. On the other hand, how sure are you that 8858289582116283904726618947467287383847 isn’t prime?
For a die in a thought experiment – sure, it can’t be 7. But in a physical universe, a die could indeed surprise you with a 7.
More to the point, why do you believe the probability that hallucinations as a problem will be solved (at least to the point that they are rare and mild enough not to matter) is literally 0? Do you think that the existence of fanatical AI zealots makes it less likely?
Okay, so by your logic the probability of literally everything is 1. That’s absurd and that’s not how Laplace’s law of succession is supposed to be applied. The point I’m trying to make is that some things are literally impossible, you can’t just hand-wave that!
And I’m not saying that solving hallucinations is impossible! What I’m saying that it could be impossible and am criticizing your blind faith in progress because you just believe the probability is literally 1. I can’t say, for sure, that it’s impossible. At the same time you can’t say, for sure, that it is possible. You can’t just assume the problem will inevitably be fixed, otherwise you’ve talked yourself into a cult.
You do not know that it is nonzero, that’s just an assumption you made up.
Also, Laplace’s law of succession necessarily implies that, over an infinite number of attempts and as long as there is a possibility of success, the probability that the next attempt results in success approaches 1.
No, Laplace’s law of succession states that the (observer’s posterior) probability that the next attempt results in a success approaches the true probability. If it really isn’t possible, then Laplace’s law predicts that as more attempts are made, the observer will predict that the next result is increasingly unlikely to be a success. In other words, the observer’s estimate of the probability approaches 0.
I know that it is possible that it might not be possible. To be clear: in the case that someone isn’t sure whether something is possible or impossible, and has no reason to believe one of those options is more likely, then to them the probability is 50%. Saying “it might be 0 or 1 but I don’t know which!” is the same as saying 50%. If you can predict something no better than a coin flip, then it’s a coin flip. This is basic Bayesian probability theory.
(Laplace’s law merely takes into account that repeat attempts might or might not be correlated – if you flip a coin a hundred times and get tails each time, you’re not going to think it’s 50/50 anymore by then.)
A Bayesian statistician believes that, in our real physical imperfect universe, a six-sided die rolled once will yield each number with 1/6 probability (the probability of a 1 is 1/6; the probability of a 2 is 1/6, and so on) because the Bayesian statistician doesn’t have any way to accurately predict the muscle movements of the person rolling the die, nor the way the die will bounce when it hits the table. (They might reserve a tiny fraction of probability space for esoteric results like landing on a corner or the die quantum-morphing into a neon sign of the number 7.) In contrast, a frequentist statistician will say, “It could end up a 1 or 2 or … or 6, but I can’t tell you which it will be without more information about how exactly the die is rolled. I’m not a physicist! Why can’t we imagine an abstract die instead and analyze that?” This is very unhelpful. If you are applying this perspective to science – which it seems you are if you’re so concerned about the possibility that the probability might be 0 and we don’t know if it is or isn’t so we can’t reason about this yet! – but not to the die, then you need to rethink your philosophy.
If the probability is 0 then it will never be 1
Therefore, there must be some probability of success.
It may help to distinguish between the “true” probability of an event and the observer’s internal probability for that event. If the observer’s probability is 0 or 1 then you’re right, it can never change. This is why your prior should never be 0 or 1 for anything.
For anything? Are you sure about that?
Because I say there’s 0 probability that six sided dice will ever produce a 7.
A better example of this is “how sure are you that 2+2=4 ?” It makes sense to assign a prior probability of 1 to such mathematical certainties, because they don’t depend on our uncertain world. On the other hand, how sure are you that 8858289582116283904726618947467287383847 isn’t prime?
For a die in a thought experiment – sure, it can’t be 7. But in a physical universe, a die could indeed surprise you with a 7.
More to the point, why do you believe the probability that hallucinations as a problem will be solved (at least to the point that they are rare and mild enough not to matter) is literally 0? Do you think that the existence of fanatical AI zealots makes it less likely?
Okay, so by your logic the probability of literally everything is 1. That’s absurd and that’s not how Laplace’s law of succession is supposed to be applied. The point I’m trying to make is that some things are literally impossible, you can’t just hand-wave that!
And I’m not saying that solving hallucinations is impossible! What I’m saying that it could be impossible and am criticizing your blind faith in progress because you just believe the probability is literally 1. I can’t say, for sure, that it’s impossible. At the same time you can’t say, for sure, that it is possible. You can’t just assume the problem will inevitably be fixed, otherwise you’ve talked yourself into a cult.
I’m not saying the probability of literally everything is 1. I am saying nonzero. 0.00003 is not 1 nor 0.
I am not assuming the problem will inevitably be fixed. I think 0.5 is a reasonable p for most.
You do not know that it is nonzero, that’s just an assumption you made up.
Also, Laplace’s law of succession necessarily implies that, over an infinite number of attempts and as long as there is a possibility of success, the probability that the next attempt results in success approaches 1.
No, Laplace’s law of succession states that the (observer’s posterior) probability that the next attempt results in a success approaches the true probability. If it really isn’t possible, then Laplace’s law predicts that as more attempts are made, the observer will predict that the next result is increasingly unlikely to be a success. In other words, the observer’s estimate of the probability approaches 0.
I know that it is possible that it might not be possible. To be clear: in the case that someone isn’t sure whether something is possible or impossible, and has no reason to believe one of those options is more likely, then to them the probability is 50%. Saying “it might be 0 or 1 but I don’t know which!” is the same as saying 50%. If you can predict something no better than a coin flip, then it’s a coin flip. This is basic Bayesian probability theory.
(Laplace’s law merely takes into account that repeat attempts might or might not be correlated – if you flip a coin a hundred times and get tails each time, you’re not going to think it’s 50/50 anymore by then.)
A Bayesian statistician believes that, in our real physical imperfect universe, a six-sided die rolled once will yield each number with 1/6 probability (the probability of a 1 is 1/6; the probability of a 2 is 1/6, and so on) because the Bayesian statistician doesn’t have any way to accurately predict the muscle movements of the person rolling the die, nor the way the die will bounce when it hits the table. (They might reserve a tiny fraction of probability space for esoteric results like landing on a corner or the die quantum-morphing into a neon sign of the number 7.) In contrast, a frequentist statistician will say, “It could end up a 1 or 2 or … or 6, but I can’t tell you which it will be without more information about how exactly the die is rolled. I’m not a physicist! Why can’t we imagine an abstract die instead and analyze that?” This is very unhelpful. If you are applying this perspective to science – which it seems you are if you’re so concerned about the possibility that the probability might be 0 and we don’t know if it is or isn’t so we can’t reason about this yet! – but not to the die, then you need to rethink your philosophy.