Very interesting essay! It's pretty mind-bendy to try to mathematically model the totality of human mathematical modeling. Neat to see a stab at formalizing the error-prone, human-driven practice of mathematics.
A couple technical nitpicks. I'm not sure how justifiable "sequential information source" property is, but it may be that I just misunderstand. The net effect of it looks like it makes your stochastic process into a Markov/'memoryless' process, which is a pretty strong assumption.
Also, is it safe to assume that the claims probability distribution converges in the many iteration limit? In noisy gene regulatory biology, where probability distributions completely characterize stochastic systems, it is common to observe oscillatory behavior in the long-time limit that prevents P_inf from being mathematically well-defined.
Here's an admittedly kind of dumb situation where I think P_inf wouldn't be well-defined, just to illustrate that it's conceivable. Imagine a universe (perhaps a desert island) consisting of people that hatch out of eggs, one at a time. Only one human/mathematician lives at a time. When they die, the next egg hatches, and another human/mathematician pops out. On this island is a computer whose screen displays, forever, a single mathematical claim: "P is ____". At one time, it says "P is true". The human alive at this time unreservedly believes the claim, so the mathematical knowledge of the world at this time is just that "P is true" with 100% belief. Unfortunately, the computer is glitchy, and happens to switch "true" for "false" and vice versa every time a mathematician dies. The next mathematician believes "P is false" unreservedly. The next the opposite, and so on. Here, there is no convergence, except possibly in the long time limit where everyone is dead, and there is no mathematical knowledge anymore.
Now for some more philosophical questions. I think there's an underlying semantic issue about what you 'mean' by mathematics. Full disclosure, I'm on the side that mathematical truth is independent of human mathematicians, and can thought to 'exist' in some Platonic sense. I think well-formed claims within well-defined axiomatic systems are either true, false, or undecidable, and that this is completely independent of our reasoning abilities. If you believe this to be true, mathematics is not stochastic, but our confidence in results (especially controversial results with very complicated proofs, like the classification of finite simple groups or more recently the abc conjecture) and choice of what problems to work on certainly is.
I find it hard to make the leap that, because human mathematical reasoners are fallible, and cannot be completely confident in every "proved" mathematical result, physical universes equivalent to formal systems must be stochastic. Again, while our reasoning about these systems is certainly prone to error, it's much harder to imagine the universe itself is prone to error when computing the consequences of its own rules. What would that even look like? Would it be possible to 'observe' the universe making a mistake?
Still, regardless of where my beliefs might differ, I admit that it's logically possible that some scenario like this could be true, even if it's hard to imagine. Very thought-provoking essay overall.