scratch.tex

RATIONAL GROUNDS FOR CHOOSING EPSILON

Standard probability theory implies that if a theory becomes overwhelmingly unlikely then evidence which might at first seem to confirm it actually turns out to deny it. In [TODO: CITATION JAYNES] he uses the example of Mrs. Smith, an apparent clairvoyant who was the subject of [CITATION: ESP PAPER]. 

At first, the result of CITATION seem to confirm that Mrs. Smith possessed psychic powers. She correctly identified k cards out of n, which given the assumption of random and independent card-choice, would be massively unlikely under the null hypothesis of ``Mrs. Smith doesn't have psychic powers''. It is, however, quite likely under the complementary hypothesis ``Mrs Smith has psychic powers'' and therefore her performance in the card-guessing experiment would seem to confirm the psychich hypothesis over the null hypothesis.

However, as Jaynes points out, this is a naive application of probability theory in that it does not account for the range of \textit{other} hypotheses which could have also generated the same data. Such hypotheses include [TODO: quote from Jaynes about deception hypotheses]. When the full range of possible hypotheses is considered, Bayesian updating confirms the deception hypotheses over both the null \textit{and} psychic hypothesis. This demonstrates how an observation we first took as evidence for an unlikely theory can actually be evidence \textit{against} it.

In general, if a theory's prior probability is low enough, and deception easy enough to perform successfully, observing a prediction of the unlikely theory will actually confirm the deception hypothesis. Deception hypotheses are always possible, as they include systematic flaws in an agent's perception, cognitive errors, hallucinations, dreams etc. Even a perfectly rational agent needs to consider these deception hypotheses \textemdash after all, mere rationality does not protect you from a brain aneurism, a RAM or hard drive corruption, or a Godlike agent interfering with your vision, all of which provide grounds for disbelieving your senses. 

This implies that agents have some theories which are, in practice, so unlikely as to be unconfirmable. If a hypothesis H is extraordinarily less likely than someone spending a lot of time and effort trying to convince you of H, then observations predicted by H will actually confirm deception(H). As Jaynes says, ``until we can rule out the possibility of deception, evidence will not convince us of ESP'' [TODO: look up actual quote]. But, as agents cannot practically rule out \textit{some} form of deception, we will always have to live with unconfirmable hypotheses.

Probability theory (if one assumes deception is possible) tells us that some extraordinarily unlikely hypotheses can't practically be confirmed. RNP tells us to choose a cutoff and disregard outcomes less likely than it. I propose agents combine these two theories:

RNP* When facing a decision problem, if there is a hypothesis they could not reasonably confirm through observing its predictions, choose a probability cutoff \epsilon to be the prior probability of this theory.

=== 

Concerns:
 - How do you gather evidence about something
 - Can you ever disconfirm deception? Have we smuggled skepticism in?
 - What if an agent becomes convinced of ESP? How could any evidence disconfirm it? How does the inverse irrational case work? Doesn't Bayesian updating say you should eventually converge on the truth?
 - Does this contradict Bayesian updating (which always converges on truth?)
 - Suppose epsilon should rationally be 10^-22 for black hole reasons. This still implies EU(St. Petersburg) > \$1,000,000,000,000. Your theory still overestimates the value of St. Petersburg. My response: true, it does, but I'm not looking at the specifics of a given paradox, I'm looking at solving HULP problems more generally. Any finite value for SPP will be enough to stop HULP exploitation, and that is what my theory does.

===

TODO:
[ ] Follow RNP math as applied to PW or SPP
[ ] Link dead hypotheses to Pascal's Mugging/Wager

Criticism:
[ ]

(1) You should choose epsilon >= P(CSH)
(2) P(R) < P(CSH)
 ∴  P(R) < epsilon

===

MC: “So, \(\epsilon\) is fixed and no problem can ever consider states whose probability is lower than it, regardless of context. It seems like this theory will give some bad advice in risk analysis. For example, the risk of a weld failure in nuclear reactors is arguably lower than the probability of CSH.”

My response: \(\epsilon\) is fixed for a SINGLE AGENT in a SINGLE ENVIRONMENT. But if you need to consider events less likely than \(\epsilon\), you could always consult multiple agents. They could rationally consider events less than the product of their epsilons.