The sunrise problem is a problem first considered by Laplace that asks for the probability that the sun will rise tomorrow given a history of sunrises. Though this may seem like a silly problem it can serve to illustrate fundamental differences between Frequentists and Bayesians.

While Frequentists use probability *only* to model processes broadly described as ‘sampling’, Bayesians use probability to model both sampling and their ‘degree of belief’.

First, let’s consider the Frequentist approach. Now, the Frequentist guy has to

cheat in some way as this problem isn’t well-defined in the Frequentist

framework since ‘tomorrow’ is a sample of size one(and infinite standard

deviation). Some Frequentists try to define this probability by assuming that

there are many worlds with a sun potentially rising on each. But this is a really

silly bastardization of Laplace’s principle of insufficent reason. In all honesty,

the best this guy can do is to calculate the probability that the sun will rise on

any day, and not the probability that the sun rises on a *particular *day. Here we go:

1) Let’s assume that this phenomenon can be modeled as i.i.d. draws from a binomial distribution(i.e. a Bernoulli trial) where is the sum of

Bernoulli random variables and represents the number of days that the sun

rises out of observations.

2) By the Law of Large Numbers converges to the expected

number of sunrises, where is the probability that the sun rises on any day and is our estimate of this probability.

3) By the Central Limit Theorem, for large the sunrises should be normally distributed with mean and variance

4) Furthermore, for large we may construct confidence intervals

with coverage at least :

where is the percentile of the standard normal distribution. So, after a large number of sunrises the Frequentist can give a reasonable answer for the probability that the sun would rise on any day provided that his assumption holds true.

Now, compare this with the Bayesian solution which allows consideration of such questions:

1) is defined exactly as the Frequentist had it defined but in addition we can

define ,the event that the sun rises tomorrow, where equals 1 or 0.

2) Let be the probability of a sunrise on any given day.

3) We assume that before observing we had no prior information concerning

. Hence, by the principle of insufficient reason we may assume that

our prior is uniformly distributed on .

4) Now for the calculation of :

Note: the Bayesian approach isn’t better than the Frequentist solution. But, building a statistical model without any domain knowledge is doomed for failure whatever your approach…Bayesian, Frequentist, or otherwise.