Yudkowsky on science and religion

This post had escaped my attention and is very much worth a read. A (large) excerpt:

Back in the old days, saying the local religion “could not be proven” would have gotten you burned at the stake. One of the core beliefs of Orthodox Judaism is that God appeared at Mount Sinai and said in a thundering voice, “Yeah, it’s all true.” From a Bayesian perspective that’s some darned unambiguous evidence of a superhumanly powerful entity. (Albeit it doesn’t prove that the entity is God per se, or that the entity is benevolent – it could be alien teenagers.) The vast majority of religions in human history – excepting only those invented extremely recently – tell stories of events that would constitute completely unmistakable evidence if they’d actually happened. The orthogonality of religion and factual questions is a recent and strictly Western concept. The people who wrote the original scriptures didn’t even know the difference.

[...]

Not only did religion used to make claims about factual and scientific matters, religion used to make claims about everything. Religion laid down a code of law – before legislative bodies; religion laid down history – before historians and archaeologists; religion laid down the sexual morals – before Women’s Lib; religion described the forms of government – before constitutions; and religion answered scientific questions from biological taxonomy to the formation of stars. The Old Testament doesn’t talk about a sense of wonder at the complexity of the universe – it was busy laying down the death penalty for women who wore men’s clothing, which was solid and satisfying religious content of that era. The modern concept of religion as purely ethical derives from every other area having been taken over by better institutions. Ethics is what’s left.

Or rather, people think ethics is what’s left. Take a culture dump from 2,500 years ago. Over time, humanity will progress immensely, and pieces of the ancient culture dump will become ever more glaringly obsolete. Ethics has not been immune to human progress – for example, we now frown upon such Bible-approved practices as keeping slaves. Why do people think that ethics is still fair game?

Intrinsically, there’s nothing small about the ethical problem with slaughtering thousands of innocent first-born male children to convince an unelected Pharaoh to release slaves who logically could have been teleported out of the country. It should be more glaring than the comparatively trivial scientific error of saying that grasshoppers have four legs. And yet, if you say the Earth is flat, people will look at you like you’re crazy. But if you say the Bible is your source of ethics, women will not slap you.

[...]

The idea that religion is a separate magisterium which cannot be proven or disproven is a Big Lie – a lie which is repeated over and over again, so that people will say it without thinking; yet which is, on critical examination, simply false. It is a wild distortion of how religion happened historically, of how all scriptures present their beliefs, of what children are told to persuade them, and of what the majority of religious people on Earth still believe.

UPDATE: Today Eliezer elaborates:

Absence of proof is not proof of absence. In logic, A->B, “A implies B”, is not equivalent to ~A->~B, “not-A implies not-B”.

But in probability theory, absence of evidence is always evidence of absence. If E is a binary event and P(H|E) > P(H), “seeing E increases the probability of H”; then P(H|~E) < P(H), “failure to observe E decreases the probability of H”. P(H) is a weighted mix of P(H|E) and P(H|~E), and necessarily lies between the two.

UPDATE #2: Even more here:

The rule that “absence of evidence is evidence of absence” is a special case of a more general law, which I would name Conservation of Expected Evidence: The expectation of the posterior probability, after viewing the evidence, must equal the prior probability.

P(H) = P(H)
P(H) = P(H,E) + P(H,~E)
P(H) = P(H|E)*P(E) + P(H|~E)*P(~E)

Therefore, for every expectation of evidence, there is an equal and opposite expectation of counterevidence.

If you expect a strong probability of seeing weak evidence in one direction, it must be balanced by a weak expectation of seeing strong evidence in the other direction. If you’re very confident in your theory, and therefore anticipate seeing an outcome that matches your hypothesis, this can only provide a very small increment to your belief (it is already close to 1); but the unexpected failure of your prediction would (and must) deal your confidence a huge blow. On average, you must expect to be exactly as confident as when you started out. Equivalently, the mere expectation of encountering evidence – before you’ve actually seen it – should not shift your prior beliefs.