Why is Harry safest at the Dursley's?

Here's some speculations about why Dumbledore placed Harry at the Dursleys'

• I believe the speculation about Mrs. Figg that lived a couple streets over from the Dursleys being a witch is true. But I don't think an old witch alone explains why Harry is safest there. I think that maybe the Fidelius Charm (like the one that was performed on Lily and James' house in Godric's Hollow) was performed on the Dursley's house. I suspect maybe Mrs. Figg is the secret keeper, but that is a pretty thin speculation. We do know that she is part of the "old crowd" of Dumbledore's from the end on Book 4, and that implies those are people that Dumbledore trusts completely and will trust with about anything, so maybe Mrs. Figg being the secret keeper is not as thin as I thought. If it is not the Fidelius Charm, then there must be some sort of magical protection there. (Elangomatt)

• "Dumbledore invoked an ancient magic, to ensure the boy's protection as long as he is in his relations' care. Not even I can touch him there..." Voldemort in Goblet of Fire "Ancient magic" is also a term used to describe the protection Lily bequeathed to Harry when she died. This suggests that Petunia is more a part of the wizarding world than she would care to admit. That she (grudgingly) undertook the care of Harry despite what were probably strenuous objectios from Vernon at Dumbledore's request indicates that she knew him and either respected or feared him enough to stand up on that issue.. (Robert Cox)