Apparently it's not very clear, at least according to the OED. It could be connected to the Welsh bwg, meaning "ghost" or "goblin", or it could be related to the German bögge or boggel-mann. At any rate we've got bog/bug, bogle/boggle, boggard/boggart, and bogey/boogey/bogeyman/boogeyman, all of which carry the sense of some sort of scary specter.