OK, look. The ghost story or horror story as an artifice designed to scare you only really develops in societies that don't believe in ghosts. In societies that do, you don't have ghost stories. You have stories about ghosts.
Do you see the difference there? If ghosts are a part of the world you live in, it's almost reportage. It's mythology. It's folklore. And folklore and fiction are different things. But if ghosts are not part of your world, then the way a good ghost story works is that during the ghost story (and if it's really good, maybe a little while after) your willing suspension of disbelief extends to doubting, just a tiny bit, just in the back of your head, in the corner of your mind, that they don't exist, accessing a fear you know is irrational. And it's in the accessing of that irrational fear, and its conflict with the conscious knowledge that of course there are no such things as ghosts, it's there that I think is where the enjoyment lies. The chill.
And you get a chill because you know there are no ghosts really, but that knowledge is butting up against the little perverse thought that you know is perverse and which will soon pass but which goes, "yeah, but what if?"
The most obvious person to talk about here if you know your supernatural fiction is HP Lovecraft.
Now, Lovecraft is interesting and horrible in all sorts of ways. He was actually genuinely afraid of two things: people who weren't of white European heritage (and, consequently the very thought of racial mixing); and marine fauna.
Lovecraft's phobia of crabs, squids, jellyfish and whatever has meant that the default for alien monsters is tentacles and eyestalks and pincers and so on. His violent Hitler-has-the-right-idea racism (and he did actually say words to the effect of, "Hitler has the right idea," I'm not joking) is poisonous and swept under the carpet far too much ("he was a man of his time," they say. No, he wasn't. He was racist even by pre-War standards, and there's no way to get around it). But. neither of those things are what is scary about his work, nor are they what his best stories are really about. Although there are stories by him about this stuff. Hell, "The Shadow Over Innsmouth" is about the intersection of those things.
Lovecraft was an atheist. Now, a lot of writing about Lovecraft goes like this: "Lovecraft's 'cosmic horror' presents the horror of a bleak uncaring universe where everything is random," and so on.
But that's actually nonsense. See, in Lovecraft's stories, God literally exists. Yes, Lovecraft's God – and he rewrites God over and over again – is an insane frothing mass at the centre of the universe; God is a malevolent tentacled monster sleeping beneath the sea, waiting to wake up and eat the world; God is a thing behind a mirror that is so horrible that even looking at it makes your tendons seize up and you spend eternity immobile, staring, mad; God is a carnival huckster who hypnotises you with his sideshow and when you look round you realise that you missed that the world ended and you're in hell.
Lovecraft's fans, and writers of the various Lovecraft-inspired roleplaying games, erase this thing from his work in the completist's urge to catalogue and make consistent, but actually, here you have an atheist writing about a universe where God exists, and can be empirically proven to do so.
And that's why the stories work. Here is a man who believes that science can understand the universe, a rational atheist (and a racist whose atheism supplied what he thought was empirical proof for his racism and misogyny – I have no doubt at all that if HP Lovecraft were alive today he'd love Richard Dawkins) who is writing scary stories that go, "but what if – what if – God exists? And he hates us all? Or worse, doesn't care?"
For a Christian (or any other sort of person who places faith in the irrational – and that's not a judgement, because I have an awful lot of time for the irrational and I've pretty much staked my life on it on many occasions), this isn't a scary thing. If you believe in a benevolent, loving God, they're just gribbly monsters.
Which is why Arthur Machen, who was an ordained priest (and a magician, too, but that's by the by), in his weird stories, is very different. His Great God Pan, although superficially like enough to Lovecraft that Lovecraft wrote a homage to it, works on a slightly different level. Both Lovecraft and Machen work on the principle that the universe isn't what we thought it was in their fiction, but while Lovecraft is all "what if God exists and he wants to eat you/ is a seething mindless chaos/ is an evil childen's entertainer?"
Machen's take is more, "what if God is not what we thought He was and all the stuff we thought about morality and sex was worthless? What if it's all urges and writhing bodies?"
It is that perversity that has people still reading Lovecraft (and Machen, although, sadly, not so much), with all his pathological hatreds and neuroses.
You can extend it to other literatures. 1984 is so scary because George Orwell, a lifelong Socialist, a man whose Socialism extended to putting his money where his politics were and going out and fighting fascists, writes a world where Socialism has become the ultimate evil. Orwell has tapped into the same cognitive perversity.
I suppose, in the end, that if you want to write something genuinely scary, it has to access ways that you know the universe doesn't work. If you don't believe in ghosts, write ghost stories. If you don't believe in God, write about a world where God exists.
Write something perverse.
(This was originally shared on Facebook and the old site of The Ghastling, but I felt it needed a more permanent home. Besides, it's the 200th anniversary of the ghost story competition that inspired Mary Shelley to write Frankenstein. Seems as good a time as any.)
