I love these forums. I thought it was just me being weird, but then I read this and realise it's not an isolated thing.
Hate driving, and that's been going on for about 15 years now, so from when I was about 35, definitely not peri at that stage. No idea why, but it's fast and bendy roads. Have convinced myself it's linked to a 'no exit fear' because I can't stop (the actual vehicle or the feeling). It's as if my veins become suddenly infused with a mercury substance. Can't explain it either, especially as most accidents happen in a town (fine driving there) and I cycle to and from work every day down extremely busy and dangerous routes. I currently don't drive on any fast or bendy roads, i.e. above 40 / 50 mph.
What you said about 'falling out', oh lordee, yes, that, except for me I couldn't figure out how the door would open, so assumed it was me, and that I simply wouldn't be able to stop myself.
On a slightly brighter note, mine got so bad I couldn't even be a passenger, so I was a bit trapped in town, wasn't going anywhere for years. I had NLP to just be able to get on a train, and that doesn't always go smoothly, but it's manageable. Anyway, recently, past year or so, it's been getting better. Managed to make it up the motorway for 15 miles with my son-in-law driving, and again with my daughter driving, and last weekend a friend gave me a lift and I realised I'd not even thought about panicking before the journey. The absolutely stupid thing is that I can get on the back of hubs massive motorbike and he can weave in and out of traffic in town til the cows come home and I'm fine - obviously, this is the real dicing with death scenario.
Short version, I think it's about not trusting myself. I'm considering hypnotherapy.