This is one of those moments where it becomes so obvious we know very little about women's bodies (sigh).
I suffered for 4 years with VA and localised oestrogen only improved things by about 30%. However, when I started systemic HRT (plus continuing the localised treatment), my symptoms are 90% resolved. But it took about 6 months at the right oestrogen dose (1 pump oestrogel for me).
The only reason I can think of for your symptoms flaring is that more nerves grow/expand in a low oestrogen environment. It's weird, but when oestrogen drops, the tissues freak out and grow a bunch of nerves to try and rescue things. When oestrogen is replaced, the nerves thin out. So just when the tissues are drying up because of low oestrogen, we're getting more nerves there so feel it way more than we ordinarily would. Cruel I know!
So, I think maybe your nerves are just freaking out and need to recalibrate to the new oestrogen-rich(er) environment. But I appreciate you can't live in pain either so maybe dropping the oestrogel is the right choice. I wish we knew more!