Ask for Ovestin or Estriol Cream instead of Vagifem, you may get on better with these. Once you have got used to one of those you could add in Balance Activ Moisture pessaries (not the BV ones, but the blue box) which may help as they contain hyaluronic acid which improves condition of the skin.
You should be using some form of local oestrogen to prevent things getting worse. Sometimes we have to push through, and it can take a while to see improvement.
JP x