Hormonal anxiety can manifest in so many different psychological ways. Avoidance becomes a bit of a default in order to protect us from what might happen, whether that's avoiding being alone or in social situations. The more we avoid, the smaller our world's become and situations that we used to feel comfortable in become uncomfortable.
If it bothers you, as some may not have any issue with becoming more reclusive, then the key is to gradually turn things around by reversing the avoidance by stepping out again. It will feel unnatural and uncomfortable to start with but eventually you will realise it's not as bad as you thought it would be - we tend to exaggerate how bad things will be when we avoid because our imagination takes precedence as we have no factual experiences to look back on (or if we do they are too long ago to relate to).
If you're starting to have panic attacks its because you have convinced your subconscious that there is something to fear, so try to rationalise what you're planning to do in order to replace the underlying thoughts - so 'it will be ok and I might even enjoy it' instead of 'I don't want to go and its going to be terrible'. The former is true and the latter is a lie after all.
Perhaps start with something that's brief, like an hour's coffee with a friend, and then when you feel comfortable doing shorter length stuff you can then move to the next level. If one to one is your worst fear then make that a future goal and start with a larger gathering where you can leave when you need to.
If you're avoiding people you are close to because you don't want to explain what you're been going through then, perhaps, pick one or two people to confide in. For all you know they are going through the same! Equally, don't feel under pressure to explain why you've not been around, you can just say you've been busy if you prefer to keep your health issues to yourself. As long as you keep the people closest to you in the loop, those who are important to you and who you don't want to risk losing, then that's where your responsibility ends. If they don't understand why you are having difficulties socialising (or any of the other issues we face) then try to explain it to them as best as you can. As long as you are communicating, they will know you care enough to share it with them.
Those who stay are the ones who you should have in your life, those who leave are just not right for you.