Not in the slightest if you are using it in your normal diet for example wherever someone else might use cow's milk.
If you were drinking gallons of it or taking it in pill form, it MIGHT, theoretically compete with estrogen at your receptors, weakening the estrogen's effects.
Even then it would not be harmful.