I don't feel equipped to answer, as I am only karyotypically female and would not feel comfortable or like a 'female space' is mine to occupy. I have no breasts, no uterus, a chemical dominance of testosterone, and a social experience (or lack thereof) clouded by autism and self-isolation just as much as it might have consciously or unconsciously been affected by growing up as a presumed female.
That said, do you mean spaces where they'd have to genetically test someone, or check inside their pants, or check their IDs, or judge them by their appearance or sound? That could certainly lead to a lot of gray areas. At the risk of appropriating the intersex experience, what would you say to an XY woman with androgen insensitivity syndrome?
The point is (and I admit I am feeling defensive because I am not sure if the question is meant to make me feel that way. Apologies if not!), we are not here to scream "transphobia!" at negative reactions to the inflammatory, conjured image of the right-wing or gender critical fear mongerer's worst fear: a cisgender man 'dressing as a woman' to invade a women's space and hurt people. (If this was actually to occur, we would not think it transphobic to be upset about, because it would have nothing to do with an actual trans person. And even if a malicious trans woman happened to do this, we likewise of course would not consider it transphobic to condemn her. Her crime would be irrelevant to her identity.)
And for one thing, it doesn’t tend to happen. For another, a man who wants to hurt women has a thousand FAR more realistic and easy ways to do it, and they already happen every day, far too often. Women having to watch their drinks closely or avoid walking home alone at night — things they already have to worry about, day in and day out — none of these require a malicious man to dress as a woman to do them and/or appropriate the transgender experience. (And I doubt a man with heinous intentions would want to give up his presentation of masculinity to carry out his deeds. That would be tough on the ego.)
Anyway, if I think about a 'female only space', I would think of something inclusive of trans and cis women alike, with empathy and face-to-face, case-by-case, getting-a-feel-for-each-other sorts of belonging and (if ever actually necessary for safety's sake) gatekeeping — of any truly harmful, problematic woman in the space, cis or trans. I think about empathy and meeting people where they are - not a black and white, reactionary impulse that makes trans people into bogeymen.
Wow...I have been a bit long-winded, but here is the best suggested reading I can think of right now: https://medium.com/@sophia-burns/your-culture-war-against-transphobia-harms-trans-people-aa6747799f79
And if you have made it this far but feel skeptical about the trans experience in general, I can tell you why it's important, too, in a lot more (and better planned) words than this: https://medium.com/@normvjulian/out-of-the-gray-1af9c4378c9b