I went to an all girl's school (that is, secondary school not university) where I was... not really gender-conforming (I was a girl, but I wasn't into boys, or makeup, or pop music... I was a science nerd). I think I did find the no-boys helpful in contrast to other people's related experiences of having been ignored by science/maths teachers whose default assumption is that "boys are better at science". Later in life having lecturers and supervisors who were women helped me not to feel that women "don't do science". I know my mother suffered a lot from being the only woman on her engineering course, I never had that problem.

Of course single-sex education is much less useful for people who don't realise pre-school that they have been assigned the wrong gender; or people who can't get their parents/guardians/school system to accept that they have been assigned the wrong gender. Having faculty members who aren't all white, cis, able bodied men would probably help more people though.