As an agnostic myself.

The problem there is actually the terminology. The existing hypothesis is that the limbic system, which is part of the midbrain, is where gender identity exists. They've done brain studies on trans people, everything from MRIs to post-mortem brain dissection and have found brain structures that, to oversimplify, tend to match our identified sex far more closely than our assigned sex. What happens with trans people is that the midbrain says A, and the rest of the world sees B, starting by assignment based on looking at genitalia instead of the brain. Assignment based on genitalia, admittedly, is a much less expensive model, but if we don't admit that it's a best guess, we create stressful conditions for we don't know how many people.

How stressful? The key predictor of suicide is a feeling of disconnect from society. How disconnected would you feel if you were the only one who accepted that you were a man? That's why 30% of untreated trans people kill themselves.

So the brain is what makes up gender, an inelastic part of the brain formed at the end of the first trimester. It forms at variance with what one would expect based on genetics due to hormone variances similar to those that influence orientation. Gestational estrogen makes you attracted to females and identify as one, depending on the time of exposure, and testosterone does the opposite.

This is the existing hypothesis, at least. I hope that answers your question.