This has happened to my mom her whole life and she never had short hair. She is very tall like you and though she is not confident, she has a lot of nervous energy, which makes her take up a lot of space. People often mistake her for a man, just for a split second, and then realize their mistake and apologize. This has been happening since I was a teenager, before our culture was more aware of trans identity - which tells me that it is all about size and the space you take up, as you theorized.