I'm a Boy

The other day Jett and I were out and about when a woman stopped us to admire my new baby girl. She went on to tell me a little bit about her own newborn daughter, and then she walked on. I didn't correct her, but since then I've been wondering...

When looking at a baby's face, how do we know if it is a boy or a girl? Or can we know at all? Do infant boys and girls have different facial features? If not, when do boys and girls start looking like, well, boys and girls? Do we just guess the sex of a baby based on what he or she is wearing?

Does anyone have any idea?