It used to be thought that "for the first couple of weeks all embryos develop the same." But this has turned out not to be true.
Sex differences have been documented in the placental cells formed by male and female human blastocysts at the stage when the blastocyst implants into the uterine wall and begins growing the placenta. This is usually around 5 days after egg and sperm merged in one of the Fallopian tubes to form a zygote - aka fertilization or conception.
Study of what goes on at the cellular level shows there are numerous sex differences in human and other mammalian embryos that precede the development of the gonads. In other words, a lot of sex differences are are upstream rather than downstream of T.
Many physical sex differences in humans once assumed to be the result of sex hormones and sex hormones alone are actually the result of sex chromosomes and genetics - or of a combination of sex chromosomes, genes, sex hormones, the different sex hormone receptors that males and females have, etc.
In humans who are either 46,XX or 46,XY - which is nearly all of us - thousands of the exact same genes in the human genome have been found to "express" and behave differently depending on whether a person has one or two X chromosomes and has or doesn't have the SRY gene.
Also, just for the record: "male baby brains" don't release T during early development or any other time. The T that drives the development of human males in utero and in after they are born comes from their gonads, the testes.