If anti-Mormon critics would take the time to read the entire lecture given by Brigham Young, they would realize that his point is one that they may well agree with. President Young was addressing those who erroneously teach that the Holy Ghost begat the Son, saying this misunderstanding detracts from the Fatherhood of our Heavenly Father. Brigham Young was affirming that God the Father was the Father of Jesus--not the Holy Ghost.
The Bible abounds with scriptures attesting to the fact that Jesus was the Son of God the Father; he was not the son of the Holy Ghost. Thus it was God the Father, not the Holy Spirit, who spoke to Jesus at the time of his baptism, saying, "thou art my beloved Son; in thee I am well pleased" (Lk 3:21-23, Matt. 3:16-17). It was to his Father, God the Father, rather than to the Holy Ghost, that Jesus taught his disciples to pray (Matt. 6:6-18). "We beheld his glory, the glory as of the only begotten of the Father" (John 1:14); "For God so loved the world, that he gave his only begotten Son" (John 3:16; see also John 3:18, 1 John 4:9).
Passages like these demonstrate that Brigham Young was correct: Jesus was the begotten Son of the Father. It appears that the Holy Spirit in some miraculous way unknown to mortals, prepared Mary for the conception of God's Son. Rather than being critical of Brigham Young, our antagonists should be pleased that he defended the Biblical position that God the Father, rather than the Holy Ghost, is the father of Jesus.