Those yellow flowers
Aren't causin' your sneezin'
It is also
Those people who react to pollen in the air are usually miserable this time of year.
In order to make seeds, most flowering plants must create pollen. Pollen is very small and dust-like. Some types of pollen sticks to bees and other pollinating insects get pollen on their bodies as they travel from flower to flower. Other types of pollen are lighter and more likely moved by the wind.
The pollen on goldenrod is fairly heavy, sticky and mostly moved by animals. Ragweed pollen however is light and mostly windblown. You can easily see the difference if you shake each type of plant. Not much will happen with the goldenrod (other than perhaps the startling release of some harmless soldier beetles!) while shaking a mature ragweed will create a cloud of pollen.
So why does goldenrod get blamed? With its bright yellow flowers, goldenrod is a much showier plant than the camouflaged ragweed. And when you are miserable with watery, itchy eyes, and runny nose you are mostly likely to blame what you can see, not what is hidden in plain sight.