Most genetic variability isn't durable. Males fill the roles of profound stupidity to profound genius and women are clustered around the mean considerably more so. While intelligent people may have an intelligent child, the odds that a 2 sigma above average person can consistently meet and breed with another 2 sigma person for multiple generations is virtually nill, especially with the variance of female intelligence being smaller the subjects are either too rare or normalize back to the mean. Then also there is such a thing as mutating too far into one direction where your only career options are chess phenomenon or theoretical math jackoff fest which doesn't exactly lend to breeding.