Guys like him seem to be exactly the kind of person that drive Musk's anxiety. The guy talks incredibly confidently, probably is intelligent, but in reality he doesn't know crap about what the system will be capable of, the intentions of the all people creating it, and what will motivate it if AI reaches a level of true sentience.
I'm making no appeals in favor of transhumanism. I'm pointing out its inevitability and you're entirely avoiding Pascal's Wager which is no option.
Oh and I should point out that Musk didn't see far enough and that it isn't a negative outcome. Musk is right to fear it, but not for the right reasons. He should be more worried about the path to get there than he is about where things end up.
Finally, there's no condition wherein a sentient AI can break out of its boundaries placed by the cognitive capacities of its creators.
(post is archived)