Indeed, I am very glad that we still have a very very long ways to go in our quest for AI. I think of this seemingly â€œpessimisticâ€ view of mine as being in fact a profound kind of optimism, whereas the seemingly â€œoptimisticâ€ visions of Ray Kurzweil and others strike me as actually being a deeply pessimistic view of the nature of the human mind.
The entire interview is an excellent read, and later on the interviewer points to some similarities in both Kurzweils’ and Hofst’s view about sentience as “software”. Hofst answers with a critique that I think echoes my earlier skepticism:
Well, the problem is that a soul by itself would go crazy; it has to live in a vastly complex world, and it has to cohabit that world with many other souls, commingling with them just as we do here on earth. To be sure, Kurzweil sees those things as no problem, either â€” we’ll have virtual worlds galore, â€œup thereâ€ in Cyberheaven, and of course there will be souls by the barrelful all running on the same hardware. And Kurzweil sees the new software souls as intermingling in all sorts of unanticipated and unimaginable ways.
Well, to me, this â€œgloriousâ€ new world would be the end of humanity as we know it. If such a vision comes to pass, it certainly would spell the end of human life.
not trans-humanism, but null-humanism, indeed.
Of course, for a more rigorous critique of Singularity, the recent IEEE special issue had some excellent critical articles alongside the fluffy vision pieces. Highly recommend: “Waiting for the Rapture“, “The Consciousness Conundrum”, and best of all, Singular Simplicity. All of these pieces level specific, scientific, and physical arguments that undercut the grandiose hand-waving arguments of the singularitans.