Cf. “Artificial Intelligence as a Positive and Negative Factor in Global Risk“

I decided to google up “singularity skeptic” – to try to find refutations of the notion – and found this page.

Having a little trouble with some of the arguments presented here…

“AI is artificial by definition – a simulation of intelligence.”

I think this is some kind of semantics game. AI is real intelligence. AI is currently very good at some things like playing chess. You might say that’s not intelligence – but what about in 2025 when an AI handles 90% of the design of a new airplane on its own? Or in 2030 when it designs the first commercially viable fusion reactor? These are of course hypothetical – but saying it’s just “simulated” doesn’t mean it’s not intelligence.

“Do any AI algorithms make an attempt to include structures like an “arcuate fasciculus” or a “basal ganglia” into their model? Is there any understanding of the difference between gray and white matter?”

Yes – we don’t currently have a good enough understand of the brain to try to simulate it at all levels (or at all to any real extent). That doesn’t mean we won’t in the mid-term (or further) future.

“augment them somehow, but that requires the same knowledge of the brain as AI does, so the same skepticism applies. The other half is physical augmentation, but here we get to the question of energy source.”

Right – energy source is a question – along with a billion other things – that doesn’t mean there aren’t answers. It’s also assuming the power requirements for anything useful would be high – which I don’t think is the case at all. A computer providing a few gigabytes of storage and a few MHZ of CPU could vastly expand someone’s ability with relatively massive mathematical skills and literal photographic memory – and use very little power.

“Then what do we become? A race of gods? or just a pile of nodes, acting out virtual fantasies until the heat death of the universe pulls the plug? That’s not post- or trans-humanism, its null-humanism.”

I actually share some of your concerns here. I’m not sure what the answer is – but I suspect we’ll be facing these questions in the future – not sure how soon – it may or may not be this century. Now seems to be a good time to start thinking about this though…

Paradoxically, transhumanism’s goal can be reached only if transhumanism’s assumption of philosophical materialism is wrong, and mind is more than an epiphenomenon of matter …

Moving software around from machine to machine only works because computers are mass-produced. People, on the other hand, are unique. I expect that if there were some way to take a checkpoint of my brain, you would not be able to find another brain that could execute it.

I do expect that it will eventually be possible to move large quantities of *data* in and out; if you google Brainport you’ll find some intriguing stuff that points that way. Ultimately, I expect this to only be successful when it’s possible to install implants in infants so that as their brains learn about the world around them, they will also learn how to manipulate the implant. For them, moving large chunks of data will be as natural as speaking is for us.

I don’t expect the technology to be cheap, however, I suspect that there will be a variety of implants with different features for a range of prices. At some point, a child’s future may be determined by how far into debt their parents were willing to go to have a data port installed immediately after birth…

That, to me, is the big problem with AI. If we know how it works, we won’t consider it to be intelligent.

I totally disagree that “every thing humans do is the result if instructions given us by evolution”. Thats arguing that humans are no more sentient than machines are. If you believe that AI can happen then you have to believe in teh I part first. If you think we are just bags of meat and chemical stimuli then what point is there in developing AI?

We certainly can give a artifical organism sensors, and program it for behaviors, but the choice is going to be like your Netflix recommendations – derived from weighted averages, not a choice. Case in point is chess – computers dont play chess. They massively brute-force the game to assess eveyr possible branching move N moves donwstream of the present board, then calculate a metric of how “good” the outcome is along each of those branches, and then “choose” the highest-scoring branch. Thats chess as algorithm not chess as a game.

I will be a lot more impressed when an AI beats a 1p dan at Go.

Every single thing humans have done is carry out instructions given to us by evolution.

It cannot notice, it cannot choose to act, it cannot explore possible problem solving techniques, even if we laid those out for it.

Sure it can. If we can give it eyes, it can notice things. If we give it motivations, it can choose between them (just as we choose between eating, sex, work, etc at a given moment). Computers can already solve chess problems better than we can.

Quite a few, actually.

true, but you might as well argue that a human being is just a pile of water and some trace materials, but given a big pile of that stuff you don’t get a human, either The process by which those raw materials become more, is encoded as information in DNA, and also an emergent property. Thats all part of billions of years of evolution, and not something we can just replicate without effort. It’s like breaking an encryption code – just numbers, right? But which numbers?