Talk:Singularity Institute for Artificial Intelligence

From Wikipedia, the free encyclopedia

[edit] NPOV for Pakaran

I've taken a lot of stuff out of the article that seemed to be basically just handwaving and self-promotion. This is what it read like when I found it:

"The Singularity Institute for Artificial Intelligence is a non-profit organization that seeks to create a benevolent artificial intelligence capable of improving its own design. To this end, they have developed the ideas of seed AI and Friendly AI and are currently coordinating efforts to physically implement them. The Singularity Institute was created in the belief that the creation of smarter-than-human, kinder-than-human minds represents a tremendous opportunity to accomplish good. Artificial intelligence was chosen because the Singularity Institute views the neurological modification of human beings as a more difficult and dangerous path to transhuman intelligence."

"The Singularity Institute observes that AI systems would run on hardware that conducts computations at billions or trillions of times the characteristic rate of human neurons, resulting in a corresponding speedup of thinking speed. Transhuman AIs would be capable of developing nanotechnology and using it to accomplish real world goals, including the further enhancement of their own intelligence and the consensual intelligence enhancement of human beings. Given enough intelligence and benevolence, a transhuman AI would be able to solve many age-old human problems on very short timescales."

As it stands, that isn't a bad article, it's just that it isn't really suitable for an encyclopedia. It presents some things as fact that are clearly opinion. It makes contentious statements, such as that it originated concept of "Seed AI" (astonishing for such a new organization--I read similar ideas in Von Neumann's book in the mid-seventies, and that had been written nearly thirty years before). The claim to be "coordinating efforts to physically implement" Seed AI and Friendly AI seem to rest on fundraising and writing a lot of papers about an extremely loosely defined programming language which seems to lack even an experimental implementation.

Wikipedia isn't for original research, it isn't for us to put up our pet ideas (however valid they may be). It's to catalog human knowledge from a neutral point of view. The article as it stood was in my opinion not so much an encyclopedia article as a promotional panegyric. --Minority Report 03:07, 23 Nov 2004 (UTC)

Ok, in the interest of admitting biases, I'm a financial donor to the SIAI. It's true that there have been holdups in beginning actual development, largely because there's a need to get all the theoretical underpinnings of Friendly AI done first.

That said, claiming that the SIAI is a "religion" rather than a group (which you may or may not agree with) is intrinsically PoV. --Pakaran (ark a pan) 03:55, 23 Nov 2004 (UTC)

I agree with most of your criticisms, Minority Report, and the article was not NPOV as it existed before. The statement that they are coordinating efforts to implement seed AI is quite valid, however. SIAI is developing a smaller, less ambitious AI program, although the primary objective of its research now is formalizing the theoretical framework for Friendly AI.

Also, using the phrase "quasi-religious" to describe an institution that claims to be entirely secular is highly misleading. SIAI has no affiliation with any religion.

I'm interested in your comments regarding von Neumann's work. I was not aware that von Neumann had speculated in this area. If you can find a source perhaps it should be mentioned at Seed AI. — Schaefer 05:02, 23 Nov 2004 (UTC)

I think my use of the term "quasi-religious" was an overstatement. I was trying to encapsulate the visionary aspect of this work, and the use of language which seems to owe more to religion than to engineering. I apologise if I also mischaracterized the Seed AI stuff; from looking around the site I saw a lot of hot air and little activity. I read a few books by Von Neumann in the late seventies, and the idea of having self-improving machines was very much his aim. I'm sorry I can't recall the specific book. I thought it might be The Computer and the Brain but a glance at the contents page on Amazon doesn't offer any clues. The idea was certainly in the air in the 1970s, long before Vinge's 1993 paper. --Minority Report 11:09, 23 Nov 2004 (UTC)

I've added basic information on the SIAI-CA and removed an erroneous middle initial. I also changed the first paragraph to reflect the fact that the SIAI actually does want to build software, rather than just talk about it, and to clarify that the 'Singularity' in the name refers to influencing the outcome of a technological singularity. --Michael Wilson

[edit] Merges

I have merged in information from the previously separate items on Emerson and Yudkowsky, which amounted to about a line of exposition and a few links. Those items now redirect to this item.

Yeah, I'd like that redirect to be removed. Actually, I'm removing it now. Eliezer Yudkowsky is wikified in many articles already. There is no reason to redirect an article about a person to their association's article. Biographical articles can be fleshed out over time and as of now it *looks* like we don't have an article on Yudkowsky when in fact we did. A line would have been a good start for smeone to write more. I'm making Eliezer Yudkowsky a bio-stub. --JoeHenzi 22:52, 21 Apr 2005 (UTC)

[edit] What is "reliably altruistic AI"?

Is it behavior that promotes the survival and flourishing of others at a cost to one's own? Wouldn't this require an AI with self-awareness and free will? But if an AI has free will, would it be moral to enslave it to serve the interests of others at the cost of it's own interests? Or is this merely a nod at Azimov's science-fiction "Three Laws of Robotics"? Those are close to a robotic version of a policeman's duties which may be seen as altruistic but may also be seen fulfilling a contract for which one is compensated. Or does the statement merely envision a non-self-aware AI with an analog of what ethologists call biological altruism? Whatever SIAI has in mind, I think the article should either make it explicit or drop the sentence since, as it stands, it is difficult or impossible to know what it means. Blanchette 04:43, 25 September 2006 (UTC)

The SIAI web site spends many pages addressing that question. Unfortunately I don't think it can be concisely added to this article, which is already fairly long; interested parties will just have to follow the references. Perhaps someone could expand the 'promotion and support' section of the 'Friendly Artificial Intelligence' article to detail the SIAI's definition of the term better. --Michael Wilson

Michael, thanks for the hint that what the author of the phrase "reliably altruistic AI" had in mind was the same thing as "Friendly AI". A search of the SIAI website reveals that the phrase "reliably altruistic AI" is not used there, nor is the term "reliably altruistic" nor is "altruistic AI". So "reliably altruistic AI" looks like an attempt to define Friendly AI that leads one away from rather than closer to understanding SIAI's ideas. I have replaced it with "Friendly AI" and the curious will then understand that further information is available through the previous link to "Friendly artificial intelligence". --Blanchette 07:06, 22 November 2006 (UTC)