Artificial Neural Networks

Artificial Neural Networks

by | Nov 6, 2017 | Narrow AI

Narrow AI’s Dark Secrets

Articles about AI are published every day. The term “AI” is used in a very narrow sense in the majority of these articles: it means applications based on training artificial neural networks under the control of sophisticated algorithms to solve very particular problems.

Here is the first dark secret: This kind of AI isn’t even AI. Whatever this software has, the one thing it lacks is anything that resembles intelligence. Intelligence is what distinguishes us from the other animals as demonstrated by its product: knowledge about the world. It is our knowledge and nothing else that has made us masters of the world around us. Not our clear vision, our acute hearing, or our subtle motor control, other animals do that every bit as well or better. The developers of this technology understand that and so a term was invented some years ago to separate this kind of program with real AI; Narrow AI which is in use in contrast to Artificial General Intelligence (AGI) which is the kind that processes and creates world knowledge.

Here’s the second dark secret. The machine learning we have been hearing about isn’t learning at all in the usual sense. When a human “learns” how to ride a bicycle, they do so by practicing until the neural pathways that coordinate the interaction of the senses and muscles have been sufficiently established to allow one to stay balanced. This “neural learning” is clearly very different than the kind of “cognitive learning” we do in school which is based on the acquisition and refinement of knowledge. Neural learning cannot be explained and cannot be unlearned, no abstract knowledge of the world is produced. A circus bear can ride a bike but we don’t say it is intelligent because of that.

The third dark secret: We don’t understand how the sophisticated algorithms that control the training of these networks actually work. This fact is probably at the root of the fear that Artificial Intelligence may someday escape human control.

But if narrow AI is not real AI why is it considered AI at all? It is because of the hope that someday these narrow techniques may be extended to become the real thing and real AI is a very exciting, world-changing prospect. That makes these current efforts more glamorous to the general public, easier to hype, and easier to attract funding. But the hype has gone too far and has engendered a growing expectation that real AI is just around the corner and we had better be prepared for its civilization-changing effects.

Today, the AI community is starting to back-pedal big time. We are seeing a growing admission coming from both the big tech companies and academia that the hope that these techniques can be evolved into real AI is, if not totally forlorn, certainly not so imminent as the general public and the media have been led to believe.

Will the Future of AI Learning Depend More on Nature or Nurture?

Yann LeCun, a computer scientist at NYU and director of Facebook Artificial Intelligence Research.
“None of the AI techniques we have can build representations of the world, whether through structure or through learning, that are anywhere near what we observe in animals and humans”

Facebook’s head of AI wants us to stop using the Terminator to talk about AI

“We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do.”
“… in terms of general intelligence, we’re not even close to a rat.”
“The crucial piece of science and technology we don’t have is how we get machines to build models of the world.”
“The step beyond that is common sense, when machines have the same background knowledge as a person.”

Inside Facebook’s Artificial Intelligence Lab

“Right now, even the best AI systems are dumb, in the way that they don’t have common sense.”
“We don’t even have a basic principle on which to build this. We’re working on it, obviously, We have lots of ideas, they just don’t work that well.”

Why Google can’t tell you if it will be dark by the time you get home — and what it’s doing about it
Emmanuel Mogenet, head of Google Research Europe:

  • “But coming up with the answer is not something we’re capable of because we cannot get to the semantic meaning of this question. This is what we would like to crack.”
  • He explained that Google needs to try and build a model of the world so that computers know things like …
  • “I’ll be honest with you, I believe that solving language is equivalent to solving general artificial intelligence. I don’t think one goes without the other. But it’s a different angle of attack. I think we’re going to push towards general AI from a different direction.”

Microsoft CEO says artificial intelligence is the ‘ultimate breakthrough’
Satya Nadella, Microsoft CEO

“We should not claim that artificial general intelligence is just around the corner,”
“We shouldn’t over-hype it.”
“Ultimately, the real challenge is human language understanding – that still doesn’t exist. We are not even close to it…”

“The Real Trouble With Cognitive Computing”
 Jerome Pesenti, former vice president of the Watson team at IBM.

“When it comes to neural networks, we don’t entirely know how they work, and what’s amazing is that we’re starting to build systems we can’t fully understand.  The math and the behavior are becoming very complex and my suspicion is that as we create these networks that are ever larger and keep throwing computing power to it, …. (it) creates some interesting methodological problems.”
Read More

Calm down, Elon. Deep learning won’t make AI generally intelligent
Mark Bishop, professor of cognitive computing and a researcher at the Tungsten Centre for Intelligent Data Analytics (TCIDA) at Goldsmiths, University of London:

It’s this lack of understanding of the real world that means AI is more artificial idiot than artificial intelligence. It means that the chances of building artificial general intelligence is quite low, because it’s so difficult for computers to truly comprehend knowledge, Bishop told The Register.

The Dark Secret at the heart of AI.
Joel Dudley leads the Mount Sinai AI team.

“We can build these models,” Dudley says ruefully, “but we don’t know how they work.”

Creative Blocks, Aeon Magazine
David Deutsch, quantum computation physicist at the University of Oxford:

“Expecting to create an AGI without first understanding how it works is like expecting skyscrapers to fly if we build them tall enough.”
“No Jeopardy answer will ever be published in a journal of new discoveries.”
“What is needed is nothing less than a breakthrough in philosophy, a new epistemological theory…”

How We Got Here

How We Got Here

by | Jul 28, 2017 | sapiens

The Road To Sapiens

The body of epistemological theory and insights that have found practical application in Compact Knowledge Models is the result of over forty years of focused interest and study by New Sapience founder, Bryant Cruse. He first formulated his epistemological theories as an undergraduate at St. Johns’ College in Annapolis in 1972, inspired by the works of Plato, Aristotle, Locke, Hobbes, Descartes, and Kant, as well as the existentialists of the 19th century.

Epistemology is generally considered an obscure and esoteric branch of philosophy of interest only to academics who, traditionally, have been focused on the debate about the truth, belief, and justification of individual assertions. Cruse’s theories, which approach knowledge as an integrated model designed from the standpoint of utility, represent a clear departure from classic epistemological traditions, and his career focus has been oriented toward practical applications rather than academic publications.

As a space systems engineer on the operations team for the Hubble Space Telescope in the mid-1980s, Cruse became interested in finding a way to automate the analysis of massive amounts of telemetry data in the ground system computers. This began a path that led him to a residency in AI at the Lockheed Palo Alto research labs where he became the driving force behind the development of the first real-time expert system shell.

Rule-based systems proved not to be a practical solution for representing human knowledge, and in 1991 he led a team that succeeded in developing a much more efficient methodology by which engineers could specify their knowledge of space systems as a model that could be directly imported into a computer program. This comprehensively solved the telemetry analysis problem and provided another critical piece of the AGI puzzle: how to efficiently put human knowledge acquired through introspection into a computer program.

Putting a detailed knowledge model of the spacecraft in the software made it possible to accurately predict the behavior of the vehicle based on thousands of telemetry measurements with very simple processing algorithms. Mission control systems based on this approach were run on PCs when traditional ones were using mainframes.

Cruse noted at this point that there is an inverse relationship with respect to sophistication in algorithms and the data structures they process. You can solve problems using unstructured data if you have very sophisticated algorithms or you can solve problems using simple algorithms if you have sophisticated information structures.

In 2005, Cruse began the process of applying his practical experience in building computer knowledge models to the body of his epistemological theory, formalizing a model of “meta-knowledge” or “knowledge about knowledge.” This “epistemological kernel” is a set of organizing principles revealing that the conceptual building blocks of all knowledge come in a number of types which determine how any concepts can be combined to create arbitrary more sophisticated ones to represent sense rather than non-sense. This is highly analogous to the way types of atoms can only be combined in fixed ways to create different materials. The epistemological kernel is to knowledge representation of what the Periodic Table of the Elements is to Chemistry. It is the key to making

New Sapience was founded in 2014 after nearly ten years of R&D to develop a software technology based on the epistemological kernel, Compact Knowledge Models. Today we have a software platform, MIKOS, ready to build the world’s first practical, scalable knowledge-based systems. We now have software that can converse in everyday English and learn the meaning of new words via unscripted question-and-answer dialog or through inference based on the meaning of known words. Natural language comprehension is based on our Cognitive Core, a compact knowledge model that may be the ultimate in sophisticated data structures.

Our success has been dependent on a unique blend of formal education, philosophic inquiry, and practical engineering experience which has turned out to be exactly the right one to solve the problem of endowing computers with the capability to process human knowledge. As it has turned out, the solution to language comprehension and ultimately Artificial General Intelligence could not be found within the boundaries of computer science, it required insights grounded in a much broader perspective. Computer science provides the means to implement these insights.

Of all the pieces of the puzzle, it may be the practical experience, often missing from the labs in academia and the large government or corporate research organizations that has been crucial to our success. We asked the right question. Not, how do we build an artificial human brain? But, how can we put information into a computer and process it not as data, but as knowledge?

Knowledge and Intelligence

Knowledge and Intelligence

by | Jul 10, 2017 | AGI, AI, Foundations, sapiens

Understanding Intelligence

Alan Turing, in his 1950 paper “Computing Machinery and Intelligence,” proposed the following question: “Can machines do what we (as thinking entities) can do?” To answer it, he described his now famous test in which a human judge engages in a natural language conversation via a text interface with one human and one machine, each of which tries to appear human; if the judge cannot reliably tell which is which, then the machine is said to pass the test. The Turing Test bounds the domain of intelligence without defining what it is. We recognize intelligence by its results.

John McCarthy, who coined the term Artificial Intelligence in 1955, defined it as “the science and engineering of making intelligent machines.” A very straightforward definition, yet few terms have been more obfuscated by hype and extravagant claims, imbued with both hope and dread, or denounced as fantasy.

Over the succeeding decades, the term has been loosely applied and is now often used to refer to software that does not by anyone’s definition enable machines to “do what we (as thinking entities) can do.” The process by which this has come about is no mystery. A researcher formulates a theory about what intelligence or one of its key components is and attempts to implement it in software. “Humans are intelligent because we can employ logic” and so rule-based inference engines are developed. “We are intelligent because our brains are composed of neural networks” and so software neural networks are developed. “We are intelligent because we can reason even under uncertainly” and so programs implementing Bayesian statistics are created.

It doesn’t matter that none of these approaches ever got even to first base at passing the Turing Test, the term Artificial Intelligence was applied to them in the first place, and it stuck. Thus, the field of Artificial Intelligence has come into existence and still, the core concept of “intelligence” itself remains vague and nebulous, people have an intuitive notion that it is about consciousness, self-awareness, and autonomy. As it turns out, these intuitions, as with many such (heavier things fall faster than light ones and giant rocks don’t fall out of the sky), are wrong.

Going back to Turing, the cogency of his test rests upon the fact that we recognize intelligence when we see it or its results. We know when someone understands what we say to them and that is undeniable proof that intelligence is at work. Let’s step back and examine the process of language comprehension.

One person composes a message and sends it to another who processes it. We often talk about the meaning of words but, of course, they have no inherent meaning, they are randomly chosen symbols assigned to represent things in the world around us, or more properly, the ideas that exist in our minds of those things. The grammar of the message and the form of the words in it encode instructions on how the receiving party should make connections between the ideas corresponding to the words to recreate, or at least approximate, in the receiving mind, the meaning that the sending mind wished to convey.

Different languages have completely different vocabularies (word-symbol sets) and grammar varies greatly as well, but people can learn each other’s languages and translation is possible because humans all live in the same world and have corresponding ideas of the things that are experienced in it. Thus, any communication using symbols is dependent on corresponding sets of common referents for those symbols. These sets of common referents are our ideas and our knowledge about the world. Knowledge is not just a bag of random thoughts but an intricately connected structure that reflects many aspects of the external world that it evolved to comprehend, it is in fact a model, an internal model of the external world.

People’s world models do vary greatly but the momentous invention/discovery of language has endowed human beings with a means to transmit ideas from one to another, expanding, deepening, correcting, and enriching one another’s models.

It has often been suggested that humans cannot think without language and that language and intelligence are one and the same. It is certainly true that without language humans could not accumulate the knowledge that has resulted in civilization. Without our languages, humans would still be living in nature and it would even be harder than it is to define the essential difference between Homo sapiens and other closely related species.

It is likely that the capacity to construct ordered world models and language both depend upon a capability for symbolic processing, and it is that which lies at the root of the essential difference of humanity, but they are very distinct processes, and the model is a prerequisite for language and not the other way around. A human living alone separated from its kind without the opportunity to learn a language (the classic “raised by wolves” scenario) will still make alterations to its environment that would be impossible if it could not first have imagined them. Imagination too, requires the model as a necessary prerequisite.

That fact suggests a thought experiment that further clarifies the essence of intelligence. Imagine traveling through space and discovering a planet with structures on the surface created by an alien life form. Can we tell whether the species was intelligent by looking at the structures? If we see patterns in the structure that repeat in a way that indicates an algorithm at work then, no matter how monumental the structures, they are probably akin to beehives, ant hills, beaver dams, and bird nests, the result of instinctive repetitive behaviors. But if they reflect an individuality clearly based on considerations of utility, environment, and future events, they had to be imagined before they could be constructed and that means the builders had an internal world model. It could be a vastly alien intelligence that evolved under conditions so different from the world our model evolved to represent that we could never really communicate, but the essence of intelligence is the same and is unmistakable.

For now, the only example we have of an intelligent entity is ourselves and it is difficult to abstract the essence of intelligence from the experience of being an intelligent being. What about consciousness, self-awareness, and autonomy? Like intelligence, these things are essential to being a human being but are they one and the same?

The answer is that they may support one another but they are not the same. All these characteristics are “off-line” in a hypnotized person, yet that person can still process language, still access and update their world model. Consciousness is the experience of processing real-time data – other animals do that as well and we do not say they are intelligent because of it. Self-awareness is an examination of an entity’s model of itself. As with language and imagination, the capacity to build an internal model is a prerequisite for self-awareness but not the other way around.

Inseparable from consciousness and self-awareness, humans also experience desires and motivations. One motivation that seems particularly inseparable from the experience of being an intelligent being is the desire to use that intelligence to change the world. The perception that intelligence and the desire to employ it to alter the world are connected in human beings is correct.

Humans evolved intelligence, manual dexterity (hands), and the desire to use the first two to change and control their environment, in parallel. All three are a matched set that has everything to do with being a human being (intelligence and hands won’t help you survive if you don’t have the desire to use them) but they are three separate phenomena.

The association between intelligence and the motivation to alter our environment in humans has led to a common misconception about artificial intelligence. If it is possible to build an artificial intelligence that is as intelligent as a human, it will probably be possible to build one that is more intelligent than a human and such a superior AI could then build another superior to itself and so on. This is the so-called “singularity” popularized by Ray Kurzweil and others. People naturally feel that a vastly superior and powerful artificial intelligence must necessarily experience the desire to exercise its power, a fearful (for humans) prospect. But giving a robot hands does not automatically endow it with a desire to pick things up and throw them around and neither does endowing it with intelligence. Whatever motivations robots are given will be by the design of their builders and not some spontaneous or unintended result of its intelligence which its builders also give it.

It is now possible to define what intelligence is, not human intelligence, not alien intelligence, not artificial intelligence but the thing itself with some clarity and we can do this without the usual appeals to how humans experience the phenomena of being intelligent beings.

Intelligence is the process through which a computational information processor creates a model within processor memory of external phenomena (knowledge) of sufficient fidelity to predict the behavior of and/or control those phenomena.

The definition has both a qualitative and quantitative aspect. It is in the quantitative aspect that we can precisely relate the ancillary functions of language, imagination, consciousness, and self-awareness that are so closely related to it yet remain distinct from the core concept of model building. How much fidelity does the model have to have, how well must it predict, and how much control over the external world must it enable before we can properly call the system that processes it intelligent?

Albert Einstein is very property called a genius. Why? He created a model of the world, of the entire space-time continuum in fact, of such unprecedented fidelity that humankind’s ability to predict and control the external world was moved forward by a quantum leap. Now, what if Einstein had been the hypothetical “raised by wolves” person described earlier, without language or culture to support the development of his internal model? Maybe he would have invented a better way to chip flint but who could he tell about it?

Functional intelligence requires accumulated knowledge and that requires language. The capacity to monitor and predict the behavior of our external environment requires real-time analysis of incoming information that can be related to our stored model and that requires consciousness. Control of our environment requires that we are self-motivated and that is autonomy which implies goals that need to be pursued in order to maximize internal utility functions, a process that is meaningless without self-consciousness.

Will artificial intelligence have all these characteristics? Yes, but they won’t spontaneously appear as the result of reaching some level of complexity or a by-product of some master yet undiscovered algorithm. Each will have to be purposely designed and integrated into the core processes that support its intelligence; and the capacity to build a high-fidelity world model

Knowledge and Language

Knowledge and Language

by | Jul 10, 2017 | Foundations, sapiens

In Humans

True or False: “The Library of Congress as a great repository of knowledge.”

Who would not answer, true, without hesitation? But consider the following thought demonstration:
Suppose Socrates[i] told you he saw a cisticola while on a trip to Africa and you asked what that might be.

He answered: “A cisticola is a very small bird that eats insects.”

In an instant you know that cisticolas, have beaks, wings, and feathers, almost certainly can fly, that they have internal organs, that they have mass and hundreds of other properties that were not contained in the sentence.

Let us step through the articulation process that Socrates when through to create the specification for the creation of this new knowledge.

First, he decomposed the concept denoted by the word “cisticola” in his mind into components concepts and selected certain ones that he guesses already exist in your mind.  The key one is “bird” because if you classify cisticolas as birds you will assign them all the properties common to all birds as well as all the essential properties and attributes of animals, organisms and physical objects; a large body of knowledge.

Second, he selected concepts that will be useful for you to distinguish cisticolas from other birds, their size is very small comparatively and they are insectivorous.
He now has a parts list for the new concept to be constructed in your mind from items that you already have in your cognitive warehouse: bird, very, small, eating, insects.

Third, he provided the assembly instructions: he choose some connective words; “is,” “a,” “that” and arranged all the words in a specific grammatical order.

Thus, natural language is a communications protocol between two entities that have pre-existing knowledge. Each sentence is an instruction, not different in essence from a line of code in a computer program.

 

 

 

 

A cisticola: A picture is worth a thousand words

The process of composing the instruction by the sender is called articulation. The receiver interprets the instruction to assemble a new concept in accordance with the specification expressed by the grammar from pre-existing conceptual components. This process is called language comprehension.

All the books in the Library of Congress are records of such instructions which can only be converted to knowledge when read by someone whose pre-existing knowledge is sufficiently compatible with that of the authors. Therefore, inescapably, we must conclude that language is information not knowledge and it does not contain knowledge.

In Machines

Today programs that use human natural language in their inputs or outputs are becoming increasing popular.

products like Amazon Alexa and chatbots are commonly referred to as AIs, but they’re just sophisticated programs built on top of machine learning, natural language processing, and statistical algorithms.

Luis Perez-Breva, Head MIT’s Innovation Program

The biggest misconception that arises is that a chatbot is a bot that converses with a human in the way that another human would converse with a human…, this is simply not possible using the current technology.

https://botpress.io/learn/what-why

The “current technology” referred to in the second quotation is specified in the first: machine learning, natural language processing, and statistical algorithms.

To converse like a human, you must able to articulate and comprehend like humans as described above. Both processes depend on a pre-existing “warehouse” of concepts containing component concepts that can be assembled to create new knowledge. Such a warehouse is comprised of knowledge and is essentially a model of the world or the domain of comprehension. None of those techniques employed by chatbots possess such a “warehouse.” The very idea of a modelling knowledge in software was abandoned by the “AI” community decades ago is being too difficult.

New Sapience has succeeded in creating such a model in a machine and our software implements the processes of comprehension and articulation in the same step-by-step sequences described above. For the first time a computer program can converse with a human in the way that another human would converse with a human.  This opens a broad new market of powerful applications forever, beyond the reach of chatbot technology.

 

[i] Compare this thought demonstration with the one in the Platonic dialog Meno, where Socrates attempts to prove that all learning is memory. Through a conversation, he leads a boy to comprehend the Pythagorean theorem. We assert that the boy did not have knowledge of the theorem in the first place, but he possessed the component concepts needed to create it and Socrates provided the instructions needed to put them together.

The New Sapience Thesis

The New Sapience Thesis

by | Jul 7, 2017 | AGI, AI, sapiens

Knowledge And Intelligence

Artificial Intelligence has been considered the “holy grail” of computer science since the dawn of computing, though these days when all kinds of programs are grouped loosely together under the term “AI” it is necessary to say “real AI” or “Artificial General Intelligence” to indicate we are talking about intelligence in the same sense as human intelligence. Humans are intelligent animals. It is that one attribute, that humans possess in so much greater degree than any other known animal that it defines us.

We define ourselves by our intelligence and the experience of being thinking entities. But who knows what is going on in the minds of other creatures? Pilot whales not only have larger brains than humans but their neo-cortex, thought to be the seat of intelligence in humans, is also larger. What is truly unique about humans is the end product of our cognitive processes: knowledge. It is knowledge of the world which allows us to evaluate how different courses of action lead to different results, that has made our species masters of our world.

It takes but a moment of reflection to realize that, since the reason we build machines is to amplify our power in the world, the real goal of intelligent machines is not “thinking” in the information processing sense, computers can already reason, remember and analyze patterns superbly – in that sense they are already intelligent but – they are ignorant. Imagine if Einstein lived in Cro-Magnon times. What intellectual achievements could he have made with so little knowledge of the world to build on? It is the acquisition and comprehension of knowledge or more specifically knowledge of the world that extends and amplifies human capabilities that is the true holy grail of computing.

Knowledge and Language

When human children reach a certain point in their development, that point when they have “learned to talk,” they are ready for the next developmental step in acquiring knowledge of the world. Knowledge is built upon knowledge and when children have acquired that critical mass of knowledge sufficient to serve as a foundation for all that comes after, we teach them to read and send them off to school. It is estimated that “first graders” have a vocabulary of about 2500 words.

That vocabulary, or rather the mental cognitions that the words relate to, represents a “knowledge bootstrap program” sufficient to permit acquiring new knowledge (assuming it is presented layer by foundational layer) of arbitrary quantity and complexity through natural language. But this bootstrap capability is far more than a vocabulary sufficient for “looking up” or being told the meaning of additional words.

The vocabulary of the average college graduate is estimated to be around 30,000 words. Only a tiny fraction of the ideas these words relate to were acquired by looking them up in dictionaries or through direct pedagogic instruction. They are unconsciously “picked up” in the context of reading and conversing.

The human brain is a vast network of interconnected neurons, and so too are the information-processing organs of vast numbers of other animals. Today artificial neural networks are demonstrating some of the low-level capabilities of animal brains such as auditory discrimination and image recognition that are ubiquitous throughout the animal kingdom.

These programs, with a kind of heroic optimism, are collectively termed “Cognitive Computing” on the basis of nothing more than that the programs have processing elements fashioned in imitation of biological neurons. The programs certainly have nothing resembling actual cognition or knowledge. In any case, it is a long, long way from low-level training of an artificial neural network to the cognitive power to create predictive internal models of the external world that a human first grader possesses.

This may not be self-evident, especially in light of how egregiously these programs can be hyped by the media and/or their creators who often talk very loosely about what is going on inside. Because a program can respond to a range of inputs with outputs that a human would recognize as a correct answer in no way justifies asserting the program “comprehended” the question or “knew” the answer.

The confusion arises from the very understandable misconception that language contains knowledge. It does not. Language is information, not knowledge. It is a specification for the recreation of an idea in the mind of the sender (human or machine) from component ideas that already exist in the mind of the receiver. Read more about knowledge and language.

This is the great fallacy of using stochastic programs like neural networks to “mine” text databases. They will never understand what is in the records because they are not reading the text. They cannot because they have no pre-existing internal knowledge to refer to the words and decode the grammar against them.

We understand that the human brain becomes furnished with a critical mass of building block concepts during childhood. The internal biological processes that are responsible for this build-out remain a mystery. The brain is a product of an incredibly complex and unique evolutionary process. Because we understand how neurons work at a base level doesn’t tell us what is going on thousands of processing layers above, any more than understanding why a light bulb illuminates when you connect it to an electric circuit throws much light onto what goes on inside a microprocessor.

We understand what goes on inside a micro-processor because they are products of our own knowledge. Modern object-oriented software enables us to create data structures in computer memory that correspond to concepts in the human mind.

It is far easier to endow computers with a “knowledge boot-strap” program commensurate with a human first grader than to build an artificial human brain that can create knowledge by means of neural processing.