Random thoughts about strong A.I.

The Singularity is Near

Tag: Artificial Intelligence

Human Brain Microchip Is 9,000 Times Faster Than a PC

neurogrid_news

An interesting announcement from Stanford. Stanford bioengineers have developed a new circuit board modeled on the human brain they call the “Neurogrid”. One popular branch of A.I. is the thought that if you can model the Human brain by modeling our nerve cells. This is a bottom up approach to the problem set (and a popular one too). In this announcement, the Stanford team have developed the Neurogrid, a circuit board consisting of 16 custom-designed “Neurocore” chips. Together these 16 chips can simulate 1 million neurons and billions of synaptic connections. In comparison, that 3-lb. (2.3-kg) hunk of matter in our skulls has roughly 80 billion neurons.

Now, for a little bad news: to use this approach, “you have to know how the brain works to program one of these” (quoted from the link below). Herein lies the fundamental problem. Having lots of nerves (whether software or hardware based) does not get you a brain. You still need to program it.

Despite this current limitation, our congratulations goes out to the Stanford team in a significant achievement.

Read more about it from this link to Stanford.

Adjectives

powerful-adjectives

Adjectives are words that are used to describe nouns and pronouns. In a nutshell, Adjectives are what define nouns and give them characteristics to differentiate them from other nouns. For example:

He was wearing a blue shirt.

Here ‘blue’ is an adjective as it is describing the noun ‘shirt’ by answering the question ‘what kind of shirt?’

There are seven rooms in the house.

Here ‘Seven’ is also an adjective as it’s telling the quantity/the number of the noun ‘rooms’, answering the question ‘how many rooms?’.

One of the most difficult problems for a Strong A.I. engine is adjectives.  Adjectives modify a noun using one of the Human senses.  As in the blue shirt example above, the color blue of course comes from our ability to see.  But our A.I. engine is blind.  It can remember that a shirt is blue, but never really know what blue is.

Similarly, the A.I. engine is deaf, has no sense of smell, taste, no feeling or sense of touch (the five traditional senses of Humans). In addition, Humans have other “senses” that the A.I can’t experience, such as temperature, balance, acceleration, time, size and pain and happiness. These at least have good internal models and mathematics to accurately describe the sensation. The other traditional senses though are much more difficult to model, but is it important? After all, a blind person doesn’t know about the color blue either and can function without true understanding. The same is true of a deaf person. These senses are not a requirement for intelligence. But because Humans do have these senses, English is littered with adjectives, so we must wade through all the various types. More than likely, A.I. engines will be able to approximate these senses in the future by adding hardware sensors to the computer such as digital cameras, etc. However, as discussed herein, the lack of true understanding of these senses doesn’t not seem to inhibit intelligence, so we will not dwell on these too much in the A.I. engine.

The rest of this post is a discussion of English adjectives.

There are different types of adjectives based upon their effect on a noun and what do they tell about the noun. There are five categories of adjectives

1. Adjectives of Quality – These adjectives are used to describe the nature of a noun. They give an idea about the characteristics of the noun by answering the question ‘what kind’.

Honest, Kind, Large, Bulky, Beautiful, Ugly etc.

New Delhi is a large city with many historical monuments.

Sheila is a beautiful woman.

2. Adjectives of Quantity – These adjectives help to show the amount or the approximate amount of the noun or pronoun. These adjectives do not provide exact numbers; rather they tell us the amount of the noun in relative or whole terms.

All, Half, Many, Few, Little, No, Enough, Great etc.

They have finished most of the rice.

Many people came to visit the fair.

3. Adjectives of Number – These adjectives are used to show the number of nouns and their place in an order. There are three different sections within adjectives of number; they are –

Definite Numeral Adjective – Those which clearly denote an exact number of nouns or the order of the noun.

One, Two, Twenty, Thirty-Three etc. also known as Cardinals.

First, Second, Third, Seventh etc. also known as Ordinals.

Indefinite Numeral Adjective – Those adjectives that do not give an exact numerical amount but just give a general idea of the amount.

Some, Many, Few, Any, Several, All etc.

E.g.: There were many people present at the meeting.

Distributive Numeral Adjective -Those adjectives that are used to refer to individual nouns within the whole amount.

Either, Neither, Each, Another, Other etc.

Taxes have to be paid by every employed citizen.

4. Demonstrative Adjectives – These adjectives are used to point out or indicate a particular noun or pronoun using the adjectives – This, That, These and Those.

That bag belongs to Neil.

Try using this paintbrush in art class.

I really like those shoes.

These flowers are lovely.

5. Interrogative Adjectives – These adjectives are used to ask questions about nouns or in relation to nouns, they are – Where, What, Which and Whose.

Where did he say he was going?

What assignment did I miss out on?

Which is your favorite author?

Whose pen is this?

In some instances, we find that we need to use more than one adjective to describe a noun in a satisfactory manner. In these cases, commas are used to separate the adjectives but some series of adjectives do not require a comma. Therefore, we need to know the difference between Coordinate and Non-coordinate Adjectives –

Coordinate Adjectives – Are those words which can be re-arranged in the series easily and are still grammatically sound. This kind of series makes use of commas. This series can also insert ‘and’ between them and still be correct.

She was a kind, generous, loving human being.

She was a generous, loving, kind human being.

She was a loving, kind and generous human being.

Here we can see that all three sentences are grammatically correct. In this case, the adjectives only need to be separated by commas.

Non-coordinate Adjectives – These are those adjectives which cannot be rearranged in the series. These do not use commas to separate the adjectives. Also, this kind of series do not make sense if we insert ‘and’ between them.

She has two energetic playful dogs.

She has playful two energetic dogs.

She has energetic and playful and two dogs.

Here we see that only the first sentence makes sense and is grammatically correct. The second and third ones are incorrect. Hence, the sentence uses non-coordinate adjectives and does not need commas.

There are certain rules regarding the placement of different kinds of adjectives in a sentence. The general order followed is –

1. Determiners – These are the various articles (the, a, an), demonstratives (this, that, these, those), possessives (my, mine, your, yours, -‘s), quantifiers (all, many etc.), numerals (one, twenty, thirty-seven etc.) and distributives (each, every, neither, either)

2. Observations/Quantity and Opinion – Then come the adjectives that give a quantity (also known as post-determiners) and subjective opinion to the noun, telling ‘how much’ and ‘how was’ the noun.

Few, Most, One, Three/ Beautiful, Ugly, Difficult etc.

The beautiful house.

3. Size – The position after Observations is for the adjectives that tell about the size of the noun, they can be used for an object as well as living thing.

Huge, Little, Bulky, Thin, Vast, Tiny, Lean etc

The beautiful little house.

4. Age -Then is the turn of the Adjectives that tell about the age of a noun either by itself or in relation to another noun.

Young, Old, Teenage, Mature, Recent, Bygone etc.

The beautiful little old house.

5. Shape – Next are the adjectives that tell about the shape or appearance of the noun.

Circular, Crooked, Triangular, Oval, Wavy, Straights etc.

The beautiful little old square house.

6. Color – After that are the adjectives that tell the shade and hue of a noun.

Pastel, Red, Blue, Metallic, Colorless, Translucent etc.

The beautiful square blue colored house.

7. Origin – Next are the adjectives that show the different geographical locations associated with a noun.

Southern, Northern, Lunar, Mexican, French etc.

The beautiful blue colored Mexican house.

8. Material – Next are the adjectives that talk about the raw material or texture of the objects or the behavior of the living nouns.

Wooden, Plastic, Steely, Metallic, Cottony etc.

The beautiful Mexican limestone house.

9. Qualifier – Lastly, the qualifier or the grammatical modifier comes, which is an additional word or phrase provided to change the meaning of the noun in a sentence.

Pink + eye, Royal + treatment, Hot + fudge etc.

The beautiful Mexican limestone doll house.

Methods of Inference or Reasoning

I have created several video games over the years.  In all my games, they shared a common architecture: the game is implemented as a big while loop.  At the top of the loop, it checks for input from the gamer and then performs it’s own internal logic to move game pieces, etc.  This is where the A.I. in video games run.  Even if the user isn’t doing something, the game itself is working on how to beat the gamer.

Likewise in the A.I. engine, the most interesting part of the engine is not when it get’s input from the user, but what it does on it’s own cycles, un-obstructed from medaling humans.   So doing this period, the engine is making inferences based on the information it has at hand.  Here is our list of different ways the kinds of inference our A.I. uses:

  1. Deduction – a premise, a conclusion and a rule that the former implies the latter
  2. Induction – inference is from the specific case to the general
  3. Intuition – no proven theory
  4. Heuristics – rules of thumb based on experience
  5. Generate and test – trial and error
  6. Abduction – reasoning back from a true condition to the premises that may have caused the condition
  7. Default – absence of specific knowledge
  8. Autoepistemic – self-knowledge
  9. Nonmonotonic – previous knowledge
  10. Analogy – inferring conclusions based on similarities with other situations

We can group multiple types of inferences to connect a problem to a solution.

Parsing English: Step 1-Penn Treebank Tagging

Sentance Diagram

The first step in the A.I engine is to move from English to a form the computer can read. This means converting an English sentence to a data structure a program can use.

The A.I. engine now converts an English sentence into a tree based data structure using the Penn Treebank Tagging rules. Similar to the how you did sentence diagramming in grammar school, the A.I. engine takes apart the sentence and group words together in a tree structure.

This first step doesn’t mean the A.I. understands the sentence. However, it now can start to parse the sentence to determine its meaning.

The tag set is based on the Penn Treebank Tagging Guidelines [pdf].

Part-of-speech tags

Part-of-speech tags are assigned to a single word according to its role in the sentence. Traditional grammar classifies words based on eight parts of speech: the verb (VB), the noun (NN), the pronoun (PR+DT), the adjective (JJ), the adverb (RB), the preposition (IN), the conjunction (CC), and the interjection (UH).

Tag Description Example
CC conjunction, coordinating and, or, but
CD cardinal number five, three, 13%
DT determiner the, a, these
EX existential there there were six boys
FW foreign word mais
IN conjunction, subordinating or preposition of, on, before, unless
JJ adjective nice, easy
JJR adjective, comparative nicer, easier
JJS adjective, superlative nicest, easiest
LS list item marker  
MD verb, modal auxillary may, should
NN noun, singular or mass tiger, chair, laughter
NNS noun, plural tigers, chairs, insects
NNP noun, proper singular Germany, God, Alice
NNPS noun, proper plural we met two Christmases ago
PDT predeterminer both his children
POS possessive ending ‘s
PRP pronoun, personal me, you, it
PRP$ pronoun, possessive my, your, our
RB adverb extremely, loudly, hard
RBR adverb, comparative better
RBS adverb, superlative best
RP adverb, particle about, off, up
SYM symbol %
TO infinitival to what to do?
UH interjection oh, oops, gosh
VB verb, base form think
VBZ verb, 3rd person singular present she thinks
VBP verb, non-3rd person singular present I think
VBD verb, past tense they thought
VBN verb, past participle a sunken ship
VBG verb, gerund or present participle thinking is fun
WDT wh-determiner which, whatever, whichever
WP wh-pronoun, personal what, who, whom
WP$ wh-pronoun, possessive whose, whosever
WRB wh-adverb where, when
. punctuation mark, sentence closer .;?*
, punctuation mark, comma ,
: punctuation mark, colon :
( contextual separator, left paren (
) contextual separator, right paren )

Chunk tags

Chunk tags are assigned to groups of words that belong together (i.e. phrases). The most common phrases are the noun phrase (NP, for example the black cat) and the verb phrase (VP, for example is purring).

Tag Description Words Example %
NP noun phrase  DT+RB+JJ+NN + PR the strange bird  51
PP prepositional phrase TO+IN in between  19
VP  verb phrase RB+MD+VB  was looking
9
ADVP adverb phrase RB also
 6
ADJP adjective phrase  CC+RB+JJ warm and cosy  3
SBAR subordinating conjunction IN whether or not
3
PRT particle RP up the stairs  1
INTJ interjection UH hello
 0

The IOB prefix marks whether a word is inside or outside of a chunk.

Tag Description
I- inside the chunk
B- inside the chunk, preceding word is part of a different chunk
O not part of a chunk

A prepositional noun phrase (PNP) is a group of chunks starting with a preposition (PP) followed by noun phrases (NP), for example: under the table.

Tag Description Chunks Example
PNP prepositional noun phrase PP+NP as of today

Relation tags

Relations tags describe the relation between different chunks, and clarify the role of a chunk in that relation. The most common roles in a sentence are SBJ (subject noun phrase) and OBJ (object noun phrase). They link NP to VP chunks. The subject of a sentence is the person, thing, place or idea that is doing or being something. The object of a sentence is the person/thing affected by the action.

Tag Description Chunks Example %
-SBJ sentence subject NP the cat sat on the mat
35
-OBJ sentence object NP+SBAR the cat grabs the fish
27
-PRD predicate PP+NP+ADJP the cat feels warm and fuzzy
7
-TMP temporal PP+NP+ADVP arrive at noon
7
-CLR closely related PP+NP+ADVP work as a researcher
6
-LOC location PP  live in Belgium
4
-DIR  direction PP walk towards the door
3
-EXT extent PP+NP drop 10 %
1
-PRP purpose PP+SBAR die as a result of
1

Anchor tags

Anchor tags describe how prepositional noun phrases (PNP) are attached to other chunks in the sentence. For example, in the sentence, I eat pizza with a fork, the anchor of with a fork is eat because it answers the question: “In what way do I eat?”

Tag Description Example
A1 anchor chunks that corresponds to P1 eat with a fork
P1 PNP that corresponds to A1 eat with a fork

The Three Laws of Robotics

Isaac AsimovThe Three Laws of Robotics (often shortened to The Three Laws or Three Laws) are a set of rules devised by the science fiction author Isaac Asimov.  The rules were introduced in his 1942 short story “Runaround”, although they had been foreshadowed in a few earlier stories.

The Three Laws are:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Later, a fourth law was added: “A robot may not harm humanity, or, by inaction, allow humanity to come to harm.”

The project team has decided, at the earliest possible moment, to add the three laws into the A.I. engine.  The problem is you have to make the A.I. engine smart enough to understand the three laws and that’s likely still a ways off.

Stephen Hawking freaks out about Artificial Intelligence

Stephen Hawking, certified genius, is freaking out about our Skynet future. In an article for The Independent, the theoretical physicist and author of “A Brief History of Time” warns that the development of real artificial intelligence could be potentially our worst mistake in history.

“One can imagine such technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand. Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all.”

Read more about it using this link to the The Independent.

Day 1 of the Development of a New A.I. Engine.

RobotFirst a little bit about the people who are working on the project. My name is Peter Chapman.  I am a software engineer.  I fell in love with programming at a very early age and at a time when the computer industry was just starting to take off.

I started coding at 16 at the M.I.T. Artificial Intelligence laboratory.  The lab was run by Marvin Minsky, (the father of A.I.) who my Dad had known.  I worked on two projects: Working on the logo language itself and the other to write logo programs that would learn a maze (using floor turtles).  As part of my work, I published my first paper “Of Mice and Men” about the program that learned mazes.  The reference to Mice, was for the floor turtles, not the pointing devices of the same name that were still a number of years off before they came to market.

What became clear to me at the time was that computers didn’t have the power to implement strong A.I. and was still many years away. So, I left the field of A.I. to take my first job working on Wall Street and wrote the world’s first computerized trading system.

My goal had been to retire at forty. I had set this goal when I was 14. I have known that I wanted to be entrepreneur and have my own company. At thirty-nine, I sold my company to Thomson Financial and retired (beating my goal by only a couple of months). After several years of various non-computer related projects, I decided to build a vacation house on the shores of Lake Winnipesaukee. I built the house with my two sons from the foundation up (that’s another story for another time). The new house is 90 miles away from my home in Boston. Each day for two years, I would commute to work for an hour and a half (each way). I cherished these times because it gave me a block of time for  un-interrupted thought.  I started to think again about strong A.I. and how to implement it.  So the project had over two thousand hours in design work put into it over a period of two years.

I had a proposed design but it was still a daunting task ahead of me. I estimated there was about 5 years worth of work and only towards the end, would I know if it would work. Just as I was completing the house, I saw an online listing to help a little startup with their applications to help the blind. And more intriguing, the startup was run by the famed futurist, Ray Kurzweil.

Partly because the amount of work was so large, and working on something for social good rather than money, I joined the company with his band of merry misfits to work on their blindness applications.

Fast forward a couple of years and my day job has been working as a president of a portfolio company of Ray Kurzweil, who is now working on this problem for Google.

And now with the world of cloud computing, it seemed that we now finally have enough computing resources to build the first strong A.I. engine. I will have to admit, it was Ray’s transition to Google and reading his most recent book that got me thinking about A.I. again. He has had a good track record of timing advances in technology and leap frog technologies (as the inventor of speech recognition, OCR, Text to Speech, digital music, etc.).

And that leads to the question: Why now? After all, the strong A.I. landscape is littered with broken promises. The short answer is that many parts of my design of a couple of years back are now available in commercial form. In fact, had I started years ago, I think I would behind today and the quality of the resulting product would have suffered. The necessary pieces are now all available.

So, in my spare time and with the help of my two sons: Nick and Chris, we have started down this ambitious path.

Nick, a high school senior (and will be attending Georgetown in the fall) has created his own software consulting firm which has built a bunch of high profile websites. Like me, he has the “programming fever”. Chris, who is out of college has gone green in a big way and started his own farm. While not fooling around with dirt, is programming as well.

I expect over the coming months, we will add other crazy fools like ourselves to the project. We are in a space race against several giants such as Google, IBM, Microsoft, Nuance, etc., however, none of them have cracked this puzzle. And even if one does, that just means the others (and everyone else) will need this project. This is a problem too that has no obvious answer, so I am not too worried about the big guys. At this stage, I think their current space race is mostly about research, not a working product.

So, the game is afoot.