A man who tried to decipher the mysteries of all things ended up bearing a heavy cross for his own pursuit, which he carried even to the grave.

This is the cruel little joke of the universe.

It was in the winter of 1947 that a little boy named Geoffrey Hinton was born in Wimbledon, southwest of London.

Standing close to Hinton and Einstein, you can't find anything in common between them, except that both of them won the Nobel Prize in Physics.

102 years apart.

But if you retreat to the edge of the cliff and look far away, you may be surprised to see: the gears of history rotate over the heads of several generations. After 102 years of long march, are meshing together perfectly again. At the moment when the gears collided, the two of them got very close.

The handover of history, the loud noise of silence.

People raised their heads and everything was business as usual.

(1) Life and Machine

We are machines, just made in a biological way.

Hinton said.

Ordinary people actually don't care what they are, they only care about whether they are noble.

Machines are not noble.

Copernicus drove us out of the center of the universe, Darwin drove us into the plains of animals, Nietzsche declared that no god ever promised us to shepherd sheep, Camus said The most powerful life a person can live is like Sisyphus.

Hinton just used the sharp "truth" to add another knife to the already bloody self-esteem of mankind.

Geoffrey Hinton

In the past ten years, On days when it's not particularly cold, Hinton lives on his own island.

Yes, it is his island, an island dotted on the shores of Lake Huron in Canada, which perfectly fits people's romantic imagination of the "Godfather of Artificial Intelligence".

He bought the island when he was 65 years old. And on any day before the age of 65, he probably never imagined that he could "spend" so luxuriously.

Three decades before that, Hinton had only one identity: a computer science professor.

Now people naturally call what he does "artificial intelligence". But turn the clock back to the 1970s, and this was just an unpopular subject that only visionaries and lunatics were willing to devote themselves to.

Even the researcher himself feels that words like "artificial intelligence" are too gaudy and difficult to say, so he usually calls himself "machine learning".

What Hinton is involved in is another unpopular school in machine learning: neural networks.

To put it simply, computers are used to simulate the connections of hundreds of millions of neurons in the human brain, so that "intelligence" emerges.

How to "surge" specifically? Who knows.

In the Lo-Fi era when making long-distance calls requires manual wiring, computers have just begun to be miniaturized, and you can only rely on paper maps to go out, artificial neurons and the like His words sounded like he was talking in his sleep.

New York, 1970.

1970 IBM System 3 computer.

In 1971, mankind’s first large-scale integrated circuit processor Intel 4004 had just been born.

In 1972, when Hinton was studying for a PhD in "Neural Networks" at the University of Edinburgh, his supervisor personally reminded him every week: "You are wasting your time."

So much so that onlookers are curious, what kind of power does a person need to spend the best 10,000 days and nights of his life in such a fantasy? superior?

That power may come from a "tree".

His family tree.

Hinton’s maternal great-great-grandfather was George Boole, who invented Boolean algebra and laid the mathematical foundation for computers;

George Boole

1815-1864

Boore's wife's uncle was a geographer, and his surname Everest named Mount Everest;

George Everest

1790-1866

One ​​of Bull's daughters, Hinton's great-aunt, was Etel Lillian Voynich, the author of "The Gadfly"; (Voynich was her husband's surname that she changed after her marriage)< /p>

Ethel Lilian Voynich

1864-1960

Hinton's great-grandfather, Charles Howard Hinton, was a mathematician and fantasy writer who invented the "four-dimensional cube", which is the four-dimensional space you see in "Interstellar";< /p>

Charles Howard Hinton

1853 -1907

Hinton's cousin invented the portable X-ray machine. Hinton's cousin participated in the Manhattan Project to develop the atomic bomb; Hinton's father Howard Sheen Dayton is an entomologist and a member of the Royal Society.

Even Jeffrey Hinton's middle name is Everest, which represents Mount Everest, in order to commemorate the glory of the family.

Living in the shadow of such a fruitful family tree, a child will only feel fear - even if he becomes successful in the future, or even wins the Nobel Prize, others will Will pout, this guy is "supposed to be like this", right?

As a child, depression has been a constant presence.

His father pointed at the thin man with the hand that could only do pull-ups with one hand: "If you work twice as hard as me, you will be as old as me." At twice the time, he might be half as good as me."

His mother was much kinder and said, "There are two ways in front of you, either be a fool." Professor, or be a loser”

His way of escaping reality is to observe the world secretly.

When he was 4 years old, Hinton rode the bus with his mother. He took out a penny and placed it on the velvet seat cover. A strange phenomenon occurred:

The coin did not slide down, but moved up against gravity. .

This incident stayed in his mind for 10 years. It was not until he was a teenager that he suddenly realized: the vibration mode of the bus caused the coin to move upward. of thrust, and the fibers of the velvet seat cover are just enough to stop the coin from sliding downward.

Some people are calm when they see things they don’t understand. But I can't accept anything that violates my cognitive model of the world. I reallyCan't accept it.

Hinton later recalled.

As a biologist, Hinton's father set up a "zoo" at home. There were meerkats in the room, venomous snakes, frogs, and lizards in the pit in the garage, and a turtle he brought back from the house was soaked in the water.

Little Hinton had a lot of time to observe how a living being interacted with its environment.

8-year-old Hinton poses with a python at the zoo.

He ignorantly realized that the response pattern of life to the environment is not completely random, but follows some kind of "intuition."

This kind of intuition is neither as simple to output as a mathematical formula, nor as elusive as the "soul" promoted.

It has traces to follow.

In the 1950s, when little Hinton was squatting next to a large pit crawling with cold-blooded animals, the American psychologist Frank Rosenblatt on the other side of the ocean Take action.

He modified a huge IBM computer and simulated hundreds of neurons, hoping that it would start from recognizing letters and shapes and eventually become a life.

This is the mother machine of all subsequent "neural networks" - the perceptron.

Frank Rosenblatt

In 1958, the New York Times was shocked after interviewing the perceptron team and wrote an enthusiastic and optimistic prediction:

In a few years, Computers walk, talk, and have self-awareness!

Subsequent facts proved that the "perceptron" was a joke. It can't talk or walk, and it can't even distinguish between left and right. . .

Colleagues in the artificial intelligence community at that time looked at Rosenblatt with sympathy: Good job, don’t do it next time!

Many scholars do not doubt that life *can* be a machine, but they generally believe that creating artificial life requires "programming":

Since we humans have understood so many principles, and compiled them into "concepts and rules" and clearly listed them one by one for the computer, wouldn't it be directly intelligent?

You have to simulate neurons and let the computer build its own understanding of the world from scratch. This is a detour!

What did Hinton say?

Sorry, Hinton was not qualified to participate in this "truth" discussion at that time.

He was choking on the turbulent sea of ​​destiny.

(2) Desire but not having it

If there is any word that can sum up Hinton’s entire life In youth, that is "desiring but not getting"

Hinton is very hands-on and has loved doing carpentry since he was a child. But there is no room for a carpenter on the shining family tree.

He must be tough Scalp takes the academic path

Although he was admitted to Cambridge, there are a lot of talents here. Hinton changed his majors several times in succession. He studied physics, but his mathematics foundation was not solid enough; he studied philosophy, but it was difficult to go deep into metaphysical fantasy. In short, compared with his classmates, It’s not an enlightenment.

When respect is not provided, he must have the courage to leave the poker table and go to London to work part-time.

In Hinton's eyes, there is a deep self-denial. It is probably the painful pain that all teenagers have experienced.

The absurd thing is: a computer will never be wrong just because its calculated results are different from others.Pain, only advanced beings with "self" can experience this pain - pain is actually a by-product of intelligence.

Hinton decided to try one last major - psychology.

Through this, he met Professor Bernard Williams.

Bernard Williams

Williams was a moral philosopher who spent his life fighting one enemy: reductionism.

"Reductionism" was actually a hypothetical inference of mainstream science at that time:

< p style="text-align: left;">Everything, no matter how complex, can be broken down into "parts" with clear boundaries and specific functions.

This means that if you want to reproduce a certain system - including humans and animals - you only need to find all the parts!

A popular science wall chart of the human body based on reductionist thinking during the Republic of China: the human body is like a factory.

But in Williams' eyes, reductionism is an arrogant conceit that will only lead to complacency.

His point of view is that a system that is complex to a certain extent, especially "human morality", cannot be reduced to clear rules at all.

Our different thoughts must reflect different physical arrangements inside our brains, but this is not the same as the internal arrangement of computers The situation is completely different.

Williams said to Hinton.

Hearing this sentence, Hinton had an atomic bomb explosion in his mind.

Since you can't replicate a system by dismantling it into simple parts, what else can you do?

Yes, use one A complex system *wholly* simulates another complex system

Blade Runner (1982). )

Forcing concepts and rules into machines is classified as the "symbolism" school of artificial intelligence, which is rooted in "reductionism";

The use of artificial nervous systems to imitate the human learning process as a whole is classified as the "neural network" school, which originates from "system theory".

In the following decades, the two factions were in the same position and refused to give in. On the surface, it was a dispute over technical routes, but in fact it was a "bet" on the nature of the world.

The bet is: Is this world a bunch of neat "parts" or a "soup" that affects the whole body?

Here, we might as well clarify the fundamental difference in methodology between "symbolism" and "neural network":

In the scheme of "symbolism" Here, the smallest part is the "concept"

For example: food, sauce, condiment, sweet, flavor, red, tomato, American, French fries. , mayonnaise, mustard, these are concepts

All concepts are connected by rules to form a huge fishing net.

New concepts, such as "ketchup", can be hung on these old ones. The appropriate position in the conceptual mesh becomes the new knot.

There are endless new concepts on the fishing net. The meshes are also endless;

The old rules are not precise enough and need to be usedEndless new rules to complete.

For example: birds can fly, penguins are birds, but penguins are flightless birds.

In the "neural network" scheme, the basic elements maintained by neurons can Called "sub-concept": a concept can emerge from many sub-concepts.

This means that changes in one sub-concept will subtly affect many concepts.

For example: If I tell you that orangutans like onions, then you will definitely guess, do monkeys also like onions?

Because in your mind, although orangutans and monkeys are two concepts, they share certain "sub-concepts", such as hairiness, animals, and intelligence. , primates, feral and more.

The key here is:

Many deep sub-concepts cannot be described in words , which is essentially just a "way of combination" of neurons.

Anyone who has deeply reflected on their own thinking process will agree that "neuro "Network" is closer to the way our brains work than "symbolism" is.

However, no one stipulates that intelligence must be realized in a brain-like way. You can definitely "cut corners".

And the Creator is definitely an "anti-chicken soup person", because in many cases taking shortcuts is effective.

In the 1970s, "symbolism" had advanced by leaps and bounds and could make some decent inferences, but "neural networks" were still at the stage of mental retardation.

This is a very strong feedback. Many neural network scholars defected with hatred and joined the "reductionist" camp.

But Hinton couldn't convince himself. Just like when he was a child, he couldn't accept anything that violated his cognitive model of the world.

In 1972, he entered the University of Edinburgh to pursue a PhD, focusing on "neural networks".

If others can't find out the reason, he has to find out the reason himself.

If nothing unexpected happens, it will take longer to find the answer this time than to understand the coin climb on the bus.

In his first year as a Ph.D., Hinton saw an experiment done by another artificial intelligence group:

A computer is connected to two cameras. The system must independently control the robotic arm to build the building blocks into the shape of a car.

This was hellishly difficult given the technology at the time. Because the system vision can only recognize scattered building blocks based on their outlines, once they are stacked together, it will not recognize them.

An unforgettable moment for Hinton occurred: the robotic arm stepped back a little, and then punched the pile of blocks apart.

If someone does this, you would think he is frustrated because he "can't do it." I felt the same emotion in the robot as it punched the blocks.

Hinton said.

Having feelings is when you start to long for something you can't have.

Blade Runner (1982)

(3) Bridge

As Williams said: Different thoughts must reflect different physical arrangements within our brains.

The question before Hinton is:

"Macro-level thoughts" and "micro-level neural arrangements" are like two isolated islands separated by a turbulent and unpredictable deep sea. They need a "bridge" to connect them.

What is this bridge?

In this regard, Hinton is extremely lucky: the shoulders of countless giants are in front of him, waiting for him to climb up the stairs.

More than half a century ago, during the wave of physics launched by scientists of Einstein's generation, the "hardest heads" were developing in all directions. Knocked out some space.

In terms of elaborating the "relationship between micro and macro", the greatest founder is Ludwig Boltzmann.

Boltzmann invented an extremely simple "statistics + probability calculation" method:

As long as we know the atomic weight, charge, structure and other microscopic properties of atoms, we can calculate the physical properties of macroscopic objects composed of hundreds of millions of atoms, such as viscosity, Heat, diffusivity.

It is also using the same theoretical framework that Boltzmann explained "entropy", the underlying concept of the universe.

This is statistical mechanics.

Ludwig EduardBoltzmann

However, Boltzmann encountered fierce opposition from many scientists at the time, and even attacked him like a heretic.

An important reason is: you actually use the fuzzy methods of "statistics" and "calculation of probabilities" to explain the deterministic physical world. What does this mean? science?

The anger of the opposition scientists essentially has only three words: denial.

Does not admit that the complexity of the universe exceeds human computing power; does not admit that human beings can only grasp the world in a vague way despite their best efforts.

But the universe will not change its basic structure due to the wrath of tiny humans.

Giving up the obsession with "accuracy" is an important prerequisite for tearing through the fog and finding the bridge connecting "macro" and "micro".

But there is a problem here.

Suppose you mix inks of various colors together.

They will definitely go through a dynamic process of mixing and will eventually be completely uniform. (At this time, each molecule has the same probability of being in various possible states.)

Boltzmann's theory can only calculate the physical properties after shaking to reach a "steady state".

But the microstructure of the human brain is obviously not such a messy "final steady state".

It is very stable, but not that stable. It can be called a "meta-stable state".

As shown below:

Also composed of carbon, diamond is a metastable state, and graphite is an ultimate stable state. But due to the potential energy barrier between the two, it is difficult for diamond to *automatically* convert into graphite.

Similarly, the neuronal structure of the human brain is also in a metastable state, but it can maintain a considerable degree of stability.

Boltzmann, as a pioneer, could only give Hinton so much.

Next, the baton was handed over to another great master.

In 1982, a paper titled "Neural Networks and Physical Systems with Emergent Collective Computational Capabilities"The article shocked the entire artificial intelligence world.

And its author turned out to be a "layman" - physicist John Hopfield.

To understand Hopfield’s insight, we must first know that "minimizing freedom Energy Principle".

No matter what physical structure it is in, the system will always do its best to do external work - just like a small ball will always roll down.

After rolling to a relatively low position, the system reaches the "minimized free energy state" and achieves stability.

Now, let's imagine a bunch of magnetic atoms, which are at a specific temperature ( Curie temperature) will eventually face one direction, which is their "minimized free energy state".

This state is relatively monotonous and cannot carry complex information.

But by changing the structure between atoms through some operations, the system can eventually be stabilized in a state with atoms facing different directions - this state is already its "minimum" transformation free energy state".

This structure is called "Spin Glass".

The top of this picture shows a "spin glass" with no internal Sequence constitutes a steady state, producing the complex "energy terrain" shown below.

Hopfield's stroke of genius is:

He did not use real-world atoms to create " "Spin Glass", instead using the computer's different potentials of 0 and 1 to replace the atomic state, a "Spin Glass" was simulated in cyberspace.

It was also called the "Hopfield Network" by later generations.

This is just a schematic diagram, there are many more 0s and 1s in the real Hopfield network.

The theory is too abstract, we An analogy can be made:

If a planet is composed entirely of water, under the influence of gravity, its final stable state must be a perfect sphere. Calm and dead.

But if a planet has abundant elements, such as hydrogen, oxygen, carbon, and iron, then the structures of various properties will rub and fetter each other, and eventually stabilize into a sphere that is similar to a sphere on a macro scale, but on a micro scale The appearance of rolling hills.

The Hopfield Network is like the mountains on our planet’s surface.

Unlike the earth, the final direction of the mountains in the Hopfield network is not created by nature, but set by humans.

The setting method is "training"

For example, we use "26." Through training, the "landform" of the Hopfield network will be shaped into a specific shape and stabilized in this shape. (Because its free energy is the lowest in this shape.)

At this point, training is completed

Now it has a useful property:

Suppose we throw a small ball downward from the air, it will not stop where it is, but will eventually roll down to a relatively low position.

Due to the complex terrain, the low point where we throw the ball is also different

Then we try to take advantage of this property:

Throw small balls downward from many points at the same time, and they will eventually stay different locations.

For example, we stand above this mountain range and throw some small balls in this arrangement:

Their last position is:

Don't hold back your cheers. This is the process of an intelligent system identifying the letter "J".

Now we return to the Hohfeld network and reveal the truth:

It is a pass A computer system that simulates neurons on a microscopic level and emerges with "memory" capabilities on a macroscopic level.

Memory is a high-level storage, which is the basis of intelligence:

The world is extremely complex and open, and new things appear every day.

For example, when ancient humans met an animal, it was very likely that it was *not* exactly the same as all the animals in our memory.

But in order to survive, our ancestors had to quickly match it to the one closest to their memory in order to decide whether to attack or escape.

Hopfield network can do exactly this: classify infinitely refreshed new monsters into limited memory categories.

This is the first time in human history that a system has been built that is so close to a large category of functions of the human brain.

1982 was the "miracle year" of artificial intelligence.

The strength of the signals between biological neurons is analogous to the strength of the neural network nodes.

After successively boarding Boltzmann and Hopfield, the camera slowly rises from behind Hinton's head, the drumbeats sound from far to near, and the fog clears in front of him, revealing a majestic bridge.

Hinton's thin arms took over the blazing torch of human exploration.

(4) Chrysalis, Soup, Butterfly

Memory is not the end of intelligence, it requires understanding the information and finally responding with expressions

Have a background in psychology. Hinton soon discovered the key:

Perhaps for the sake of simplicity, perhaps because he did not go that far. In short, Hopfield assumed that the various pieces of information stored are completely independent.

That is to say: when the Hopfield network learns the alphabet, by default A is A, B is B, and C is C. If you enter a piece of information, the system will either determine it to be A or B, but will not determine it. Between A and B

This is a bit like a coin sorting machine, any coin will inevitably fall into a predetermined groove:

This seems okay, after all, there are no other letters between A and B

But if it is extended to more. Extensive "semantics", the limitations are immediately apparent:

For example, the two concepts of "good" and "bad". Doesn't something have to be bad if it is not good? There are obviously many concepts between good and bad, such as: ordinary, defective, acceptable, excellent, perfect .

You can even find that these concepts are not in a straight line with good and bad. They have intersecting meanings and some do not intersect.

They are at different positions in a huge semantic space, with no obvious boundaries like a spectrum.

That's why we need to create so many words, isn't it?

In order to express the relationship between these concepts more accurately, an important property must be introduced to the Hopfield network: probability.

For example: the word "radical" may contain 22% of "brave", 16% of "arbitrary", followed by 62% Space can list many other semantics.

This picture shows the degree of semantic association between words, the redder the color The greater the semantic association. For example, "name" and "gender" are closely related.

Thus, concepts are no longer isolated islands, but rely on probability to establish exquisite mathematical relationships, forming a "semantic space":

p>

Each word has a coordinate in the semantic space.

Semantic space is not an ordinary three-dimensional space, but a multi-dimensional space, perhaps with hundreds or thousands of dimensions.

Semantic space

This picture shows the use of 50 dimensions to depict the word on the left. Color in each dimension can be seen as the intensity of a "sub-concept".

With this "semantic space", the system can break down the concept and find its coordinates for each sub-concept "powder".

For example:

Ordinary "通" and smooth "通", It contains some common deep semantics. We can understand the sub-concepts, but it is difficult to describe.

The use of sub-concepts for learning is equivalent to entering a deeper level of "Inception". In an instant, the two channels of Ren and Du are opened, and understanding arises. .

Moreover, it can also reintegrate these sub-concepts in the deep space and spit out brand-new sentences that are different from the learning materials, that is, expressions.

That's exactly what Hinton did. In 1983, Hinton and his collaborator Terrence Shenofsky announced the new system. ——"Boltzmann machine"

Hinton (right) and Terrence Shenofsky

Because the Boltzmann machine requires a sub-conceptual thinking space that can only be understood but not expressed, Hinton changed the Hopfield network into two layers:

One ​​layer is the "visible layer": it accepts input and organizes the output according to human expression specifications.

One ​​layer is the "hidden layer": only Used for thinking, regardless of any expression standards.

A bunch of information balls hit the visible layer first, and then roll to the lowest point on the visible layer. Finally, it leaks to the hidden layer and continues to scroll. This is "understanding"

The ball starts from the hidden layer and bounces back to the visible layer. This is "expression".

This is the purpose of all artificial intelligence today. Basic structure: multi-layer neural network

Boltzmann machine: visible above. layer, below is the hidden layer

Boltzmann machine training, most of the work is actually the calculation of various probabilities, fixing the calculated parameters in the connection parameters of each neuron, so that the "landform" ultimately composed of these neurons It can approximate the "landform" implicit in the training material.

At this time, the number of neurons is already very large, and each neuron plays a specific role in the "landform". It’s hard to tell what it does.

In other words, humans have no way to directly intervene on specific neurons and can only use a certain algorithm to operate.

Hinton’s mind The "backpropagation algorithm" appeared

You may have read Kafka's "The Castle"

Land surveyor K He was employed by a castle, but when he came to the village where the castle was located, he was unable to get in touch with the real authority, but he was indeed influenced and obstructed by that superior power.

The backpropagation algorithm is like this:

< p style="text-align: left;">1. Each neuron is K, they didn’t know what they should do to achieve the meaning of power at first.

2. "Power" is the order implicit in the training corpus. No one has seen it clearly, but it will affect every K.

3. Once K does something that violates " will to power", K will be punished, but K But it can never contact the power to ask what its true will is. It can only contact some grassroots officials.

4. So the only thing K can do. That is to indirectly listen to the feelings and anger of the higher-ups conveyed by the grassroots officials. If you feel that you are wrong less, you will make less changes. If you feel that you are more wrong, you will make more changes.

until all K In the end, "do whatever you want without breaking the rules" is *as much as possible. This neural network is *supposedly* trained.

Note, the reason why I say " "As much as possible" and "Sort of", because the will to power is essentially elusive.

If you keep training, there will definitely be K The behavior will go wrong, but it is greatly reduced compared to the peak period, so backpropagation training does not have a clear end mark, it only has a "convergence" situation

From the perspective of the system as a whole, "backpropagation" is an extremely effective convergence method. This overall effectiveness can, to some extent, conceal the specific details experienced by each neuron K. Absurd.

But what is reflected in K's eyes may be the truth of the universe - we will never have the chance to fundamentally understand this world.

p>

Hinton, who loves animals, uses more positive metaphors to explain the training of neural networks:

A caterpillar is the data for training the neural network. It will turn into a pupa, and in the pupa, the original caterpillar melts into a soup, and a butterfly will eventually emerge from this soup.

blockquote>

So, what happened from the caterpillar to the butterfly? Is the butterfly still the same insect as the previous caterpillar?

These answers, such as Zhuang Zhou Mengdie is generally profound and romantic.

In the 1980s, after successively introducing Boltzmann machines and backpropagation algorithms, Hinton attracted the attention of a small circle. But the disturbance soon subsided.

However, his efforts to find the truth gave the "neural network" group a solid win.

This is an example of a Boltzmann machine: when using a two-layer neural network to recognize handwritten numbers, the real-time activation state of each neuron during the writing process.

at 80 In the 1990s, after an unsuccessful flash marriage with his junior sister and student, Hinton entered into his second marriage to molecular biologist Rosalind Zalin.

Although Hinton traveled to several cities in the United States and Canada in order to find a suitable teaching position, his spirit still seemed to be bathed in the warm spring:

In the daytime, there are smart companions marching side by side with him. At dusk, he returns to the boat like twilight, chatting happily with his lover.

The important thing is that when you stand in front of the mirror, what is reflected is a young face.

Perhaps one night, he also dreamed that his name was hanging on the Hinton family tree, shining brightly.

But as Hinton himself revealed: the predictions of machines (and of course humans) about the world are only based on simulations and probability calculations.

The castle in the clouds has no foundation.

What each neuron K experiences is the truth - the direction of the world is fundamentally unpredictable like the sun in the Trisolaran world, and absurd like a shadow Follow everyone the same.

Hinton in 1990

(5) Winter

Although the neural network based on "system theory" implied by the Boltzmann machine looks very kingly, But to create an "AI that can look at humans" requires a significant increase in human computing power.

Not a thousand times, not ten thousand times, not a hundred thousand times, but a billion times.

In the 1990s, although the computing power of global computers has taken off. But it is still to the needs of neural networks what a candlelight is to the sun.

It was just as embarrassing as when Einstein held high esteem for his theory of relativity, but was unable to verify it, which led to delays in winning the Nobel Prize.

Hinton improved the Boltzmann machine, reducing the connections between neurons and becoming a "restricted Boltzmann machine", which greatly Reduces the amount of calculation; he also designed a "model distillation method" that can transfer knowledge from large models to small models.

Even so, the computing power required is far beyond imagination.

"Desire without getting", this childhood nightmare suddenly came back.

In other words, it has neverWalk far away.

Rosalind suffered from infertility and they were unable to give birth to a child, so they finally decided to adopt two children from South America.

Rosalind was diagnosed with ovarian cancer just when her two children came into the house.

The nightmare experience of treating infertility made Rosalind extremely disgusted with the doctor's indifference and incompetence.

She refused surgery and chemotherapy, and stubbornly used a very unpopular "homeopathy" at home, which is to dilute the medicine to an almost undetectable level. Then enter it into the body.

"Unpopular" is just a polite way of saying - this therapy is ineffective.

Rosalind's disease progressed rapidly, with more and more tumors and more and more mental breakdown. Stubbornly believing that she could get better, she began looking for more expensive "homeopathic medicines." Until she said to Hinton in tears: "Let's sell the house."

Hinton looked at his wife, watching this person support him through the spring The couple said the cruelest words in their lives: "We will not sell the house. If you die, I have to take care of the children and they must have a place to live."

Even if 30 Years have passed, and every time Hinton thinks about this moment, his heart is still filled with various emotions, including anger, guilt, sadness, and confusion.

That is the violent reaction of an intelligent life when faced with the absurdity of this world. It is something that Hinton cannot yet understand.

Hinton stood at the pinnacle of self-doubt in his life.

Decades of observation of the world will inevitably slide into cruel "self-analysis":

If humans are just machines after all, then how deep is this bleeding emotion hidden in the neural network?

If the machine can eventually become an adult, then creating an AI so that it can eventually experience the sufferings of the human world is meaningful.What is it?

Hinton was 46 when his wife left. His son is 5 and his daughter is 3.

Rosalind's tombstone

For half of his life before that, Hinton lived in his own spiritual world, and after that, Hinton had to live in the "rolling earthly world."

Hinton's son has ADHD and learning disabilities. Even with the help of a nanny, he has to get home from get off work at 6 o'clock on time to take care of him, and has to go to the store later. Buy socks for your children.

A life as a single father that he had never imagined shattered Hinton's decades of illusion:

In the past, "living" meant realization and transcendence to him, as well as the glory of his family.

Now, "living" means existence, which means dragging yourself from today to tomorrow.

Once, when he saw a cashier in a supermarket who couldn't calculate simple numbers, he was very annoyed and thought: Can't they hire someone who can count?

Now, he will think: It is really good for a supermarket to hire him.

From that point on, I stopped being anxious to become a "better" person.

Hinton memories.

He found the "family tree" from the most secret corner of his heart and threw it into the trash heap calmly.

It was also from then on that Hinton settled in his position as a professor of computer science at the University of Toronto. In the years when neural networks gradually turned from a maze into a wasteland, , walking forward slowly and slowly when other people's attitudes change from understanding to forgetfulness.

The years are long enough for Hinton to educate many students.

But at the turn of the century, the Internet began to flourish and people began to become restless.

Students who feel they have business acumen are "abandoned" one after another, giving up on artificial intelligence that will never succeed, and jumping into the entrepreneurial wave. Facts have also proved that they made the right choice, and most of them gained a lot.

In the most deserted time, only Hinton and a few of his students were left walking on this road - the one in "The Last Supper". A long table that can seat all the disciples of "Neural Network".

Hinton isn't sure why these students remain in the field.

But unlike his teachers who often poured cold water on him, Hinton was willing to give his students some meager light.

He always said: "Neural networks are not a dead end."

But at the same time, they are not a dead end. I forgot to add: "This thing may not be possible until a century later. I am afraid that neither I nor you will be able to witness it with your own eyes."

It is not surprising that humans are machines. .

What is really strange is that a machine can calmly wait for something that will only come after it turns into powder.

Hinton is ready to live his life in peace.

But the world seems not to want to give him this opportunity.

(Six) Spring

The booming development of the Internet has plunged the world into an unprecedented hunger for computing power.

Business is the most powerful aphrodisiac in the universe. Moore's Law is at full throttle. Not only is the computing power of CPUs used for scientific computing rolling upward, but the computing power of GPUs used for graphics computing is also booming.

If you run a computer on a state-of-the-art computer in 1985,Calculate, non-stop until this moment. It only takes 1 second to switch to the best computer at the moment to do the same number of calculations.

The best prophets did not dare to imagine that as time flies over the decades, the candlelight of computing power has really turned into a dazzling sun.

Leave the original grass, just waiting for a spark.

A Beijing-born woman ignited the flames.

Stanford University professor Li Feifei led the team to spend 800 days and nights to manually annotate 14 million images, divided them into 20,000 categories, and founded the company in 2010 The ImageNet Image Recognition Challenge encourages researchers around the world to use AI to classify more images.

This This competition cannot pay that much prize money.

The prize is - honor.

In the winter of 2012, the awards for that year were announced, and the champion belonged to a system called AlexNet.

Its error rate for image recognition is as low as 15.3%, which is 10.8 percentage points lower than the error rate of the second place.

Contest results

The structure of AlexNet has 650,000 neurons, 60 million parameters, and 8 layers of neural network. The most special thing is that it uses 4 GPUs for calculations, while the second-place Google uses 16,000 CPUs.

People quickly understood that AlexNet and its competitors were not the same species.

AlexNThe structure of et

The authors of this system are a "triple group":

Asia Alex Krizevski, Ilya Suzkver, and their mutual mentor, 65-year-old Jeffrey Hinton.

Hinton, slowly steps back into the spotlight in this way.

He stood humbly behind the two closed disciples, with white hair on his head and wrinkles across his face, and his eyes were the same as before.

The current academic circle recognizes that moment as: "The first time AI looked at humans."

Two months later, the trio presented their use of The algorithms used by GPUs to do AI are overflowing with commercial value that they themselves have not yet fully realized - with the computing power held by giant companies at the time, it was enough to use neural networks to make practical AI!

Acquisition invitations from all over the world are coming like a snowflake: How much does your company offer? I buy it!

The three masters and apprentices realized that they should set up a company.

At the end of 2012, the hastily established DNNresearch company faced four ultimate buyers: Google, Microsoft, DeepMind, and Baidu.

They decided to hold an auction.

The skinny Hinton lay in the back seat of a taxi heading to the auction location. When he was 19, he injured his spine while helping his mother move a heater, and the condition gradually worsened over the decades. At this point, he could no longer sit down and could only stand or lie down.

DeepMind is a startup company. It can only use its own shares to make quotations and quickly withdrew from the competition. Microsoft also withdrew from the competition after offering $22 million. Only Google and Baidu kept increasing their prices. From early morning to midnight, the quotations continued to rise steeply, as if they would face the end of the world if they did not buy the Hinton trio.

Since it was a remote auction, Hinton was lying on the bed in the hotel to discuss with the two students.

The next morning, a new round of bids The fight continued, and Google had already offered $44 million. Hinton decided to stop the auction. His 65-year-old age and fragile spine could not support his work on the other side of the world.

He decided to sell the company to Google.

What Google bought for 44 million is like an "empty shell", with only the intellectual property rights in the hands of Hinton and their commitment to work at Google in the next few years.

But just like Darwin, Copernicus, Camus, and Einstein, what these three people hold in their hands is not only intellectual property, but also some kind of truth

The truth is the most dignified thing in the world. It is as valuable as a thousand pieces of gold, and it should be worth a thousand pieces of gold.

Hinton suggested that the three people should divide the shares equally. , each took 33%. The two students refused and insisted on letting Hinton take 40%.

left;">This is the most famous photo of the trio, with a rare smile on their faces at the same time.

Hinton got an unprecedented Money, praise and attention from all over the world, it seems to be a reward, or "repayment" for a long life of silent waiting.

Standing on the bank of history, he once again felt absurd.

Howard Hinton, his caustic father who predicted he would eventually be half as good as he was, has died 35 years ago. Hinton isn't even sure whether his father would have felt honored, scorned, or envious had he lived.< /p>

Rosalind, the wife who accompanied him through his prime years, could not witness his glory after all. What would she say if she were alive? Me, kiss me? Will she cry?

Hinton could not imagine, did not even dare to imagine

Because Rosalind always looked young.The ground stopped in place, but he had to drag his extremely real body and two children away, little by little, to remarry another woman.

Jacqueline Ford is Hinton's third wife, an art historian, and the person who truly accompanies her in the eyes of her two children. Growing up mom.

Hinton bought an island and gave it to Jacqueline.

"That was the only real indulgence in my life," Hinton said.

There are snakes, birds, insects, and various trees that Hinton needs for his carpentry work on the island. There are the sun, moon and stars rolling, and there is peace. The waves have time to flow quietly and unhurriedly.

The spring on the island is like poetry.

Just two years after Hinton and Jacqueline Ford moved to the island, Jacqueline was diagnosed with cancer again.

This "one more time" is for Hinton.

Georgian Bay in Lake Huron

(7) Ring

Jacqueline said to Hinton:

"I feel sad, but I know that I must use the remaining time to enjoy life and try to arrange everything for you and others."

While walking on the island, they stumbled upon the wreckage of a small boat. Jacqueline found some female workers and renovated the boat into a burgundy canoe.

"She made her maiden voyage," Hinton recalled, "and then, no one ever used it again."

In 2017, Jacqueline, who was nearing the end of her life, witnessed Hinton winning the highest award in the computing world: the Turing Award.

Hinton, who became famous, tried to use everything he had just to snatch people from the hands of death.

With the support of Canada, he quickly established the "Vector Institute", which gathered the world's top artificial intelligence talents. The first project was: applying AI to medical treatment diagnosis.

But a few months later, Jacqueline passed away.

Hinton thought of the robot that couldn't distinguish building blocks many years ago. A furious punch, a crushing desire.

He carefully stored the photos on his computer.

One ​​of the pictures is of his and Jacqueline's wedding, exchanging vows in the living room of a neighbor's house. Hinton was in all his glory that day, and Jacqueline held one of his hands in hers.

There is also a photo of Jacqueline staring at the camera on a burgundy canoe, with mottled water and breeze blowing.

"I want you to get to know Roz and Jackie, because they are my life "An important part of that," Hinton told New Yorker reporter Joshua Rothman in 2023.

But in fact, this also has a lot to do with artificial intelligence.

There are two attitudes towards artificial intelligence. One is denial and the other is perseverance.

Everyone's first reaction to artificial intelligence is "We have to stop it", just like everyone's first reaction to cancer is "We have to stop it" How to cut it off."

He continued.

But excision may be just a fantasy.

The reason why Hinton said this was because of what happened later in the "AlexNet trio":

Ilya Suzkver left Google in 2015. Under Musk’s guidance, he served as chief scientistCo-founded a company called OpenAI with Sam Altman.

Ilya Sutskever & Sam Altman

From here on, Hinton's life is connected with the drama that most people know about chasing light.

In OpenAI, Suzkwer took the Boltzmann machine to the extreme and became the "big model" of deep neural network. The big model became the basis of ChatGPT. The foundation has also become the soul of all AI that has the ability to communicate and understand today.

For the first time, humans have created an AI that can pass the Turing test.

In the past ten thousand years, the boundaries between humans and machines have never been so blurred.

In 1950, Turing published a paper proposing the "imitation game", that is, Turing test.

Just when ordinary people are beginning to worry about AI taking over the world, many AI practitioners are unusually calm - they feel that they know everything about the large models they have made with their own hands.

One ​​of the reasons is that its essence is based on probability calculation. To put it bluntly, it is just "predicting the next word."

But Hinton disagrees.

He believes it is dangerous to belittle this "predictive ability" before thoroughly understanding how the brain works.

Let us analyze it. If your job is to predict the next word, and you want to be very good at it, you have to understand what's being said.

This is the only way.

He said.

The second reason is: it will produce "hallucinations", that is, fabricating facts when outputting, which is embarrassing and useful.

Hinton I don’t agree either.

He thinks this is an arrogant inference based on "human exceptionalism" because humans can also hallucinate (based on the same principle).

Hinton mentioned the Watergate incident. White House counselor John Dean made up a lot of content during the interview, and the details were also full of errors and omissions, confusing different people's words.

But the gist of what he said is right. In our minds, there is no line between making things up and telling the truth. Telling the truth is just making it right. p>

he said.

Looking at it this way, ChatGPT's fabrication is both a flaw and encouraging evidence that it resembles the human brain.

p>

In 1973, John Dean was examined by the Commission of Inquiry.

From ChatGPT At the beginning, artificial intelligence research advanced rapidly, but researchers have always avoided using some seemingly unscientific words, such as "intuition".

They tried to use various theories to break it down. But Hinton has exclaimed on many occasions: “AI is more intuitive than we admit. ”

Once you try to dismantle your intuition into tiny parts, you will take a shortcut of "reductionism".

Symbolism always says that our essence is a reasoning machine, which is complete nonsense. Our essence is an analogy machine.

Maybe add a little bit of reasoning to it so you can notice and correct wrong answers when analogies come up.

Hinton said.

Admitting that you are an intuitive machine is costly.

This means that you must accept that what you think you understand about the world is essentially just a probabilistic prediction;

It means that you must admit that the memory that you think is solid is actually just an illusion;

It means that you must completely put aside your arrogance and face the absurd situation. Move forward in the world and accept what fate gives you.

Not everyone can afford this price.

In the beginning, people just wanted to understand the world;

In order to understand the world, we have to understand ourselves;

And in order to understand ourselves, we create another self;

< p style="text-align: left;">We create another version of ourselves, proving that we can never understand the world.

What Hinton has done in his life is to stand in this absurd circle and point it out to the world.

15-year-old Alan Turing

And outside this ring, there is a larger ring.

That can be called the fate of history - technologies that truly change the world will inevitably be used in wars.

At this moment, militaries around the world are developing AI-driven war robots, which they quietly call "autonomous weapons."

Einstein's theory of relativity finally detonated the atomic bomb, and the world entered the era of nuclear deterrence. Human beings born after that were essentiallyIt’s just one generation.

The only technologies that are truly expected to break through nuclear deterrence are more powerful AI-controlled anti-missile systems and thousands of "not afraid of death" and "can "Mass-produced" and "extremely intuitive" AI army.

When the AI ​​army grows, will they demand corresponding dignity, rights, and resources like the Crusaders?

Don’t forget, if there is no difference between machines and people. It means that they can also experience the pain of "desiring but not getting". The actions that such pain may inspire are completely unpredictable.

In other words, it is completely predictable.

How many examples have you seen of intelligent things being controlled by less intelligent things? And AI does not have to use force. As long as it can speak, it has every way to control humans.

Hinton said.

The heavy cross on Einstein’s shoulders is slowly being transferred to countless artificial intelligence scientists since Hinton.

1963, Castro and Khrushchev after the Cuban Missile Crisis .

In 2023, Ilya Suzkvir believed that OpenAI CEO Sam Altman ignored the safety construction of artificial intelligence, setting off a palace fight.

After the failure, he left OpenAI, which he founded, and established Safe Superintelligence.

Hinton publicly expressed his praise for his disciples, but he was not sure whether Sutskville would succeed.

To make an atom bomb just make it explode, but make sure something doesn't explode,Much harder.

Hinton said.

In 2023, Hinton resigned from Google, the latest of the three. The reason he gave for his resignation was: This can make criticism more neutral. The dangers of artificial intelligence.

Like the era of nuclear deterrence, the new era will probably be based on a new balance of deterrence.

From this we can understand Hinton’s strange metaphor: think of AI as a tumor.

If all the tumors can be removed, why does Hinton have to bear so much sadness in his life?

Rather than fantasizing about excision in vain, it is better to investigate a more active strategy of living with it.

But what is this strategy?

"No one knows the answer."

Hinton sighed.

(8) Liberation

In 2024, Hinton won the Nobel Prize in Physics.

He shared this honor with Hopfield.

The public finds it incredible when they see an artificial intelligence expert win the Nobel Prize in Physics. But to understand the work of Hopfield and Hinton, you will know that their theory is indeed born out of physics.

In other words, it is derived from the operating rules of the world itself.

People are willing to believe this story: a persistent scientist who has been consistent for decades Sun moves forward in suffering, and what he believes in finally turns into a reward, like the heavy rain in "The Shawshank Redemption".

But this does not seem to be true.

Hinton's life has endured the same absurdity as you and me. And the vast majority of ordinary people in the world, We can only move forward in the years and bear everything that fate has to offer.

There is no reward. ">No reward until they reach the other side, or fail to reach the other side.

A Nobel Prize, hanging on Hinton's family tree, seems qualified, even too qualified.

< p style="text-align: left;">I often think, I like carpentry, would I be happier as an architect because I don't have to force myself to do anything?

However, for science, I have to force myself all the time, and because of family reasons, I must succeed in science. Of course there is happiness in these years of scientific research, but mainly anxiety.

< p style="text-align: left;">Now that I'm successful, it's a huge relief

Hinton said.

"Relief" took a person 77 years of his life.

But even so, Hinton may be lucky.

We might as well look back at those people on the road

Boltzmann, the one who first built it. The person who bridges the macro and micro worlds

He was regarded as a heretic throughout his life. Under fierce criticism from academic opponents, and unable to prove himself, he finally lost confidence in himself.

In his later years, he became increasingly irritable and suffered from severe bipolar disorder. At the age of 62, he finally ended his life with a rope in a hotel.

This is his "relief"”.

And Alex, the first author of AlexNet, this legendary era began with his name.

< p style="text-align: left;">Alex was born in the former Soviet-era Ukraine and moved to Canada as a child. He was a quiet and resilient kid who, in Suzkovel's words, "had the ability." Keep studying a problem until it is solved."

In 2017, Alex also left Google and joined a startup company because he couldn't bear the style of a large company. Then he became unknown, stayed away from the public eye, and lived quietly.

But on academic websites, you can also see that he publishes some cutting-edge papers every year, and his collaborators are almost all Sutzkwer and Hinton.

Someone posted an article to discuss: What is Alex’s position in the history of artificial intelligence?

His conclusion is:

Alex is like a firework shining at a critical moment in history, but has not established a foothold in this field. , disappeared into the crowd

But maybe, he himself never wanted to be a "leader". He is a visionary, and that is enough.

The public doesn't know where Alex is, but friends say he now likes hiking.

This is his "relief".< /p>

In recent years, Hinton has been living with retired sociologist Rosemary Gartner.

"I think he is that The kind of person who needs company at all times. " Rosemary said softly.

Rosemary set rules for her "old carpenter boyfriend" Hinton: Never cut down trees when alone on the island. , in case no one saves him if he cuts off his arm.

In the record of New Yorker reporter Rothman, there is a scene:

Shinton sailed ashore that day, waiting for Rosemary to give her He brought supplies to the island.

He went to the store to buy a light bulb, but when he came out, he ducked into the green plants at the door of the store. He stood up, holding up a black and yellow snake.

It twists and turns, about a meter long

"A gift for you! He held it up to Rosemary boldly, "I found it in the bushes." ”

Rosemary smiled.

He poured the snake from his left hand to his right hand, and the two It was all sticky on my hands. Let Rosemary smell it, and it was full of a peculiar mineral smell that was unique to this kind of snake.

" Your shirt is dirty. said Rosemary.

"Because I had to catch it." Hinton explained.

Then, Hinton put the snake down and watched with satisfaction as it burrowed back into the grass.

: left;">"The weather is so nice today, let's go sailing! "He said.

Hinton is in love again.

At this moment, on the island In the cabin above, the burgundy canoe was shining brightly in the sunlight that filtered through the windows. Several chairs were placed around it, facing the shimmering lake in the distance, and some magazines were scattered on the table.

That is a beautiful cabin.

Human thinking is not just reasoning after all. We think about time, life and death, and the things we have passed by Everything gathers meaning like gravity, trying to give the final answer.

Does artificial intelligence also need such a cabin?

Sail through the foggy Sinton, one dayWe will reunite with the ultimate question: If human life is full of suffering and separation, what is the point of creating more human-like AI?

He may still have no answer.

But at least, between suffering and suffering, there is something else.

Just like between winter and winter, there is spring.

"People are machines. But people are special, wonderful machines." Hinton said.

At the age of 77, Hinton was stunned to usher in an unprecedented bright pursuit of light and the desire of the world to know and listen to himself. He allowed people to pluck all kinds of meanings from his body, like an autumn tree shedding golden light.

But aren't those meanings the self-projection of those who love or ignore him?

Hinton just lived like this for 77 years, winter passed, spring came, spring passed, winter came.

In order to move forward in the cruel time, he must fight against the pain, and in order to fight against the pain, he becomes brilliant.

He is a machine like you and me, and like you and me, he has a mysterious dream. Whether it was luck or misfortune, half of his dreams went into spring and half were buried in the cold winter.

But that may not matter.

Because all the condensed things will be washed away again by the torrent of time, just like the rampant giant beasts turn into silent fossils, just like the rushing tears disappear in the rain.

The important thing is that those who have looked at him may sigh softly for a moment:

"Hey, he is a gentle person"