Artificial neural networks are computational models inspired by the brains of animals, intended to replicate their ability to learn and recognize patterns.
Some believe that outperforming humans is the ultimate goal of artificial intelligence (AI), regardless of the task we may apply it to.
Even though this is not yet a reality in every field, the use of AI to optimize tasks in scientific research is rapidly growing, boosting discoveries and the development of new technologies.
In this context, the UK-based firm DeepMind is prominent for its achievements in developing and applying AI to solve real-world problems. The company was featured in another Quantum Record article earlier this year for its AI that predicts the three-dimensional structures of proteins with unprecedented accuracy, an extremely relevant application for many fields, from material science to medicine. More recently, they managed to raise the bar even higher, taking humankind one step closer to sustainable energy in our quest to harness nuclear fusion.
Nuclear fusion is the power that fuels stars and, if used on Earth, it may be our best hope for a virtually unlimited source of clean energy. During fusion, the nuclei of two lighter atoms are joined, or fused, into a heavier nucleus, liberating an enormous amount of energy. To put it in perspective, one kilogram of fuel used in nuclear fusion can produce as much energy as 10 million kilograms of coal, oil, or gas. The process also does not emit any long-lived radioactive waste or even greenhouse gases.
However, replicating the process that happens spontaneously in the core of stars involves several challenges that make some researchers doubt if it will ever be a reality. Atomic nuclei repel each other, and beating this force to make them fuse requires huge amounts of energy, which can be very tricky to do safely. To induce nuclear fusion, scientists have to create and control extremely high temperatures, up to hundreds of millions of degrees, in which matter enters the state of plasma – a superheated, extremely difficult to control soup of electrically-charged particles.
Thanks to their enormous masses, stars rely on gravity to hold the plasma together in their cores, but inside fusion reactors, researchers have to find their own strategies to accomplish this. Among the existing techniques, magnetic confinement uses magnetic fields to trap the plasma, holding its form and keeping it from touching the reactor walls. This requires an operator to constantly monitor the plasma and adjust the magnetic field accordingly, which is extremely difficult, especially considering a wide range of plasma configurations.
AI speed controls fusion reactions
Variations in the plasma happen too fast for a human to react, making it impossible for a human operator to do that task in real-time. Additionally, the stakes are very high – if the plasma touches the walls of the vessel, it will not only cool down the plasma but also damage the reactor. The conventional solution to this obstacle involves solving intricate mathematical problems to precompute a set of commands, designing independent controllers to stabilize and control the plasma, and creating a control loop based on real-time estimations for the plasma shape. This solution is usually effective, but it is overly complex and more so when the target plasma configuration changes.
Now, researchers successfully developed and used an AI algorithm to do this job. In collaboration with the Swiss Plasma Center at EPFL, DeepMind developed and trained a deep reinforcement learning algorithm to operate the coils that control the magnetic field to shape and control the plasma in real-time. In the article describing the achievement, published in the journal Nature, the team states that this “is one of the most challenging real-world systems to which reinforcement learning has been applied.”
“Reinforcement learning is the training of machine learning models to make a sequence of decisions. The agent learns to achieve a goal in an uncertain, potentially complex environment. In reinforcement learning, an artificial intelligence faces a game-like situation. The computer employs trial and error to come up with a solution to the problem. To get the machine to do what the programmer wants, the artificial intelligence gets either rewards or penalties for the actions it performs. Its goal is to maximize the total reward. […] the designer […] gives the model no hints or suggestions for how to solve the game. It’s up to the model to figure out how to perform the task to maximize the reward, starting from totally random trials and finishing with sophisticated tactics and superhuman skills. By leveraging the power of search and many trials, reinforcement learning is currently the most effective way to hint machine’s creativity.” — Deepsense.
The researchers trained the algorithm in a simulation until it had learned to control the shape of the plasma in a virtual reactor, then gave it control of a real reactor to see how it would translate.
The fusion reactor used was the Variable Configuration Tokamak (TCV), an experimental reactor in Lausanne, Switzerland. The AI successfully controlled the plasma in the TCV without any additional fine-tuning from the researchers, proving to be a more simple, straightforward, and overall better approach to the job than previous methods.
Interior of the tokamak fusion reactor at the MIT Plasma Science and Fusion Center
The algorithm significantly outperformed the previous methods of controlling plasma inside reactors, with unprecedented flexibility and generality.
The AI successfully produced and controlled several plasma configurations while meeting objectives specified at a high level and obeying physical and operational constraints. The shapes ranged from simple elongated shapes to more advanced ones such as negative triangularity, snowflake, and even sustained droplets – separate plasma spheres maintained simultaneously in the reactor.
In order to produce such impressive results, the algorithm goes beyond the point where any human operator or even sophisticated models could ever go. It takes in 90 different measurements of the shape and position of the plasma ten thousand times a second, adjusting the magnetic field accordingly through the 19 coils of the TCV. To optimize the speed of the network, the researchers split it into two: a larger one called the critic, which learned how to control the reactor inside the simulator; and a smaller, faster one called the actor, which runs on the TCV using the knowledge produced by the critic.
The extraordinary success of this experiment points to an exciting future in both the fields of nuclear fusion and AI.
Researchers expect it will be easier to safely experiment with different configurations and conditions inside the reactor using an AI operator, which would boost developments in nuclear fusion research. Also, because the AI learns to control the plasma without guidance on how to do it, it can design new strategies and configurations that exceed human capacity.
Finally, this development highlights the power of AI technology to tackle real-world problems and shows that even though there are problems that we humans may not be able to solve, we still have the imagination to create something that has the capability to do what we can’t.
