“The oxygen you breathe, the fluorine in your toothpaste, the calcium in your bones, many of the elements that we’re very familiar with are created either in a massive star before the explosion or during the explosion itself, and the explosion ejects this material to litter the interstellar medium with these products of existence,” explains Adam Burrows, a professor at Princeton University who is using supercomputers at the U.S. Department of Energy’s (DOE) Argonne National Laboratory to model supernova explosions in three dimensions (3D).

Supernovae shape our universe and life as we know it, but how exactly a massive star explodes as a supernova remains a mystery. Images from NASA’s James Webb Space Telescope are providing an unprecedented view of supernovae remnants and other mysterious cosmological phenomena, but scientists need to peer deep inside massive stars to understand the internal mechanisms behind the distant cosmic explosions. That’s where supercomputers at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science user facility, come in.

“The goal is to understand how stars with different solar masses explode – a kind of experiment that you cannot perform in a lab,” says Marta García Martínez, an Argonne computational scientist working with Burrows’ team. “Simulation is an amazing way to get the problem solved.”

For the past few years, Burrows has been leading a project to model supernova core collapse at the ALCF, where they were able to produce one of the largest collections of 3D supernova simulations. Ultimately, the team’s research aims to illuminate how and why a massive star goes supernova.

“The mechanism by which they explode has not been firmly established,” Burrows says. “The thermonuclear processes that attend stellar evolution need to be understood so you know the object that will eventually, by dynamical and violent means, produce a supernova explosion.”

Going 3D

The theory describing supernovae has evolved alongside advances in physics and advances in computer hardware and software. It has taken 60 years of pursuing this problem for computational capacity to reach the fidelity necessary for modeling these events in a way that emulates nature.

In past efforts, computer simulations modeling supernovae in one dimension frequently failed to explode as expected. The shockwaves modeled in one dimension stall because they have to contend with all the material continuing to fall into the core of the star, becoming an accretion shock rather than an explosion, showing that something was missing from the simulations. This was evidence that unstable internal structures that can only be modeled in three dimensions occur, and that the turbulence these instabilities cause are necessary for the explosion.

Increasing the dimensions of the simulation, however, required a massive increase in computational power. Every dimension added to the computation increases the complexity by at least a factor of 100, so going from a one-dimensional simulation to a 3D simulation increases the complexity by a factor of 10,000, says Burrows. In addition to that, researchers wanted to understand the radiation field in more detail, adding further complexity. In a way, Burrows says, “It’s a seven-dimensional problem, time and space and then three spatial and three momentum directions, along with multiple neutrino species.” In order to incorporate all of the physics needed to simulate the explosions in 3D, the team created a code called FORNAX.

To be able to run such computationally intensive explosions with FORNAX, the ALCF’s high performance computing (HPC) resources were needed. “Big iron HPC resources such as Theta, Polaris and Aurora are central to our ability to actually simulate this complicated problem,” says Burrows.

The structure of stars’ interiors, including instabilities that can only be looked at in 3D, are very important in determining whether stars explode. The interior structure varies with the evolution of the stars and is not monotonic with the mass of the stars.

Whether or not a star will explode is also in part dependent on how the convective shells and core burning interact at its last stages. What kind of structure it will leave behind is hard to predict—it is not smoothly varying with stellar mass and depends upon rotation and metallicity.

The aim of Burrows’ project is “to bring together all these disparate strands to put together a theory” to explain the explosions, a theory that would tell us what stars will explode and what stars will produce neutron stars and black holes.

The major breakthrough that has been achieved with modeling supernovae in 3D is that “they explode naturally and in the past they didn’t,” Burrows says. The models are now behaving the way supernovae behave in nature, so investigation into how they explode and what their cores become grows closer to describing and predicting what really occurs.

The impact of neutrinos

Several aspects of physics must be understood to describe the mechanisms behind supernovae, including the particle physics of neutrino matter interaction, nuclear physics, and gravity according to general relativity because the objects are so compact. Burrows explains that “you have to know particle physics because particles such as neutrinos turn out to be important. The high densities and temperatures in the center of the stars mean light can’t move around at all. It’s just trapped and can't move energy through the matter. The neutrinos though can, and so the inner cores of massive stars at their death become in a sense neutrino stars. So, the neutrinos are energetically important in driving the supernova explosion. Moreover, neutrinos also drive the star's evolution just before the dynamical violent event that gives birth to the supernova.”

During core collapse the object created in the center of the star is extremely hot and extremely dense and produces neutrinos at a high rate. The interior of the core becomes opaque to neutrinos, while the rest of the star, which is far larger is not, so neutrinos outside of the core continue to diffuse through the less dense material while the interior becomes a neutrino star. Some of the neutrinos get absorbed behind the shockwave, adding more energy.

“So the question is, do the neutrinos deposit enough energy during this phase which only lasts a few seconds to actually reignite the supernova?” Burrows asks. It turns out that most of the time the neutrinos do not deposit enough energy for this, but they do heat the matter and provide convection resulting in turbulent stress. This turbulence seems to be “necessary to get them to explode most of the time.”

Results of Burrows’ work with 3D simulations advancing the theory of how supernovae occur have been published in an article in Nature, highlighting the neutrino-heating mechanism as a key driver of supernova simulations.

Collaboration with the ALCF

Burrows gained access to ALCF computing resources through an award from DOE’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. In addition to providing computing time, the INCITE program pairs each project with a computational scientist to help the team maximize the use of DOE’s leadership supercomputers.

“Over the course of the project, you get to know the users and the code and find what are the elements that can help them achieve their scientific goals faster,” says García Martínez, who is the ALCF computational scientist for Burrows’ INCITE project.

García Martínez explains that the simulations look at a very brief moment leading up to the explosion, around half a second of physical time, and the researchers are looking to extend that amount of time up to 4 or 5 seconds with their current INCITE allocation.

To help aid in the analysis of the data generated on Theta, the team worked with ALCF visualization experts to produce images and animations of the supernova simulations.

“Over the years, I will say it's getting more and more important for our users to take advantage of our visualization support. With the extreme scale and complexity of the data they produce, visualizing it is increasingly the best, and often the only, way to gain insight from it,“ says Joseph Insley, who leads the ALCF Visualization and Data Analytics team.

The 3D simulations depend upon an extremely complex set of calculations, which resulted in the visualizations being quite complex as well. To produce the visualizations, Insley’s team injected particles in post processing to create trails to show their path over time. These particles were traced for a finite amount of time while new particles were being added and traced. “In order to do that,” Insley says “and combine them all together, took a bit of development and exploration on our part.”

The team’s latest visualization, which shows the results of 3D simulations of a supernova explosion of a 25-solar-mass star and the simultaneous birth of a neutron star, was recognized as a finalist in the Scientific Visualization & Data Analytics Showcase at the SC22 conference.

Burrows and his team continue to evolve and improve the FORNAX code since there are ambiguities in the equation of state—which is the connection between density, pressure, composition, temperature, energy, and entropy. The researchers are working to incorporate constraints from nuclear physics experiments, and from measurements of neutron stars. They are collaborating with experimentalists whose efforts in nuclear physics and with particle accelerators determine ground truths. These ground truths will allow simulations of the unobservable—the moments leading to the explosion of massive stars to get even closer to representing reality.

==========

The Argonne Leadership Computing Facility provides supercomputing capabilities to the scientific and engineering community to advance fundamental discovery and understanding in a broad range of disciplines. Supported by the U.S. Department of Energy’s (DOE’s) Office of Science, Advanced Scientific Computing Research (ASCR) program, the ALCF is one of two DOE Leadership Computing Facilities in the nation dedicated to open science.

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science.

The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science