Computation propels particle physicists in quest for discovery

In the High Energy Physics Building, teams of scientists work on two so-called trigger systems for the world’s largest accelerator, the Large Hadron Collider in Geneva, Switzerland. Triggers quickly help identify the most important experimental data to record before the vast majority of it almost instantly disappears.

And next door in the sub-basement of the Accelerator Building sits a server room with special electrical and cooling capacity that houses a large-scale computing center to support ATLAS scientists.

One of the UChicago trigger projects is called the Fast TracKer (FTK), which now is being tested and commissioned. The other one, called the global feature extraction (gFEX) electronics board, is planned for the LHC’s Run 3 operations, which will begin later this decade.

Leading the FTK project are Young-Kee Kim, the Louis Block Professor in Physics; and Melvyn Shochet, the Elaine M. and Samuel D. Kersten Jr. Distinguished Service Professor in Physics.

“We have finished producing the boards that are needed for this year’s commissioning operation with the ATLAS detector and are now installing and testing the system,” Shochet said.

All the data produced in particle collisions are difficult to store because there are several tens of millions of collisions per second, Kim explains. “All we can save or store is about a thousand events per second, so we have to make a quick decision.”

Increasing intensity

At the beginning of LHC’s first run, which began in 2009, each collision of protons produced five particle interactions. By the end of the first run, the intensity had increased to between 20 and 25 interactions per collision.

“We can expect upwards of maybe as high as 70 as we get toward the end of run two,” says Mark Oreglia, a professor in physics and a member of UChicago’s ATLAS group, which conducts research at the LHC. “In order to perform under the conditions of the higher interaction rate, we need faster and more intelligent electronics on the detector.”

Kim and Shochet, who are also members of the UChicago ATLAS group, have designed the FTK to more efficiently identify and store the most interesting but ephemeral signals emerging from the experiment. These include potential decays of the Higgs boson into dark matter or bottom quarks. A leading candidate for dark matter, the Weakly Interacting Massive Particles (WIMPs) would zip through the detector like a ghost, leaving behind a significant but missing quantity of energy and momentum in their direction of travel.

“Usually the signal for dark matter is missing energy,” Kim says. And quarks only travel in packs of other subatomic particles, making it challenging to identify them individually.

Another high-profile LHC electronics program at UChicago involves upgrading the tile calorimeter, the detector that measures particle jets. Oreglia’s TileCal upgrade specifications passed the National Science Foundation’s Design Review earlier this year and will receive $4 million in research and development funds over the next four years.

Oreglia and his associates plan to replace copper wire in the calorimeter’s 90-meter trigger cables with fiber optics that will eliminate most of the extra background noise that affects the instrument’s formerly state-of-the-art electronics.

Previously, the trigger system could only identify interesting jets based on their high energy. But now, if the system identifies an interesting jet, “We can examine the region around it and say, ‘Was there more activity there or was it isolated?” Oreglia says. That would be the signature of a clean, important event that would lead to more intelligent trigger decisions.

Improved electronics

“With faster, lower noise, and more intelligent electronics, we hope to be able to tune the trigger on jets so that we can have a higher rate of interesting physics,” Oreglia says. “Let’s say you have an interesting particle like the Higgs. You’re going to get many more events where it’s decaying into jets than into simple things like electrons.”

Physicists prefer to see the Higgs decay into electrons because there is less background noise to obscure the data. But that only happens a tiny fraction of the time. Phenomena such as Higgs bosons and other exotic particles usually decay into jets. “The better you can measure jets, the better your chances of discovering new physics,” Oreglia says.

The gFEX electronics board will bring new capabilities to the ATLAS trigger system, says David Miller, assistant professor in physics.

“It’s a fun project. We’re specifically responsible for the brains of the board that does a lot of the monitoring and that controls the system itself,” Miller says.

The first postdoctoral scholar to join Miller’s group, Reina Camacho, has been testing the ultra-high speed optical communications system for gFEX. “The tests at the end of January/beginning of February were extremely successful, and we are moving full speed ahead with the next prototypes,” Miller says.

Miller also is in charge of the ATLAS experiment’s jet trigger system, which provides a way to skim off the most interesting data using more complicated algorithms that operate at lower speeds. He likened ATLAS’s huge, cylindrical detector to a camera that takes 40 million photographs per second in three dimensions.

Operating in the sub-basement of the Accelerator Building, meanwhile, are Tier 2 and Tier 3 computing clusters for the ATLAS experiment, which enjoy support from UChicago’s Computation Institute. “Because of the excellent network connections between the university and the nation’s major research and education networks, including ESnet and Internet2, we are able to play a leading role in processing data from ATLAS,” says Senior Scientist Rob Gardner, who heads up the facility, which is comprised of more than 15,000 processor cores and holds more than 8,000 terabytes of ATLAS data.

UChicago’s high-energy physicists heavily use the Tier 3 cluster for in-house ATLAS computing, while over 1,500 physicists from the international collaboration tap into the Tier 2 cluster in a given run of the experiment. Because of the ready availability of these resources, Miller notes, “The ATLAS group at Chicago has been able to analyze new LHC data faster and more consistently than many comparable institutions.”