U.S. Particle Accelerator School
U.S. Particle Accelerator School
Education in Beam Physics and Accelerator Technology

Optimization and Machine Learning for Accelerators

Sponsoring University:

Stony Brook University

Course Name:

Optimization and Machine Learning for Accelerators
This class is full. Please contact USPAS to be placed on the waiting list.

Instructors:
Remi Lehe, Lawrence Berkeley National Lab; Auralee Edelen, Nicole Neveu and Christopher Mayes, SLAC



Purpose and Audience
Optimization techniques are key to both the design and operation of contemporary charged particle accelerator systems. In addition, machine learning techniques are now being increasingly used, either to augment the capabilities of standard optimization (e.g. through surrogate modeling), or to address entirely new tasks (e.g. anomaly detection, fault classification).

This course will introduce a number of optimization and machine learning techniques that are commonly used for particle accelerators, as well as their range of applicability and limitations. The course is suitable for graduate students in physics or engineering as well as postdocs, and staff members at laboratories or companies.

Prerequisites
Some familiarity with accelerator science and technology at the level of USPAS Fundamentals of Accelerator Physics and Technology with Simulations and Measurements Lab or USPAS graduate-level Accelerator Physics is strongly recommended. For example, prospective students should be familiar with standard accelerator system terms such as ‘quadrupole magnet’,’ rf cavity’, ‘emittance’, ‘bunch length’.

Experience with Python 3 basics (numpy, h5py, matplotlib) is required. All homework and laboratory assignments will use Python 3 based iPython notebooks. Python basics will NOT be covered during the course. There will NOT be a python tutorial on the first day. Students can both assess their experience and prepare if they have any specific needs by viewing the short tutorials and background reading in the Reading Requirements below. It is essential that enrolled students meet these requirements. There will not be time during the course to catch up if unfamiliar.

It is the responsibility of the student to ensure that they meet the course prerequisites or have equivalent experience.

Objectives
The course will cover the basics of various optimization and machine learning techniques that are presently being used or are under consideration for applications in the context of particle accelerators. The aim is to give the students an overview of these techniques, and of the specific accelerator-related task that they can solve. In particular, for a given task (e.g. optimization, classification), the relative benefits and limitations of different techniques will be emphasized (e.g. sensitivity to hyperparameters, robustness to local minima, etc.). A large fraction of the course will be dedicated to hands-on programming sessions. All the sessions will use open-source, widely-supported Python packages for optimization and machine learning. Packages to be employed include: numpy, h5py, matplotlib, keras, and tensorflow. At the end of the course, attendants should be able to directly apply these tools to their respective topics of research.

Instructional Method
The morning and afternoon sessions will be split into lecture and computer laboratory time. Short lectures on the lab subjects will be given, then students will work on the laboratory assignment during class time with instructors present. Students will use their laptops to complete the laboratory assignments. Homework will be assigned each day, and instructors will be available to help answer questions during the evening homework sessions.

Course Content
The course topics include:

Optimization

  1. Single-objective and multi-objective optimization
  2. Gradient-based methods (1st and 2nd order)
  3. Heuristic methods (e.g. Genetic Algorithms, Particle Swarm Optimization)
  4. Model-based methods (e.g. Bayesian optimization)

Machine Learning (ML) Techniques

  1. Brief ML background
    1. Learning paradigms: Supervised learning, Unsupervised learning, Reinforcement Learning
    2. Kinds of tasks: classification, clustering, regression, dimensionality reduction
    3. Survey of common algorithms
  2. ML for applications in particle accelerators
    1. Design optimization
    2. Anomaly detection
    3. Fault classification / prediction
    4. System modeling (surrogate models, models from data)
    5. Control / tuning

Highlights of recent ML results in accelerators

Reading Requirements
All materials needed for the course will be provided by the instructors.

Computer Requirements
This course will be taught using iPython notebooks, with computing provided for free by RadiaSoft. Students should bring a laptop or tablet computer with an html5 compatible browser. If you are not able to bring your own computer please contact the USPAS as soon as possible to arrange a loaner.

Perspective students must prepare for the course in advance by reviewing the following:

Python: https://www.learnpython.org 
GitHub: https://github.com
Scipy lecture notes: https://scipy-lectures.org/ Students must be competent with materials at the level presented and must carry out independent study in advance to rectify any needs in cases of deficiencies.

Credit Requirements
Students will be evaluated based on performance: homework assignments (60% grade), and laboratory notebooks (40% grade). 

Stony Brook University course number:
Indiana University course number: Physics 671 Advanced Topics in Physics of Beams
Michigan State University course number: PHY 963 
MIT course number: 8.790 Accelerator Physics