Multi-threaded parallelization

For shared memory and multi-CPU computers - available from Asap version 2.18

Multi-threaded support must be enabled at compile-time!

Asap can used multiple CPUs simultaneously when calculating the energy and the forces on the atoms. This is almost transparent for the user, once enabled in the beginning of a script nothing more needs to be changed.

The main purpose of introducing this parallelization paradigm from Asap version 2.18 is not performance: do not expect significantly better performance than with Message Passing parallelization. The main motivation is ease-of-use, as multithreaded parallelization is invisible to the Python script.

To enable multi-threading, place this command in the beginning of your script:

AsapThreads()

this will cause Asap to run on all available CPUs.

If you want to run on a different number of CPUs, or if Asap the command above fails because Asap cannot determine the number of CPUs, you can specify the number of threads (and thus CPUs) to use with an argument:

AsapThreads(4)   # Use 4 threads

Asap will not let you use more threads than you have CPUs/cores, unless it is unable to determine the number of CPUs/cores.

Warning:

On supercomputers, clusters and other setups with a queueing system, it is possible to accidentally use more CPUs than the queueing system has allocated to you. This will make you very unpopular. Be sure to reserve all CPUs on a given node if you use multithreading.

On the Niflheim cluster, AsapThreads() will check if you are accidentally trying to run on more CPUs than allocated by the PBS queueing system, and abort the job if that is the case. This may also work on other clusters with a sufficiently similar PBS setup.

Molecular Dynamics when running multi-threaded

To get decent performance when running multi-threaded, please observe the following.

  • Only the Asap calculators EMT, EMT2011 and LennardJones support multithreading. User-defined calculators based on these but doing post-processing of the forces in Python will work, but as the Python part will not be multithreaded performance may suffer.

  • For best performance, use VelocityVerlet or Langevin dynamics, and import it from asap3.md, not from ase.md. Using ASE dynamics will still give a performance benefit, but it will be smaller.

  • If you use the FixAtoms constraint, import it from asap3.constraints and use only one constraint. Using more than one constraint or using any other type of constraint will still work, but it will result in somewhat reduced performance as the ASE version of the dynamics will be used automatically.

  • Do not use multi-threading with less than 1000 atoms, performance may actually be worse than running on a single CPU. The Asap-optimized dynamics detect this situation and temporarily disable multithreading.

Limitations

The ASAP-optimized dynamics modules have the following extra limitations, compared to the normal ASE versions.

  • Atoms.set_calculator() must be called before the dynamics object is created, and the calculator cannot later be changed.

  • Constraints should be set before the dynamics object is created, and the constraints should not be changed thereafter, nor should constraints be added or deleted.

  • An observer should not add or remove observers when it is being called (this is a pretty absurd thing to do, but technically allowed by the ASE dynamics objects).

Note that it is OK to add or remove observers while the dynamics object is being used, as long as it is done between calls to dynamics.run().

Breaking these rules may not be detected, and thus result in incorrect results.

Expect strange crashes

The multithreading support is based on the OpenMP standard. Unfortunately, OpenMP and C++ exceptions are incompatible. As a workaround, any error detected by Asap inside parallel regions will be printed to stderr immediately, but stored and only reported when control flow leaves the parallel region. Unfortunately, some such errors may result in the program crashing or deadlocking when it tries to continue. Please report such incidents so they can be fixed.

Unfortunately, this delaying of the errors is necessary as soon as Asap is compiled with OpenMP support, even if running on a single thread.

Performance

See the main Performance page.