I am attaching my slides from 25/03/2026, Belfast. Don’t hesitate to reach out if you want to discuss any aspect of the included content.
Content
1 - Motivation
2 - Introduction to Granular LAMMPS & Kokkos
3 - Float Precision
4 - Non-dimensional DEM 5 LAMMPS/Kokkos code compilation and calculations
All the best,
Dariel
NE4_KOKKOS.pdf (7.5 MB)
Following up on the KOKKOS/LAMMPS topic, I want to share an easy guide to install kokkos-tool ( GitHub - kokkos/kokkos-tools: Kokkos C++ Performance Portability Programming Ecosystem: Profiling and Debugging Tools · GitHub ). This package enables simple profiling and access to KOKKOS tools. While you can find a range of tools, I have been using only the space-time-stack tool for profiling.
The installation is very short (< 30 s) and relatively straightforward.
git clone https://github.com/kokkos/kokkos-tools.git
cd kokkos-tools/profiling/space-time-stack/
make -j
export KOKKOS_TOOLS_LIBS=~/kokkos-tools/profiling/space-timestack/kp_space_time_stack.so
-If you do not want to add KOKKOS_TOOLS_LIBS to your system PATH variable, you would need to execute the last command in every session you want to use kp_space_time_stack.so
-As a prerequisite, it is required that your LAMMPS binary has been compiled with the option -DKokkos_ENABLE_LIBRT=on.
After executing a simulation, you will find a report containing relevant information about the execution performance.
Total time - self-explanatory total time of execution. My experience so far is that when Kokkos_ENABLE_LIBRT=on is enabled, the reported total time is almost identical to the traditional Total wall time reported by LAMMPS.
BEGIN KOKKOS PROFILING REPORT:
TOTAL TIME: 319.832 seconds
TOP-DOWN TIME TREE and BOTTOM-UP TIME TREE- they provide a list of the individual kernel(code functions) performance, sorted from the most time-consuming one to the least. The information is provided in different columns for [type]
Allocation Memory- finally, the report includes a detailed breakdown of the occupied memory on the CPU and GPU spaces and the percentage occupied per variable.
Example:
KOKKOS HOST SPACE:
===================
MAX MEMORY ALLOCATED: 21145.5 kB
ALLOCATIONS AT TIME OF HIGH WATER MARK:
29.1% neighbor_history:firstvalue_mirror
29.1% neighbor_history:valuepartner_mirror
14.5% neighbor_history:valuepartner_mirror
4.8% neighbor_history:firstflag_mirror
4.8% neighbor_history:partner_mirror
1.8% atom:x_mirror
1.8% atom:v_mirror
1.8% atom:f_mirror
1.8% atom:omega_mirror
1.8% atom:torque_mirror
1.8% neigh:xhold_mirror
1.1% comm:k_buf_send_mirror
1.1% comm:sendlist_mirror
0.6% atom:radius_mirror
0.6% atom:rmass_mirror
0.6% coord/atom:cvec_mirror
0.6% comm:k_buf_recv_mirror
0.4% Neighbor::bins_mirror
0.3% atom:tag_mirror
0.3% atom:type_mirror
0.3% atom:mask_mirror
0.3% atom:image_mirror
0.3% neighbor_history:npartner_mirror
KOKKOS CUDA SPACE:
===================
MAX MEMORY ALLOCATED: 22964.6 kB
ALLOCATIONS AT TIME OF HIGH WATER MARK:
26.8% neighbor_history:firstvalue
26.8% neighbor_history:valuepartner
13.4% neighbor_history:valuepartner
4.5% neighbor_history:firstflag
4.5% neighbor_history:partner
2.2% neighbor_history:partner
1.7% atom:f
1.7% atom:omega
1.7% atom:torque
1.7% neigh:xhold
1.7% VerletKokkos::f_merge_copy
1.7% neigh:xhold
1.7% atom:x
1.7% atom:v
1.0% comm:k_buf_send
1.0% comm:sendlist
0.6% atom:radius
0.6% atom:rmass
0.6% coord/atom:cvec
0.5% comm:k_buf_recv
0.5% neighlist:neighbors
0.5% pair:neighbors_touch
0.5% neighlist:neighbors
0.4% Neighbor::bins
0.3% neighbor_history:npartner
0.3% atom:tag
0.3% atom:type
0.3% atom:mask
0.3% atom:image
0.1% pair:normalstrain_atom
0.1% pair:shearstrain_atom
0.1% pair:dissipfriction_atom
0.1% pair:dissipdampn_atom
0.1% pair:dissipdampt_atom
Thanks,
Dariel