Skip to Main Content U.S. Department of Energy
Computational Sciences & Mathematics

Software Catalog


CM4 both enables and contributes to a variety of open-source software designed to impact fundamental mathematics and materials science.

CAS–Concurrent Adaptive Sampling

The Concurrent Adaptive Sampling, or CAS, algorithm is an enhanced sampling technique for discovering transition states, pathways, and new metastable conformations in molecular dynamics (MD) simulations. It also produces free energy plots and kinetic rates. The CAS algorithm adaptively constructs discrete macrostates and builds a graph connecting these states based on observed transition rates. Importantly, it requires minimal knowledge about the system. The input typically is given as a number of collective variables (functions of the atomic positions), from a few up to 10-20. These variables can be arbitrarily defined. For example, CAS supports integer values and discontinuous variables. The algorithm generates parallel walkers that sample the conformational space. The algorithm strictly controls the computational cost while allowing the system to efficiently explore phase space and make progress toward the product states. CAS is written in Python and Bash. The CAS package can use any MD simulation program in a black box manner (no modification of the MD code required). CAS has a large amount of parallelism by running each simulation across different cores on a computer cluster.

The code is available for download at:


Nanorod CAS Simulations

ISPH–Implict SPH

The ISPH package solves large-scale multi-physics mesoscale flow problems using implicit smoothed particle hydrodynamics methods. It relies on an incremental pressure correction scheme (effective for flows with moderately small Reynolds numbers) and differential operator renormalizations to achieve second-order accuracy in both time and space. ISPH, built on LAMMPS and Trilinos, provides a massively parallel scalable simulation capability for incompressible, multiphase flow coupled with electrostatics and has been used to perform implicit SPH simulations of over 150 M particles.


ISPH simulation of porous media flow at pore scale. High-resolution magnetic resonance imaging of 6864 packed polystyrene beads was used to construct the pore geometry, and steady-state flow was simulated using 165 M particles on the Edison Cray XC30 at NERSC. The figures show steady-state flow profiles, where color indicates the velocity magnitude.


  • N. Trask, M. Maxey, K. Kim, M. Perego, M.L. Parks, K. Yang, and J. Xu, A scalable consistent second-order SPH solver for unsteady low Reynolds number flows, Computer Methods in Applied Mechanics and Engineering, 289, pp. 155-178, 2015. DOI: 10.1016/j.cma.2014.12.027.
  • X. Yang, Y. Mehmani, W.A. Perkins, A. Pasquali, M. Schönherr, K. Kim, M. Perego, M.L. Parks, N. Trask, M.T. Balhoff, M.C. Richmond, M. Geier, M. Krafczyk, L-S. Luo, A.M. Tartakovsky, and T.D. Scheibe, Intercomparison of 3D Pore-scale Flow and Solute Transport Simulation Methods, Advances in Water Resources, In Press, 2015.
  • N. Trask, K. Kim, A. Tartakovsky, M. Perego, M. L. Parks, A Highly-Scalable Implicit SPH Code for Simulating Single- and Multi-phase Flows in Geometrically Complex Bounded Domains, Journal of Computational Physics, Sandia Report SAND2015-5407C,,2015.
  • W. Pan, K. Kim, M. Perego, A. M. Tartakovsky, M. L. Parks, Modeling Electrokinetic Flows by Implicit Incompressible Smoothed Particle Hydrodynamics, Journal of Computational Physics, Submitted, 2015.


    This package provides methods for stochastic hybrid particle-continuum simulations based on fluctuating hydrodynamics in LAMMPS. “SELM” refers to the stochastic Eulerian Lagrangian method that allows for coupling particle-based simulations to continuum stochastic fields. The package includes momentum-conserving thermostats for dynamic implicit-solvent coarse-grained simulations, functionality to perform simulations subject to Lees-Edwards boundary conditions for rheological studies of emergent properties of soft materials, and a graphical user interface, referred to as “MANGO,” for setting up models.

    The package is available for download at:



    • Fluctuating Hydrodynamics Methods for Dynamic Coarse-Grained Implicit-Solvent Simulations in LAMMPS, Y. Wang, J. K. Sigurdsson, and P.J. Atzberger, SIAM J. Sci. Comp., Vol. 38, pp. S62-S77, (2016). DOI: 10.1137/15M1026390.
    • Stochastic Reductions for Inertial Fluid-Structure Interactions Subject to Thermal Fluctuations, G. Tabak and P.J. Atzberger, SIAM J. Appl. Math., 75(4), 1884–1914, (2015). DOI: 10.1137/15M1019088.
    • Incorporating Shear into Stochastic Eulerian Lagrangian Methods for Rheological Studies of Complex Fluids and Soft Materials., P.J. Atzberger, Physica D, Vol. 265, pg. 57–70, (2013). DOI: 10.1016/j.physd.2013.09.002.
    • Stochastic Eulerian Lagrangian Methods for Fluid Structure Interactions with Thermal Fluctuations, P.J. Atzberger, J. of Comp. Phys., 230, pp. 2821-2837, (2011). DOI: 10.1016/


    The USERMESO package of LAMMPS is a fully GPU-accelerated package for dissipative particle dynamics. Instead of merely being a translation of the conventional molecular dynamics, the package integrates several innovations that specifically targets CUDA devices, including:

    • Atomics-free warp-synchronous neighbor list construction algorithm
    • Two-level particle reordering that aligns with the cell list lattice boundaries for generating strictly monotonic neighbor list
    • Locally transposed neighbor list
    • Redesigned non-branching transcendental functions (sin, cos, pow, log, exp, etc.)
    • Overlapped pairwise force evaluation and halo exchange using CUDA streams for hiding communication and kernel launch latency
    • Radix sort with GPU stream support
    • Pairwise random number generation based on per-timestep binary particle signatures and the Tiny Encryption Algorithm.

    It can achieve tens of times speedup on a single CUDA GPU over 8-16 CPU cores. The work is featured in a NVIDIA Parallel Forall blog article, “Accelerating Dissipative Particle Dynamics Simulation on Tesla GPUs.”

    The package can be freely used and redistributed under the GPL v3 license, preferably with the following paper citation:

    Tang, Yu-Hang, and George Em Karniadakis. “Accelerating dissipative particle dynamics simulations on GPUs: Algorithms, numerics and applications.” Computer Physics Communications 185.11 (2014):2809-2822.

    The code is available for download at:

    Compilation Guide
    NVCC and an MPI implementation is required for compilation of the code.

    cd <working_copy>/src
    make yes-molecule
    make yes-user-meso
    make meso ARCH=[sm_30|sm_35|sm_52|sm_60|...]

    To run a simple example of DPD fluid

    cd <working_copy>/example/simple
    ../../src/lmp_meso -in [|] -var case [25|48|64] single-precision double-precison
    case: simulation box size (number density = 4)


    Amphiphilic polymers in microfluidic channel
    Membrane wrapping into a vesicle Multi-layer vesicle

    MUI–Multiscale Universal Interface

    Concurrently coupled numerical simulations using heterogeneous solvers are powerful tools for modeling multiscale phenomena. However, major modifications to existing codes often are required to enable them, posing significant difficulties. Multiscale Universal Interface (MUI) is capable of facilitating the coupling effort for a range of multiscale simulations. The library adopts a header-only form with minimal external dependency and can be easily dropped into existing codes. A data samplerconcept is introduced, combined with a hybrid dynamic/static typing mechanism, to create an easily customizable framework for solver-independent data interpretation. The library integrates MPI MPMD support and an asynchronous communication protocol to handle inter-solver information exchange irrespective of the solvers’ own MPI awareness. Template metaprogramming is heavily employed to simultaneously improve runtime performance and code flexibility. The library was validated by solving three different multiscale problems, which also serve to demonstrate the framework’s flexibility in handling heterogeneous models and solvers. In the first example, a Couette flow was simulated using two concurrently coupled smoothed particle hydrodynamics (SPH) simulations of different spatial resolutions. In the second example, the deterministic SPH method was coupled with the stochastic dissipative particle dynamics (DPD) method to study the effect of surface grafting on the surface’s hydrodynamics properties. The third example conjugates heat transfer between a solid and fluid domain by coupling the particle-based energy-conserving DPD (eDPD) method with the finite element method (FEM).

    The source code is dual-licensed under both GPL v3 and Apache v2.

    The code is available for download at:


    Tang Y.-H., Kudo, S., Bian, X., Li, Z., and Karniadakis, G. E. “Multiscale Universal Interface: A Concurrent Framework for Coupling Heterogeneous Solvers,” Journal of Computational Physics, 2015, 297.15, 13-31.

    Should you have any questions, please contact .


    Concurrently coupled simulation of conjugate heat transfer using FEM-DPD
    DPD-SPH coupled simulation of flow past a polymer-grafted surface

Collaboratory on Mathematics for Mesoscopic Modeling of Materials (CM4)


Seminar Series

Computing Research at PNNL

Science at PNNL

Computing Research

View All Highlights