Feature pass MPI comm from Python script to AMReX during initialization (#2034)

* Added functionality to pass mpi comm from python script to amrex during initialization

* Fixed missing _ in MPI._sizeof()

* Added functions to get the current processor's rank and total number of processors

* Renamed MPI_Comm to _MPI_Comm_type and defined _MPI_Comm_type in except statement

* Updated comment to explain why mpi4py needs to be imported before loading libwarpx

* Removed ifdef flags that prevent amrex_init_with_inited_mpi from being declared when MPI is off

* Changed amrex_init_with_inited_mpi to be declared even when not using mpi, but will be defined to be functionally the same as amrex_init

* Defined MPI = None to signify whether MPI is used, to add another check when initializing amrex

* Changed ifdef blocks in WarpXWrappers.cpp/h to fix compile errors.
Added ifdef block to conditionally declare amrex_init_with_inited_mpi in WarpXWrappers.h to prevent compile error when not using MPI. Removed ifdef block to declare/define same function in WarpXWrappers.cpp since function needs to be declared even when MPI is not used, but will never be called in that case.

* Changed BL_USE_MPI to AMREX_USE_MPI and removed incorrect MPI=None statement

* Changed BL_USE_MPI to AMREX_USE_MPI

* Added test to verify correct passing of MPI communicator to amrex

* Added ability to pass mpi_comm to sim.step

* Change test to check for differeing outputs when passed different inputs

* Removed obsolete comments refactored program to use more shared code

* Refactored comments

* Updated description to match test

* Removed unecessary imports and updated comments

1 files changed, 8 insertions, 2 deletions

diff --git a/Source/Python/WarpXWrappers.cpp b/Source/Python/WarpXWrappers.cpp
index 60d675d3a..47942a55b 100644
--- a/Source/Python/WarpXWrappers.cpp
+++ b/Source/Python/WarpXWrappers.cpp
@@ -142,12 +142,10 @@ extern "C"
         warpx_amrex_init(argc, argv);
     }
 
-#ifdef BL_USE_MPI
     void amrex_init_with_inited_mpi (int argc, char* argv[], MPI_Comm mpicomm)
     {
         warpx_amrex_init(argc, argv, true, mpicomm);
     }
-#endif
 
     void amrex_finalize (int /*finalize_mpi*/)
     {
@@ -496,6 +494,14 @@ extern "C"
         return warpx.finestLevel ();
     }
 
+    int warpx_getMyProc () {
+        return amrex::ParallelDescriptor::MyProc();
+    }
+
+    int warpx_getNProcs () {
+        return amrex::ParallelDescriptor::NProcs();
+    }
+
     void mypc_Redistribute () {
         auto & mypc = WarpX::GetInstance().GetPartContainer();
         mypc.Redistribute();