aboutsummaryrefslogtreecommitdiff
path: root/Source/Parallelization (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-11-191D3V Cartesian Support (#2307)Gravatar Prabhat Kumar 3-15/+134
* Build System: Add 1D Geometry * test PR * test PR * 1D cartesian yee algorithm * fixed typo * Fixes for PML * 1D support related multiple changes * Fix compilation * change 1D to 1D_Z * 1D Field Gather and typo fix * 1D Charge Deposition * Particle Pusher * multiple changes related to 1D * 1D diagnostics and initialization * PlasmaInjector and PEC fixes for 1D * clean-up delete diags file * mobility 1D laser particle and bilinear filter * deleted diags files * update laser particle weight formula * delete diags files * Azure: Add 1D Cartesian Runner * 1D fixes for FieldProbe * Update Docs/source/developers/dimensionality.rst Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> * 1d laser injection and langmuir test input files * 1d tests * clean up : delete print statements * analyse simulation result for laser injection and Langmuir tests * EOL * delete input files for which there are no automated tests * delete input files for which there are no automated tests * add ignore_unused to remove warnings * remove space * Fix compilation issues * fix error : macro name must be an identifier * Small bug fix * cleanup Python script for analysis * bug fix * bug fix * Update ParticleProbe: Check 1D in-domain * Update Source/Make.WarpX * Update .azure-pipelines.yml * Add USE_OPENPMD=FALSE to .azure-pipeline.yml * resolve conflict * resolve conflict * fix typo * Correct out-of-bound access * Fix Particle BC in WarpXParticleContainer and correct path to checksumAPI in python analysis scripts * EOL * Fix bug : accessing out of bound index of cell in 1D * remove 1d test for cartesian3d * Fix CI check * Slight style change * Address review comments * Fix GPU compilation Filter.cpp * Fix CI * Fix Indentation * Address review comments * More consistent ifdef for dimension bigger than 1 * Update Examples/Tests/Langmuir/analysis_langmuir_multi_1d.py Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update GNUmakefile Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Regression/prepare_file_ci.py Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/FieldSolver/FiniteDifferenceSolver/FiniteDifferenceAlgorithms/CartesianCKCAlgorithm.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/FieldSolver/FiniteDifferenceSolver/FiniteDifferenceAlgorithms/CartesianNodalAlgorithm.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Initialization/PlasmaInjector.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Initialization/PlasmaInjector.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * add comment inline to explain twice push_back * Add amrex::Abort for NCIGodfreyFilter Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Prabhat Kumar <prabhatkumar@kraken.dhcp.lbl.gov> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Remi Lehe <rlehe@lbl.gov>
2021-11-18WarpXComm.cpp: Use Explicit Types, not `auto` (#2578)Gravatar Edoardo Zoni 1-26/+29
* WarpXComm.cpp: Use Explicit Types, not `auto` * Use MultiFab* const instead of MultiFab* const&
2021-11-18Bug fixes and cleanup in load balancing (#2563)Gravatar Roelof Groenewald 1-162/+66
* added helper function to rebuild MultiFabs and iMultiFabs during load balancing and included rebuilding of EB multifabs * added redistribute call for the particle boundary buffer during load balancing * consistently use DistribtionMap rather than dmap in ElectrostaticSolver.cpp * applied suggested changes from code review by Phil Miller * removed default argument for redistribute in RemakeMultiFab * removed RemakeMultiFab() as a member of WarpX * Only remake EB multifabs if they are used Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> * adapted existing particle scraping test (PICMI version) to also cover the redistribution of particle buffers from load balancing * added redeclaring of m_borrowing * Move redeclaring of m_borrow inside if statement for ECT solver algorihtm Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> * added calls to MarkCells and ComputeFaceExtensions * fixed issue causing CI test to fail and copied conditionals from WarpXInitData.cpp to recompute EB quantities * Guard cells communication for EB data when re-gridding (#105) * Add 2D circle EB test (#2538) * Added embedded_circle test * Add embedded_circle test files * Removed diag files * removed PICMI input file * Update to use default regression analysis * Added line breaks for spacing * Added description * Fixed benchmark file * Added load balancing to test * Commented out load_balancing portion of test. This will be added back in once load balancing is fixed. Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Added guard cells communication for EB data in regridding Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * moved all EB grid data calculations to a new function InitializeEBGridData() which is now called by both WarpX::InitLevelData and WarpX::RemakeLevel * Fix typo in doc string. Co-authored-by: Phil Miller <unmobile+gh@gmail.com> Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Phil Miller <unmobile+gh@gmail.com>
2021-11-15Bugfix in load balancing routine (#2555)Gravatar Roelof Groenewald 1-0/+8
* add remake of phi_fp during load balancing RemakeLevel * added phi_cp remake to RemakeLevel function * revert changes from previous commit
2021-11-09Fix some issues with Fujitsu compiler (#2529)Gravatar Luca Fedeli 1-2/+4
* make some code compilable with Fujitsu compiler in clang mode * update documentation
2021-10-29FieldProbe using Particle (#2430)Gravatar Tiberius Rheaume 1-0/+10
* FieldProbe using Particle Update FieldProbe.cpp Update FieldProbeParticleContainer.H Updates FieldProbe and FieldProbeParticleContainer * Make <diag>.integrate optional The param parser query keeps te default value if no entry is found. * Fixed number particle needed for AddNParticles * Removing unnecessary type definition * Added Doxygen-style comments to FieldProbe.cpp Corrected Poynting calculation by implementing vacuum permeability * Added Doxygen comments * Implement virtual function ReducedDiags::AllocData() + comments * InitData implemented * Fixed Doxygen commenting. * Now uses WarpX physics constant for vaccuum permeability * forgotton comments to MultiReducedDiags * Update Source/Diagnostics/ReducedDiags/FieldProbe.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbe.H * Update FieldProbe.cpp * Update Source/Diagnostics/ReducedDiags/ReducedDiags.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/MultiReducedDiags.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/MultiReducedDiags.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbeParticleContainer.H * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbeParticleContainer.cpp * Update FieldProbe.cpp * Update FieldProbe.H * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbeParticleContainer.cpp * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/ReducedDiags.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Changed enumerated class to struct w/ enumeration. Can remove "static_cast<int>" * FieldProbeParticleContainer::iterator implemented * Cleaned up output += operator, fixed output comments * style fix * Replaces Tabs with 4 spaces * Defined modes and interp order to avoid GPU compilation errors * 1 more tab fix * EoL white spaces * fixed a typoX * Explicitly capturing "this" in parallel for to combat error saying "error #3223-D: Implicit capture of 'this' in extended lambda expression" * removed unncessacesy double define * moved output out of ParallelFor. temp variable for integrate * Parse integrate, integrate all time steps, output setup for integrate and regular * Fixed integrate bug. * ammend header. integreate variable name change. * Integrate values in input file * updates to timing for integrate * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * whitespace * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Diagnostics/ReducedDiags/FieldProbeParticleContainer.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Apply suggestions from code review - Style Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update reduce_diag_names * field_probe_integrate change * review amends * Apply suggestions from code review - Style Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Vectors + AddNParticles Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbe.cpp * Update FieldProbe.H * bug fix and inputs * reintroduce raw_fields functionality * docs update and correction * whitespaces * Fix GPU Compile (raw_fields) * changed f_probe to m_probe apropriately * Typos Co-authored-by: David Grote <dpgrote@lbl.gov> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Better name for ParticleVal * used map for observables and units * Apply suggestions from code review Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Simplified output. Fixed double integration error * Update FieldProbe.H Removed unneeded variable * comments and fixed rawFields * white spaces * Apply suggestions from code review Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update FieldProbe.H * Guard on write * Update Source/Diagnostics/ReducedDiags/FieldProbe.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Fix Syntax Error in Write * Fix Init: Only 1 Particle (MPI) Only one MPI rank adds a particle, which we done distribute into the right rank. * Fix MPI Deadlock: No Early Return We just want to skip the write to `m_data`, not the rest of the logic. * Fix Probe in Domain Logic General global check, not only on a single rank. * Container: Add `const_iterator` * Fix MPI Comms * Cleaning * Remove PrintAll Leftover * Reduced Diags: Support LoadBalance * Cleaning of "Definitions ()" Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: David Grote <dpgrote@lbl.gov>
2021-10-25Bugfixes for ref_ratio = 4 in the moving window direction. (#2452)Gravatar Andrew Myers 2-11/+18
* Bugfixes for ref_ratio = 4 in the moving window direction. * == should be <= * only adjust guard cells for moving window by ref_ratio if nlevs > 1 * restore logic to always have at least 2 grow cells if moving window is on * remove comment about this not being needed on level 0 - it does.
2021-10-18Option to do single precision mesh communication. (#2294)Gravatar Andrew Myers 6-90/+442
* option to use single precision guard cell exhanges * add missing files * fix namespace issue * change precision of comms to float * ParmParse the single_precision_comms flag * set back to real * test * make sure dst is filled * nGrow -> nGrowVect * restore float * don't override valid cells * single precision mesh * whitespace * wrap SumBoundary * Wrap additional uses of FillBoundary. * catch missing copies of ParallelCopy * missing OverrideSyncs * add wrapper for iMultifab * fix typo * moar typos * typo * strip single_precision_mesh option * fix original copy * update fusible syntax Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> * Fix: Single Precision Builds Should not copy around data for `do_single_precision_comms` * Docs: warpx.do_single_precision_comms * initialize this tmp multifab to 0.0 * fix tiny profile label * remove orig copies, they are only correct for FillBoundary * loosen tolerance for space charge tests for single precision * missing some setVal * another missing setVal * missing setVal * add wrapper for new version of sumboundary * add explicit cast to silence compiler warning * add a test for single precision comms * revert change to test precision * add benchmark for single precision comms test * restore tolerance I removed by mistake * tolerance * copyright headers * drop tolerance for single precision tests in default analysis script * missing python module * bump tol again * fix bad merge * Apply suggestions from code review Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov>
2021-10-18Multi-J Algo: Synchronize Nodal Points of F,G (#2434)Gravatar Edoardo Zoni 1-0/+18
2021-10-12Fix PML Behavior in FillBoundary for F,G (#2383)Gravatar Edoardo Zoni 1-49/+27
* Fix PML Behavior in FillBoundary for F,G * Update Benchmark of Test pml_psatd_dive_divb_cleaning
2021-10-12Increase number of guard cells allocated with multi-J algorithm (#2377)Gravatar Neïl Zaim 2-4/+22
2021-10-11Use parser for more input parameters (#2386)Gravatar Neïl Zaim 2-7/+8
* Use parser for more input parameters * Fix PSATD compilation and apply suggestions from code review * Avoid out of bound array access for num_particles_per_cell_each_dim * Fix few input files with respect to num_particles_per_cell_each_dim * Fix get that incorrectly became query
2021-10-06FillBoundaryG: Add PML Exchange (#2381)Gravatar Edoardo Zoni 1-2/+10
2021-10-06NodalSync: Less Duplicate Code (#2375)Gravatar Edoardo Zoni 1-58/+14
2021-09-27SumBoundary: Use Fewer Guard Cells for J/rho (#2323)Gravatar Edoardo Zoni 2-21/+65
* SumBoundary: Use Fewer Guard Cells for J/rho * Account for Stencil of Bilinear Filter * Revert Last Commit, Replace SumBoundary/Copy With ParallelAdd * Account for Stencil of Bilinear Filter * Reset Benchmarks of Averaged Galilean Hybrid Tests * Revert Last Commit This reverts commit 7963176b46708eb08e7c41e67a3b0a6fe6cb2147. * Account for Stencil of Current Centering
2021-09-27Allow more work for OpenMP threads in `WarpX::UpdateAuxilaryData` (#2342)Gravatar Remi Lehe 1-7/+7
* Allow more work for OpenMP threads in AuxilaryData * Better parallelization in stagger-to-nodal current interpolation
2021-09-24Exchange fewer guard cells with standard PSATD solver (#2266)Gravatar Neïl Zaim 2-0/+7
* Try to exchange fewer guard cells with standard PSATD solver * Add ng_afterPushPSATD variable in GuardCellManager
2021-08-30ECT conformal solver (#1923)Gravatar Lorenzo Giacomel 1-1/+2
* adding base implementation of the conformal solver * adding some preprocessor directives * qualifying the isnan's correctly * getting rid of some unused fields * computing S_stab on the fly * using empty in length check * Source/FieldSolver/FiniteDifferenceSolver/EvolveB.cpp * replaced a looponcpu with a parallelfor * trying to not pass lending/borrowing info by reference in evolveB * passing data using dataPtr * converting inds into a device vector * simplifying some += * extracting the inds BaseFabs as DataPtr * Revert "extracting the inds BaseFabs as DataPtr" This reverts commit bc709d2fcaa347c119514de79a3f57169a05af65. * implementing new way ogf handling cell extensions (gpu compatible) * fixing some white spaces * removed a typo * made a function static * tidying up * tidying up * getting rid of the rho vector * moving the rho update to evolveE * refectoring the cell extension * small chenge in input parameters * updating WarpX.H * bug fix * fixing another bug * rotating the cube * updated gitignore * using the conformal soler in the cube test * fixing the signature of a function * trying to fix the setVal problem * Revert "trying to fix the setVal problem" This reverts commit c7d0e5e3709b730ff570183b2a6df5f587ca4640. * trying to fix the setVal problem * cleaning some comments * removing an abort in device code * Including geometric information in the external field initializer * cleaned up the test * moving the geometric data initialization before the external fields initialization * changing way of saving info about intruded cells * fixed some white spaces * Using S_mod instead of S_red and S_enl * substituting a std::vector with amrex::Array1D * bug fix in the uint8_t -> coords maps * Condensed ect cell info into one single MultiFab * fixing some includes * fixing some more includes * fixed a typo * making some functions available in gpu code * using tilebox instead of validbox for the geometry initialization * communicating the geometric info in the guard cells * fixing the guard cells initialization for ect * fixing an unused parameter * fixing an unused parameter * ignored some unused variables when EB is not supported * cleaning up * cleaning up * ignored some unused variables when EB is not supported * evolving rho just for ect * handling some more unused variables * removing some white spaces * adding the rotated cube test * removed some white spaces * change cells into faces * small fix in unused parameters * fixed a comment * for safety for now just initialize the geometric data when lev==maxLevel * adding some preprocessor directives to isolate EB code * updating Makefile stuff * Improving the rotated cube test * bug fix in mesh refinement * fixing boundary conditions in rotated cube test * deactivating MPI in rotated cube test * activating mpi in test * improving outputs of cells extension * improved the documentation * Update Source/EmbeddedBoundary/WarpXInitEB.cpp Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> * Update Source/EmbeddedBoundary/WarpXInitEB.cpp Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> * Removed some obsolete isnan's * undone change to gitignore * adding some missing AMREX_GPU_DEVICEs * Changing some amrex::Real(0) into 0. Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Making some variables const Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Adding amrex:: to some ignore_unused Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Enhancing readability of some parallelfor's Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Removed some unecessary includes * Fixing some tags Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Fixing the function CountExtFaces * added a comment for Rhofield * Fixed a typo in CountExtCells * trying to remove accesses to private members in ComputeEdgeLengths * making some functions public * undoing some useless changes * undoing some useless changes * adding some AMREX_GPU_DEVICEs * adding some unused variables * adding some AMREX_GPU_DEVICEs to fix warnings * fixing relative error * Making several variables const Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Fixed a comment Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * changing some double to amrex::Real * removing commented lines * changing some double to amrex::Real * removing commented lines * inizialing nelems_x,y,z to avoid a warning * Fixing number of cells * Adding missing analysis routine in rotated cube test Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * Making some variables constexpr Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> * Improving some reduce operations Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> * reading time directly from the output * fixed a few data types * fixing another int * Replacing or->||, and->&&, not->! * Adding license info * Adding a white space Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * removed some unused imports * Moving the doxygen documentation * Including AMReX_LayoutData.H * removing some useless parameters * adding ect solver to the doc * Removing some useless reductions * Small change for consistency * Handling the resizing of arrays * Handling correctly the resing of arrays * Fixing some whitespaces * Fixing an indentation * Improving a comment * Removed a useless initialization * Renamed Rhofield to ECTRhofield * Renamed Rhofield to ECTRhofield * Added some if conditions to isolate some EB related code * Improved some comments Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> * Made some functions not member of WarpX anymore * Isolated some EB-only code * Isolated some EB-only code * Using the square brackets to access vectors * Ignoring some unused variables * Bug fix in initialization * Removed a redundant if * Using enumeration for connectivity and improved names * Added a comment * added a few comments * Removed some useless template parameters * Printing some info only in verbose mode * Revert "Removed some useless template parameters" This reverts commit 2c527980e6872c0212767c27f00e2b53ecbcfd0a. * Fixed a bug with the neighbours enumeration * Initializing geom data alsoo in the ghost cells Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org>
2021-08-24WarpXComm.cpp: Fix Bug (Uninitialized MultiFabs) (#2230)Gravatar Edoardo Zoni 1-0/+6
2021-08-16Use less guard cells in ParallelCopy of refined level's UpdateAuxilaryData ↵Gravatar Neïl Zaim 1-8/+24
(#2144) * Use less guard cells in ParallelCopy of refined level's UpdateAuxilaryData * Update Source/Parallelization/WarpXComm.cpp * Add inline comments and use explicit type for ng_src Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com>
2021-07-16Infrastructure for interacting particles with embedded boundary walls (#2069)Gravatar Andrew Myers 1-0/+1
* Infrastructure for interacting particles with embedded boundary walls * remove debug prints * protect with AMREX_USE_EB * fix for 2D XZ * also update level set when regridding * rename level set to 'DistanceToEB' * add docstring for scrape particles. * add assertion on maxLevel() since EB does not work with mesh refinement right now. * m_eb_if_parser no longer exists * add test for particle aborption at embedded boundaries * fix bug I introduced refactoring * add new test to suite * fix test names * fix 2D * rookie python error * fix filename in test * fix script * fix unused * make sure we turn EB on in test
2021-07-13when regridding (say after a load balance), we need to regenerate the edge ↵Gravatar Andrew Myers 1-0/+4
and face multifabs. (#2079)
2021-07-12Add Cost Calculations for Cartesian/RZ Filtering (#2074)Gravatar Edoardo Zoni 1-8/+8
* Add Cost Calculations to ApplyFilter * Add Cost Calculations to ApplyStencil * Update Doxygen
2021-06-28Multi-J scheme (#1828)Gravatar Remi Lehe 1-0/+2
* Introduce new option skip_deposition * Properly implement the option to skip deposition * Skip deposition for electrostatic solver * Correct typo * Add Index Enumerator and Equations for F/G Without Averaging * Define new functions for current deposition and charge deposition * Disable interpolation between levels * Correct compilation error in RZ mode * Add argument for relative time * Add Index Enumerator and Equations for F/G With Averaging * [skip ci] Add new OneStep function * Fix compilation errors * Correct more compilation errors * [skip ci] Fix compilation * Split the PSATD push into separate functions * Add guards for rho field * [skip ci] Use new functions in OneStep * [skip ci] Separate the inverse transform of E_avg, B_avg * Add deposition of rho * [skip ci] Prevent deposition of rho if unallocated * Fix error in deposition function * Add subcycling of current deposition * [skip ci] Add inverse transform of averaged fields * [skip ci] Move component of rho * Add function to copy J * Temporarily deactivate contribution from F * [skip ci] Implement call to linear in J * Add psatd time averaging for multiJ * [skip ci] Fix implementation of averaging * [skip ci] Implement divE cleaning * Fix Bug for RZ Builds * Fix Bug for RZ Builds * Fix Bug in Init of PML Spectral Solvers * Cleaning * Cleaning * Add div(B) Cleaning (G Equation) * Multi-J Not Implemented with Galilean PSATD or PMLs * Add 2D CI Test Using Multi-J Scheme * Add More Inline Comments * Add More Inline Comments & Doxygen * Add Doxygen for Constructor of SpectralSolver * More Doxygen in Class SpectralSolver * Add Doxygen for New Functions in WarpXPushFieldsEM.cpp * Add Doxygen for New Functions in WarpX/MultiParticleContainer * do_dive/b_cleaning Must Be True With linear_in_J Option * Cast multij_n_depose to Real in Divisions * New Input Syntax * Add const where Possible, Fix Warnings * Docs for New Input Syntax, Fix Abort Messages * Consistent Use of Idx, IdxAvg, IdxLin * Improve Documentation of psatd.J_linear_in_time * Use const Type Qualifier whenever Possible * Simplify Initialization of Pointer ion_lev * Improve Doxygen * Update documentation * Add Note on NCI to Docs * Make warpx.do_multi_J_n_depositions Not Optional * Simplify Logic in getRequiredNumberOfFields * Use More const Type Qualifiers Co-authored-by: Edoardo Zoni <ezoni@lbl.gov>
2021-06-24Use "Include What You Use" on WarpX (#1947)Gravatar Luca Fedeli 7-19/+83
* initial tests with IWYU * added a couple of forward declarations * used iwyu on more files * progress * used iwyu on more files * progress with iwyu * progress with iwyu * fixed bug * progress with iwyu * progress with IWYU * progress with IWYU * fixed bug * fixed bug * progress with IWYU * progress with IWYU + use forward declarations in WarpX.H * first try with .def files * fix bugs * progress with IWYU * progress with IWYU * progress with iwyu * correct copyright * fixed bug * fixed bugs * fix missing include * fixed bug * fix bug * fix bug introduced during last bugfix * use iwyu on newly added files * add space * fix bug * fix missing include * fix missing include * fix missing include * fixed bugs * fixed bug * attempt at fixing issue with math functions * added missing include * fixed missing include * using _fwd.H * fixed bug * progress with iwyu * update AMReX branch * enforce alphabetic order * progress with iwyu * use right version of amrex * use right version of amrex for tests * fixed bug * fix another bug * fix missing include * fix missing include * fix missing include * updated amrex * initial work to document new include strategy * updated documentation * Fix rst & private includes * Remove accidentially added files * Fix rst code blocks * one more rst block Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2021-06-08Fix kernel timer: use after free (#2006)Gravatar Axel Huebl 1-5/+6
* KernelTimer: Fix Cost Use-After-Free AMReX' ParallelFor is async, the free of the cost function occurs too early can can lead to use-after-free's from the kernel. * CMake: WarpX_GPUCLOCK on CUDA/HIP Make conditional on supported backends. * Add addition to global cost array As we abstracted this from the original research prototypes (https://doi.org/10.5281/zenodo.4708449) to merge this to mainline in 1406, we forgot to add the addition to `*costs` at this point. Measurements were done with the code linked in the zenodo archive. * Cleaning: Move m_cost assignment more in Not too important, but with a default nullptr now possible. * synchronize -> streamSynchronize Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov>
2021-05-19Set guard cells for allocation using field solver stencil and particle shape ↵Gravatar Revathi Jambunathan 1-10/+36
factor (#1969)
2021-05-03Implement div(B) Cleaning With FDTD (#1829)Gravatar Edoardo Zoni 3-1/+72
* Implement div(B) Cleaning With FDTD * Add CI Test * Clean Up
2021-04-21Fix LoadBalance + MR: Rebuild Particle Masks (#1904)Gravatar Axel Huebl 1-2/+6
In MR simulations, particle masks are used to determine if a fine-patch particle is in the field gather and/or current deposition buffer regions near the coarse/fine boundaries. This is needed because some particles deposit to / gather from the coarse level, since they are near the level boundaries. On regriding during load-balancing, we can avoid to communicate the `MultiFab`s for those masks (gather and deposit) if we locally re-build the mask in the new distribution mapping. We forgot to trigger the rebuild, which caused illegal memory accesses in the particle evolve (partition) down the road.
2021-04-20Fix logic error when load balancing with more than 1 level. (#1903)Gravatar Andrew Myers 1-2/+6
* Fix logic error when load balancing with more than 1 level. * 0 -> false
2021-04-19Divergence cleaning for PSATD in PML (#1600)Gravatar Edoardo Zoni 1-0/+4
* Add div(E)/div(B) cleaning options for PSATD in PMLs * Pass missing flags to spectral solver in PML * Duplicate MPI exchange and communication functions for G * Use separate parameters for div cleaning in PMLs * Add asserts for features that are not implemented * Do not need to duplicate MPI exchange functions for G * Add short documentation for new input parameters * Set new parameters true by default with PSATD solver * Add CI test for PML div cleaning with PSATD * Use new syntax <diag_name>.intervals in new input file * Reset benchmark of new CI test * Always synchronize nodal points of G MultiFab * Fix few warnings in 2D build * Update Benchmark of pml_psatd_dive_divb_cleaning * Improve Documentation of warpx.do_pml_dive_cleaning Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com> * Improve Documentation of warpx.do_pml_divb_cleaning Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com> * Improve Abort Message * Clean Up Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com>
2021-04-15Hybrid Galilean: Same Guard Cells in z as Nodal (#1887)Gravatar Edoardo Zoni 1-11/+25
* Galilean Staggered: Same Guard Cells in z as Nodal * Update Benchmarks of Hybrid Tests
2021-04-12Add nodal synchronization of E and B fields in the main grid (#1817)Gravatar Neïl Zaim 1-0/+66
* Add nodal synchronization of E and B fields in the main grid * Update benchmarks * Update value in pml analysis file and put synchronization between PSATD push and FillBoundary * Update PML benchmarks and put the synchronization back in its original position * Bogus change in benchmark to trigger CI again * Update benchmarks
2021-04-08Fix Out-Of-Bound Access In Some Interpolation Functions (#1885)Gravatar Edoardo Zoni 1-12/+48
2021-04-05Compile-time option to use `algo.load_balance_costs_update = gpuclock` (#1838)Gravatar Michael E Rowan 1-12/+20
* compile time option for gpuclock eol move directives eol spaces assert fix GPU clock: CMake option * GPUCLOCK: ON by default Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2021-03-31Move KernelTimer to Parallelization (#1856)Gravatar Michael E Rowan 1-0/+75
2021-03-29Regrid spectral solver during load balance (#1815)Gravatar Michael E Rowan 1-8/+76
* add PSATD support when remaking level * eol * update_with_rho * Update WarpXRegrid.cpp * Upate WarpXRegrid.cpp * Update WarpXRegrid.cpp * WIP * Cleanup * Remove * Add function to allocate spectral solver * EOL * spaces * Remove comment * Update documentation * eol * Remove function arguments * fix * Move lines into function * eol * amrex::Real * args
2021-03-29Add option for finite-order centering of currents (nodal to staggered) (#1763)Gravatar Edoardo Zoni 2-82/+184
* Start adding centering of current * Implement arbitrary order centering and split inputs * No need to define a brand new interpolation function * Update input file of hybrid CI tests * Clean up * Clean up more * Fix bug and clean up * Use current centering in two existing CI tests * Update documentation * Move Calls To UpdateCurrentNodalToStag Into SyncCurrent * Add Doxygen For New Function UpdateCurrentNodalToStag * Add Doxygen For New Functions Used For Stencil Coefficients * Finite-Order Centering of Currents Not Implemented With MR
2021-03-18Use Clear Naming Standard for ParmParse Variables (#1809)Gravatar Edoardo Zoni 1-4/+4
2021-03-17Replaced almost all nGrow with nGrowVect (#1801)Gravatar David Grote 1-2/+2
2021-03-04Implement averaged algo on staggered grids & merge spectral classes (#1544)Gravatar Edoardo Zoni 1-12/+17
* Refactor and clean up some spectral classes * Abort when current correction or Vay deposition are not implemented * Implement general equations for averaged Galilean * Allocate averaged MultiFabs also when aux_is_nodal=1 and do_nodal=0 * Allocate +ngextra guard cells also for averaged MultiFabs * Make alias MultiFabs for averaged aux data * With averaging, interpolate from avg_fp (not fp) to aux * Fix some limits of the coefficients * Fix bug causing NaNs in spectral coefficients * Add 2D CI test with same analysis as nodal test * Add 3D CI test with same analysis as nodal test * Add limit that was not covered (knorm=0 && knorm_c!=0 && nu=0) * Allocate T2_coef only if Galilean algorithm is used * Allocate X4_coef only if Galilean algorithm is used * Remove extra ghost cell from 'avg_fp' MultiFabs
2021-03-04Remove extra ghost cell used with momentum-conserving gathering (#1758)Gravatar Edoardo Zoni 3-30/+16
2021-03-02CI: Always with FFTW/PSATD (#1587)Gravatar Axel Huebl 2-70/+69
* CI: Always with FFTW/PSATD Reduce build time by always building with FFTW and unifying PSATD tests. This also adds coverage for the case that we forget to make `#ifdef WARPX_USE_PSATD` runtime blocks, which can potentially be seen in FDTD runs now. * Remove worker with environment USE_WARPX_PSATD * Fix warpx_interp: Unconditional PSATD Fix the `warpx_interp` function to properly enable/disable PSATD logic based on runtime logic. Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com>
2021-02-25Finite-order field centering: few small optimizations (#1714)Gravatar Edoardo Zoni 1-49/+52
* Use re-ordered stencil coefficients to optimize interpolation * Few more small optimizations and clean-up
2021-02-12Fix bug in loop over boxes in WarpX::UpdateAuxilaryDataStagToNodal (#1691)Gravatar Edoardo Zoni 2-17/+23
* Clarify loop over boxes in WarpX::UpdateAuxilaryDataStagToNodal * Loop over fabbox instead and pad with zeros beyond ghost cells * Clean up names of source and destination arrays
2021-02-03Reduced diagnostic for load balance efficiency (#1673)Gravatar Michael E Rowan 1-0/+5
* load balance efficiency reduced diagnostic * eol * minor * eol * docs * Docs -1 explanation Add explanation about load balance efficiency before costs are recorded * eol
2021-02-02Refactor finite-order interpolation functions for momentum-conserving field ↵Gravatar Edoardo Zoni 2-303/+196
gathering (#1653) * Unify high-order Fornberg interpolation functions * Add Doxygen documentation for new function * Unify also FDTD linear interpolation functions * Improve new implementation * Compute stencil coefficients only at initialization * Small clean-up and optimization
2021-02-02Always synchronize nodal points of PML MultiFabs (#1669)Gravatar Edoardo Zoni 1-0/+35
* Always synchronize nodal points of PML MultiFabs * Reset benchmark for CI test pml_x_psatd * Use new separate functions NodalSyncPML
2021-01-29make sure we redefine the tmp particle tiles when we load balance. (#1658)Gravatar Andrew Myers 1-0/+1
2021-01-27Take time step into account to compute guard cells for J and rho (#1607)Gravatar Edoardo Zoni 2-8/+27
* Use IntVect for ng_J and ng_rho * Compute guard cells for J and rho based on dt * Reset some CI benchmarks * Fix rebase commit * Add back +1 cell for rho: fix remaining out-of-bound accesses * Simplify ASSERTS using new interface of amrex::numParticlesOutOfRange