Nektar++
Protected Member Functions | Protected Attributes | List of all members
Nektar::SolverUtils::DriverParallelInTime Class Reference

Base class for the development of parallel-in-time solvers. More...

#include <DriverParallelInTime.h>

Inheritance diagram for Nektar::SolverUtils::DriverParallelInTime:
[legend]

Protected Member Functions

SOLVER_UTILS_EXPORT DriverParallelInTime (const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
 Constructor. More...
 
SOLVER_UTILS_EXPORT ~DriverParallelInTime () override=default
 Destructor. More...
 
SOLVER_UTILS_EXPORT void v_InitObject (std::ostream &out=std::cout) override
 Virtual function for initialisation implementation. More...
 
SOLVER_UTILS_EXPORT void v_Execute (std::ostream &out=std::cout) override
 Virtual function for solve implementation. More...
 
void SetParallelInTimeEquationSystem (std::string AdvectiveType)
 
void GetParametersFromSession (void)
 
void InitialiseEqSystem (bool turnoff_output)
 
void InitialiseInterpolationField (void)
 
void PrintSolverInfo (std::ostream &out=std::cout)
 
void PrintHeader (const std::string &title, const char c)
 
void RecvFromPreviousProc (Array< OneD, Array< OneD, NekDouble > > &array, int &convergence)
 
void RecvFromPreviousProc (Array< OneD, NekDouble > &array)
 
void SendToNextProc (Array< OneD, Array< OneD, NekDouble > > &array, int &convergence)
 
void SendToNextProc (Array< OneD, NekDouble > &array)
 
void CopySolutionVector (const Array< OneD, const Array< OneD, NekDouble > > &in, Array< OneD, Array< OneD, NekDouble > > &out)
 
void CopyFromPhysField (const size_t timeLevel, Array< OneD, Array< OneD, NekDouble > > &out)
 
void CopyToPhysField (const size_t timeLevel, const Array< OneD, const Array< OneD, NekDouble > > &in)
 
void UpdateFieldCoeffs (const size_t timeLevel, const Array< OneD, const Array< OneD, NekDouble > > &in=NullNekDoubleArrayOfArray)
 
void EvaluateExactSolution (const size_t timeLevel, const NekDouble &time)
 
void SolutionConvergenceMonitoring (const size_t timeLevel, const size_t iter)
 
void SolutionConvergenceSummary (const size_t timeLevel)
 
void UpdateErrorNorm (const size_t timeLevel, const bool normalized)
 
void PrintErrorNorm (const size_t timeLevel, const bool normalized)
 
NekDouble vL2ErrorMax (void)
 
NekDouble EstimateCommunicationTime (Array< OneD, Array< OneD, NekDouble > > &buffer1, Array< OneD, Array< OneD, NekDouble > > &buffer2)
 
void Interpolate (const Array< OneD, MultiRegions::ExpListSharedPtr > &infield, const Array< OneD, MultiRegions::ExpListSharedPtr > &outfield, const Array< OneD, Array< OneD, NekDouble > > &inarray, Array< OneD, Array< OneD, NekDouble > > &outarray)
 
- Protected Member Functions inherited from Nektar::SolverUtils::Driver
 Driver (const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
 Initialises EquationSystem class members. More...
 
virtual SOLVER_UTILS_EXPORT void v_InitObject (std::ostream &out=std::cout)
 Virtual function for initialisation implementation. More...
 
virtual SOLVER_UTILS_EXPORT void v_Execute (std::ostream &out=std::cout)=0
 Virtual function for solve implementation. More...
 

Protected Attributes

NekDouble m_totalTime
 Total time integration interval. More...
 
NekDouble m_chunkTime
 Time integration interval per chunk. More...
 
NekDouble m_time
 Local time. More...
 
size_t m_numChunks
 Number of time chunks. More...
 
size_t m_chunkRank
 Rank in time. More...
 
size_t m_iterMaxPIT
 Maximum number of parallel-in-time iteration. More...
 
size_t m_numWindowsPIT
 
bool m_exactSolution
 Using exact solution to compute error norms. More...
 
NekDouble m_tolerPIT
 ParallelInTime tolerance. More...
 
size_t m_nVar
 Number of variables. More...
 
size_t m_nTimeLevel
 Number of time levels. More...
 
Array< OneD, size_t > m_nsteps
 Number of time steps for each time level. More...
 
Array< OneD, NekDoublem_timestep
 Time step for each time level. More...
 
Array< OneD, size_t > m_npts
 Number of dof for each time level. More...
 
Array< OneD, std::shared_ptr< UnsteadySystem > > m_EqSys
 Equation system to solve. More...
 
Array< OneD, NekDoublem_vL2Errors
 Storage for parallel-in-time iteration. More...
 
Array< OneD, NekDoublem_vLinfErrors
 
Array< OneD, Array< OneD, NekDouble > > m_exactsoln
 
- Protected Attributes inherited from Nektar::SolverUtils::Driver
LibUtilities::CommSharedPtr m_comm
 Communication object. More...
 
LibUtilities::SessionReaderSharedPtr m_session
 Session reader object. More...
 
LibUtilities::SessionReaderSharedPtr session_LinNS
 Coupling between SFD and arnoldi. More...
 
SpatialDomains::MeshGraphSharedPtr m_graph
 MeshGraph object. More...
 
Array< OneD, EquationSystemSharedPtrm_equ
 Equation system to solve. More...
 
int m_nequ
 number of equations More...
 
enum EvolutionOperatorType m_EvolutionOperator
 Evolution Operator. More...
 

Additional Inherited Members

- Public Member Functions inherited from Nektar::SolverUtils::Driver
virtual ~Driver ()=default
 Destructor. More...
 
SOLVER_UTILS_EXPORT void InitObject (std::ostream &out=std::cout)
 Initialise Object. More...
 
SOLVER_UTILS_EXPORT void Execute (std::ostream &out=std::cout)
 Execute driver. More...
 
SOLVER_UTILS_EXPORT Array< OneD, EquationSystemSharedPtrGetEqu ()
 
- Static Protected Attributes inherited from Nektar::SolverUtils::Driver
static std::string evolutionOperatorLookupIds []
 
static std::string evolutionOperatorDef
 
static std::string driverDefault
 

Detailed Description

Base class for the development of parallel-in-time solvers.

Definition at line 45 of file DriverParallelInTime.h.

Constructor & Destructor Documentation

◆ DriverParallelInTime()

Nektar::SolverUtils::DriverParallelInTime::DriverParallelInTime ( const LibUtilities::SessionReaderSharedPtr  pSession,
const SpatialDomains::MeshGraphSharedPtr  pGraph 
)
protected

Constructor.

Definition at line 55 of file DriverParallelInTime.cpp.

58 : Driver(pSession, pGraph)
59{
60}
Driver(const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
Initialises EquationSystem class members.
Definition: Driver.cpp:75

◆ ~DriverParallelInTime()

SOLVER_UTILS_EXPORT Nektar::SolverUtils::DriverParallelInTime::~DriverParallelInTime ( )
overrideprotecteddefault

Destructor.

Member Function Documentation

◆ CopyFromPhysField()

void Nektar::SolverUtils::DriverParallelInTime::CopyFromPhysField ( const size_t  timeLevel,
Array< OneD, Array< OneD, NekDouble > > &  out 
)
protected

Definition at line 428 of file DriverParallelInTime.cpp.

430{
431 for (size_t i = 0; i < m_nVar; ++i)
432 {
433 m_EqSys[timeLevel]->CopyFromPhysField(i, out[i]);
434 }
435}
Array< OneD, std::shared_ptr< UnsteadySystem > > m_EqSys
Equation system to solve.

References m_EqSys, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ CopySolutionVector()

void Nektar::SolverUtils::DriverParallelInTime::CopySolutionVector ( const Array< OneD, const Array< OneD, NekDouble > > &  in,
Array< OneD, Array< OneD, NekDouble > > &  out 
)
protected

Definition at line 415 of file DriverParallelInTime.cpp.

418{
419 for (size_t i = 0; i < m_nVar; ++i)
420 {
421 Vmath::Vcopy(in[i].size(), in[i], 1, out[i], 1);
422 }
423}
void Vcopy(int n, const T *x, const int incx, T *y, const int incy)
Definition: Vmath.hpp:825

References m_nVar, and Vmath::Vcopy().

Referenced by Nektar::SolverUtils::DriverPFASST::RestrictResidual(), and Nektar::SolverUtils::DriverPFASST::RestrictSolution().

◆ CopyToPhysField()

void Nektar::SolverUtils::DriverParallelInTime::CopyToPhysField ( const size_t  timeLevel,
const Array< OneD, const Array< OneD, NekDouble > > &  in 
)
protected

Definition at line 440 of file DriverParallelInTime.cpp.

442{
443 for (size_t i = 0; i < m_nVar; ++i)
444 {
445 m_EqSys[timeLevel]->CopyToPhysField(i, in[i]);
446 m_EqSys[timeLevel]->UpdateFields()[i]->SetPhysState(true);
447 }
448}

References m_EqSys, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ EstimateCommunicationTime()

NekDouble Nektar::SolverUtils::DriverParallelInTime::EstimateCommunicationTime ( Array< OneD, Array< OneD, NekDouble > > &  buffer1,
Array< OneD, Array< OneD, NekDouble > > &  buffer2 
)
protected

Definition at line 567 of file DriverParallelInTime.cpp.

570{
571 if (m_numChunks == 1)
572 {
573 return 0.0;
574 }
575 else
576 {
577 // Average communication time over niter iteration.
578 size_t niter = 20;
580 for (size_t n = 0; n <= niter; n++)
581 {
582 if (n == 1)
583 {
584 timer.Start(); // Ignore the first iteration
585 }
586
587 if (m_chunkRank == 0)
588 {
589 for (size_t i = 0; i < buffer1.size(); ++i)
590 {
591 m_comm->GetTimeComm()->Send(m_numChunks - 1, buffer1[i]);
592 }
593 }
594
595 if (m_chunkRank == m_numChunks - 1)
596 {
597 for (size_t i = 0; i < buffer2.size(); ++i)
598 {
599 m_comm->GetTimeComm()->Recv(0, buffer2[i]);
600 }
601 }
602 }
603 timer.Stop();
604 return timer.Elapsed().count() / niter;
605 }
606}
LibUtilities::CommSharedPtr m_comm
Communication object.
Definition: Driver.h:80

References Nektar::LibUtilities::Timer::Elapsed(), m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, Nektar::LibUtilities::Timer::Start(), and Nektar::LibUtilities::Timer::Stop().

◆ EvaluateExactSolution()

void Nektar::SolverUtils::DriverParallelInTime::EvaluateExactSolution ( const size_t  timeLevel,
const NekDouble time 
)
protected

Definition at line 471 of file DriverParallelInTime.cpp.

473{
474 for (size_t i = 0; i < m_nVar; ++i)
475 {
476 m_EqSys[timeLevel]->EvaluateExactSolution(i, m_exactsoln[i], time);
477 }
478}
Array< OneD, Array< OneD, NekDouble > > m_exactsoln

References m_EqSys, m_exactsoln, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ GetParametersFromSession()

void Nektar::SolverUtils::DriverParallelInTime::GetParametersFromSession ( void  )
protected

Definition at line 251 of file DriverParallelInTime.cpp.

252{
253 // Parallel-in-Time iteration parameters.
254 m_tolerPIT = m_session->DefinesParameter("PITToler")
255 ? m_session->GetParameter("PITToler")
256 : 1.0E-16;
257 m_iterMaxPIT = m_session->DefinesParameter("PITIterMax")
258 ? m_session->GetParameter("PITIterMax")
259 : m_numChunks;
260 m_numWindowsPIT = m_session->DefinesParameter("NumWindows")
261 ? m_session->GetParameter("NumWindows")
262 : 1;
263
264 // Other parameters.
265 m_exactSolution = m_session->DefinesParameter("ExactSolution")
266 ? m_session->GetParameter("ExactSolution")
267 : 0;
268}
LibUtilities::SessionReaderSharedPtr m_session
Session reader object.
Definition: Driver.h:83
size_t m_iterMaxPIT
Maximum number of parallel-in-time iteration.
NekDouble m_tolerPIT
ParallelInTime tolerance.
bool m_exactSolution
Using exact solution to compute error norms.

References m_exactSolution, m_iterMaxPIT, m_numChunks, m_numWindowsPIT, Nektar::SolverUtils::Driver::m_session, and m_tolerPIT.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ InitialiseEqSystem()

void Nektar::SolverUtils::DriverParallelInTime::InitialiseEqSystem ( bool  turnoff_output)
protected

Definition at line 273 of file DriverParallelInTime.cpp.

274{
275 m_nVar = m_EqSys[0]->GetNvariables();
276
277 // Initialize fine solver.
278 if (turnoff_output)
279 {
280 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
281 {
282 m_EqSys[timeLevel]->SetInfoSteps(0);
283 m_EqSys[timeLevel]->SetCheckpointSteps(0);
284 }
285 }
286 m_EqSys[0]->DoInitialise(true);
287
288 // Initialize coarse solver(s).
289 for (size_t timeLevel = 1; timeLevel < m_nTimeLevel; timeLevel++)
290 {
291 m_EqSys[timeLevel]->DoInitialise(false);
292 }
293
294 // Initialize time stepping parameters.
295 m_timestep = Array<OneD, NekDouble>(m_nTimeLevel);
296 m_nsteps = Array<OneD, size_t>(m_nTimeLevel);
297 m_npts = Array<OneD, size_t>(m_nTimeLevel);
298 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
299 {
300 m_timestep[timeLevel] =
301 m_EqSys[timeLevel]->EquationSystem::GetTimeStep();
302 m_nsteps[timeLevel] = m_EqSys[timeLevel]->EquationSystem::GetSteps();
303 m_npts[timeLevel] = m_EqSys[timeLevel]->EquationSystem::GetNpoints();
304 }
305
306 // Initialize errors.
307 m_exactsoln = Array<OneD, Array<OneD, NekDouble>>(m_nVar);
308 for (size_t i = 0; i < m_nVar; ++i)
309 {
310 m_exactsoln[i] = Array<OneD, NekDouble>(m_npts[0], 0.0);
311 }
312 m_vL2Errors = Array<OneD, NekDouble>(m_nVar, 0.0);
313 m_vLinfErrors = Array<OneD, NekDouble>(m_nVar, 0.0);
314}
Array< OneD, NekDouble > m_vL2Errors
Storage for parallel-in-time iteration.
Array< OneD, size_t > m_nsteps
Number of time steps for each time level.
Array< OneD, size_t > m_npts
Number of dof for each time level.
Array< OneD, NekDouble > m_timestep
Time step for each time level.

References m_EqSys, m_exactsoln, m_npts, m_nsteps, m_nTimeLevel, m_nVar, m_timestep, m_vL2Errors, and m_vLinfErrors.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ InitialiseInterpolationField()

void Nektar::SolverUtils::DriverParallelInTime::InitialiseInterpolationField ( void  )
protected

◆ Interpolate()

void Nektar::SolverUtils::DriverParallelInTime::Interpolate ( const Array< OneD, MultiRegions::ExpListSharedPtr > &  infield,
const Array< OneD, MultiRegions::ExpListSharedPtr > &  outfield,
const Array< OneD, Array< OneD, NekDouble > > &  inarray,
Array< OneD, Array< OneD, NekDouble > > &  outarray 
)
protected

Definition at line 611 of file DriverParallelInTime.cpp.

616{
617 if (infield.size() != outfield.size())
618 {
619 NEKERROR(ErrorUtil::efatal, "not the same number of variables")
620 }
621
622 for (size_t n = 0; n < infield.size(); ++n)
623 {
624 // Interpolation from infield -> outfield assuming that infield and
625 // outfield are the same explists, but at potentially different
626 // polynomial orders.
627 if (infield[n]->GetExpSize() != outfield[n]->GetExpSize())
628 {
629 NEKERROR(ErrorUtil::efatal, "not the same mesh")
630 }
631
632 // Assign input/output array.
633 auto inphys = (inarray == NullNekDoubleArrayOfArray)
634 ? infield[n]->UpdatePhys()
635 : inarray[n];
636 auto outphys = (outarray == NullNekDoubleArrayOfArray)
637 ? outfield[n]->UpdatePhys()
638 : outarray[n];
639
640 // If same polynomial orders, simply copy solution.
641 if (infield[n]->GetTotPoints() == outfield[n]->GetTotPoints())
642 {
643 Vmath::Vcopy(infield[n]->GetTotPoints(), inphys, 1, outphys, 1);
644 }
645 // If different polynomial orders, interpolate solution.
646 else
647 {
648 for (size_t i = 0; i < infield[n]->GetExpSize(); ++i)
649 {
650 // Get the elements.
651 auto inElmt = infield[n]->GetExp(i);
652 auto outElmt = outfield[n]->GetExp(i);
653
654 // Get the offset of elements in the storage arrays.
655 size_t inoffset = infield[n]->GetPhys_Offset(i);
656 size_t outoffset = outfield[n]->GetPhys_Offset(i);
657
658 // Transform solution from physical to coefficient space.
659 Array<OneD, NekDouble> incoeff(inElmt->GetNcoeffs());
660 inElmt->FwdTrans(inphys + inoffset, incoeff);
661
662 // Interpolate elements.
664 if (inElmt->DetShapeType() == LibUtilities::Seg)
665 {
666 expPtr = std::make_shared<StdRegions::StdSegExp>(
667 LibUtilities::BasisKey(
668 inElmt->GetBasis(0)->GetBasisType(),
669 inElmt->GetBasis(0)->GetNumModes(),
670 outElmt->GetBasis(0)->GetPointsKey()));
671 }
672 else if (inElmt->DetShapeType() == LibUtilities::Quad)
673 {
674 expPtr = std::make_shared<StdRegions::StdQuadExp>(
675 LibUtilities::BasisKey(
676 inElmt->GetBasis(0)->GetBasisType(),
677 inElmt->GetBasis(0)->GetNumModes(),
678 outElmt->GetBasis(0)->GetPointsKey()),
679 LibUtilities::BasisKey(
680 inElmt->GetBasis(1)->GetBasisType(),
681 inElmt->GetBasis(1)->GetNumModes(),
682 outElmt->GetBasis(1)->GetPointsKey()));
683 }
684 else if (inElmt->DetShapeType() == LibUtilities::Tri)
685 {
686 expPtr = std::make_shared<StdRegions::StdTriExp>(
687 LibUtilities::BasisKey(
688 inElmt->GetBasis(0)->GetBasisType(),
689 inElmt->GetBasis(0)->GetNumModes(),
690 outElmt->GetBasis(0)->GetPointsKey()),
691 LibUtilities::BasisKey(
692 inElmt->GetBasis(1)->GetBasisType(),
693 inElmt->GetBasis(1)->GetNumModes(),
694 outElmt->GetBasis(1)->GetPointsKey()));
695 }
696 else if (inElmt->DetShapeType() == LibUtilities::Hex)
697 {
698 expPtr = std::make_shared<StdRegions::StdHexExp>(
699 LibUtilities::BasisKey(
700 inElmt->GetBasis(0)->GetBasisType(),
701 inElmt->GetBasis(0)->GetNumModes(),
702 outElmt->GetBasis(0)->GetPointsKey()),
703 LibUtilities::BasisKey(
704 inElmt->GetBasis(1)->GetBasisType(),
705 inElmt->GetBasis(1)->GetNumModes(),
706 outElmt->GetBasis(1)->GetPointsKey()),
707 LibUtilities::BasisKey(
708 inElmt->GetBasis(2)->GetBasisType(),
709 inElmt->GetBasis(2)->GetNumModes(),
710 outElmt->GetBasis(2)->GetPointsKey()));
711 }
712 else if (inElmt->DetShapeType() == LibUtilities::Prism)
713 {
714 expPtr = std::make_shared<StdRegions::StdPrismExp>(
715 LibUtilities::BasisKey(
716 inElmt->GetBasis(0)->GetBasisType(),
717 inElmt->GetBasis(0)->GetNumModes(),
718 outElmt->GetBasis(0)->GetPointsKey()),
719 LibUtilities::BasisKey(
720 inElmt->GetBasis(1)->GetBasisType(),
721 inElmt->GetBasis(1)->GetNumModes(),
722 outElmt->GetBasis(1)->GetPointsKey()),
723 LibUtilities::BasisKey(
724 inElmt->GetBasis(2)->GetBasisType(),
725 inElmt->GetBasis(2)->GetNumModes(),
726 outElmt->GetBasis(2)->GetPointsKey()));
727 }
728 else if (inElmt->DetShapeType() == LibUtilities::Pyr)
729 {
730 expPtr = std::make_shared<StdRegions::StdPyrExp>(
731 LibUtilities::BasisKey(
732 inElmt->GetBasis(0)->GetBasisType(),
733 inElmt->GetBasis(0)->GetNumModes(),
734 outElmt->GetBasis(0)->GetPointsKey()),
735 LibUtilities::BasisKey(
736 inElmt->GetBasis(1)->GetBasisType(),
737 inElmt->GetBasis(1)->GetNumModes(),
738 outElmt->GetBasis(1)->GetPointsKey()),
739 LibUtilities::BasisKey(
740 inElmt->GetBasis(2)->GetBasisType(),
741 inElmt->GetBasis(2)->GetNumModes(),
742 outElmt->GetBasis(2)->GetPointsKey()));
743 }
744 else if (inElmt->DetShapeType() == LibUtilities::Tet)
745 {
746 expPtr = std::make_shared<StdRegions::StdTetExp>(
747 LibUtilities::BasisKey(
748 inElmt->GetBasis(0)->GetBasisType(),
749 inElmt->GetBasis(0)->GetNumModes(),
750 outElmt->GetBasis(0)->GetPointsKey()),
751 LibUtilities::BasisKey(
752 inElmt->GetBasis(1)->GetBasisType(),
753 inElmt->GetBasis(1)->GetNumModes(),
754 outElmt->GetBasis(1)->GetPointsKey()),
755 LibUtilities::BasisKey(
756 inElmt->GetBasis(2)->GetBasisType(),
757 inElmt->GetBasis(2)->GetNumModes(),
758 outElmt->GetBasis(2)->GetPointsKey()));
759 }
760
761 // Transform solution from coefficient to physical space.
762 Array<OneD, NekDouble> tmp = outphys + outoffset;
763 expPtr->BwdTrans(incoeff, tmp);
764 }
765 }
766 }
767}
#define NEKERROR(type, msg)
Assert Level 0 – Fundamental assert which is used whether in FULLDEBUG, DEBUG or OPT compilation mode...
Definition: ErrorUtil.hpp:202
std::shared_ptr< StdExpansion > StdExpansionSharedPtr
static Array< OneD, Array< OneD, NekDouble > > NullNekDoubleArrayOfArray

References Nektar::ErrorUtil::efatal, Nektar::LibUtilities::Hex, NEKERROR, Nektar::NullNekDoubleArrayOfArray, Nektar::LibUtilities::Prism, Nektar::LibUtilities::Pyr, Nektar::LibUtilities::Quad, Nektar::LibUtilities::Seg, Nektar::LibUtilities::Tet, Nektar::LibUtilities::Tri, and Vmath::Vcopy().

Referenced by Nektar::SolverUtils::DriverParareal::InterpolateCoarseSolution(), Nektar::SolverUtils::DriverParareal::UpdateInitialConditionFromSolver(), and Nektar::SolverUtils::DriverParareal::UpdateSolverInitialCondition().

◆ PrintErrorNorm()

void Nektar::SolverUtils::DriverParallelInTime::PrintErrorNorm ( const size_t  timeLevel,
const bool  normalized 
)
protected

Definition at line 517 of file DriverParallelInTime.cpp.

519{
520 if (m_chunkRank == m_numChunks - 1 &&
521 m_comm->GetSpaceComm()->GetRank() == 0)
522 {
523 for (size_t i = 0; i < m_nVar; ++i)
524 {
525 if (normalized)
526 {
527 std::cout << "L2 error (variable "
528 << m_EqSys[timeLevel]->GetVariable(i)
529 << ") : " << m_vL2Errors[i] << std::endl
530 << std::flush;
531 std::cout << "Linf error (variable "
532 << m_EqSys[timeLevel]->GetVariable(i)
533 << ") : " << m_vLinfErrors[i] << std::endl
534 << std::flush;
535 }
536 else
537 {
538 std::cout << "L 2 error (variable "
539 << m_EqSys[timeLevel]->GetVariable(i)
540 << ") : " << m_vL2Errors[i] << std::endl
541 << std::flush;
542 std::cout << "L inf error (variable "
543 << m_EqSys[timeLevel]->GetVariable(i)
544 << ") : " << m_vLinfErrors[i] << std::endl
545 << std::flush;
546 }
547 }
548 }
549}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, m_numChunks, m_nVar, m_vL2Errors, and m_vLinfErrors.

Referenced by SolutionConvergenceMonitoring(), SolutionConvergenceSummary(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ PrintHeader()

void Nektar::SolverUtils::DriverParallelInTime::PrintHeader ( const std::string &  title,
const char  c 
)
protected

Definition at line 343 of file DriverParallelInTime.cpp.

344{
345 if (m_chunkRank == m_numChunks - 1 &&
346 m_comm->GetSpaceComm()->GetRank() == 0)
347 {
348 std::cout << std::endl;
349 std::cout << std::string(43, c) << std::endl << std::flush;
350 std::cout << title << std::endl << std::flush;
351 std::cout << std::string(43, c) << std::endl << std::flush;
352 }
353}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, and CG_Iterations::title.

Referenced by SolutionConvergenceMonitoring(), Nektar::SolverUtils::DriverParareal::v_Execute(), Nektar::SolverUtils::DriverPFASST::v_Execute(), and Nektar::SolverUtils::DriverParareal::WriteTimeChunkOuput().

◆ PrintSolverInfo()

void Nektar::SolverUtils::DriverParallelInTime::PrintSolverInfo ( std::ostream &  out = std::cout)
protected

Definition at line 319 of file DriverParallelInTime.cpp.

320{
321 if (m_chunkRank == 0 && m_comm->GetSpaceComm()->GetRank() == 0)
322 {
323 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
324 {
325 std::cout << std::string(71, '=') << std::endl << std::flush;
326 std::cout << "=========================== TIME LEVEL " +
327 std::to_string(timeLevel) +
328 " INFO "
329 "========================="
330 << std::endl
331 << std::flush;
332
333 m_EqSys[timeLevel]->PrintSummary(out);
334
335 std::cout << std::endl << std::flush;
336 }
337 }
338}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, and m_nTimeLevel.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ RecvFromPreviousProc() [1/2]

void Nektar::SolverUtils::DriverParallelInTime::RecvFromPreviousProc ( Array< OneD, Array< OneD, NekDouble > > &  array,
int &  convergence 
)
protected

Definition at line 358 of file DriverParallelInTime.cpp.

360{
361 if (m_chunkRank > 0)
362 {
363 if (!convergence)
364 {
365 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, convergence);
366 for (size_t i = 0; i < m_nVar; ++i)
367 {
368 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, array[i]);
369 }
370 }
371 }
372}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ RecvFromPreviousProc() [2/2]

void Nektar::SolverUtils::DriverParallelInTime::RecvFromPreviousProc ( Array< OneD, NekDouble > &  array)
protected

Definition at line 377 of file DriverParallelInTime.cpp.

378{
379 if (m_chunkRank > 0)
380 {
381 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, array);
382 }
383}

References m_chunkRank, and Nektar::SolverUtils::Driver::m_comm.

◆ SendToNextProc() [1/2]

void Nektar::SolverUtils::DriverParallelInTime::SendToNextProc ( Array< OneD, Array< OneD, NekDouble > > &  array,
int &  convergence 
)
protected

Definition at line 388 of file DriverParallelInTime.cpp.

390{
391 if (m_chunkRank < m_numChunks - 1)
392 {
393 m_comm->GetTimeComm()->Send(m_chunkRank + 1, convergence);
394 for (size_t i = 0; i < m_nVar; ++i)
395 {
396 m_comm->GetTimeComm()->Send(m_chunkRank + 1, array[i]);
397 }
398 }
399}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ SendToNextProc() [2/2]

void Nektar::SolverUtils::DriverParallelInTime::SendToNextProc ( Array< OneD, NekDouble > &  array)
protected

Definition at line 404 of file DriverParallelInTime.cpp.

405{
406 if (m_chunkRank < m_numChunks - 1)
407 {
408 m_comm->GetTimeComm()->Send(m_chunkRank + 1, array);
409 }
410}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, and m_numChunks.

◆ SetParallelInTimeEquationSystem()

void Nektar::SolverUtils::DriverParallelInTime::SetParallelInTimeEquationSystem ( std::string  AdvectiveType)
protected

Set the ParallelInTime (coarse solver) session file

Definition at line 127 of file DriverParallelInTime.cpp.

129{
130 // Retrieve the equation system to solve.
131 ASSERTL0(m_session->DefinesSolverInfo("EqType"),
132 "EqType SolverInfo tag must be defined.");
133 std::string vEquation = m_session->DefinesSolverInfo("SolverType")
134 ? m_session->GetSolverInfo("SolverType")
135 : m_session->GetSolverInfo("EqType");
136
137 // Check such a module exists for this equation.
138 ASSERTL0(GetEquationSystemFactory().ModuleExists(vEquation),
139 "EquationSystem '" + vEquation +
140 "' is not defined.\n"
141 "Ensure equation name is correct and module is compiled.\n");
142
143 // Set fine parallel-in-time solver.
144 m_session->SetTag("AdvectiveType", AdvectiveType);
145 m_session->SetTag("ParallelInTimeSolver", "TimeLevel0");
147 m_graph);
148
149 // Define argument for the coarse parallel-in-time solver.
150 int npx = m_session->DefinesCmdLineArgument("npx")
151 ? m_session->GetCmdLineArgument<int>("npx")
152 : 1;
153 int npy = m_session->DefinesCmdLineArgument("npy")
154 ? m_session->GetCmdLineArgument<int>("npy")
155 : 1;
156 int npz = m_session->DefinesCmdLineArgument("npz")
157 ? m_session->GetCmdLineArgument<int>("npz")
158 : 1;
159 int nsz = m_session->DefinesCmdLineArgument("nsz")
160 ? m_session->GetCmdLineArgument<int>("nsz")
161 : 1;
162 int npt = m_session->DefinesCmdLineArgument("npt")
163 ? m_session->GetCmdLineArgument<int>("npt")
164 : 1;
165
166 // Convert into string.
167 std::string npx_string = std::to_string(npx);
168 std::string npy_string = std::to_string(npy);
169 std::string npz_string = std::to_string(npz);
170 std::string nsz_string = std::to_string(nsz);
171 std::string npt_string = std::to_string(npt);
172 std::string driver_string = "Driver=" + m_session->GetSolverInfo("Driver");
173
174 // use-opt-file
175 bool useOptFile = m_session->DefinesCmdLineArgument("use-opt-file");
176 std::string optfilename = useOptFile ? m_session->GetFilenames()[0] : "";
177
178 char *argv[] = {const_cast<char *>("Solver"), // this is just a place holder
179 const_cast<char *>("--solverinfo"),
180 const_cast<char *>(driver_string.c_str()),
181 const_cast<char *>("--npx"),
182 const_cast<char *>(npx_string.c_str()),
183 const_cast<char *>("--npy"),
184 const_cast<char *>(npy_string.c_str()),
185 const_cast<char *>("--npz"),
186 const_cast<char *>(npz_string.c_str()),
187 const_cast<char *>("--nsz"),
188 const_cast<char *>(nsz_string.c_str()),
189 const_cast<char *>("--npt"),
190 const_cast<char *>(npt_string.c_str()),
191 const_cast<char *>("-f"),
192 const_cast<char *>("--use-opt-file"),
193 const_cast<char *>(optfilename.c_str()),
194 nullptr};
195
196 size_t argc = useOptFile ? 16 : 14;
197
198 // Get list of session file names.
199 std::vector<std::string> sessionFileNames;
200 for (auto &filename : m_session->GetFilenames())
201 {
202 // Remove optfile name, if necessary.
203 if (filename.substr(filename.find_last_of(".") + 1) != "opt")
204 {
205 sessionFileNames.push_back(filename);
206 }
207 }
208
209 // Set session for coarse solver.
210 for (size_t timeLevel = 1; timeLevel < m_nTimeLevel; timeLevel++)
211 {
213 argc, argv, sessionFileNames, m_session->GetComm(), timeLevel);
214
215 // Set graph for coarse solver.
218
219 // Set BndRegionOrdering (necessary for DG with periodic BC) FIXME
220 graph->SetBndRegionOrdering(m_graph->GetBndRegionOrdering());
221
222 // Set CompositeOrdering (necessary for DG with periodic BC) FIXME
223 graph->SetCompositeOrdering(m_graph->GetCompositeOrdering());
224
225 // Retrieve the equation system to solve.
226 ASSERTL0(session->DefinesSolverInfo("EqType"),
227 "EqType SolverInfo tag must be defined.");
228 auto vEquation = session->DefinesSolverInfo("SolverType")
229 ? session->GetSolverInfo("SolverType")
230 : session->GetSolverInfo("EqType");
231
232 // Check such a module exists for this equation.
233 ASSERTL0(
234 GetEquationSystemFactory().ModuleExists(vEquation),
235 "EquationSystem '" + vEquation +
236 "' is not defined.\n"
237 "Ensure equation name is correct and module is compiled.\n");
238
239 // Set coarse parallel-in-time solver.
240 session->SetTag("AdvectiveType", AdvectiveType);
241 session->SetTag("ParallelInTimeSolver",
242 "TimeLevel" + std::to_string(timeLevel));
244 vEquation, session, graph);
245 }
246}
#define ASSERTL0(condition, msg)
Definition: ErrorUtil.hpp:208
tBaseSharedPtr CreateInstance(tKey idKey, tParam... args)
Create an instance of the class referred to by idKey.
static SessionReaderSharedPtr CreateInstance(int argc, char *argv[])
Creates an instance of the SessionReader class.
SpatialDomains::MeshGraphSharedPtr m_graph
MeshGraph object.
Definition: Driver.h:89
Array< OneD, EquationSystemSharedPtr > m_equ
Equation system to solve.
Definition: Driver.h:92
static MeshGraphSharedPtr Read(const LibUtilities::SessionReaderSharedPtr pSession, LibUtilities::DomainRangeShPtr rng=LibUtilities::NullDomainRangeShPtr, bool fillGraph=true, SpatialDomains::MeshGraphSharedPtr partitionedGraph=nullptr)
Definition: MeshGraphIO.cpp:53
static DomainRangeShPtr NullDomainRangeShPtr
Definition: DomainRange.h:65
EquationSystemFactory & GetEquationSystemFactory()

References ASSERTL0, Nektar::LibUtilities::SessionReader::CreateInstance(), Nektar::LibUtilities::NekFactory< tKey, tBase, tParam >::CreateInstance(), Nektar::SolverUtils::GetEquationSystemFactory(), Nektar::SolverUtils::Driver::m_equ, Nektar::SolverUtils::Driver::m_graph, m_nTimeLevel, Nektar::SolverUtils::Driver::m_session, Nektar::LibUtilities::NullDomainRangeShPtr, and Nektar::SpatialDomains::MeshGraphIO::Read().

Referenced by v_InitObject().

◆ SolutionConvergenceMonitoring()

void Nektar::SolverUtils::DriverParallelInTime::SolutionConvergenceMonitoring ( const size_t  timeLevel,
const size_t  iter 
)
protected

Definition at line 483 of file DriverParallelInTime.cpp.

485{
486 PrintHeader((boost::format("ITERATION %1%") % iter).str(), '-');
487 UpdateErrorNorm(timeLevel, true);
488 PrintErrorNorm(timeLevel, true);
489}
void PrintErrorNorm(const size_t timeLevel, const bool normalized)
void PrintHeader(const std::string &title, const char c)
void UpdateErrorNorm(const size_t timeLevel, const bool normalized)

References CellMLToNektar.pycml::format, PrintErrorNorm(), PrintHeader(), and UpdateErrorNorm().

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ SolutionConvergenceSummary()

void Nektar::SolverUtils::DriverParallelInTime::SolutionConvergenceSummary ( const size_t  timeLevel)
protected

Definition at line 494 of file DriverParallelInTime.cpp.

495{
496 UpdateErrorNorm(timeLevel, false);
497 PrintErrorNorm(timeLevel, false);
498}

References PrintErrorNorm(), and UpdateErrorNorm().

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ UpdateErrorNorm()

void Nektar::SolverUtils::DriverParallelInTime::UpdateErrorNorm ( const size_t  timeLevel,
const bool  normalized 
)
protected

Definition at line 503 of file DriverParallelInTime.cpp.

505{
506 for (size_t i = 0; i < m_nVar; ++i)
507 {
508 m_vL2Errors[i] =
509 m_EqSys[timeLevel]->L2Error(i, m_exactsoln[i], normalized);
510 m_vLinfErrors[i] = m_EqSys[timeLevel]->LinfError(i, m_exactsoln[i]);
511 }
512}

References m_EqSys, m_exactsoln, m_nVar, m_vL2Errors, and m_vLinfErrors.

Referenced by SolutionConvergenceMonitoring(), and SolutionConvergenceSummary().

◆ UpdateFieldCoeffs()

void Nektar::SolverUtils::DriverParallelInTime::UpdateFieldCoeffs ( const size_t  timeLevel,
const Array< OneD, const Array< OneD, NekDouble > > &  in = NullNekDoubleArrayOfArray 
)
protected

Definition at line 453 of file DriverParallelInTime.cpp.

455{
456 for (size_t i = 0; i < m_nVar; ++i)
457 {
459 {
460 m_EqSys[timeLevel]->CopyToPhysField(i, in[i]);
461 }
462 m_EqSys[timeLevel]->UpdateFields()[i]->FwdTrans(
463 m_EqSys[timeLevel]->UpdateFields()[i]->GetPhys(),
464 m_EqSys[timeLevel]->UpdateFields()[i]->UpdateCoeffs());
465 }
466}

References m_EqSys, m_nVar, and Nektar::NullNekDoubleArrayOfArray.

Referenced by Nektar::SolverUtils::DriverPFASST::WriteOutput(), and Nektar::SolverUtils::DriverParareal::WriteTimeChunkOuput().

◆ v_Execute()

void Nektar::SolverUtils::DriverParallelInTime::v_Execute ( std::ostream &  out = std::cout)
overrideprotectedvirtual

Virtual function for solve implementation.

Implements Nektar::SolverUtils::Driver.

Reimplemented in Nektar::SolverUtils::DriverParareal, and Nektar::SolverUtils::DriverPFASST.

Definition at line 120 of file DriverParallelInTime.cpp.

121{
122}

◆ v_InitObject()

void Nektar::SolverUtils::DriverParallelInTime::v_InitObject ( std::ostream &  out = std::cout)
overrideprotectedvirtual

Virtual function for initialisation implementation.

Reimplemented from Nektar::SolverUtils::Driver.

Reimplemented in Nektar::SolverUtils::DriverParareal, and Nektar::SolverUtils::DriverPFASST.

Definition at line 65 of file DriverParallelInTime.cpp.

66{
67 try
68 {
69 // Retrieve the type of evolution operator to use
71 m_session->GetSolverInfoAsEnum<EvolutionOperatorType>(
72 "EvolutionOperator");
73
74 m_nTimeLevel = 2; // Only two time levels currently implemented.
75
76 m_equ = Array<OneD, EquationSystemSharedPtr>(m_nTimeLevel);
77
78 // Set the AdvectiveType tag and create EquationSystem objects.
79 switch (m_EvolutionOperator)
80 {
81 case eNonlinear:
83 break;
84 case eDirect:
86 break;
87 case eAdjoint:
89 break;
90 case eSkewSymmetric:
91 SetParallelInTimeEquationSystem("SkewSymmetric");
92 break;
93 default:
94 ASSERTL0(false, "Unrecognised evolution operator.");
95 }
96
97 // Set pointers.
98 m_EqSys = Array<OneD, std::shared_ptr<UnsteadySystem>>(m_nTimeLevel);
99 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
100 {
101 m_EqSys[timeLevel] =
102 std::dynamic_pointer_cast<SolverUtils::UnsteadySystem>(
103 m_equ[timeLevel]);
104 }
105
106 // Set time communication parameters.
107 m_numChunks = m_comm->GetTimeComm()->GetSize();
108 m_chunkRank = m_comm->GetTimeComm()->GetRank();
109 }
110 catch (int e)
111 {
112 ASSERTL0(e == -1, "No such class class defined.");
113 out << "An error occurred during driver initialisation." << std::endl;
114 }
115}
enum EvolutionOperatorType m_EvolutionOperator
Evolution Operator.
Definition: Driver.h:98
void SetParallelInTimeEquationSystem(std::string AdvectiveType)

References ASSERTL0, Nektar::SolverUtils::eAdjoint, Nektar::SolverUtils::eDirect, Nektar::SolverUtils::eNonlinear, Nektar::SolverUtils::eSkewSymmetric, m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, Nektar::SolverUtils::Driver::m_equ, Nektar::SolverUtils::Driver::m_EvolutionOperator, m_nTimeLevel, m_numChunks, Nektar::SolverUtils::Driver::m_session, and SetParallelInTimeEquationSystem().

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ vL2ErrorMax()

NekDouble Nektar::SolverUtils::DriverParallelInTime::vL2ErrorMax ( void  )
protected

Definition at line 554 of file DriverParallelInTime.cpp.

555{
556 NekDouble L2Error = 0.0;
557 for (size_t i = 0; i < m_nVar; ++i)
558 {
559 L2Error = std::max(L2Error, m_vL2Errors[i]);
560 }
561 return L2Error;
562}
double NekDouble

References m_nVar, and m_vL2Errors.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

Member Data Documentation

◆ m_chunkRank

size_t Nektar::SolverUtils::DriverParallelInTime::m_chunkRank
protected

◆ m_chunkTime

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_chunkTime
protected

◆ m_EqSys

Array<OneD, std::shared_ptr<UnsteadySystem> > Nektar::SolverUtils::DriverParallelInTime::m_EqSys
protected

◆ m_exactsoln

Array<OneD, Array<OneD, NekDouble> > Nektar::SolverUtils::DriverParallelInTime::m_exactsoln
protected

◆ m_exactSolution

bool Nektar::SolverUtils::DriverParallelInTime::m_exactSolution
protected

Using exact solution to compute error norms.

Definition at line 144 of file DriverParallelInTime.h.

Referenced by GetParametersFromSession(), and Nektar::SolverUtils::DriverParareal::v_Execute().

◆ m_iterMaxPIT

size_t Nektar::SolverUtils::DriverParallelInTime::m_iterMaxPIT
protected

Maximum number of parallel-in-time iteration.

Definition at line 138 of file DriverParallelInTime.h.

Referenced by GetParametersFromSession(), Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ m_npts

Array<OneD, size_t> Nektar::SolverUtils::DriverParallelInTime::m_npts
protected

◆ m_nsteps

Array<OneD, size_t> Nektar::SolverUtils::DriverParallelInTime::m_nsteps
protected

◆ m_nTimeLevel

size_t Nektar::SolverUtils::DriverParallelInTime::m_nTimeLevel
protected

◆ m_numChunks

size_t Nektar::SolverUtils::DriverParallelInTime::m_numChunks
protected

◆ m_numWindowsPIT

size_t Nektar::SolverUtils::DriverParallelInTime::m_numWindowsPIT
protected

◆ m_nVar

size_t Nektar::SolverUtils::DriverParallelInTime::m_nVar
protected

◆ m_time

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_time
protected

◆ m_timestep

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_timestep
protected

◆ m_tolerPIT

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_tolerPIT
protected

◆ m_totalTime

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_totalTime
protected

Total time integration interval.

Definition at line 123 of file DriverParallelInTime.h.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ m_vL2Errors

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_vL2Errors
protected

◆ m_vLinfErrors

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_vLinfErrors
protected