Nektar++
Protected Member Functions | Protected Attributes | List of all members
Nektar::SolverUtils::DriverParallelInTime Class Reference

Base class for the development of parallel-in-time solvers. More...

#include <DriverParallelInTime.h>

Inheritance diagram for Nektar::SolverUtils::DriverParallelInTime:
[legend]

Protected Member Functions

SOLVER_UTILS_EXPORT DriverParallelInTime (const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
 Constructor. More...
 
SOLVER_UTILS_EXPORT ~DriverParallelInTime () override=default
 Destructor. More...
 
SOLVER_UTILS_EXPORT void v_InitObject (std::ostream &out=std::cout) override
 Virtual function for initialisation implementation. More...
 
SOLVER_UTILS_EXPORT void v_Execute (std::ostream &out=std::cout) override
 Virtual function for solve implementation. More...
 
void SetParallelInTimeEquationSystem (std::string AdvectiveType)
 
void GetParametersFromSession (void)
 
void InitialiseEqSystem (bool turnoff_output)
 
void InitialiseInterpolationField (void)
 
void PrintSolverInfo (std::ostream &out=std::cout)
 
void PrintHeader (const std::string &title, const char c)
 
void RecvFromPreviousProc (Array< OneD, Array< OneD, NekDouble > > &array, int &convergence)
 
void RecvFromPreviousProc (Array< OneD, NekDouble > &array)
 
void SendToNextProc (Array< OneD, Array< OneD, NekDouble > > &array, int &convergence)
 
void SendToNextProc (Array< OneD, NekDouble > &array)
 
void CopySolutionVector (const Array< OneD, const Array< OneD, NekDouble > > &in, Array< OneD, Array< OneD, NekDouble > > &out)
 
void CopyFromPhysField (const size_t timeLevel, Array< OneD, Array< OneD, NekDouble > > &out)
 
void CopyToPhysField (const size_t timeLevel, const Array< OneD, const Array< OneD, NekDouble > > &in)
 
void UpdateFieldCoeffs (const size_t timeLevel, const Array< OneD, const Array< OneD, NekDouble > > &in=NullNekDoubleArrayOfArray)
 
void EvaluateExactSolution (const size_t timeLevel, const NekDouble &time)
 
void SolutionConvergenceMonitoring (const size_t timeLevel, const size_t iter)
 
void SolutionConvergenceSummary (const size_t timeLevel)
 
void UpdateErrorNorm (const size_t timeLevel, const bool normalized)
 
void PrintErrorNorm (const size_t timeLevel, const bool normalized)
 
NekDouble vL2ErrorMax (void)
 
NekDouble EstimateCommunicationTime (Array< OneD, Array< OneD, NekDouble > > &buffer1, Array< OneD, Array< OneD, NekDouble > > &buffer2)
 
void Interpolate (const Array< OneD, MultiRegions::ExpListSharedPtr > &infield, const Array< OneD, MultiRegions::ExpListSharedPtr > &outfield, const Array< OneD, Array< OneD, NekDouble > > &inarray, Array< OneD, Array< OneD, NekDouble > > &outarray)
 
- Protected Member Functions inherited from Nektar::SolverUtils::Driver
 Driver (const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
 Initialises EquationSystem class members. More...
 
virtual SOLVER_UTILS_EXPORT void v_InitObject (std::ostream &out=std::cout)
 Virtual function for initialisation implementation. More...
 
virtual SOLVER_UTILS_EXPORT void v_Execute (std::ostream &out=std::cout)=0
 Virtual function for solve implementation. More...
 

Protected Attributes

NekDouble m_totalTime
 Total time integration interval. More...
 
NekDouble m_chunkTime
 Time integration interval per chunk. More...
 
NekDouble m_time
 Local time. More...
 
size_t m_numChunks
 Number of time chunks. More...
 
size_t m_chunkRank
 Rank in time. More...
 
size_t m_iterMaxPIT
 Maximum number of parallel-in-time iteration. More...
 
size_t m_numWindowsPIT
 
bool m_exactSolution
 Using exact solution to compute error norms. More...
 
NekDouble m_tolerPIT
 ParallelInTime tolerance. More...
 
size_t m_nVar
 Number of variables. More...
 
size_t m_nTimeLevel
 Number of time levels. More...
 
Array< OneD, size_t > m_nsteps
 Number of time steps for each time level. More...
 
Array< OneD, NekDoublem_timestep
 Time step for each time level. More...
 
Array< OneD, size_t > m_npts
 Number of dof for each time level. More...
 
Array< OneD, std::shared_ptr< UnsteadySystem > > m_EqSys
 Equation system to solve. More...
 
Array< OneD, NekDoublem_vL2Errors
 Storage for parallel-in-time iteration. More...
 
Array< OneD, NekDoublem_vLinfErrors
 
Array< OneD, Array< OneD, NekDouble > > m_exactsoln
 
- Protected Attributes inherited from Nektar::SolverUtils::Driver
LibUtilities::CommSharedPtr m_comm
 Communication object. More...
 
LibUtilities::SessionReaderSharedPtr m_session
 Session reader object. More...
 
LibUtilities::SessionReaderSharedPtr session_LinNS
 Coupling between SFD and arnoldi. More...
 
SpatialDomains::MeshGraphSharedPtr m_graph
 MeshGraph object. More...
 
Array< OneD, EquationSystemSharedPtrm_equ
 Equation system to solve. More...
 
int m_nequ
 number of equations More...
 
enum EvolutionOperatorType m_EvolutionOperator
 Evolution Operator. More...
 

Additional Inherited Members

- Public Member Functions inherited from Nektar::SolverUtils::Driver
virtual ~Driver ()
 Destructor. More...
 
SOLVER_UTILS_EXPORT void InitObject (std::ostream &out=std::cout)
 Initialise Object. More...
 
SOLVER_UTILS_EXPORT void Execute (std::ostream &out=std::cout)
 Execute driver. More...
 
SOLVER_UTILS_EXPORT Array< OneD, EquationSystemSharedPtrGetEqu ()
 
- Static Protected Attributes inherited from Nektar::SolverUtils::Driver
static std::string evolutionOperatorLookupIds []
 
static std::string evolutionOperatorDef
 
static std::string driverDefault
 

Detailed Description

Base class for the development of parallel-in-time solvers.

Definition at line 45 of file DriverParallelInTime.h.

Constructor & Destructor Documentation

◆ DriverParallelInTime()

Nektar::SolverUtils::DriverParallelInTime::DriverParallelInTime ( const LibUtilities::SessionReaderSharedPtr  pSession,
const SpatialDomains::MeshGraphSharedPtr  pGraph 
)
protected

Constructor.

Definition at line 54 of file DriverParallelInTime.cpp.

57 : Driver(pSession, pGraph)
58{
59}
Driver(const LibUtilities::SessionReaderSharedPtr pSession, const SpatialDomains::MeshGraphSharedPtr pGraph)
Initialises EquationSystem class members.
Definition: Driver.cpp:74

◆ ~DriverParallelInTime()

SOLVER_UTILS_EXPORT Nektar::SolverUtils::DriverParallelInTime::~DriverParallelInTime ( )
overrideprotecteddefault

Destructor.

Member Function Documentation

◆ CopyFromPhysField()

void Nektar::SolverUtils::DriverParallelInTime::CopyFromPhysField ( const size_t  timeLevel,
Array< OneD, Array< OneD, NekDouble > > &  out 
)
protected

Definition at line 427 of file DriverParallelInTime.cpp.

429{
430 for (size_t i = 0; i < m_nVar; ++i)
431 {
432 m_EqSys[timeLevel]->CopyFromPhysField(i, out[i]);
433 }
434}
Array< OneD, std::shared_ptr< UnsteadySystem > > m_EqSys
Equation system to solve.

References m_EqSys, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ CopySolutionVector()

void Nektar::SolverUtils::DriverParallelInTime::CopySolutionVector ( const Array< OneD, const Array< OneD, NekDouble > > &  in,
Array< OneD, Array< OneD, NekDouble > > &  out 
)
protected

Definition at line 414 of file DriverParallelInTime.cpp.

417{
418 for (size_t i = 0; i < m_nVar; ++i)
419 {
420 Vmath::Vcopy(in[i].size(), in[i], 1, out[i], 1);
421 }
422}
void Vcopy(int n, const T *x, const int incx, T *y, const int incy)
Definition: Vmath.hpp:825

References m_nVar, and Vmath::Vcopy().

Referenced by Nektar::SolverUtils::DriverPFASST::RestrictResidual(), and Nektar::SolverUtils::DriverPFASST::RestrictSolution().

◆ CopyToPhysField()

void Nektar::SolverUtils::DriverParallelInTime::CopyToPhysField ( const size_t  timeLevel,
const Array< OneD, const Array< OneD, NekDouble > > &  in 
)
protected

Definition at line 439 of file DriverParallelInTime.cpp.

441{
442 for (size_t i = 0; i < m_nVar; ++i)
443 {
444 m_EqSys[timeLevel]->CopyToPhysField(i, in[i]);
445 }
446}

References m_EqSys, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ EstimateCommunicationTime()

NekDouble Nektar::SolverUtils::DriverParallelInTime::EstimateCommunicationTime ( Array< OneD, Array< OneD, NekDouble > > &  buffer1,
Array< OneD, Array< OneD, NekDouble > > &  buffer2 
)
protected

Definition at line 565 of file DriverParallelInTime.cpp.

568{
569 if (m_numChunks == 1)
570 {
571 return 0.0;
572 }
573 else
574 {
575 // Average communication time over niter iteration.
576 size_t niter = 20;
578 for (size_t n = 0; n <= niter; n++)
579 {
580 if (n == 1)
581 {
582 timer.Start(); // Ignore the first iteration
583 }
584
585 if (m_chunkRank == 0)
586 {
587 for (size_t i = 0; i < buffer1.size(); ++i)
588 {
589 m_comm->GetTimeComm()->Send(m_numChunks - 1, buffer1[i]);
590 }
591 }
592
593 if (m_chunkRank == m_numChunks - 1)
594 {
595 for (size_t i = 0; i < buffer2.size(); ++i)
596 {
597 m_comm->GetTimeComm()->Recv(0, buffer2[i]);
598 }
599 }
600 }
601 timer.Stop();
602 return timer.Elapsed().count() / niter;
603 }
604}
LibUtilities::CommSharedPtr m_comm
Communication object.
Definition: Driver.h:80

References Nektar::LibUtilities::Timer::Elapsed(), m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, Nektar::LibUtilities::Timer::Start(), and Nektar::LibUtilities::Timer::Stop().

◆ EvaluateExactSolution()

void Nektar::SolverUtils::DriverParallelInTime::EvaluateExactSolution ( const size_t  timeLevel,
const NekDouble time 
)
protected

Definition at line 469 of file DriverParallelInTime.cpp.

471{
472 for (size_t i = 0; i < m_nVar; ++i)
473 {
474 m_EqSys[timeLevel]->EvaluateExactSolution(i, m_exactsoln[i], time);
475 }
476}
Array< OneD, Array< OneD, NekDouble > > m_exactsoln

References m_EqSys, m_exactsoln, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ GetParametersFromSession()

void Nektar::SolverUtils::DriverParallelInTime::GetParametersFromSession ( void  )
protected

Definition at line 250 of file DriverParallelInTime.cpp.

251{
252 // Parallel-in-Time iteration parameters.
253 m_tolerPIT = m_session->DefinesParameter("PITToler")
254 ? m_session->GetParameter("PITToler")
255 : 1.0E-16;
256 m_iterMaxPIT = m_session->DefinesParameter("PITIterMax")
257 ? m_session->GetParameter("PITIterMax")
258 : m_numChunks;
259 m_numWindowsPIT = m_session->DefinesParameter("NumWindows")
260 ? m_session->GetParameter("NumWindows")
261 : 1;
262
263 // Other parameters.
264 m_exactSolution = m_session->DefinesParameter("ExactSolution")
265 ? m_session->GetParameter("ExactSolution")
266 : 0;
267}
LibUtilities::SessionReaderSharedPtr m_session
Session reader object.
Definition: Driver.h:83
size_t m_iterMaxPIT
Maximum number of parallel-in-time iteration.
NekDouble m_tolerPIT
ParallelInTime tolerance.
bool m_exactSolution
Using exact solution to compute error norms.

References m_exactSolution, m_iterMaxPIT, m_numChunks, m_numWindowsPIT, Nektar::SolverUtils::Driver::m_session, and m_tolerPIT.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ InitialiseEqSystem()

void Nektar::SolverUtils::DriverParallelInTime::InitialiseEqSystem ( bool  turnoff_output)
protected

Definition at line 272 of file DriverParallelInTime.cpp.

273{
274 m_nVar = m_EqSys[0]->GetNvariables();
275
276 // Initialize fine solver.
277 if (turnoff_output)
278 {
279 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
280 {
281 m_EqSys[timeLevel]->SetInfoSteps(0);
282 m_EqSys[timeLevel]->SetCheckpointSteps(0);
283 }
284 }
285 m_EqSys[0]->DoInitialise(true);
286
287 // Initialize coarse solver(s).
288 for (size_t timeLevel = 1; timeLevel < m_nTimeLevel; timeLevel++)
289 {
290 m_EqSys[timeLevel]->DoInitialise(false);
291 }
292
293 // Initialize time stepping parameters.
294 m_timestep = Array<OneD, NekDouble>(m_nTimeLevel);
295 m_nsteps = Array<OneD, size_t>(m_nTimeLevel);
296 m_npts = Array<OneD, size_t>(m_nTimeLevel);
297 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
298 {
299 m_timestep[timeLevel] =
300 m_EqSys[timeLevel]->EquationSystem::GetTimeStep();
301 m_nsteps[timeLevel] = m_EqSys[timeLevel]->EquationSystem::GetSteps();
302 m_npts[timeLevel] = m_EqSys[timeLevel]->EquationSystem::GetNpoints();
303 }
304
305 // Initialize errors.
306 m_exactsoln = Array<OneD, Array<OneD, NekDouble>>(m_nVar);
307 for (size_t i = 0; i < m_nVar; ++i)
308 {
309 m_exactsoln[i] = Array<OneD, NekDouble>(m_npts[0], 0.0);
310 }
311 m_vL2Errors = Array<OneD, NekDouble>(m_nVar, 0.0);
312 m_vLinfErrors = Array<OneD, NekDouble>(m_nVar, 0.0);
313}
Array< OneD, NekDouble > m_vL2Errors
Storage for parallel-in-time iteration.
Array< OneD, size_t > m_nsteps
Number of time steps for each time level.
Array< OneD, size_t > m_npts
Number of dof for each time level.
Array< OneD, NekDouble > m_timestep
Time step for each time level.

References m_EqSys, m_exactsoln, m_npts, m_nsteps, m_nTimeLevel, m_nVar, m_timestep, m_vL2Errors, and m_vLinfErrors.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ InitialiseInterpolationField()

void Nektar::SolverUtils::DriverParallelInTime::InitialiseInterpolationField ( void  )
protected

◆ Interpolate()

void Nektar::SolverUtils::DriverParallelInTime::Interpolate ( const Array< OneD, MultiRegions::ExpListSharedPtr > &  infield,
const Array< OneD, MultiRegions::ExpListSharedPtr > &  outfield,
const Array< OneD, Array< OneD, NekDouble > > &  inarray,
Array< OneD, Array< OneD, NekDouble > > &  outarray 
)
protected

Definition at line 609 of file DriverParallelInTime.cpp.

614{
615 if (infield.size() != outfield.size())
616 {
617 NEKERROR(ErrorUtil::efatal, "not the same number of variables")
618 }
619
620 for (size_t n = 0; n < infield.size(); ++n)
621 {
622 // Interpolation from infield -> outfield assuming that infield and
623 // outfield are the same explists, but at potentially different
624 // polynomial orders.
625 if (infield[n]->GetExpSize() != outfield[n]->GetExpSize())
626 {
627 NEKERROR(ErrorUtil::efatal, "not the same mesh")
628 }
629
630 // Assign input/output array.
631 auto inphys = (inarray == NullNekDoubleArrayOfArray)
632 ? infield[n]->UpdatePhys()
633 : inarray[n];
634 auto outphys = (outarray == NullNekDoubleArrayOfArray)
635 ? outfield[n]->UpdatePhys()
636 : outarray[n];
637
638 // If same polynomial orders, simply copy solution.
639 if (infield[n]->GetTotPoints() == outfield[n]->GetTotPoints())
640 {
641 Vmath::Vcopy(infield[n]->GetTotPoints(), inphys, 1, outphys, 1);
642 }
643 // If different polynomial orders, interpolate solution.
644 else
645 {
646 // Assign memory for coefficient space.
647 Array<OneD, NekDouble> incoeff(infield[n]->GetNcoeffs());
648
649 // Transform solution from physical to coefficient space.
650 infield[n]->FwdTransLocalElmt(inphys, incoeff);
651
652 for (size_t i = 0; i < infield[n]->GetExpSize(); ++i)
653 {
654 // Get the elements.
655 auto inElmt = infield[n]->GetExp(i);
656 auto outElmt = outfield[n]->GetExp(i);
657
658 // Get the offset of elements in the storage arrays.
659 size_t inoffset = infield[n]->GetCoeff_Offset(i);
660 size_t outoffset = outfield[n]->GetPhys_Offset(i);
661
662 // Interpolate elements.
663 Array<OneD, NekDouble> tmp;
665 if (inElmt->DetShapeType() == LibUtilities::Seg)
666 {
667 expPtr = std::make_shared<StdRegions::StdSegExp>(
668 LibUtilities::BasisKey(
669 inElmt->GetBasis(0)->GetBasisType(),
670 inElmt->GetBasis(0)->GetNumModes(),
671 outElmt->GetBasis(0)->GetPointsKey()));
672 }
673 else if (inElmt->DetShapeType() == LibUtilities::Quad)
674 {
675 expPtr = std::make_shared<StdRegions::StdQuadExp>(
676 LibUtilities::BasisKey(
677 inElmt->GetBasis(0)->GetBasisType(),
678 inElmt->GetBasis(0)->GetNumModes(),
679 outElmt->GetBasis(0)->GetPointsKey()),
680 LibUtilities::BasisKey(
681 inElmt->GetBasis(1)->GetBasisType(),
682 inElmt->GetBasis(1)->GetNumModes(),
683 outElmt->GetBasis(1)->GetPointsKey()));
684 }
685 else if (inElmt->DetShapeType() == LibUtilities::Tri)
686 {
687 expPtr = std::make_shared<StdRegions::StdTriExp>(
688 LibUtilities::BasisKey(
689 inElmt->GetBasis(0)->GetBasisType(),
690 inElmt->GetBasis(0)->GetNumModes(),
691 outElmt->GetBasis(0)->GetPointsKey()),
692 LibUtilities::BasisKey(
693 inElmt->GetBasis(1)->GetBasisType(),
694 inElmt->GetBasis(1)->GetNumModes(),
695 outElmt->GetBasis(1)->GetPointsKey()));
696 }
697 else if (inElmt->DetShapeType() == LibUtilities::Hex)
698 {
699 expPtr = std::make_shared<StdRegions::StdHexExp>(
700 LibUtilities::BasisKey(
701 inElmt->GetBasis(0)->GetBasisType(),
702 inElmt->GetBasis(0)->GetNumModes(),
703 outElmt->GetBasis(0)->GetPointsKey()),
704 LibUtilities::BasisKey(
705 inElmt->GetBasis(1)->GetBasisType(),
706 inElmt->GetBasis(1)->GetNumModes(),
707 outElmt->GetBasis(1)->GetPointsKey()),
708 LibUtilities::BasisKey(
709 inElmt->GetBasis(2)->GetBasisType(),
710 inElmt->GetBasis(2)->GetNumModes(),
711 outElmt->GetBasis(2)->GetPointsKey()));
712 }
713 else if (inElmt->DetShapeType() == LibUtilities::Prism)
714 {
715 expPtr = std::make_shared<StdRegions::StdPrismExp>(
716 LibUtilities::BasisKey(
717 inElmt->GetBasis(0)->GetBasisType(),
718 inElmt->GetBasis(0)->GetNumModes(),
719 outElmt->GetBasis(0)->GetPointsKey()),
720 LibUtilities::BasisKey(
721 inElmt->GetBasis(1)->GetBasisType(),
722 inElmt->GetBasis(1)->GetNumModes(),
723 outElmt->GetBasis(1)->GetPointsKey()),
724 LibUtilities::BasisKey(
725 inElmt->GetBasis(2)->GetBasisType(),
726 inElmt->GetBasis(2)->GetNumModes(),
727 outElmt->GetBasis(2)->GetPointsKey()));
728 }
729 else if (inElmt->DetShapeType() == LibUtilities::Pyr)
730 {
731 expPtr = std::make_shared<StdRegions::StdPyrExp>(
732 LibUtilities::BasisKey(
733 inElmt->GetBasis(0)->GetBasisType(),
734 inElmt->GetBasis(0)->GetNumModes(),
735 outElmt->GetBasis(0)->GetPointsKey()),
736 LibUtilities::BasisKey(
737 inElmt->GetBasis(1)->GetBasisType(),
738 inElmt->GetBasis(1)->GetNumModes(),
739 outElmt->GetBasis(1)->GetPointsKey()),
740 LibUtilities::BasisKey(
741 inElmt->GetBasis(2)->GetBasisType(),
742 inElmt->GetBasis(2)->GetNumModes(),
743 outElmt->GetBasis(2)->GetPointsKey()));
744 }
745 else if (inElmt->DetShapeType() == LibUtilities::Tet)
746 {
747 expPtr = std::make_shared<StdRegions::StdTetExp>(
748 LibUtilities::BasisKey(
749 inElmt->GetBasis(0)->GetBasisType(),
750 inElmt->GetBasis(0)->GetNumModes(),
751 outElmt->GetBasis(0)->GetPointsKey()),
752 LibUtilities::BasisKey(
753 inElmt->GetBasis(1)->GetBasisType(),
754 inElmt->GetBasis(1)->GetNumModes(),
755 outElmt->GetBasis(1)->GetPointsKey()),
756 LibUtilities::BasisKey(
757 inElmt->GetBasis(2)->GetBasisType(),
758 inElmt->GetBasis(2)->GetNumModes(),
759 outElmt->GetBasis(2)->GetPointsKey()));
760 }
761 expPtr->BwdTrans(incoeff + inoffset, tmp = outphys + outoffset);
762 }
763 }
764 }
765}
#define NEKERROR(type, msg)
Assert Level 0 – Fundamental assert which is used whether in FULLDEBUG, DEBUG or OPT compilation mode...
Definition: ErrorUtil.hpp:202
std::shared_ptr< StdExpansion > StdExpansionSharedPtr
static Array< OneD, Array< OneD, NekDouble > > NullNekDoubleArrayOfArray

References Nektar::ErrorUtil::efatal, Nektar::LibUtilities::Hex, NEKERROR, Nektar::NullNekDoubleArrayOfArray, Nektar::LibUtilities::Prism, Nektar::LibUtilities::Pyr, Nektar::LibUtilities::Quad, Nektar::LibUtilities::Seg, Nektar::LibUtilities::Tet, Nektar::LibUtilities::Tri, and Vmath::Vcopy().

Referenced by Nektar::SolverUtils::DriverParareal::InterpolateCoarseSolution(), Nektar::SolverUtils::DriverParareal::UpdateInitialConditionFromSolver(), and Nektar::SolverUtils::DriverParareal::UpdateSolverInitialCondition().

◆ PrintErrorNorm()

void Nektar::SolverUtils::DriverParallelInTime::PrintErrorNorm ( const size_t  timeLevel,
const bool  normalized 
)
protected

Definition at line 515 of file DriverParallelInTime.cpp.

517{
518 if (m_chunkRank == m_numChunks - 1 &&
519 m_comm->GetSpaceComm()->GetRank() == 0)
520 {
521 for (size_t i = 0; i < m_nVar; ++i)
522 {
523 if (normalized)
524 {
525 std::cout << "L2 error (variable "
526 << m_EqSys[timeLevel]->GetVariable(i)
527 << ") : " << m_vL2Errors[i] << std::endl
528 << std::flush;
529 std::cout << "Linf error (variable "
530 << m_EqSys[timeLevel]->GetVariable(i)
531 << ") : " << m_vLinfErrors[i] << std::endl
532 << std::flush;
533 }
534 else
535 {
536 std::cout << "L 2 error (variable "
537 << m_EqSys[timeLevel]->GetVariable(i)
538 << ") : " << m_vL2Errors[i] << std::endl
539 << std::flush;
540 std::cout << "L inf error (variable "
541 << m_EqSys[timeLevel]->GetVariable(i)
542 << ") : " << m_vLinfErrors[i] << std::endl
543 << std::flush;
544 }
545 }
546 }
547}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, m_numChunks, m_nVar, m_vL2Errors, and m_vLinfErrors.

Referenced by SolutionConvergenceMonitoring(), SolutionConvergenceSummary(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ PrintHeader()

void Nektar::SolverUtils::DriverParallelInTime::PrintHeader ( const std::string &  title,
const char  c 
)
protected

Definition at line 342 of file DriverParallelInTime.cpp.

343{
344 if (m_chunkRank == m_numChunks - 1 &&
345 m_comm->GetSpaceComm()->GetRank() == 0)
346 {
347 std::cout << std::endl;
348 std::cout << std::string(43, c) << std::endl << std::flush;
349 std::cout << title << std::endl << std::flush;
350 std::cout << std::string(43, c) << std::endl << std::flush;
351 }
352}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, and CG_Iterations::title.

Referenced by SolutionConvergenceMonitoring(), Nektar::SolverUtils::DriverParareal::v_Execute(), Nektar::SolverUtils::DriverPFASST::v_Execute(), and Nektar::SolverUtils::DriverParareal::WriteTimeChunkOuput().

◆ PrintSolverInfo()

void Nektar::SolverUtils::DriverParallelInTime::PrintSolverInfo ( std::ostream &  out = std::cout)
protected

Definition at line 318 of file DriverParallelInTime.cpp.

319{
320 if (m_chunkRank == 0 && m_comm->GetSpaceComm()->GetRank() == 0)
321 {
322 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
323 {
324 std::cout << std::string(71, '=') << std::endl << std::flush;
325 std::cout << "=========================== TIME LEVEL " +
326 std::to_string(timeLevel) +
327 " INFO "
328 "========================="
329 << std::endl
330 << std::flush;
331
332 m_EqSys[timeLevel]->PrintSummary(out);
333
334 std::cout << std::endl << std::flush;
335 }
336 }
337}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, and m_nTimeLevel.

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ RecvFromPreviousProc() [1/2]

void Nektar::SolverUtils::DriverParallelInTime::RecvFromPreviousProc ( Array< OneD, Array< OneD, NekDouble > > &  array,
int &  convergence 
)
protected

Definition at line 357 of file DriverParallelInTime.cpp.

359{
360 if (m_chunkRank > 0)
361 {
362 if (!convergence)
363 {
364 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, convergence);
365 for (size_t i = 0; i < m_nVar; ++i)
366 {
367 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, array[i]);
368 }
369 }
370 }
371}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ RecvFromPreviousProc() [2/2]

void Nektar::SolverUtils::DriverParallelInTime::RecvFromPreviousProc ( Array< OneD, NekDouble > &  array)
protected

Definition at line 376 of file DriverParallelInTime.cpp.

377{
378 if (m_chunkRank > 0)
379 {
380 m_comm->GetTimeComm()->Recv(m_chunkRank - 1, array);
381 }
382}

References m_chunkRank, and Nektar::SolverUtils::Driver::m_comm.

◆ SendToNextProc() [1/2]

void Nektar::SolverUtils::DriverParallelInTime::SendToNextProc ( Array< OneD, Array< OneD, NekDouble > > &  array,
int &  convergence 
)
protected

Definition at line 387 of file DriverParallelInTime.cpp.

389{
390 if (m_chunkRank < m_numChunks - 1)
391 {
392 m_comm->GetTimeComm()->Send(m_chunkRank + 1, convergence);
393 for (size_t i = 0; i < m_nVar; ++i)
394 {
395 m_comm->GetTimeComm()->Send(m_chunkRank + 1, array[i]);
396 }
397 }
398}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_numChunks, and m_nVar.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ SendToNextProc() [2/2]

void Nektar::SolverUtils::DriverParallelInTime::SendToNextProc ( Array< OneD, NekDouble > &  array)
protected

Definition at line 403 of file DriverParallelInTime.cpp.

404{
405 if (m_chunkRank < m_numChunks - 1)
406 {
407 m_comm->GetTimeComm()->Send(m_chunkRank + 1, array);
408 }
409}

References m_chunkRank, Nektar::SolverUtils::Driver::m_comm, and m_numChunks.

◆ SetParallelInTimeEquationSystem()

void Nektar::SolverUtils::DriverParallelInTime::SetParallelInTimeEquationSystem ( std::string  AdvectiveType)
protected

Set the ParallelInTime (coarse solver) session file

Definition at line 126 of file DriverParallelInTime.cpp.

128{
129 // Retrieve the equation system to solve.
130 ASSERTL0(m_session->DefinesSolverInfo("EqType"),
131 "EqType SolverInfo tag must be defined.");
132 std::string vEquation = m_session->DefinesSolverInfo("SolverType")
133 ? m_session->GetSolverInfo("SolverType")
134 : m_session->GetSolverInfo("EqType");
135
136 // Check such a module exists for this equation.
137 ASSERTL0(GetEquationSystemFactory().ModuleExists(vEquation),
138 "EquationSystem '" + vEquation +
139 "' is not defined.\n"
140 "Ensure equation name is correct and module is compiled.\n");
141
142 // Set fine parallel-in-time solver.
143 m_session->SetTag("AdvectiveType", AdvectiveType);
144 m_session->SetTag("ParallelInTimeSolver", "TimeLevel0");
146 m_graph);
147
148 // Define argument for the coarse parallel-in-time solver.
149 int npx = m_session->DefinesCmdLineArgument("npx")
150 ? m_session->GetCmdLineArgument<int>("npx")
151 : 1;
152 int npy = m_session->DefinesCmdLineArgument("npy")
153 ? m_session->GetCmdLineArgument<int>("npy")
154 : 1;
155 int npz = m_session->DefinesCmdLineArgument("npz")
156 ? m_session->GetCmdLineArgument<int>("npz")
157 : 1;
158 int nsz = m_session->DefinesCmdLineArgument("nsz")
159 ? m_session->GetCmdLineArgument<int>("nsz")
160 : 1;
161 int npt = m_session->DefinesCmdLineArgument("npt")
162 ? m_session->GetCmdLineArgument<int>("npt")
163 : 1;
164
165 // Convert into string.
166 std::string npx_string = std::to_string(npx);
167 std::string npy_string = std::to_string(npy);
168 std::string npz_string = std::to_string(npz);
169 std::string nsz_string = std::to_string(nsz);
170 std::string npt_string = std::to_string(npt);
171 std::string driver_string = "Driver=" + m_session->GetSolverInfo("Driver");
172
173 // use-opt-file
174 bool useOptFile = m_session->DefinesCmdLineArgument("use-opt-file");
175 std::string optfilename = useOptFile ? m_session->GetFilenames()[0] : "";
176
177 char *argv[] = {const_cast<char *>("Solver"), // this is just a place holder
178 const_cast<char *>("--solverinfo"),
179 const_cast<char *>(driver_string.c_str()),
180 const_cast<char *>("--npx"),
181 const_cast<char *>(npx_string.c_str()),
182 const_cast<char *>("--npy"),
183 const_cast<char *>(npy_string.c_str()),
184 const_cast<char *>("--npz"),
185 const_cast<char *>(npz_string.c_str()),
186 const_cast<char *>("--nsz"),
187 const_cast<char *>(nsz_string.c_str()),
188 const_cast<char *>("--npt"),
189 const_cast<char *>(npt_string.c_str()),
190 const_cast<char *>("-f"),
191 const_cast<char *>("--use-opt-file"),
192 const_cast<char *>(optfilename.c_str()),
193 nullptr};
194
195 size_t argc = useOptFile ? 16 : 14;
196
197 // Get list of session file names.
198 std::vector<std::string> sessionFileNames;
199 for (auto &filename : m_session->GetFilenames())
200 {
201 // Remove optfile name, if necessary.
202 if (filename.substr(filename.find_last_of(".") + 1) != "opt")
203 {
204 sessionFileNames.push_back(filename);
205 }
206 }
207
208 // Set session for coarse solver.
209 for (size_t timeLevel = 1; timeLevel < m_nTimeLevel; timeLevel++)
210 {
212 argc, argv, sessionFileNames, m_session->GetComm(), timeLevel);
213
214 // Set graph for coarse solver.
217
218 // Set BndRegionOrdering (necessary for DG with periodic BC) FIXME
219 graph->SetBndRegionOrdering(m_graph->GetBndRegionOrdering());
220
221 // Set CompositeOrdering (necessary for DG with periodic BC) FIXME
222 graph->SetCompositeOrdering(m_graph->GetCompositeOrdering());
223
224 // Retrieve the equation system to solve.
225 ASSERTL0(session->DefinesSolverInfo("EqType"),
226 "EqType SolverInfo tag must be defined.");
227 auto vEquation = session->DefinesSolverInfo("SolverType")
228 ? session->GetSolverInfo("SolverType")
229 : session->GetSolverInfo("EqType");
230
231 // Check such a module exists for this equation.
232 ASSERTL0(
233 GetEquationSystemFactory().ModuleExists(vEquation),
234 "EquationSystem '" + vEquation +
235 "' is not defined.\n"
236 "Ensure equation name is correct and module is compiled.\n");
237
238 // Set coarse parallel-in-time solver.
239 session->SetTag("AdvectiveType", AdvectiveType);
240 session->SetTag("ParallelInTimeSolver",
241 "TimeLevel" + std::to_string(timeLevel));
243 vEquation, session, graph);
244 }
245}
#define ASSERTL0(condition, msg)
Definition: ErrorUtil.hpp:208
tBaseSharedPtr CreateInstance(tKey idKey, tParam... args)
Create an instance of the class referred to by idKey.
Definition: NekFactory.hpp:143
static SessionReaderSharedPtr CreateInstance(int argc, char *argv[])
Creates an instance of the SessionReader class.
SpatialDomains::MeshGraphSharedPtr m_graph
MeshGraph object.
Definition: Driver.h:89
Array< OneD, EquationSystemSharedPtr > m_equ
Equation system to solve.
Definition: Driver.h:92
static MeshGraphSharedPtr Read(const LibUtilities::SessionReaderSharedPtr pSession, LibUtilities::DomainRangeShPtr rng=LibUtilities::NullDomainRangeShPtr, bool fillGraph=true, SpatialDomains::MeshGraphSharedPtr partitionedGraph=nullptr)
Definition: MeshGraph.cpp:115
static DomainRangeShPtr NullDomainRangeShPtr
Definition: DomainRange.h:65
EquationSystemFactory & GetEquationSystemFactory()

References ASSERTL0, Nektar::LibUtilities::SessionReader::CreateInstance(), Nektar::LibUtilities::NekFactory< tKey, tBase, tParam >::CreateInstance(), Nektar::SolverUtils::GetEquationSystemFactory(), Nektar::SolverUtils::Driver::m_equ, Nektar::SolverUtils::Driver::m_graph, m_nTimeLevel, Nektar::SolverUtils::Driver::m_session, Nektar::LibUtilities::NullDomainRangeShPtr, and Nektar::SpatialDomains::MeshGraph::Read().

Referenced by v_InitObject().

◆ SolutionConvergenceMonitoring()

void Nektar::SolverUtils::DriverParallelInTime::SolutionConvergenceMonitoring ( const size_t  timeLevel,
const size_t  iter 
)
protected

Definition at line 481 of file DriverParallelInTime.cpp.

483{
484 PrintHeader((boost::format("ITERATION %1%") % iter).str(), '-');
485 UpdateErrorNorm(timeLevel, true);
486 PrintErrorNorm(timeLevel, true);
487}
void PrintErrorNorm(const size_t timeLevel, const bool normalized)
void PrintHeader(const std::string &title, const char c)
void UpdateErrorNorm(const size_t timeLevel, const bool normalized)

References CellMLToNektar.pycml::format, PrintErrorNorm(), PrintHeader(), and UpdateErrorNorm().

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute().

◆ SolutionConvergenceSummary()

void Nektar::SolverUtils::DriverParallelInTime::SolutionConvergenceSummary ( const size_t  timeLevel)
protected

Definition at line 492 of file DriverParallelInTime.cpp.

493{
494 UpdateErrorNorm(timeLevel, false);
495 PrintErrorNorm(timeLevel, false);
496}

References PrintErrorNorm(), and UpdateErrorNorm().

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ UpdateErrorNorm()

void Nektar::SolverUtils::DriverParallelInTime::UpdateErrorNorm ( const size_t  timeLevel,
const bool  normalized 
)
protected

Definition at line 501 of file DriverParallelInTime.cpp.

503{
504 for (size_t i = 0; i < m_nVar; ++i)
505 {
506 m_vL2Errors[i] =
507 m_EqSys[timeLevel]->L2Error(i, m_exactsoln[i], normalized);
508 m_vLinfErrors[i] = m_EqSys[timeLevel]->LinfError(i, m_exactsoln[i]);
509 }
510}

References m_EqSys, m_exactsoln, m_nVar, m_vL2Errors, and m_vLinfErrors.

Referenced by SolutionConvergenceMonitoring(), and SolutionConvergenceSummary().

◆ UpdateFieldCoeffs()

void Nektar::SolverUtils::DriverParallelInTime::UpdateFieldCoeffs ( const size_t  timeLevel,
const Array< OneD, const Array< OneD, NekDouble > > &  in = NullNekDoubleArrayOfArray 
)
protected

Definition at line 451 of file DriverParallelInTime.cpp.

453{
454 for (size_t i = 0; i < m_nVar; ++i)
455 {
457 {
458 m_EqSys[timeLevel]->CopyToPhysField(i, in[i]);
459 }
460 m_EqSys[timeLevel]->UpdateFields()[i]->FwdTrans(
461 m_EqSys[timeLevel]->UpdateFields()[i]->GetPhys(),
462 m_EqSys[timeLevel]->UpdateFields()[i]->UpdateCoeffs());
463 }
464}

References m_EqSys, m_nVar, and Nektar::NullNekDoubleArrayOfArray.

Referenced by Nektar::SolverUtils::DriverPFASST::WriteOutput(), and Nektar::SolverUtils::DriverParareal::WriteTimeChunkOuput().

◆ v_Execute()

void Nektar::SolverUtils::DriverParallelInTime::v_Execute ( std::ostream &  out = std::cout)
overrideprotectedvirtual

Virtual function for solve implementation.

Implements Nektar::SolverUtils::Driver.

Reimplemented in Nektar::SolverUtils::DriverParareal, and Nektar::SolverUtils::DriverPFASST.

Definition at line 119 of file DriverParallelInTime.cpp.

120{
121}

◆ v_InitObject()

void Nektar::SolverUtils::DriverParallelInTime::v_InitObject ( std::ostream &  out = std::cout)
overrideprotectedvirtual

Virtual function for initialisation implementation.

Reimplemented from Nektar::SolverUtils::Driver.

Reimplemented in Nektar::SolverUtils::DriverParareal, and Nektar::SolverUtils::DriverPFASST.

Definition at line 64 of file DriverParallelInTime.cpp.

65{
66 try
67 {
68 // Retrieve the type of evolution operator to use
70 m_session->GetSolverInfoAsEnum<EvolutionOperatorType>(
71 "EvolutionOperator");
72
73 m_nTimeLevel = 2; // Only two time levels currently implemented.
74
75 m_equ = Array<OneD, EquationSystemSharedPtr>(m_nTimeLevel);
76
77 // Set the AdvectiveType tag and create EquationSystem objects.
78 switch (m_EvolutionOperator)
79 {
80 case eNonlinear:
82 break;
83 case eDirect:
85 break;
86 case eAdjoint:
88 break;
89 case eSkewSymmetric:
90 SetParallelInTimeEquationSystem("SkewSymmetric");
91 break;
92 default:
93 ASSERTL0(false, "Unrecognised evolution operator.");
94 }
95
96 // Set pointers.
97 m_EqSys = Array<OneD, std::shared_ptr<UnsteadySystem>>(m_nTimeLevel);
98 for (size_t timeLevel = 0; timeLevel < m_nTimeLevel; timeLevel++)
99 {
100 m_EqSys[timeLevel] =
101 std::dynamic_pointer_cast<SolverUtils::UnsteadySystem>(
102 m_equ[timeLevel]);
103 }
104
105 // Set time communication parameters.
106 m_numChunks = m_comm->GetTimeComm()->GetSize();
107 m_chunkRank = m_comm->GetTimeComm()->GetRank();
108 }
109 catch (int e)
110 {
111 ASSERTL0(e == -1, "No such class class defined.");
112 out << "An error occurred during driver initialisation." << std::endl;
113 }
114}
enum EvolutionOperatorType m_EvolutionOperator
Evolution Operator.
Definition: Driver.h:98
void SetParallelInTimeEquationSystem(std::string AdvectiveType)

References ASSERTL0, Nektar::SolverUtils::eAdjoint, Nektar::SolverUtils::eDirect, Nektar::SolverUtils::eNonlinear, Nektar::SolverUtils::eSkewSymmetric, m_chunkRank, Nektar::SolverUtils::Driver::m_comm, m_EqSys, Nektar::SolverUtils::Driver::m_equ, Nektar::SolverUtils::Driver::m_EvolutionOperator, m_nTimeLevel, m_numChunks, Nektar::SolverUtils::Driver::m_session, and SetParallelInTimeEquationSystem().

Referenced by Nektar::SolverUtils::DriverParareal::v_InitObject(), and Nektar::SolverUtils::DriverPFASST::v_InitObject().

◆ vL2ErrorMax()

NekDouble Nektar::SolverUtils::DriverParallelInTime::vL2ErrorMax ( void  )
protected

Definition at line 552 of file DriverParallelInTime.cpp.

553{
554 NekDouble L2Error = 0.0;
555 for (size_t i = 0; i < m_nVar; ++i)
556 {
557 L2Error = std::max(L2Error, m_vL2Errors[i]);
558 }
559 return L2Error;
560}
double NekDouble

References m_nVar, and m_vL2Errors.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

Member Data Documentation

◆ m_chunkRank

size_t Nektar::SolverUtils::DriverParallelInTime::m_chunkRank
protected

◆ m_chunkTime

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_chunkTime
protected

◆ m_EqSys

Array<OneD, std::shared_ptr<UnsteadySystem> > Nektar::SolverUtils::DriverParallelInTime::m_EqSys
protected

◆ m_exactsoln

Array<OneD, Array<OneD, NekDouble> > Nektar::SolverUtils::DriverParallelInTime::m_exactsoln
protected

◆ m_exactSolution

bool Nektar::SolverUtils::DriverParallelInTime::m_exactSolution
protected

Using exact solution to compute error norms.

Definition at line 144 of file DriverParallelInTime.h.

Referenced by GetParametersFromSession(), and Nektar::SolverUtils::DriverParareal::v_Execute().

◆ m_iterMaxPIT

size_t Nektar::SolverUtils::DriverParallelInTime::m_iterMaxPIT
protected

Maximum number of parallel-in-time iteration.

Definition at line 138 of file DriverParallelInTime.h.

Referenced by GetParametersFromSession(), Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ m_npts

Array<OneD, size_t> Nektar::SolverUtils::DriverParallelInTime::m_npts
protected

◆ m_nsteps

Array<OneD, size_t> Nektar::SolverUtils::DriverParallelInTime::m_nsteps
protected

◆ m_nTimeLevel

size_t Nektar::SolverUtils::DriverParallelInTime::m_nTimeLevel
protected

◆ m_numChunks

size_t Nektar::SolverUtils::DriverParallelInTime::m_numChunks
protected

◆ m_numWindowsPIT

size_t Nektar::SolverUtils::DriverParallelInTime::m_numWindowsPIT
protected

◆ m_nVar

size_t Nektar::SolverUtils::DriverParallelInTime::m_nVar
protected

◆ m_time

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_time
protected

◆ m_timestep

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_timestep
protected

◆ m_tolerPIT

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_tolerPIT
protected

◆ m_totalTime

NekDouble Nektar::SolverUtils::DriverParallelInTime::m_totalTime
protected

Total time integration interval.

Definition at line 123 of file DriverParallelInTime.h.

Referenced by Nektar::SolverUtils::DriverParareal::v_Execute(), and Nektar::SolverUtils::DriverPFASST::v_Execute().

◆ m_vL2Errors

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_vL2Errors
protected

◆ m_vLinfErrors

Array<OneD, NekDouble> Nektar::SolverUtils::DriverParallelInTime::m_vLinfErrors
protected