Nektar++
Public Member Functions | Static Public Member Functions | Private Member Functions | Private Attributes | Static Private Attributes | Friends | List of all members
Nektar::Thread::ThreadManagerBoost Class Reference

Implementation of ThreadManager using Boost threads. More...

#include <ThreadBoost.h>

Inheritance diagram for Nektar::Thread::ThreadManagerBoost:
Inheritance graph
[legend]
Collaboration diagram for Nektar::Thread::ThreadManagerBoost:
Collaboration graph
[legend]

Public Member Functions

 ThreadManagerBoost (unsigned int numWorkers)
 Constructs a ThreadManagerBoost. More...
 
virtual ~ThreadManagerBoost ()
 Shuts down threading. More...
 
virtual void QueueJobs (std::vector< ThreadJob * > &joblist)
 Pass a list of tasklets to the master queue. More...
 
virtual void QueueJob (ThreadJob *job)
 Pass a single job to the master queue. More...
 
virtual unsigned int GetNumWorkers ()
 Return the number of active workers. More...
 
virtual unsigned int GetWorkerNum ()
 Returns the worker number of the executing thread. More...
 
virtual void SetNumWorkers (unsigned int num)
 Sets the number of active workers. More...
 
virtual void SetNumWorkers ()
 Sets the number of active workers to the maximum. More...
 
virtual unsigned int GetMaxNumWorkers ()
 Gets the maximum available number of threads. More...
 
virtual void Wait ()
 Waits until all queued jobs are finished. More...
 
virtual void SetChunkSize (unsigned int chnk)
 Controls how many jobs are sent to each worker at a time. More...
 
virtual void SetSchedType (SchedType s)
 Sets the current scheduling algorithm. More...
 
virtual bool InThread ()
 Indicates whether the code is in a worker thread or not. More...
 
virtual void Hold ()
 A calling threads holds until all active threads call this method. More...
 
virtual const std::string & GetType () const
 Returns a description of the type of threading. More...
 
- Public Member Functions inherited from Nektar::Thread::ThreadManager
virtual ~ThreadManager ()
 Destructor. More...
 
virtual bool IsInitialised ()
 ThreadManager implementation. More...
 
int GetThrFromPartition (int pPartition)
 
int GetRankFromPartition (int pPartition)
 
int GetPartitionFromRankThr (int pRank, unsigned int pThr)
 

Static Public Member Functions

static ThreadManagerSharedPtr Create (unsigned int numT)
 Called by the factory method. More...
 

Private Member Functions

 ThreadManagerBoost ()
 
 ThreadManagerBoost (const ThreadManagerBoost &)
 
bool IsWorking ()
 
void SetNumWorkersImpl (const unsigned int num)
 

Private Attributes

const unsigned int m_numThreads
 
unsigned int m_numWorkers
 
std::queue< ThreadJob * > m_masterQueue
 
boost::mutex m_masterQueueMutex
 
boost::mutex m_masterActiveMutex
 
boost::condition_variable m_masterQueueCondVar
 
boost::condition_variable m_masterActiveCondVar
 
ThreadWorkerBoost ** m_threadList
 
boost::thread ** m_threadThreadList
 
boost::thread::id m_masterThreadId
 
bool * m_threadBusyList
 
bool * m_threadActiveList
 
unsigned int m_chunkSize
 
SchedType m_schedType
 
boost::barrier * m_barrier
 
std::map< boost::thread::id, unsigned int > m_threadMap
 
std::string m_type
 

Static Private Attributes

static std::string className
 

Friends

class ThreadWorkerBoost
 

Detailed Description

Implementation of ThreadManager using Boost threads.

Definition at line 59 of file ThreadBoost.h.

Constructor & Destructor Documentation

Nektar::Thread::ThreadManagerBoost::ThreadManagerBoost ( unsigned int  numT)

Constructs a ThreadManagerBoost.

Parameters
numWorkersThe number of threads to start (including master thread).
Note
Do not use, use factory instead.

Definition at line 56 of file ThreadBoost.cpp.

References m_barrier, m_masterThreadId, m_numThreads, m_numWorkers, m_threadActiveList, m_threadBusyList, m_threadList, m_threadMap, m_threadThreadList, m_type, and ThreadWorkerBoost.

56  :
57  m_numThreads(numT), m_numWorkers(numT), m_masterQueue(),
60  m_threadMap()
61 {
62  using namespace std;
63  try
64  {
66  m_threadThreadList = new boost::thread*[m_numThreads];
67  m_threadBusyList = new bool[m_numThreads];
68  m_threadActiveList = new bool[m_numThreads];
69  }
70  catch (exception &e)
71  {
72  cerr << "Exception while allocating thread storage: "
73  << e.what() << endl;
74  abort();
75  }
76  unsigned int i = 0;
77  while (i < m_numThreads)
78  {
80  try
81  {
82  tw = new ThreadWorkerBoost(this, i);
83  }
84  catch (exception &e)
85  {
86  cerr << "Exception while allocating worker threads: "
87  << e.what() << endl;
88  abort();
89  }
90 
91  m_threadList[i] = tw;
92  m_threadBusyList[i] = false;
93  m_threadActiveList[i] = true;
94 
95  try
96  {
97  m_threadThreadList[i] = new boost::thread(boost::ref(*tw));
98  boost::thread::id id = m_threadThreadList[i]->get_id();
99  m_threadMap[id] = i;
100  }
101  catch (...)
102  {
103  std::cerr << "Exception while creating worker threads" << std::endl;
104  abort();
105  }
106  i++;
107  }
108  m_masterThreadId = boost::this_thread::get_id();
109  m_barrier = new boost::barrier(m_numWorkers > 0 ? m_numWorkers : 1);
110  m_type = "Threading with Boost";
111 }
boost::thread::id m_masterThreadId
Definition: ThreadBoost.h:109
STL namespace.
boost::condition_variable m_masterQueueCondVar
Definition: ThreadBoost.h:105
std::map< boost::thread::id, unsigned int > m_threadMap
Definition: ThreadBoost.h:115
std::queue< ThreadJob * > m_masterQueue
Definition: ThreadBoost.h:102
ThreadWorkerBoost ** m_threadList
Definition: ThreadBoost.h:107
boost::condition_variable m_masterActiveCondVar
Definition: ThreadBoost.h:106
Nektar::Thread::ThreadManagerBoost::~ThreadManagerBoost ( )
virtual

Shuts down threading.

Terminates all running threads (they will finish their current job), releases resources and destructs.

Definition at line 118 of file ThreadBoost.cpp.

References m_barrier, m_masterActiveCondVar, m_masterQueueCondVar, m_numThreads, m_threadActiveList, m_threadBusyList, m_threadList, m_threadThreadList, and Nektar::Thread::ThreadWorkerBoost::Stop().

119 {
120  // This is an immediate teardown. We attempt to kill everything.
121  // we daren't lock anything as we may cause a deadlock
122  for (unsigned int i=0; i<m_numThreads; i++)
123  {
124  m_threadList[i]->Stop();
125  }
126 
127  m_masterQueueCondVar.notify_all();
128  m_masterActiveCondVar.notify_all();
129  for (unsigned int i=0; i<m_numThreads; i++)
130  {
131  m_threadThreadList[i]->join();
132  delete m_threadThreadList[i];
133  delete m_threadList[i];
134  }
135 
136  delete []m_threadList;
137  delete []m_threadThreadList;
138  delete []m_threadActiveList;
139  delete []m_threadBusyList;
140  delete m_barrier;
141 }
boost::condition_variable m_masterQueueCondVar
Definition: ThreadBoost.h:105
void Stop()
A signal to shut down.
Definition: ThreadBoost.h:150
ThreadWorkerBoost ** m_threadList
Definition: ThreadBoost.h:107
boost::condition_variable m_masterActiveCondVar
Definition: ThreadBoost.h:106
Nektar::Thread::ThreadManagerBoost::ThreadManagerBoost ( )
private

Referenced by Create().

Nektar::Thread::ThreadManagerBoost::ThreadManagerBoost ( const ThreadManagerBoost )
private

Member Function Documentation

static ThreadManagerSharedPtr Nektar::Thread::ThreadManagerBoost::Create ( unsigned int  numT)
inlinestatic

Called by the factory method.

Definition at line 87 of file ThreadBoost.h.

References ThreadManagerBoost().

88  {
89  return boost::shared_ptr<ThreadManager>(
90  new ThreadManagerBoost(numT));
91  }
unsigned int Nektar::Thread::ThreadManagerBoost::GetMaxNumWorkers ( )
virtual

Gets the maximum available number of threads.

Returns
The maximum number of workers.

Implements Nektar::Thread::ThreadManager.

Definition at line 299 of file ThreadBoost.cpp.

References m_numThreads.

300 {
301  return m_numThreads;
302 }
unsigned int Nektar::Thread::ThreadManagerBoost::GetNumWorkers ( )
virtual

Return the number of active workers.

Active workers are threads that are either running jobs or are waiting for jobs to be queued.

Implements Nektar::Thread::ThreadManager.

Definition at line 236 of file ThreadBoost.cpp.

References m_numWorkers.

237 {
238  return m_numWorkers;
239 }
const std::string & Nektar::Thread::ThreadManagerBoost::GetType ( ) const
virtual

Returns a description of the type of threading.

E.g. "Threading with Boost"

Implements Nektar::Thread::ThreadManager.

Definition at line 317 of file ThreadBoost.cpp.

References m_type.

318 {
319  return m_type;
320 }
unsigned int Nektar::Thread::ThreadManagerBoost::GetWorkerNum ( )
virtual

Returns the worker number of the executing thread.

Returns an unsigned int between 0 and N-1 where N is the number of active worker threads. Repeated calls from within this thread will always return the same value and the value will be the same as returned from ThreadJob.GetWorkerNum(). The same thread will run a job until it finishes.

Although if there are active threads then thread 0 is always one of them, it is possible that thread 0 does not run for a given set of jobs. For example, if there are 4 active threads and 3 jobs are submitted with a e_static scheduling strategy and a chunksize of 1, then it is possible that threads 1,2, and 3 pick up the jobs and thread 0 remains idle.

Returns 0 if called by non-thread.

Implements Nektar::Thread::ThreadManager.

Definition at line 245 of file ThreadBoost.cpp.

References m_threadMap.

246 {
247  boost::thread::id id = boost::this_thread::get_id();
248  return m_threadMap[id];
249 }
std::map< boost::thread::id, unsigned int > m_threadMap
Definition: ThreadBoost.h:115
void Nektar::Thread::ThreadManagerBoost::Hold ( )
virtual

A calling threads holds until all active threads call this method.

When called, the calling thread will sleep until all active workers have called this method. Once all have done so all threads awake and continue execution.

Note
Behaviour is likely undefined if the number of active workers is altered after a thread has called this method. It is only safe to call SetNumWorkers() when no threads are holding.

Implements Nektar::Thread::ThreadManager.

Definition at line 308 of file ThreadBoost.cpp.

References m_barrier.

309 {
310  m_barrier->wait();
311 }
bool Nektar::Thread::ThreadManagerBoost::InThread ( )
virtual

Indicates whether the code is in a worker thread or not.

Returns
True if the caller is in a worker thread.

Implements Nektar::Thread::ThreadManager.

Definition at line 206 of file ThreadBoost.cpp.

References m_masterThreadId.

207 {
208  boost::thread::id id = boost::this_thread::get_id();
209  return (id != m_masterThreadId);
210 }
boost::thread::id m_masterThreadId
Definition: ThreadBoost.h:109
bool Nektar::Thread::ThreadManagerBoost::IsWorking ( )
private

Definition at line 171 of file ThreadBoost.cpp.

References m_masterActiveMutex, m_numWorkers, and m_threadBusyList.

Referenced by Wait().

172 {
173  bool working = false;
174  Lock masterActiveLock(m_masterActiveMutex);
175  for (unsigned int i = 0; i < m_numWorkers; i++)
176  {
177  working = working || m_threadBusyList[i];
178  }
179  return working;
180 }
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52
void Nektar::Thread::ThreadManagerBoost::QueueJob ( ThreadJob job)
virtual

Pass a single job to the master queue.

Parameters
jobA pointer to a ThreadJob subclass.

The job may become available for running immediately. If this is an issue then suspend the workers with SetNumWorkers(0) until the jobs are queued.

Implements Nektar::Thread::ThreadManager.

Definition at line 160 of file ThreadBoost.cpp.

References m_masterQueue, m_masterQueueCondVar, and m_masterQueueMutex.

Referenced by QueueJobs().

161 {
162  Lock masterQueueLock(m_masterQueueMutex); // locks the queue
163  m_masterQueue.push(job);
164  m_masterQueueCondVar.notify_all(); // alert a waiting thread.
165 } // queue unlocked
boost::condition_variable m_masterQueueCondVar
Definition: ThreadBoost.h:105
std::queue< ThreadJob * > m_masterQueue
Definition: ThreadBoost.h:102
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52
void Nektar::Thread::ThreadManagerBoost::QueueJobs ( std::vector< ThreadJob * > &  joblist)
virtual

Pass a list of tasklets to the master queue.

Parameters
joblistVector of ThreadJob pointers.

The list of jobs is copied into the master queue. Jobs may be available for running immediately, even before the list has been fully copied. This can have consequences for the scheduling. If this is an issue then suspend the workers with SetNumWorkers(0) until the jobs are queued.

See also
SchedType

Implements Nektar::Thread::ThreadManager.

Definition at line 147 of file ThreadBoost.cpp.

References Nektar::iterator, and QueueJob().

148 {
150  for (it=joblist.begin(); it<joblist.end(); ++it)
151  {
152  QueueJob(*it);
153  }
154 }
virtual void QueueJob(ThreadJob *job)
Pass a single job to the master queue.
StandardMatrixTag boost::call_traits< LhsDataType >::const_reference rhs typedef NekMatrix< LhsDataType, StandardMatrixTag >::iterator iterator
void Nektar::Thread::ThreadManagerBoost::SetChunkSize ( unsigned int  chnk)
virtual

Controls how many jobs are sent to each worker at a time.

The exact meaning of this parameter depends on the current scheduling algorithm.

See also
SchedType
SetSchedType()

Implements Nektar::Thread::ThreadManager.

Definition at line 186 of file ThreadBoost.cpp.

References m_chunkSize, and m_masterQueueMutex.

187 {
188  Lock masterQueueLock(m_masterQueueMutex); // locks the queue
189  m_chunkSize = std::max(chnk, 1U);
190 }
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52
void Nektar::Thread::ThreadManagerBoost::SetNumWorkers ( unsigned int  num)
virtual

Sets the number of active workers.

Parameters
numThe number of active workers.

Active workers are threads that are either running jobs or are waiting for jobs to be queued.

If num is greater than the maximum allowed number of active workers, then the maximum value will be used instead.

Implements Nektar::Thread::ThreadManager.

Definition at line 279 of file ThreadBoost.cpp.

References m_numThreads, and SetNumWorkersImpl().

280 {
281  num = std::min(num, m_numThreads);
282  num = std::max(num, static_cast<unsigned int>(0));
283  SetNumWorkersImpl(num);
284 }
void SetNumWorkersImpl(const unsigned int num)
void Nektar::Thread::ThreadManagerBoost::SetNumWorkers ( )
virtual

Sets the number of active workers to the maximum.

Sets the number of active workers to the maximum available.

Implements Nektar::Thread::ThreadManager.

Definition at line 290 of file ThreadBoost.cpp.

References m_numThreads, and SetNumWorkersImpl().

291 {
293 }
void SetNumWorkersImpl(const unsigned int num)
void Nektar::Thread::ThreadManagerBoost::SetNumWorkersImpl ( const unsigned int  num)
private

Definition at line 255 of file ThreadBoost.cpp.

References m_barrier, m_masterActiveCondVar, m_masterActiveMutex, m_numThreads, m_numWorkers, and m_threadActiveList.

Referenced by SetNumWorkers().

256 {
257  Lock masterActiveLock(m_masterActiveMutex); // locks the active
258 
259  if (m_numWorkers == num)
260  {
261  return;
262  }
263 
264  delete m_barrier;
265  m_barrier = new boost::barrier(num > 0 ? num : 1);
266 
267  m_numWorkers = num;
268  for (unsigned int i = 0; i < m_numThreads; i++)
269  {
270  m_threadActiveList[i] = i < m_numWorkers ? true : false;
271  }
272  m_masterActiveCondVar.notify_all();
273 } // Lock on active released here
boost::condition_variable m_masterActiveCondVar
Definition: ThreadBoost.h:106
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52
void Nektar::Thread::ThreadManagerBoost::SetSchedType ( SchedType  s)
virtual

Sets the current scheduling algorithm.

See also
SetChunkSize()

Implements Nektar::Thread::ThreadManager.

Definition at line 196 of file ThreadBoost.cpp.

References m_masterQueueMutex, and m_schedType.

197 {
198  Lock masterQueueLock(m_masterQueueMutex); // locks the queue
199  m_schedType = s;
200 }
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52
void Nektar::Thread::ThreadManagerBoost::Wait ( )
virtual

Waits until all queued jobs are finished.

If there are no jobs running or queued this method returns immediately. Otherwise it blocks until the queue is empty and the worker threads are idle.

Implementations must ensure that trivial deadlocks are not possible from this method, that is, that this code:

// assume ThreadManager* tm
// assume SomeJob is subclass of ThreadJob
// assume SomeJob job
tm->SetNumWorkers(0);
tm->QueueJob(job);
tm->Wait();

does not wait forever. Since the master thread is counted in the number of worker threads, implementations should increase the number of active workers by 1 on entering Wait().

Implements Nektar::Thread::ThreadManager.

Definition at line 216 of file ThreadBoost.cpp.

References IsWorking(), m_masterActiveCondVar, m_masterQueue, m_masterQueueCondVar, and m_masterQueueMutex.

217 {
218  bool working;
219  Lock masterQueueLock(m_masterQueueMutex); // locks the queue
220  working = IsWorking();
221  m_masterActiveCondVar.notify_all();
222  m_masterQueueCondVar.notify_all();
223  while (!m_masterQueue.empty() || working)
224  {
225  // while waiting, master queue is unlocked
226  m_masterQueueCondVar.wait(masterQueueLock);
227  // on exiting wait master queue is locked again
228  working = IsWorking();
229  }
230 }
boost::condition_variable m_masterQueueCondVar
Definition: ThreadBoost.h:105
std::queue< ThreadJob * > m_masterQueue
Definition: ThreadBoost.h:102
boost::condition_variable m_masterActiveCondVar
Definition: ThreadBoost.h:106
boost::unique_lock< boost::mutex > Lock
Definition: ThreadBoost.h:52

Friends And Related Function Documentation

friend class ThreadWorkerBoost
friend

So the workers can access the master queue and locks.

Definition at line 64 of file ThreadBoost.h.

Referenced by ThreadManagerBoost().

Member Data Documentation

std::string Nektar::Thread::ThreadManagerBoost::className
staticprivate
Initial value:
=
ThreadManagerBoost::Create, "Threading using Boost.")

Definition at line 116 of file ThreadBoost.h.

boost::barrier* Nektar::Thread::ThreadManagerBoost::m_barrier
private

Definition at line 114 of file ThreadBoost.h.

Referenced by Hold(), SetNumWorkersImpl(), ThreadManagerBoost(), and ~ThreadManagerBoost().

unsigned int Nektar::Thread::ThreadManagerBoost::m_chunkSize
private

Definition at line 112 of file ThreadBoost.h.

Referenced by Nektar::Thread::ThreadWorkerBoost::GetNumToLoad(), and SetChunkSize().

boost::condition_variable Nektar::Thread::ThreadManagerBoost::m_masterActiveCondVar
private
boost::mutex Nektar::Thread::ThreadManagerBoost::m_masterActiveMutex
private
std::queue<ThreadJob*> Nektar::Thread::ThreadManagerBoost::m_masterQueue
private
boost::condition_variable Nektar::Thread::ThreadManagerBoost::m_masterQueueCondVar
private
boost::mutex Nektar::Thread::ThreadManagerBoost::m_masterQueueMutex
private
boost::thread::id Nektar::Thread::ThreadManagerBoost::m_masterThreadId
private

Definition at line 109 of file ThreadBoost.h.

Referenced by InThread(), and ThreadManagerBoost().

const unsigned int Nektar::Thread::ThreadManagerBoost::m_numThreads
private
unsigned int Nektar::Thread::ThreadManagerBoost::m_numWorkers
private
SchedType Nektar::Thread::ThreadManagerBoost::m_schedType
private

Definition at line 113 of file ThreadBoost.h.

Referenced by Nektar::Thread::ThreadWorkerBoost::GetNumToLoad(), and SetSchedType().

bool* Nektar::Thread::ThreadManagerBoost::m_threadActiveList
private
bool* Nektar::Thread::ThreadManagerBoost::m_threadBusyList
private
ThreadWorkerBoost** Nektar::Thread::ThreadManagerBoost::m_threadList
private

Definition at line 107 of file ThreadBoost.h.

Referenced by ThreadManagerBoost(), and ~ThreadManagerBoost().

std::map<boost::thread::id, unsigned int> Nektar::Thread::ThreadManagerBoost::m_threadMap
private

Definition at line 115 of file ThreadBoost.h.

Referenced by GetWorkerNum(), and ThreadManagerBoost().

boost::thread** Nektar::Thread::ThreadManagerBoost::m_threadThreadList
private

Definition at line 108 of file ThreadBoost.h.

Referenced by ThreadManagerBoost(), and ~ThreadManagerBoost().

std::string Nektar::Thread::ThreadManagerBoost::m_type
private

Definition at line 117 of file ThreadBoost.h.

Referenced by GetType(), and ThreadManagerBoost().