Intel(R) Threading Building Blocks Doxygen Documentation  version 4.2.3
tbb::internal::governor Class Reference

The class handles access to the single instance of market, and to TLS to keep scheduler instances. More...

#include <governor.h>

Collaboration diagram for tbb::internal::governor:

Static Public Member Functions

static unsigned default_num_threads ()
 
static void one_time_init ()
 
static generic_schedulerinit_scheduler (int num_threads, stack_size_type stack_size, bool auto_init)
 Processes scheduler initialization request (possibly nested) in a master thread. More...
 
static generic_schedulerinit_scheduler_weak ()
 Automatic initialization of scheduler in a master thread with default settings without arena. More...
 
static bool terminate_scheduler (generic_scheduler *s, bool blocking)
 Processes scheduler termination request (possibly nested) in a master thread. More...
 
static void sign_on (generic_scheduler *s)
 Register TBB scheduler instance in thread-local storage. More...
 
static void sign_off (generic_scheduler *s)
 Unregister TBB scheduler instance from thread-local storage. More...
 
static bool is_set (generic_scheduler *s)
 Used to check validity of the local scheduler TLS contents. More...
 
static void assume_scheduler (generic_scheduler *s)
 Temporarily set TLS slot to the given scheduler. More...
 
static uintptr_t tls_value_of (generic_scheduler *s)
 Computes the value of the TLS. More...
 
static generic_schedulertls_scheduler_of (uintptr_t v)
 Converts TLS value to the scheduler pointer. More...
 
static generic_schedulerlocal_scheduler ()
 Obtain the thread-local instance of the TBB scheduler. More...
 
static generic_schedulerlocal_scheduler_weak ()
 
static generic_schedulerlocal_scheduler_if_initialized ()
 
static void terminate_auto_initialized_scheduler ()
 Undo automatic initialization if necessary; call when a thread exits. More...
 
static void print_version_info ()
 
static void initialize_rml_factory ()
 
static bool does_client_join_workers (const tbb::internal::rml::tbb_client &client)
 
static bool speculation_enabled ()
 
static bool rethrow_exception_broken ()
 

Static Private Member Functions

static void acquire_resources ()
 Create key for thread-local storage and initialize RML. More...
 
static void release_resources ()
 Destroy the thread-local storage key and deinitialize RML. More...
 
static rml::tbb_server * create_rml_server (rml::tbb_client &)
 
static void auto_terminate (void *scheduler)
 The internal routine to undo automatic initialization. More...
 

Static Private Attributes

static basic_tls< uintptr_t > theTLS
 TLS for scheduler instances associated with individual threads. More...
 
static unsigned DefaultNumberOfThreads
 Caches the maximal level of parallelism supported by the hardware. More...
 
static rml::tbb_factory theRMLServerFactory
 
static bool UsePrivateRML
 
static bool is_speculation_enabled
 
static bool is_rethrow_broken
 

Friends

class __TBB_InitOnce
 
class market
 

Detailed Description

The class handles access to the single instance of market, and to TLS to keep scheduler instances.

It also supports automatic on-demand initialization of the TBB scheduler. The class contains only static data members and methods.

Definition at line 52 of file governor.h.

Member Function Documentation

◆ acquire_resources()

void tbb::internal::governor::acquire_resources ( )
staticprivate

Create key for thread-local storage and initialize RML.

Definition at line 71 of file governor.cpp.

71  {
72 #if USE_PTHREAD
73  int status = theTLS.create(auto_terminate);
74 #else
75  int status = theTLS.create();
76 #endif
77  if( status )
78  handle_perror(status, "TBB failed to initialize task scheduler TLS\n");
81 }
static bool is_rethrow_broken
Definition: governor.h:69
bool gcc_rethrow_exception_broken()
Definition: tbb_misc.cpp:189
static bool is_speculation_enabled
Definition: governor.h:68
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
static void auto_terminate(void *scheduler)
The internal routine to undo automatic initialization.
Definition: governor.cpp:217
bool cpu_has_speculation()
check for transaction support.
Definition: tbb_misc.cpp:221
void __TBB_EXPORTED_FUNC handle_perror(int error_code, const char *aux_info)
Throws std::runtime_error with what() returning error_code description prefixed with aux_info.
Definition: tbb_misc.cpp:78

References auto_terminate(), tbb::internal::cpu_has_speculation(), tbb::internal::basic_tls< T >::create(), tbb::internal::gcc_rethrow_exception_broken(), tbb::internal::handle_perror(), is_rethrow_broken, is_speculation_enabled, and theTLS.

Referenced by tbb::internal::__TBB_InitOnce::add_ref().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ assume_scheduler()

void tbb::internal::governor::assume_scheduler ( generic_scheduler s)
static

Temporarily set TLS slot to the given scheduler.

Definition at line 120 of file governor.cpp.

120  {
121  theTLS.set( tls_value_of(s) );
122 }
static uintptr_t tls_value_of(generic_scheduler *s)
Computes the value of the TLS.
Definition: governor.cpp:114
void set(T value)
Definition: tls.h:60
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
void const char const char int ITT_FORMAT __itt_group_sync s

References s, tbb::internal::basic_tls< T >::set(), theTLS, and tls_value_of().

Referenced by auto_terminate(), tbb::internal::market::cleanup(), init_scheduler(), tbb::internal::generic_scheduler::nested_arena_entry(), sign_off(), sign_on(), and tbb::internal::nested_arena_context::~nested_arena_context().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ auto_terminate()

void tbb::internal::governor::auto_terminate ( void scheduler)
staticprivate

The internal routine to undo automatic initialization.

The signature is written with void* so that the routine can be the destructor argument to pthread_key_create.

Definition at line 217 of file governor.cpp.

217  {
218  generic_scheduler* s = tls_scheduler_of( uintptr_t(arg) ); // arg is equivalent to theTLS.get()
219  if( s && s->my_auto_initialized ) {
220  if( !--(s->my_ref_count) ) {
221  // If the TLS slot is already cleared by OS or underlying concurrency
222  // runtime, restore its value.
223  if( !is_set(s) )
225  s->cleanup_master( /*blocking_terminate=*/false );
226  __TBB_ASSERT( is_set(NULL), "cleanup_master has not cleared its TLS slot" );
227  }
228  }
229 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:120
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:119
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124

References __TBB_ASSERT, assume_scheduler(), is_set(), s, and tls_scheduler_of().

Referenced by acquire_resources(), and terminate_auto_initialized_scheduler().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ create_rml_server()

rml::tbb_server * tbb::internal::governor::create_rml_server ( rml::tbb_client &  client)
staticprivate

Definition at line 96 of file governor.cpp.

96  {
97  rml::tbb_server* server = NULL;
98  if( !UsePrivateRML ) {
99  ::rml::factory::status_type status = theRMLServerFactory.make_server( server, client );
100  if( status != ::rml::factory::st_success ) {
101  UsePrivateRML = true;
102  runtime_warning( "rml::tbb_factory::make_server failed with status %x, falling back on private rml", status );
103  }
104  }
105  if ( !server ) {
106  __TBB_ASSERT( UsePrivateRML, NULL );
107  server = rml::make_private_server( client );
108  }
109  __TBB_ASSERT( server, "Failed to create RML server" );
110  return server;
111 }
void __TBB_EXPORTED_FUNC runtime_warning(const char *format,...)
Report a runtime warning.
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:63
static bool UsePrivateRML
Definition: governor.h:65
tbb_server * make_private_server(tbb_client &client)
Factory method called from task.cpp to create a private_server.

References __TBB_ASSERT, tbb::internal::rml::make_private_server(), tbb::internal::runtime_warning(), theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::market::market().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ default_num_threads()

static unsigned tbb::internal::governor::default_num_threads ( )
inlinestatic

Definition at line 85 of file governor.h.

85  {
86  // No memory fence required, because at worst each invoking thread calls AvailableHwConcurrency once.
89  }
int AvailableHwConcurrency()
Returns maximal parallelism level supported by the current OS configuration.
static unsigned DefaultNumberOfThreads
Caches the maximal level of parallelism supported by the hardware.
Definition: governor.h:61

References tbb::internal::AvailableHwConcurrency(), and DefaultNumberOfThreads.

Referenced by tbb::internal::calc_workers_soft_limit(), tbb::internal::rml::private_server::default_concurrency(), tbb::task_scheduler_init::default_num_threads(), tbb::internal::allowed_parallelism_control::default_value(), tbb::internal::DoOneTimeInitializations(), tbb::internal::market::global_market(), tbb::internal::tbb_thread_v3::hardware_concurrency(), init_scheduler(), tbb::interface7::internal::task_arena_base::internal_initialize(), and tbb::interface7::internal::task_arena_base::internal_max_concurrency().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ does_client_join_workers()

bool tbb::internal::governor::does_client_join_workers ( const tbb::internal::rml::tbb_client &  client)
static

Definition at line 296 of file market.cpp.

296  {
297  return ((const market&)client).must_join_workers();
298 }
friend class market
Definition: governor.h:55

Referenced by tbb::internal::rml::private_worker::start_shutdown(), and tbb::internal::rml::private_worker::wake_or_launch().

Here is the caller graph for this function:

◆ init_scheduler()

generic_scheduler * tbb::internal::governor::init_scheduler ( int  num_threads,
stack_size_type  stack_size,
bool  auto_init 
)
static

Processes scheduler initialization request (possibly nested) in a master thread.

If necessary creates new instance of arena and/or local scheduler. The auto_init argument specifies if the call is due to automatic initialization.

Definition at line 176 of file governor.cpp.

176  {
177  one_time_init();
178  if ( uintptr_t v = theTLS.get() ) {
179  generic_scheduler* s = tls_scheduler_of( v );
180  if ( (v&1) == 0 ) { // TLS holds scheduler instance without arena
181  __TBB_ASSERT( s->my_ref_count == 1, "weakly initialized scheduler must have refcount equal to 1" );
182  __TBB_ASSERT( !s->my_arena, "weakly initialized scheduler must have no arena" );
183  __TBB_ASSERT( s->my_auto_initialized, "weakly initialized scheduler is supposed to be auto-initialized" );
184  s->attach_arena( market::create_arena( default_num_threads(), 1, 0 ), 0, /*is_master*/true );
185  __TBB_ASSERT( s->my_arena_index == 0, "Master thread must occupy the first slot in its arena" );
186  s->my_arena_slot->my_scheduler = s;
187  s->my_arena->my_default_ctx = s->default_context(); // it also transfers implied ownership
188  // Mark the scheduler as fully initialized
189  assume_scheduler( s );
190  }
191  // Increment refcount only for explicit instances of task_scheduler_init.
192  if ( !auto_init ) s->my_ref_count += 1;
193  __TBB_ASSERT( s->my_arena, "scheduler is not initialized fully" );
194  return s;
195  }
196  // Create new scheduler instance with arena
197  if( num_threads == task_scheduler_init::automatic )
198  num_threads = default_num_threads();
199  arena *a = market::create_arena( num_threads, 1, stack_size );
200  generic_scheduler* s = generic_scheduler::create_master( a );
201  __TBB_ASSERT(s, "Somehow a local scheduler creation for a master thread failed");
202  __TBB_ASSERT( is_set(s), NULL );
203  s->my_auto_initialized = auto_init;
204  return s;
205 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
static generic_scheduler * create_master(arena *a)
Initialize a scheduler for a master thread.
Definition: scheduler.cpp:1252
static const int automatic
Typedef for number of threads that is automatic.
static unsigned default_num_threads()
Definition: governor.h:85
static arena * create_arena(int num_slots, int num_reserved_slots, size_t stack_size)
Creates an arena object.
Definition: market.cpp:300
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:120
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:119
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
static void one_time_init()
Definition: governor.cpp:160
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124

References __TBB_ASSERT, assume_scheduler(), tbb::task_scheduler_init::automatic, tbb::internal::market::create_arena(), tbb::internal::generic_scheduler::create_master(), default_num_threads(), tbb::internal::basic_tls< T >::get(), is_set(), one_time_init(), s, theTLS, and tls_scheduler_of().

Referenced by tbb::task_scheduler_init::initialize(), and local_scheduler().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ init_scheduler_weak()

generic_scheduler * tbb::internal::governor::init_scheduler_weak ( )
static

Automatic initialization of scheduler in a master thread with default settings without arena.

Definition at line 168 of file governor.cpp.

168  {
169  one_time_init();
170  __TBB_ASSERT( is_set(NULL), "TLS contains a scheduler?" );
171  generic_scheduler* s = generic_scheduler::create_master( NULL ); // without arena
172  s->my_auto_initialized = true;
173  return s;
174 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
static generic_scheduler * create_master(arena *a)
Initialize a scheduler for a master thread.
Definition: scheduler.cpp:1252
static void one_time_init()
Definition: governor.cpp:160
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124

References __TBB_ASSERT, tbb::internal::generic_scheduler::create_master(), is_set(), one_time_init(), and s.

Referenced by local_scheduler_weak().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ initialize_rml_factory()

void tbb::internal::governor::initialize_rml_factory ( )
static

Definition at line 244 of file governor.cpp.

244  {
245  ::rml::factory::status_type res = theRMLServerFactory.open();
246  UsePrivateRML = res != ::rml::factory::st_success;
247 }
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:63
static bool UsePrivateRML
Definition: governor.h:65

References theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::DoOneTimeInitializations().

Here is the caller graph for this function:

◆ is_set()

bool tbb::internal::governor::is_set ( generic_scheduler s)
static

Used to check validity of the local scheduler TLS contents.

Definition at line 124 of file governor.cpp.

124  {
125  return theTLS.get() == tls_value_of(s);
126 }
static uintptr_t tls_value_of(generic_scheduler *s)
Computes the value of the TLS.
Definition: governor.cpp:114
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
void const char const char int ITT_FORMAT __itt_group_sync s

References tbb::internal::basic_tls< T >::get(), s, theTLS, and tls_value_of().

Referenced by auto_terminate(), tbb::internal::generic_scheduler::cleanup_master(), tbb::internal::generic_scheduler::generic_scheduler(), init_scheduler(), init_scheduler_weak(), tbb::internal::generic_scheduler::local_spawn(), tbb::internal::generic_scheduler::local_spawn_root_and_wait(), tbb::internal::custom_scheduler< SchedulerTraits >::local_wait_for_all(), tbb::internal::arena::process(), tbb::internal::market::process(), sign_off(), sign_on(), and terminate_scheduler().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ local_scheduler()

static generic_scheduler* tbb::internal::governor::local_scheduler ( )
inlinestatic

Obtain the thread-local instance of the TBB scheduler.

If the scheduler has not been initialized yet, initialization is done automatically. Note that auto-initialized scheduler instance is destroyed only when its thread terminates.

Definition at line 126 of file governor.h.

126  {
127  uintptr_t v = theTLS.get();
128  return (v&1) ? tls_scheduler_of(v) : init_scheduler( task_scheduler_init::automatic, 0, /*auto_init=*/true );
129  }
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:119
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
static generic_scheduler * init_scheduler(int num_threads, stack_size_type stack_size, bool auto_init)
Processes scheduler initialization request (possibly nested) in a master thread.
Definition: governor.cpp:176

References tbb::task_scheduler_init::automatic, tbb::internal::basic_tls< T >::get(), init_scheduler(), theTLS, and tls_scheduler_of().

Referenced by tbb::internal::arena::advertise_new_work(), tbb::internal::generic_scheduler::enqueue(), tbb::internal::get_initial_auto_partitioner_divisor(), tbb::internal::affinity_partitioner_base_v3::resize(), tbb::internal::generic_scheduler::spawn(), tbb::task::spawn_and_wait_for_all(), tbb::internal::generic_scheduler::spawn_root_and_wait(), and tbb::internal::custom_scheduler< SchedulerTraits >::wait_for_all().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ local_scheduler_if_initialized()

static generic_scheduler* tbb::internal::governor::local_scheduler_if_initialized ( )
inlinestatic

◆ local_scheduler_weak()

static generic_scheduler* tbb::internal::governor::local_scheduler_weak ( )
inlinestatic

Definition at line 131 of file governor.h.

131  {
132  uintptr_t v = theTLS.get();
133  return v ? tls_scheduler_of(v) : init_scheduler_weak();
134  }
static generic_scheduler * init_scheduler_weak()
Automatic initialization of scheduler in a master thread with default settings without arena.
Definition: governor.cpp:168
static generic_scheduler * tls_scheduler_of(uintptr_t v)
Converts TLS value to the scheduler pointer.
Definition: governor.h:119
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58

References tbb::internal::basic_tls< T >::get(), init_scheduler_weak(), theTLS, and tls_scheduler_of().

Referenced by tbb::internal::allocate_additional_child_of_proxy::allocate(), tbb::internal::allocate_root_proxy::allocate(), tbb::internal::allocate_continuation_proxy::allocate(), tbb::internal::allocate_child_proxy::allocate(), tbb::interface5::internal::task_base::destroy(), tbb::internal::allocate_additional_child_of_proxy::free(), tbb::internal::allocate_root_proxy::free(), tbb::internal::allocate_continuation_proxy::free(), tbb::internal::allocate_child_proxy::free(), tbb::interface7::internal::task_arena_base::internal_execute(), tbb::interface7::internal::task_arena_base::internal_initialize(), tbb::interface7::internal::task_arena_base::internal_wait(), and tbb::task::self().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ one_time_init()

void tbb::internal::governor::one_time_init ( )
static

Definition at line 160 of file governor.cpp.

160  {
163 #if __TBB_SURVIVE_THREAD_SWITCH
164  atomic_do_once( &initialize_cilk_interop, cilkrts_load_state );
165 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
166 }
static bool initialization_done()
Definition: tbb_main.h:68
void atomic_do_once(const F &initializer, atomic< do_once_state > &state)
One-time initialization function.
Definition: tbb_misc.h:210
void DoOneTimeInitializations()
Performs thread-safe lazy one-time general TBB initialization.
Definition: tbb_main.cpp:218

References tbb::internal::atomic_do_once(), tbb::internal::DoOneTimeInitializations(), and tbb::internal::__TBB_InitOnce::initialization_done().

Referenced by init_scheduler(), init_scheduler_weak(), and tbb::interface7::internal::task_arena_base::internal_initialize().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ print_version_info()

void tbb::internal::governor::print_version_info ( )
static

Definition at line 231 of file governor.cpp.

231  {
232  if ( UsePrivateRML )
233  PrintExtraVersionInfo( "RML", "private" );
234  else {
235  PrintExtraVersionInfo( "RML", "shared" );
236  theRMLServerFactory.call_with_server_info( PrintRMLVersionInfo, (void*)"" );
237  }
238 #if __TBB_SURVIVE_THREAD_SWITCH
239  if( watch_stack_handler )
240  PrintExtraVersionInfo( "CILK", CILKLIB_NAME );
241 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
242 }
void PrintRMLVersionInfo(void *arg, const char *server_info)
A callback routine to print RML version information on stderr.
Definition: tbb_misc.cpp:213
void PrintExtraVersionInfo(const char *category, const char *format,...)
Prints arbitrary extra TBB version information on stderr.
Definition: tbb_misc.cpp:202
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:63
static bool UsePrivateRML
Definition: governor.h:65

References tbb::internal::PrintExtraVersionInfo(), tbb::internal::PrintRMLVersionInfo(), theRMLServerFactory, and UsePrivateRML.

Referenced by tbb::internal::DoOneTimeInitializations().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ release_resources()

void tbb::internal::governor::release_resources ( )
staticprivate

Destroy the thread-local storage key and deinitialize RML.

Definition at line 83 of file governor.cpp.

83  {
84  theRMLServerFactory.close();
86 #if TBB_USE_ASSERT
88  runtime_warning( "TBB is unloaded while tbb::task_scheduler_init object is alive?" );
89 #endif
90  int status = theTLS.destroy();
91  if( status )
92  runtime_warning("failed to destroy task scheduler TLS: %s", strerror(status));
94 }
void __TBB_EXPORTED_FUNC runtime_warning(const char *format,...)
Report a runtime warning.
static rml::tbb_factory theRMLServerFactory
Definition: governor.h:63
static bool initialization_done()
Definition: tbb_main.h:68
static basic_tls< uintptr_t > theTLS
TLS for scheduler instances associated with individual threads.
Definition: governor.h:58
void destroy_process_mask()
Definition: tbb_misc.h:263
void dynamic_unlink_all()

References tbb::internal::basic_tls< T >::destroy(), tbb::internal::destroy_process_mask(), dynamic_unlink_all(), tbb::internal::basic_tls< T >::get(), tbb::internal::__TBB_InitOnce::initialization_done(), tbb::internal::runtime_warning(), theRMLServerFactory, and theTLS.

Referenced by tbb::internal::__TBB_InitOnce::remove_ref().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ rethrow_exception_broken()

static bool tbb::internal::governor::rethrow_exception_broken ( )
inlinestatic

Definition at line 156 of file governor.h.

156 { return is_rethrow_broken; }
static bool is_rethrow_broken
Definition: governor.h:69

References is_rethrow_broken.

◆ sign_off()

void tbb::internal::governor::sign_off ( generic_scheduler s)
static

Unregister TBB scheduler instance from thread-local storage.

Definition at line 149 of file governor.cpp.

149  {
151  __TBB_ASSERT( is_set(s), "attempt to unregister a wrong scheduler instance" );
152  assume_scheduler(NULL);
153 #if __TBB_SURVIVE_THREAD_SWITCH
154  __cilk_tbb_unwatch_thunk &ut = s->my_cilk_unwatch_thunk;
155  if ( ut.routine )
156  (*ut.routine)(ut.data);
157 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
158 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
void suppress_unused_warning(const T1 &)
Utility template function to prevent "unused" warnings by various compilers.
Definition: tbb_stddef.h:381
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:120
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124
__cilk_tbb_pfn_unwatch_stacks routine

References __TBB_ASSERT, assume_scheduler(), __cilk_tbb_unwatch_thunk::data, is_set(), __cilk_tbb_unwatch_thunk::routine, s, and tbb::internal::suppress_unused_warning().

Referenced by tbb::internal::generic_scheduler::free_scheduler().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ sign_on()

void tbb::internal::governor::sign_on ( generic_scheduler s)
static

Register TBB scheduler instance in thread-local storage.

Definition at line 128 of file governor.cpp.

128  {
129  __TBB_ASSERT( is_set(NULL) && s, NULL );
130  assume_scheduler( s );
131 #if __TBB_SURVIVE_THREAD_SWITCH
132  if( watch_stack_handler ) {
134  o.routine = &stack_op_handler;
135  o.data = s;
136  if( (*watch_stack_handler)(&s->my_cilk_unwatch_thunk, o) ) {
137  // Failed to register with cilkrts, make sure we are clean
138  s->my_cilk_unwatch_thunk.routine = NULL;
139  }
140 #if TBB_USE_ASSERT
141  else
142  s->my_cilk_state = generic_scheduler::cs_running;
143 #endif /* TBB_USE_ASSERT */
144  }
145 #endif /* __TBB_SURVIVE_THREAD_SWITCH */
146  __TBB_ASSERT( is_set(s), NULL );
147 }
__cilk_tbb_pfn_stack_op routine
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
static void assume_scheduler(generic_scheduler *s)
Temporarily set TLS slot to the given scheduler.
Definition: governor.cpp:120
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124

References __TBB_ASSERT, assume_scheduler(), __cilk_tbb_stack_op_thunk::data, is_set(), __cilk_tbb_stack_op_thunk::routine, and s.

Referenced by tbb::internal::generic_scheduler::create_master(), and tbb::internal::generic_scheduler::create_worker().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ speculation_enabled()

static bool tbb::internal::governor::speculation_enabled ( )
inlinestatic

Definition at line 155 of file governor.h.

155 { return is_speculation_enabled; }
static bool is_speculation_enabled
Definition: governor.h:68

References is_speculation_enabled.

Referenced by tbb::mutex::internal_destroy().

Here is the caller graph for this function:

◆ terminate_auto_initialized_scheduler()

static void tbb::internal::governor::terminate_auto_initialized_scheduler ( )
inlinestatic

Undo automatic initialization if necessary; call when a thread exits.

Definition at line 141 of file governor.h.

141  {
143  }
static generic_scheduler * local_scheduler_if_initialized()
Definition: governor.h:136
static void auto_terminate(void *scheduler)
The internal routine to undo automatic initialization.
Definition: governor.cpp:217

References auto_terminate(), and local_scheduler_if_initialized().

Referenced by tbb::internal::__TBB_InitOnce::~__TBB_InitOnce().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ terminate_scheduler()

bool tbb::internal::governor::terminate_scheduler ( generic_scheduler s,
bool  blocking 
)
static

Processes scheduler termination request (possibly nested) in a master thread.

Definition at line 207 of file governor.cpp.

207  {
208  bool ok = false;
209  __TBB_ASSERT( is_set(s), "Attempt to terminate non-local scheduler instance" );
210  if (0 == --(s->my_ref_count)) {
211  ok = s->cleanup_master( blocking );
212  __TBB_ASSERT( is_set(NULL), "cleanup_master has not cleared its TLS slot" );
213  }
214  return ok;
215 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
void const char const char int ITT_FORMAT __itt_group_sync s
static bool is_set(generic_scheduler *s)
Used to check validity of the local scheduler TLS contents.
Definition: governor.cpp:124

References __TBB_ASSERT, is_set(), and s.

Referenced by tbb::task_scheduler_init::internal_terminate().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ tls_scheduler_of()

static generic_scheduler* tbb::internal::governor::tls_scheduler_of ( uintptr_t  v)
inlinestatic

Converts TLS value to the scheduler pointer.

Definition at line 119 of file governor.h.

119  {
120  return (generic_scheduler*)(v & ~uintptr_t(1));
121  }

Referenced by auto_terminate(), init_scheduler(), local_scheduler(), local_scheduler_if_initialized(), and local_scheduler_weak().

Here is the caller graph for this function:

◆ tls_value_of()

uintptr_t tbb::internal::governor::tls_value_of ( generic_scheduler s)
static

Computes the value of the TLS.

Definition at line 114 of file governor.cpp.

114  {
115  __TBB_ASSERT( (uintptr_t(s)&1) == 0, "Bad pointer to the scheduler" );
116  // LSB marks the scheduler initialized with arena
117  return uintptr_t(s) | uintptr_t((s && (s->my_arena || s->is_worker()))? 1 : 0);
118 }
#define __TBB_ASSERT(predicate, comment)
No-op version of __TBB_ASSERT.
Definition: tbb_stddef.h:169
void const char const char int ITT_FORMAT __itt_group_sync s

References __TBB_ASSERT, and s.

Referenced by assume_scheduler(), and is_set().

Here is the caller graph for this function:

Friends And Related Function Documentation

◆ __TBB_InitOnce

friend class __TBB_InitOnce
friend

Definition at line 54 of file governor.h.

◆ market

friend class market
friend

Definition at line 55 of file governor.h.

Member Data Documentation

◆ DefaultNumberOfThreads

unsigned tbb::internal::governor::DefaultNumberOfThreads
staticprivate

Caches the maximal level of parallelism supported by the hardware.

Definition at line 61 of file governor.h.

Referenced by default_num_threads().

◆ is_rethrow_broken

bool tbb::internal::governor::is_rethrow_broken
staticprivate

Definition at line 69 of file governor.h.

Referenced by acquire_resources(), and rethrow_exception_broken().

◆ is_speculation_enabled

bool tbb::internal::governor::is_speculation_enabled
staticprivate

Definition at line 68 of file governor.h.

Referenced by acquire_resources(), and speculation_enabled().

◆ theRMLServerFactory

rml::tbb_factory tbb::internal::governor::theRMLServerFactory
staticprivate

◆ theTLS

basic_tls< uintptr_t > tbb::internal::governor::theTLS
staticprivate

TLS for scheduler instances associated with individual threads.

Definition at line 58 of file governor.h.

Referenced by acquire_resources(), assume_scheduler(), init_scheduler(), is_set(), local_scheduler(), local_scheduler_if_initialized(), local_scheduler_weak(), and release_resources().

◆ UsePrivateRML

bool tbb::internal::governor::UsePrivateRML
staticprivate

The documentation for this class was generated from the following files:

Copyright © 2005-2019 Intel Corporation. All Rights Reserved.

Intel, Pentium, Intel Xeon, Itanium, Intel XScale and VTune are registered trademarks or trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

* Other names and brands may be claimed as the property of others.