CVS log for libraries/libldap_r/tpool.c

Up to [OpenLDAP] / libraries / libldap_r

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Happy New Year

happy belated New Year

Cleanup & slight speedup (no real change):
Remove '#if 0 / broken code / #endif.  Rearrange pool_wrapper() to avoid
decrement-increment(ltp_active_count) when more tasks available.

Update last commit: avoid LDAP_STAILQ_REMOVE in ldap_pvt_thread_pool_retract

Fix prev commit, note *task* not *thread* ...

Add ldap_pvt_thread_pool_retract() to cancel pending threads

Update copyright notices

Avoid locking up slapd when paused during shutdown (ITS#5841), reverting
r1.81

ITS#5407 cleanup (make pool_pause & pool_pausecheck wrappers for handle_pause)

ITS#5407 more checks for pool pausing

ITS#5364 cleanup
Fix rev 1.91 patch: Reset ltp_pending_count when flushing ltp_pending_list.
Remove flush_pending_list() again and the now-unnecessary 2nd call to it.
Help the compiler a little.
Move ltp_work_list in case it makes a difference for caching.
Move mutex unlock in pool_destroy() to make concurrency debuggers happier.

ITS#5364, reduce work with ltp_mutex locked:
Negate ltp_open_count when paused, avoids an ltp_pause test in pool_submit().

ITS#5364: introduce ltp_work_list, drop 1st ltp_pause loop in pool_wrapper().

ITS#5364: maintain value ltp_vary_open_count

ITS#5364, reduce work with ltp_mutex locked:
Replace ltp_state with ltp_finishing.  Drop state
LDAP_INT_THREAD_POOL_STOPPING, flush pending list instead.
ltp_max_pending = default value instead of 0, and negative when finishing.

ITS#5364, thread pool efficiency:
Add ldap_pvt_thread_pool_pausing(): pause check for slapd without locking.
Make counters int instead of long; INT_MAX pending tasks is enough.
Nitpick cleanup: goto failure instead of if() to not-failure in _submit().

Warning cleanup: function ptr <=> void* at ldap_pvt_thread_pool_<set/get>key()

Happy New Year (belated)

rename ldap_pvt_thread_pool_setkey_x() to ldap_pvt_thread_pool_setkey() (as part of ITS#5309)

new ldap_pvt_thread_pool_setkey API

Fix typo in prev commit

Use thread-specific data in pool_context()

Added native thread-specific data support

ITS#5194 zero out pool when destroying

In pool_resume don't touch the condvar if the pool is tearing down.

ITS#4188 check for pool pause every 64 entries. (probably should make
this number tunable.)

Ensure ltp_max_pending >= 0; negative values confused pool_query().
Remove accidentally added #define LDAP_MAX_PENDING from last commit.

Remove bogus pause wait near end of pool_wrapper().  Add/fix comments.

Rename thread contexts to tasks, leaving user contexts the only context.
Remove ldap_int_thread_ctx_t.ltc_next.al (active list) left over from rev 1.75.

For ITS#4943: Axe thread pool semaphore code

Remove ltp_active_list.  It became unneeded when rev 1.24 added thread_keys[].
Tighten loop in pool_pause(): Omit unneeded ltp_pcond signals since rev 1.64.
Fix comment.

Simplify: Remove tid_zero, thread_keys[].id and ldap_int_main_tid.
Not needed after ldap_int_thread_userctx_t.ltu_id was added and
_pool_wrapper took care not to update thread_keys[] during pauses.
Fix bogus thread_keys[].ctx comment.

Add comments. Constify tid_zero. Shut up gcc -Wswitch. if(!tpool)->assert().

Add variable ldap_int_has_thread_pool, assert(at most one pool);

In pool_submit():
- Move problematic mutex_unlock()-mutex_lock() into the unused #ifdef
  LDAP_PVT_THREAD_POOL_SEM_LOAD_CONTROL and add a FIXME comment inside.
- Delay ltp_starting--; until pool_wrapper().  Nonzero value could otherwise
  no longer be exposed when !defined LDAP_PVT_THREAD_POOL_SEM_LOAD_CONTROL.

Preserve key order when deleting, so context_reset will free last keys first.

Comment fixes.
Corrected rev 1.67 cvs comment: ltu_key[] is NULL-terminated _when not full_.

Since we only have LDAP_MAXTHR thread_keys, allow max LDAP_MAXTHR
threads per pool, even when ltp_max_count <= 0 ("unlimited").
Keep ltp_max_count in range [0, LDAP_MAXTHR].

Make ldap_int_thread_userctx_t.ltu_key[] a proper NULL-terminated array.
(setkey/getkey expected that, but purgekey could set a NULL in the middle.)
Added some checks for input key!=NULL.

API changes, need review - I'm not sure what's indented here:
- setkey(data=NULL, kfree!=NULL) searched as if intended to reset
  the key, but updated by setting the key.  Now always updates.
- setkey(key=<not found>, data=NULL) could return either success or
  failure.  Now succeeds iff (data == NULL && kfree == NULL).

thread_keys is a (poor) open-addessed hash table, but it lacked a
"deleted item" mark.  Add DELETED_THREAD_CTX.
Also improve the hash function a bit, and make the hash unsigned.

Protect thread_keys[] with ldap_pvt_thread_pool_mutex, except in
ldap_pvt_thread_pool_purgekey() which may only be called during pauses.

Thus, also wait for pauses to finish before accessing thread_keys in
ldap_int_thread_pool_wrapper().  This may prevent pending tasks from
being started when a pause had been requested, which seems to have been
possible.  If that was a feature, we can split ltp_pause==1 in 2 states:
in pause (causes wait), and pause requested.

Also move 'thread_keys[].id = <thread id>' from pool_submit to
pool_wrapper.  Until pool_wrapper set the ctx as well, thread context
lookup would just return NULL anyway.

Replace state LDAP_INT_THREAD_POOL_PAUSING with member ltp_pause,
so a pause will work during states FINISHING and STOPPING.

Add missing waits and signals, and move waits in pool_wrapper().
Replace if(test) with while(test) when waiting for the multi-purpose
condition variable ltp_cond.

ITS#4943:
In ldap_pvt_thread_pool_submit(), when backing out of thread creation:
  ltp_pending_count '++' -> '--'.  Signal if there are no more threads.
In ldap_int_thread_pool_wrapper():
  if() -> assert() where false would result in eternal loop.

ITS#4912 fix typo in prev commit

ITS#4899 fix for keys not getting cleared

Fix prev commit again

ITS#4821 partially revert prev patch

Calling setkey() with NULL data and kfree should erase the key.

Free thread keys in reverse order of allocation

happy new year

monitor __ALL__ thread parameters

Add ldap_pvt_thread_pool_tid()

Fix tpool_shutdown, don't remove pool from list since pool_destroy
already does it.

Warnings: Unused vars. funcptr=0, not =NULL. if(b=...) -> if((b=...) != NULL).

ITS#4368 unlock pool ltp_mutex before destroying it

ITS#4349 don't wait on condition if there are no threads in the pool.

Get rid of trivial yield, for completeness' sake.

ITS#3950 delete gratuitous yield. (could use pthread_cond_timedwait
if we ever wanted to time out idle threads.)

Happy new year!

Fix prev commit, spawns unnecessary threads.

Fix pool_submit, create threads as long as we're under the max count.

silence SLAP_SEM_LOAD_CONTROL related warnings...

disable SLAP_SEM_LOAD_CONTROL by default, needs additional review/work
Misc. other cleanup

Correct last commit message: lightweight listener (ITS#3975

Align Operation buffers using LBER_ALIGNED_BUFFER

Add thread debugging wrapper thr_debug.c and ldap_thr_debug.h in libldap_r/,
enabled with LDAP_THREAD_DEBUG (cpp macro and environment variable):

Move any ldap_pvt_* definitions from ldap_int_thread.h to ldap_pvt_thread.h.
#define ldap_int_thread_equal/ldap_pvt_thread_equal instead of tpool.c:TID_EQ.
Define some ldap_debug_*_t types, and LDAP_UINTPTR_T, in ldap_int_thread.h.
ldap_int_thread.h/ldap_pvt_thread.h can now be included multiple times, giving
different results depending on whether libldap_r/ldap_thr_debug.h was included.
Add some cleanup and some preprocessor hacks.
#define LDAP_THREAD*_IMPLEMENTATION in libldap_r/*.c, used by ldap_thr_debug.h.
Add PTHREAD_MUTEX_ERRORCHECK/PTHREAD_MUTEX_ERRORCHECK_NP in thr_posix.c.

address ITS#3925; please test

wait also if PAUSING; silence warnings and improve readability with gdb

ITS#3961 better fix - provide a context for the main thread. The context
must be reset by the caller to clear out temp allocations etc.

ITS#3961 provide ldap_pvt_thread_pool_fake_context_init/destroy, don't
use arbitrary context pointers.

ITS#3847 silence warnings

ISO C chokes on comma at end of enum list

Add ldap_pvt_thread_pool_purgekey to free all instances of key across
all threads.

Added ldap_pvt_thread_pool_pause/resume

Happy New Year!

ITS#2997 change MAXTHREADS to LDAP_MAXTHR

Happy new year

Updated notices and acknowledgements

a runqueue for periodic thread execution (for syncrepl)

Use pthread_equal to compare thread IDs

Fix prev commit - hash bytes of ldap_pvt_thread_t to find proper slot

New ldap_pvt_thread_pool_context()

ITS#2404 keep the per-thread context in the individual threads...

Don't spawn threads based on ltp_pending_count, it doesn't get decremented
right away due to ltp_mutex still being held. Just see if all open threads
are active... Use cond_broadcast to signal all waiting threads. Make sure
to remove correct ctx from active list after the thread returns.

Don't free active thread list

Added ldap_pvt_thread_self() and ldap_pvt_thread_pool_context() API.

Happy new year

Added thread-pool getkey/setkey functions

Fix unprotected list modification

ldap_int_thread_pool_wrapper was called with wrong prototype  (ITS#1673)

                        ================
Written by Hallvard B. Furuseth and placed into the public domain.
This software is not subject to any license of the University of Oslo.
                        ================

ldap_int_thread_pool_wrapper() was defined with an parameter
struct ldap_int_thread_pool_s* but called with a void* argument.
Since these may have different representations, it can crash.


Hallvard B. Furuseth <h.b.furuseth@usit.uio.no>, March 2002.

Update copyright statements

Use LDAP_FREE instead of free

Use ldap_queue.h instead of <ac/queue.h>

Rework filter code
Misc cleanup / lint removal

Fix typo

Moved free list to avoid additional mutex

Fix previous commit

Use queue-compat.h for queue management. Cache ctx structures on a free
list instead of always freeing them.

Regenerate configure and fix AC header inclusion

wrap unused function, ldap_int_thread_onlist() in #if 0

specifically check for NULL return from ldap_int_thread_delist(). Also, fixup comment to get rid of compiler warning.

Added "threads" config to slapd.conf.

Fix for thread/fork problem.  Don't start a worker thread until one is
 needed.

Put cond_signal loop inside of mutex'ed area to fix race bug (seen on
Digital OSF 4.0e on an Alpha?)

Reworked thread code to better support thread-library specific
r/w locks and thread pools.  Hide internal structures (using
pthread'ish technics).  Place common code in threads.c.  Move
no-thread code to thr_stub.c.  Move thread pool code to tpool.c.
Removed setconcurrency call from initializer, added 'concurrency'
directive to slapd.  Tested code under pthreads, pth, and no-threads.

Update copyrights for year 2002

wrap unused function, ldap_int_thread_onlist() in #if 0

Import changes from devel including:
+       Fixed -lldap SASL got_max_ssf bug
+       Fixed -lldap TCP_NODELAY bug
+       Fixed -llutil sa_flags never set bug
+       Removed lint
plus
	clients/tools ldap_init() fixes
	-lldap SASL fixes
	binary syntax ";binary" transfer change

Import misc changes from devel including client/tools
ldap_result() call fixes

Import Mark's thread pool max threads changes from devel

Import from devel

Import last minute thread pool fix from devel

Merge latest devel codes into releng 2 branch.
Remove back-perl, back-sql, back-tcl from branch as they have yet
to be updated.
Additional changes are needed prior to release of alpha4.

ITS#2820 fix from HEAD - fix Draft 4 pthread_create, use pthread_equal

ITS#2404 fix from HEAD

Happy (belated) new year

Misc tweaks from HEAD

Changes
    Updated libldap TLS certificate checking
    Updated client tool argument handling
    Updated test suite
    Updated liblutil detach handling
    Added libldap ldap_whoami routines
    Added liblber ber_flatten2 routine
    Added liblutil passwd sanity checks
    Fixed liblber PROTOS bugs
    Fixed ber_flush debug level
    Fixed libldap NULL cred bug
    Build Environment
        Check back-bdb requirement for BDB 4.1

Sync with HEAD, including
	- subclass indexing
	- bdb locker id fixes

updates from HEAD

Suck in HEAD changes since 2.1alpha

Happy new year!

sync with HEAD

Happy new year

Sync with HEAD

Happy new year! (belated)

ITS#4899 fix from HEAD

fix tpool key reset (ITS#4855)

ITS#4805 free thread keys in reverse order

happy new year

tpool cleanup

Misc library fixes from HEAD

Changes from HEAD
+       Update libldap_r thread pool destroy hang fix (ITS#4368)
+       Fixed slapd slap_daemon destroy issue (ITS#4370)

From HEAD:
+       Fixed libldap disable DH key exchange with DH params (ITS#4354)
+       Fixed libldap_r pool destroy hang (ITS#4349)
+       Fixed slapd(8) solaris select(2) issue (ITS#4357)

Updates from HEAD

Happy new year!

Sync with HEAD

Sync with HEAD
Changes needs updating for release...

Sync changes from yesterday

Sync with HEAD (need to update CHANGES)

Sync with HEAD

Sync with HEAD

Initial round of changes for 2.3 beta

Sync with HEAD

Sync with HEAD

Happy New Year!

happy belated new year

Cleanup & slight speedup (no real change):
Remove '#if 0 / broken code / #endif.  Rearrange pool_wrapper() to avoid
decrement-increment(ltp_active_count) when more tasks available.

avoid LDAP_STAILQ_REMOVE in ldap_pvt_thread_pool_retract

Add ldap_pvt_thread_pool_retract() to cancel pending tasks

Update copyright for next release

ITS#5841

ITS#5364, #5407 from HEAD

Happy new year! (belated)

Warning cleanup: function ptr <=> void* at ldap_pvt_thread_pool_<set/get>key()

ITS#5309 work

new ldap_pvt_thread_pool_setkey API

Use per-thread slap_counters to eliminate lock contention
Unifdef SLAP_LIGHTWEIGHT_DISPATCHER
Use thread-specific data in pool_context()
Added native thread-specific data support
Concurrency tweaks:
  store conn->c_sd, don't use ber_sockbuf_ctrl to retrieve it.
  use per-thread free lists for operations, no mutexes needed.

ITS#5194 from HEAD

pool_resume fix from HEAD

ITS#4188

Sync with HEAD for OL 2.4.5

Quick merge: everything from HEAD

Sync with HEAD

Sync with HEAD