ThreadPoolExecutor

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

ThreadPoolExecutor

superuser
Hi, great work on the util.concurrent package, we have converted from to
Oswego package entirely, however there remains one missing piece that I
could not figure out how to do. Basically, we need a ThreadPoolExecutor
with the following properties:
 
1) {infinite | bounded} linked list request queue, where the bound is
really high to avoid OOM (100000+)
2) MIN of 0 threads in pool
3) MAX # of threads in pool, configurable on pool create
4) reuse existing threads if available
5) allow pool to shrink to zero threads if there are no outstanding
requests
 

This was possible in the old Oswego package, but now in the JDK, these
all seem possible except for #4, which conflicts with the documentation
(and class ThreadPoolExecutor) here:

"When a new task is submitted in method execute(java.lang.Runnable)
<http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPool
Executor.html#execute%28java.lang.Runnable%29> , and fewer than
corePoolSize threads are running, a new thread is created to handle the
request, even if other worker threads are idle. If there are more than
corePoolSize but less than maximumPoolSize threads running, a new thread
will be created only if the queue is full"



Because LinkedBlockingQueue is never full (or when it is its already
HUGE), no combination of corePoolSize, maximumPoolSize seems to allow
this.

Basically, the problem is:

If corePoolSize == MAX, idle threads are ignored, and we will always
climb up to MAX threads very quickly, even though we only wan MAX
threads under heavy load
If corePoolSize < MAX, with an infinite (or effectively infinite)
request queue, we have effectively reduced MAX to corePoolSize.
 

Currently we have a hacked-up JDK 1.6 ThreadPoolExecutor to give us #4
(with corePoolSize==MAX and allowCoreThreadTimeout==true), IE reuse
existing idle threads before trying to create new ones. I'm not certain
if our implementation is correct, it is most certainly ugly:


Hacked into "public void execute(Runnable command)" right after the "if
(runState != RUNNING)" block:
------------------------------------------------------------------------
------------------
//NOTE: Added this for less thread-crazy behaviour
if (workersBlocked.get() > 0) {
        if(workQueue.offer(command)) {
                //HACK: this should protect against the pool shrinking,
should be very rare...
                if (poolSize == 0)
                        addIfUnderCorePoolSize(new Runnable(){ public
void run() {} });

                return;
        }
}
------------------------------------------------------------------------
------------------
 
 

Everything is working correctly as per the docs AFAIK, just seemingly
counterintuitively. It seems quite pointless and lower performant to be
creating new threads while existing ones are idle and available to
process work. Is this just a bad interaction between ThreadPoolExecutor
and LinkedBlockingQueue?  Is there another queue type that will work for
me or thread pool option I am missing?

 

Thanks in advance and please let me know if I am off base or incorrect
here.


Best regards,

Patrick


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: ThreadPoolExecutor

Joe Bowbeer
I've probably missed something but it occurs to me that you want to be using maxPoolSize instead of corePoolSize.

In other words, have you tried the following?

corePoolSize = 0
maxPoolSize = MAX
keepAliveTime = tuned so that pool shrinks to 0 when executor is idle


On 10/5/05, Patrick Eger <[hidden email]> wrote:
Hi, great work on the util.concurrent package, we have converted from to
Oswego package entirely, however there remains one missing piece that I
could not figure out how to do. Basically, we need a ThreadPoolExecutor
with the following properties:

1) {infinite | bounded} linked list request queue, where the bound is
really high to avoid OOM (100000+)
2) MIN of 0 threads in pool
3) MAX # of threads in pool, configurable on pool create
4) reuse existing threads if available
5) allow pool to shrink to zero threads if there are no outstanding
requests


This was possible in the old Oswego package, but now in the JDK, these
all seem possible except for #4, which conflicts with the documentation
(and class ThreadPoolExecutor) here:

"When a new task is submitted in method execute(java.lang.Runnable)
<http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPool
Executor.html#execute%28java.lang.Runnable%29> , and fewer than
corePoolSize threads are running, a new thread is created to handle the
request, even if other worker threads are idle. If there are more than
corePoolSize but less than maximumPoolSize threads running, a new thread
will be created only if the queue is full"



Because LinkedBlockingQueue is never full (or when it is its already
HUGE), no combination of corePoolSize, maximumPoolSize seems to allow
this.

Basically, the problem is:

If corePoolSize == MAX, idle threads are ignored, and we will always
climb up to MAX threads very quickly, even though we only wan MAX
threads under heavy load
If corePoolSize < MAX, with an infinite (or effectively infinite)
request queue, we have effectively reduced MAX to corePoolSize.


Currently we have a hacked-up JDK 1.6 ThreadPoolExecutor to give us #4
(with corePoolSize==MAX and allowCoreThreadTimeout==true), IE reuse
existing idle threads before trying to create new ones. I'm not certain
if our implementation is correct, it is most certainly ugly:


Hacked into "public void execute(Runnable command)" right after the "if
(runState != RUNNING)" block:
------------------------------------------------------------------------
------------------
//NOTE: Added this for less thread-crazy behaviour
if (workersBlocked.get() > 0) {
        if(workQueue.offer(command)) {
                //HACK: this should protect against the pool shrinking,
should be very rare...
                if (poolSize == 0)
                        addIfUnderCorePoolSize(new Runnable(){ public
void run() {} });

                return;
        }
}
------------------------------------------------------------------------
------------------



Everything is working correctly as per the docs AFAIK, just seemingly
counterintuitively. It seems quite pointless and lower performant to be
creating new threads while existing ones are idle and available to
process work. Is this just a bad interaction between ThreadPoolExecutor
and LinkedBlockingQueue?  Is there another queue type that will work for
me or thread pool option I am missing?



Thanks in advance and please let me know if I am off base or incorrect
here.


Best regards,

Patrick




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

superuser
In reply to this post by superuser
Thank you for the quick response.  Looking at the code for ThreadPoolExecutor (from JDK 1.6):
 
public void execute(Runnable command), line ~876
-------------------------------------------------
Runnable r = addIfUnderMaximumPoolSize(command);
-------------------------------------------------
 
This code is basically the first thing that is run with the below settings. So basically it will operate the same way it seems, IE adding a new thread unconditionally and not reusing existing idle threads.  I will write up a quick test to confirm this behaviour but i'm pretty sure we explored this option as well.
 
Thanks again for your help.
 
 
 
Best regards,
 
Patrick


From: Joe Bowbeer [mailto:[hidden email]]
Sent: Wednesday, October 05, 2005 1:38 PM
To: Patrick Eger
Cc: [hidden email]
Subject: Re: [concurrency-interest] ThreadPoolExecutor

I've probably missed something but it occurs to me that you want to be using maxPoolSize instead of corePoolSize.

In other words, have you tried the following?

corePoolSize = 0
maxPoolSize = MAX
keepAliveTime = tuned so that pool shrinks to 0 when executor is idle


On 10/5/05, Patrick Eger <[hidden email]> wrote:
Hi, great work on the util.concurrent package, we have converted from to
Oswego package entirely, however there remains one missing piece that I
could not figure out how to do. Basically, we need a ThreadPoolExecutor
with the following properties:

1) {infinite | bounded} linked list request queue, where the bound is
really high to avoid OOM (100000+)
2) MIN of 0 threads in pool
3) MAX # of threads in pool, configurable on pool create
4) reuse existing threads if available
5) allow pool to shrink to zero threads if there are no outstanding
requests


This was possible in the old Oswego package, but now in the JDK, these
all seem possible except for #4, which conflicts with the documentation
(and class ThreadPoolExecutor) here:

"When a new task is submitted in method execute(java.lang.Runnable)
<http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPool
Executor.html#execute%28java.lang.Runnable%29> , and fewer than
corePoolSize threads are running, a new thread is created to handle the
request, even if other worker threads are idle. If there are more than
corePoolSize but less than maximumPoolSize threads running, a new thread
will be created only if the queue is full"



Because LinkedBlockingQueue is never full (or when it is its already
HUGE), no combination of corePoolSize, maximumPoolSize seems to allow
this.

Basically, the problem is:

If corePoolSize == MAX, idle threads are ignored, and we will always
climb up to MAX threads very quickly, even though we only wan MAX
threads under heavy load
If corePoolSize < MAX, with an infinite (or effectively infinite)
request queue, we have effectively reduced MAX to corePoolSize.


Currently we have a hacked-up JDK 1.6 ThreadPoolExecutor to give us #4
(with corePoolSize==MAX and allowCoreThreadTimeout==true), IE reuse
existing idle threads before trying to create new ones. I'm not certain
if our implementation is correct, it is most certainly ugly:


Hacked into "public void execute(Runnable command)" right after the "if
(runState != RUNNING)" block:
------------------------------------------------------------------------
------------------
//NOTE: Added this for less thread-crazy behaviour
if (workersBlocked.get() > 0) {
        if(workQueue.offer(command)) {
                //HACK: this should protect against the pool shrinking,
should be very rare...
                if (poolSize == 0)
                        addIfUnderCorePoolSize(new Runnable(){ public
void run() {} });

                return;
        }
}
------------------------------------------------------------------------
------------------



Everything is working correctly as per the docs AFAIK, just seemingly
counterintuitively. It seems quite pointless and lower performant to be
creating new threads while existing ones are idle and available to
process work. Is this just a bad interaction between ThreadPoolExecutor
and LinkedBlockingQueue?  Is there another queue type that will work for
me or thread pool option I am missing?



Thanks in advance and please let me know if I am off base or incorrect
here.


Best regards,

Patrick




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: ThreadPoolExecutor

Joe Bowbeer
Patrick Eger <[hidden email]> wrote:
>
> This code is basically the first thing that is run with the  below settings.

Scratch those settings...

Though I think workQueue.offer is executed when corePoolSize == 0.

  if (poolSize < corePoolSize && addIfUnderCorePoolSize(command))
      return;
  if (workQueue.offer(command))
      return;
  int status = addIfUnderMaximumPoolSize(command);
  if (status > 0)      // created new thread
      return;
  if (status == 0) {   // failed to create thread
      reject(command);
      return;
  }

SO you want a decent core pool size, and maxPoolSize is mostly
irrelevant because the queue is effectively unbounded.

If you want the thread pool to shrink to 0 when idle, then you can use

allowCoreThreadTimeOut(true)

which was added in 1.6.


On 10/5/05, Patrick Eger <[hidden email]> wrote:

>
> Thank you for the quick response.
> Looking at the code  for ThreadPoolExecutor (from JDK 1.6):
>
> public void execute(Runnable command), line  ~876
> -------------------------------------------------
> Runnable r =  addIfUnderMaximumPoolSize(command);
> -------------------------------------------------
>
> This code is basically the first thing that is run with the below settings.
> So basically it will operate the same way it seems, IE adding a
> new thread unconditionally and not reusing existing idle threads.
> I will  write up a quick test to confirm this behaviour but i'm pretty
> sure we explored  this option as well.
>
> Thanks again for your help.
>
> Best regards,
>
> Patrick
>
>  ________________________________
 From: Joe Bowbeer [mailto:[hidden email]]

> Sent: Wednesday, October 05, 2005 1:38 PM
> To: Patrick  Eger
> Cc: [hidden email]
> Subject:  Re: [concurrency-interest] ThreadPoolExecutor
>
> I've probably missed something but it occurs to me that you want to
> be using maxPoolSize instead of corePoolSize.
>
> In other words, have you  tried the following?
>
> corePoolSize = 0
> maxPoolSize =  MAX
> keepAliveTime = tuned so that pool shrinks to 0 when executor is  idle
>
> On 10/5/05, Patrick  Eger <[hidden email]>  wrote:
> Basically, we need a ThreadPoolExecutor with the following properties:
> >
> > 1) {infinite | bounded} linked list request    queue, where the bound is
> > really high to avoid OOM (100000+)
> > 2) MIN of 0    threads in pool
> > 3) MAX # of threads in pool, configurable on pool create
> > 4) reuse existing threads if available
> > 5) allow pool to shrink to zero    threads if there are no outstanding
> > requests
> >
> >

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

David Holmes
In reply to this post by superuser
Patrick,

Your requirements can't be met directly by the current implementation. The
current design works, as you know, by having a core-pool that can either be
pre-started or lazily created, and which always stays around (in 1.6 idle
core threads can timeout but that just takes you back to the lazy startup
mode). Once you have your core pool you only create more threads (up to max)
if your queue fills up - if the queue never fills then no more threads will
be created. If you reach the maximum size with a full queue the task is
rejected.

> 1) {infinite | bounded} linked list request queue, where the bound is
> really high to avoid OOM (100000+)
> 2) MIN of 0 threads in pool
> 3) MAX # of threads in pool, configurable on pool create
> 4) reuse existing threads if available
> 5) allow pool to shrink to zero threads if there are no outstanding
> requests

Requirement (4) requires a way to detect if threads are "available". But
what does this mean? A worker thread is either executing a task or blocked
waiting for a task to appear in the BlockingQueue. If it is blocked then it
is "available", but to use it you have to submit your task to the queue. To
know if it is blocked you need to keep track of it, which is presumably what
this code is doing:

> //NOTE: Added this for less thread-crazy behaviour
> if (workersBlocked.get() > 0) {
> if(workQueue.offer(command)) {
> //HACK: this should protect against the pool shrinking,
> should be very rare...
> if (poolSize == 0)
> addIfUnderCorePoolSize(new Runnable(){ public
> void run() {} });
>
> return;
> }
> }

However this sort of check requires atomicity that isn't normally present in
the ThreadPoolExecutor. So to do this right requires additional locking
otherwise two incoming tasks can see one available worker and assume the
worker will run their task, when it fact one task will remain in the queue
and the pool could have queued tasks but less than max (or even core)
threads.

So if you really want this you have to pay a price to get it.

> Everything is working correctly as per the docs AFAIK, just seemingly
> counterintuitively. It seems quite pointless and lower performant to be
> creating new threads while existing ones are idle and available to
> process work.

The assumption is that the core pool will be quite steady so if you don't
create a core thread this time, the expected usage means you are going to
create it very soon anyway. If you pre-start the core then you don't create
new threads until your queue is full.

> Is this just a bad interaction between ThreadPoolExecutor
> and LinkedBlockingQueue?  Is there another queue type that will work for
> me or thread pool option I am missing?

It seems to me - and I could be misunderstanding things - that what you want
is a "dual-mode" queue. Set the core size at zero and what you want is to
submit the task to the queue, if a thread is waiting, else create a thread.
This is where you need a synchronous queue - it has no capacity, so if no
thread is waiting then offer() will fail and from the pool's perspective the
queue is "full" and so a new thread will be created if under max. But when
max is reached you now want a queue that has capacity. You could use the
RejectedExecutionHandler to then make your queue (you'd need to define a
custom queue for this) switch to act as a (finite) linked blocking queue. If
the normal queue detects it is empty then it switches back to synchronous
mode. I *think* that would meet your requirements *but* I don't know if what
I just described can actually be implemented. Interesting to think about it
anyway :-)

Cheers,
David Holmes

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

superuser
In reply to this post by superuser
Hello and thank you for your response.  I forgot {to mention} the extra
"workersBlocked" variable I added to keep track of the # of threads
waiting for a task to execute.  This obviously adds the extra expense of
the atomic inc/dec pair for every execute() (told you it was hacky :-).
There are definite race issues here as well, but it think it is safe in
that the only problem would be if the pool shrinks to zero size after we
offer it to the queue.  "if (poolSize == 0)" clause below protects
against this case I think...

....
private final AtomicInteger workersBlocked = new AtomicInteger(0);
....
workersBlocked.incrementAndGet();
try {
        // untimed wait if core and not allowing core timeout
        if (poolSize <= corePoolSize && !allowCoreThreadTimeOut)
      return workQueue.take();
   
      long timeout = keepAliveTime;
        if (timeout <= 0) // die immediately for 0 timeout
      return null;
      Runnable r = workQueue.poll(timeout, TimeUnit.NANOSECONDS);
        if (r != null)
      return r;
        if (poolSize > corePoolSize || allowCoreThreadTimeOut)
      return null; // timed out
      // Else, after timeout, the pool shrank. Retry
      break;
} finally {
        workersBlocked.decrementAndGet();
}
....


It uses the "workersBlocked" as an estimate of whether or not a thread
is immediately available to execute the Runnable, though this will be
incorrect under heavily concurrent calls to execute().  I would lkove to
find a better way to do this, maybe an atomic decrement semaphore-style
"reservation" would allow correctness in the face of multiple concurrent
executes()? This would still be an additional atomic dec/inc pair (one
in execute(), one in the executor thread when it picks the task up).

Your queue approach is also interesting but sound much more complex and
error prone, plus it would really complexify the (currently simple)
interface between the thread pool and the queue.

I would gladly pay this inc/dec cost (as a configurable option, off by
default for backwards compatibility sake and so should not effect
existing users) for the better (IMO of course) thread creation
behaviour.

Do you guys see this as generally useful?  Would a patch using the
(configurable, off by default and so would only add a cachable read to
the fastpath) inc/dec reservation-style approach be considered for
review (the above is a bit of a hack and probably has some bad
characteristics under heavy concurrent use as David has pointed out)? Or
should I pursue the queue-based approach as suggested?

Thanks again for everyone's help!

P.S. I have confirmed via a small test case that a corePoolSize of zero
will result in submitted tasks *never* executing.  They are infinitely
offer()ed up to the queue but there are no threads available to process.
In effect the corePoolSize becomes the "max" pool size with an infinite
queue, which does not give me the behaviour I desire.

P.P.S. Am I still confused and not knowing what I want?  I assumed this
behaviour is what most people would want for a dynamic work queue
(0-1000 threads) with bursty request patterns (0-1 million+ on the queue
ant any given point), but I cannot find much in the archives...

Best regards,

Patrick


-----Original Message-----
From: David Holmes [mailto:[hidden email]]
Sent: Wednesday, October 05, 2005 5:18 PM
To: Patrick Eger; [hidden email]
Subject: RE: [concurrency-interest] ThreadPoolExecutor

Patrick,

Your requirements can't be met directly by the current implementation.
The current design works, as you know, by having a core-pool that can
either be pre-started or lazily created, and which always stays around
(in 1.6 idle core threads can timeout but that just takes you back to
the lazy startup mode). Once you have your core pool you only create
more threads (up to max) if your queue fills up - if the queue never
fills then no more threads will be created. If you reach the maximum
size with a full queue the task is rejected.

> 1) {infinite | bounded} linked list request queue, where the bound is
> really high to avoid OOM (100000+)
> 2) MIN of 0 threads in pool
> 3) MAX # of threads in pool, configurable on pool create
> 4) reuse existing threads if available
> 5) allow pool to shrink to zero threads if there are no outstanding
> requests

Requirement (4) requires a way to detect if threads are "available". But
what does this mean? A worker thread is either executing a task or
blocked waiting for a task to appear in the BlockingQueue. If it is
blocked then it is "available", but to use it you have to submit your
task to the queue. To know if it is blocked you need to keep track of
it, which is presumably what this code is doing:

> //NOTE: Added this for less thread-crazy behaviour if
> (workersBlocked.get() > 0) {
> if(workQueue.offer(command)) {
> //HACK: this should protect against the pool shrinking,
should be
> very rare...
> if (poolSize == 0)
> addIfUnderCorePoolSize(new Runnable(){ public
void run() {} });
>
> return;
> }
> }

However this sort of check requires atomicity that isn't normally
present in the ThreadPoolExecutor. So to do this right requires
additional locking otherwise two incoming tasks can see one available
worker and assume the worker will run their task, when it fact one task
will remain in the queue and the pool could have queued tasks but less
than max (or even core) threads.

So if you really want this you have to pay a price to get it.

> Everything is working correctly as per the docs AFAIK, just seemingly
> counterintuitively. It seems quite pointless and lower performant to
> be creating new threads while existing ones are idle and available to
> process work.

The assumption is that the core pool will be quite steady so if you
don't create a core thread this time, the expected usage means you are
going to create it very soon anyway. If you pre-start the core then you
don't create new threads until your queue is full.

> Is this just a bad interaction between ThreadPoolExecutor and
> LinkedBlockingQueue?  Is there another queue type that will work for
> me or thread pool option I am missing?

It seems to me - and I could be misunderstanding things - that what you
want is a "dual-mode" queue. Set the core size at zero and what you want
is to submit the task to the queue, if a thread is waiting, else create
a thread.
This is where you need a synchronous queue - it has no capacity, so if
no thread is waiting then offer() will fail and from the pool's
perspective the queue is "full" and so a new thread will be created if
under max. But when max is reached you now want a queue that has
capacity. You could use the RejectedExecutionHandler to then make your
queue (you'd need to define a custom queue for this) switch to act as a
(finite) linked blocking queue. If the normal queue detects it is empty
then it switches back to synchronous mode. I *think* that would meet
your requirements *but* I don't know if what I just described can
actually be implemented. Interesting to think about it anyway :-)

Cheers,
David Holmes



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

David Holmes
Patrick,

> There are definite race issues here as well, but it think it is safe in
> that the only problem would be if the pool shrinks to zero size after we
> offer it to the queue.  "if (poolSize == 0)" clause below protects
> against this case I think...

The approximation involved in using the workersBlocked count would
invalidate one of the basic guarantees regarding the creation of core
threads - ie that one will always be created if needed. Without this, for
example, barrier designs that use a pool wouldn't work because tasks may be
left in the queue with all existing workers blocked on the barrier waiting
for the other tasks to arrive - which they won't because they are in the
queue and less than coreSize threads have been created. You would have to
guarantee atomicity using locks.

People have enough trouble understanding the relationship between coreSize,
queue length and maxSize, as it is! Adding another variable to the mix would
be a recipe for potential disaster. That's not to say that there couldn't be
a different TPE that worked as you would like, but it seems unlikely to be a
j.u.c class. I wish there were a clean way to abstract these policy choices
out so that subclasses could modify them, but I can't see a way of doing
this. So cut-paste-modify seems the simplest solution.

> Your queue approach is also interesting but sound much more complex and
> error prone, plus it would really complexify the (currently simple)
> interface between the thread pool and the queue.

Hmmm. There wouldn't be any change in the interface between them - as far as
I can see. You give the pool your special queue and install the queue's
rejected execution handler, then everything "just works". I'm not sure how
the task that triggers the switch from synchronous to non-synchronous mode
gets resubmitted: maybe a simple MyQueue.this.put(), or if necessary hook
the queue to the TPE and let the handler reinvoke execute. I don't *think*
it is that complicated but I won't know unless I try to implement it.

> P.S. I have confirmed via a small test case that a corePoolSize of zero
> will result in submitted tasks *never* executing.

As you would expect - threads beyond the core size only get created when the
queue is full. If the queue can't fill then no threads ever get created. Do
you think this combination should be detected and prohibited?

> P.P.S. Am I still confused and not knowing what I want?  I assumed this
> behaviour is what most people would want for a dynamic work queue
> (0-1000 threads) with bursty request patterns (0-1 million+ on the queue
> ant any given point), but I cannot find much in the archives...

The current design assumes that the core is set such that the expected load
is accommodated, with room for increasing the worker set under heavy load,
by limiting the buffering done by the queue. Having large numbers of threads
in the pool is generally detrimental to overall performce, but may be
necessary if the tasks are expected to block for reasonable periods.
Otherwise with a small core size you would not generally be concerned about
the startup overhead of these threads - whether done eagerly or on demand.

Do you really expect 1000 active threads? How many CPU's are you running on?
Even on a 384-way Azul box I wouldn't expect to need a pool that large. :)

Cheers,
David Holmes

PS. Doug Lea is obviously incommunicado right now or else I'm certain he
would have chipped in - and I expect he will when he can.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

superuser
In reply to this post by superuser
Sorry forgot to include the list


... Snipped
>
> > P.S. I have confirmed via a small test case that a corePoolSize of
> > zero will result in submitted tasks *never* executing.
>
> As you would expect - threads beyond the core size only get created
> when the queue is full. If the queue can't fill then no threads ever
> get created. Do you think this combination should be detected and
> prohibited?
>

Just a little surprising is all, though I'm not sure how the
ThreadPoolExecutor could know that a queue is non-synchronous?  Actually
isn't corePoolSize==0 an invalid usage pattern if the queue has *any*
capacity at all?  Otherwise the risk of tasks sitting on the queue never
being executed (or later being executed after a possibly lengthy wait)
is very high as I see it.  Once the queue overflows though (ie offer()
returns false), a core thread will be created and will flush out the
queue.  Not sure if this has bitten anyone but perhaps it has and no one
has noticed...


> > P.P.S. Am I still confused and not knowing what I want?  I assumed
> > this behaviour is what most people would want for a dynamic
> work queue
> > (0-1000 threads) with bursty request patterns (0-1 million+ on the
> > queue ant any given point), but I cannot find much in the
> archives...
>
> The current design assumes that the core is set such that the expected

> load is accommodated, with room for increasing the worker set under
> heavy load, by limiting the buffering done by the queue. Having large
> numbers of threads in the pool is generally detrimental to overall
> performce, but may be necessary if the tasks are expected to block for

> reasonable periods.
> Otherwise with a small core size you would not generally be concerned
> about the startup overhead of these threads - whether done eagerly or
> on demand.
>
> Do you really expect 1000 active threads? How many CPU's are you
> running on?
> Even on a 384-way Azul box I wouldn't expect to need a pool that
> large. :)

Would love to play with such a beast but unfortunately no, we're working
on bog-standard ia32 & amd64 ;-)

What we have is lots of threads active at a time doing things like
waiting for a database or network response so our pools need to be
pretty big to accommodate all these IO blocking calls.  I'd love to move
to an event driven non-blocking IO model but unfortunately key
technologies such as JDBC don't provide any such options.  Hence my only
real method of not artificially limiting throughput is lots of threads.
Problem with the current threadpool is that the first thousand (for
example, with corePoolSize=1000) requests will each get their own new
thread, regardless of whether these requests ever had the need to
execute concurrently or not.  IE I'm wasting a bunch of threads that
potentially other request queues in the system will need (we have
multiple pools per instance).  We'd like the multiple pools within the
system to be reasonably bounded by the # of concurrent requests (with a
little leeway for thread keepalives of course), while still maintaining
maximum concurrency in the face of a large spike of requests or a
slowdown in our dependent IO operations.

I hope this all made sense & thanks for all the help.  If you don't see
generic usefulness in the core libs its no problem, I'll just maintain a
modified version for myself (and anyone else if interested, just let me
know).

>
> Cheers,
> David Holmes
>
> PS. Doug Lea is obviously incommunicado right now or else I'm certain
> he would have chipped in - and I expect he will when he can.
>
>

Thanks again!


Best regards,

Patrick


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

David Holmes
> Just a little surprising is all, though I'm not sure how the
> ThreadPoolExecutor could know that a queue is non-synchronous?

A check of BlockingQueue.remainingCapacity would tell whether the queue was
bounded or not - but it would be subjective as to how big a value
"unbounded" really is.

> Actually isn't corePoolSize==0 an invalid usage pattern if the queue has
> *any* capacity at all?

With an infinite queue, or a queue you don't expect to fill then it is a
problem. However, you could use a zero core size and a small queue to
effectively "batch" tasks.

ThreadPoolExecutor is quite flexible but a side effect of that is that some
combinations of options make little or no sense.

> What we have is lots of threads active at a time doing things like
> waiting for a database or network response so our pools need to be
> pretty big to accommodate all these IO blocking calls.  I'd love to move
> to an event driven non-blocking IO model but unfortunately key
> technologies such as JDBC don't provide any such options.

I see. At this stage "fixing" the pool is probably a better option than
redesigning the core application architecture.

> I'll just maintain a modified version for myself (and anyone else if
> interested, just let me know).

I'm intrigued enough to try and implement that dual queue set up. :)

Cheers,
David Holmes

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: ThreadPoolExecutor

Doug Lea
In reply to this post by superuser
Patrick Eger wrote:
>
>>PS. Doug Lea is obviously incommunicado right now .
>>

No, but Joe and David have handled this so well that I have nothing to
add :-)

... except to say that ThreadPoolExecutor handles as many variants
as we can support without hurting performance or adding too much for any
given mode of use. It sounds like your application might escape
the boundaries. Except that I'd be surprised if a simple
cahchedThreadPool was measurably worse that what you are doing.
Bounding to thousands of threads on a 2 or 4way machine
is not much different than not bounding at all.

-Doug

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

superuser
In reply to this post by superuser
> Patrick Eger wrote:
> >
> >>PS. Doug Lea is obviously incommunicado right now .
> >>
>
> No, but Joe and David have handled this so well that I have
> nothing to add :-)
>
> ... except to say that ThreadPoolExecutor handles as many
> variants as we can support without hurting performance or
> adding too much for any given mode of use. It sounds like
> your application might escape the boundaries. Except that I'd
> be surprised if a simple cahchedThreadPool was measurably
> worse that what you are doing.

The problem with cachedThreadPool() for our use was that the synchronous
handoff / lack of queuing makes us unable to smoothe out transient
request bursts, and lack of MAX size on the pool means potential
resource exhaustion. I couldn't see a way around the SynchronousQueue
used there...

> Bounding to thousands of threads on a 2 or 4way machine is
> not much different than not bounding at all.
>
> -Doug
>
>

Unbounded would likely work as well if threads were cheaper than they
are.  Alas, we are only able to allocate a few thousand threads per JVM
maximum, and we need it to be shared dynamically among 10-20 separate
pools.


PS As an aside, are there any known issues with the old "oswego" package
thread pools (since the new JDK5 memory model)?  I seem to remember
these giving us the desired charateristics, though i'd hate to switch
back and will more likely attempt some modification of the Mustang
sources as previously mentioned.


Again, thanks a bunch for everybody's help on this.


Best regards,

Patrick

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

RE: ThreadPoolExecutor

David Holmes
In reply to this post by David Holmes
I wrote:
> I'm intrigued enough to try and implement that dual queue set up. :)

Well the basic idea seems sound but the details are a bit tricky. To do the
queue switching you need atomicity otherwise the possible states are too
numerous for me to work out how to handle them. :) But atomicity at the
dual-queue level conflicts with the normal synchronization of the
SynchronousQueue and LinkedBlockingQueue (imagine trying to add a
synchronized wrapper to them: you hold the outer lock while blocked waiting
for space/elements). So the queue would have to be fully custom implemented,
rather than wrapping existing queue implementations. :(

Cheers,
David Holmes


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: ThreadPoolExecutor

Doug Lea
In reply to this post by superuser
Patrick Eger wrote:
>
>
> The problem with cachedThreadPool() for our use was that the synchronous
> handoff / lack of queuing makes us unable to smoothe out transient
> request bursts,

One way to deal with this is to create a custom ThreadFactory that
creates threads in batches.

> and lack of MAX size on the pool means potential
> resource exhaustion.

You can still use a max expecially if you can live with a
CallerRuns or Discard policy on saturation. Which
I'd guess that you probably need to do anyway in this
case.

>
> PS As an aside, are there any known issues with the old "oswego" package
> thread pools (since the new JDK5 memory model)?  

None that I'm aware of. That package was pretty conservative
about assumptions about JVMs.

-Doug


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest