Fixed Rate Executor Service

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Fixed Rate Executor Service

Valentin Kovalenko
Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

David Holmes-6

Hi Valentin,
 
Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.
 
From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?
 
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Valentin Kovalenko
Sent: Wednesday, 4 February 2015 9:45 AM
To: [hidden email]
Subject: [concurrency-interest] Fixed Rate Executor Service

Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Valentin Kovalenko
In reply to this post by Valentin Kovalenko
>>the task is scheduled for release at times S, S+T, S+2T etc.
This approach is only suitable for single-thread executor, which fires actions in a sequence. If executor have more that one thread it can fire actions in parallel, but must try to execute exactly RATE actions per time unit (period).

>>From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right?
Yes, but we can only do it if we have RATE (or more) threads in executor. Otherwise not all (not RATE) actions will be fired at the start of each period (obviously).

>>Or did you intend to distribute the execution of the tasks evenly throughout the time period?
That's another story of which I'm thinking, but for start I prefer to not focus on the "inside period distribution strategy".

>>Given we don't have guaranteed cancellation mechanisms
Yep, we don't.
So if we have requested rate 1 action per second, duration of the action varies [1s; 2.5s], and 2 threads in executor then:
0) at period [0s; 1s) execution rate 0 (failed rate)
1) at period [1s; 2s) execution rate 0 (failed rate)
2) at period [2s; 3s) execution rate 2, because action started at 0) have completed in say 2.2s and action started at 1) have completed in 1.5s
So at the period [2s; 3s) we have rate 2 instead of 1, is this what you've called overrun? If so then it seems like such cases should also be handled by FailedRatePolicy, so a user can be notified that he should specify rate with a more coarse grained time unit (e.g. 60 actions/minute instead of 1 action/second).

On Wed, Feb 4, 2015 at 3:08 AM, David Holmes <[hidden email]> wrote:
Hi Valentin,
 
Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.
 
From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?
 
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Valentin Kovalenko
Sent: Wednesday, 4 February 2015 9:45 AM
To: [hidden email]
Subject: [concurrency-interest] Fixed Rate Executor Service

Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

David Holmes-6

If your tasks can run in parallel (which implies they are independent and non-interfering) then you don't need RATE threads in the executor, you just need sufficient threads to execute RATE tasks within the time unit. If a task takes UNIT time to execute then you need RATE threads. If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead).
 
But simple answer: I know of no existing Executor that will do what you want.
 
David
-----Original Message-----
From: Valentin Kovalenko [mailto:[hidden email]]
Sent: Wednesday, 4 February 2015 10:51 AM
To: [hidden email]; [hidden email]
Subject: Re: [concurrency-interest] Fixed Rate Executor Service

>>the task is scheduled for release at times S, S+T, S+2T etc.
This approach is only suitable for single-thread executor, which fires actions in a sequence. If executor have more that one thread it can fire actions in parallel, but must try to execute exactly RATE actions per time unit (period).

>>From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right?
Yes, but we can only do it if we have RATE (or more) threads in executor. Otherwise not all (not RATE) actions will be fired at the start of each period (obviously).

>>Or did you intend to distribute the execution of the tasks evenly throughout the time period?
That's another story of which I'm thinking, but for start I prefer to not focus on the "inside period distribution strategy".

>>Given we don't have guaranteed cancellation mechanisms
Yep, we don't.
So if we have requested rate 1 action per second, duration of the action varies [1s; 2.5s], and 2 threads in executor then:
0) at period [0s; 1s) execution rate 0 (failed rate)
1) at period [1s; 2s) execution rate 0 (failed rate)
2) at period [2s; 3s) execution rate 2, because action started at 0) have completed in say 2.2s and action started at 1) have completed in 1.5s
So at the period [2s; 3s) we have rate 2 instead of 1, is this what you've called overrun? If so then it seems like such cases should also be handled by FailedRatePolicy, so a user can be notified that he should specify rate with a more coarse grained time unit (e.g. 60 actions/minute instead of 1 action/second).

On Wed, Feb 4, 2015 at 3:08 AM, David Holmes <[hidden email]> wrote:
Hi Valentin,
 
Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.
 
From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?
 
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Valentin Kovalenko
Sent: Wednesday, 4 February 2015 9:45 AM
To: [hidden email]
Subject: [concurrency-interest] Fixed Rate Executor Service

Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Martin Buchholz-3
In reply to this post by Valentin Kovalenko

On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko <[hidden email]> wrote:
Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Valentin Kovalenko
In reply to this post by David Holmes-6
>>then you don't need RATE threads in the executor
but if rate is for example 100_000 then it's not the right approach

>>If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead)
I believe no one can tell for any action performed in Java that it takes exactly T to execute. The best what could be told is some approximation interval [t1; t2]

On Wed, Feb 4, 2015 at 4:05 AM, David Holmes <[hidden email]> wrote:
If your tasks can run in parallel (which implies they are independent and non-interfering) then you don't need RATE threads in the executor, you just need sufficient threads to execute RATE tasks within the time unit. If a task takes UNIT time to execute then you need RATE threads. If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead).
 
But simple answer: I know of no existing Executor that will do what you want.
 
David
-----Original Message-----
From: Valentin Kovalenko [mailto:[hidden email]]
Sent: Wednesday, 4 February 2015 10:51 AM
To: [hidden email]; [hidden email]
Subject: Re: [concurrency-interest] Fixed Rate Executor Service

>>the task is scheduled for release at times S, S+T, S+2T etc.
This approach is only suitable for single-thread executor, which fires actions in a sequence. If executor have more that one thread it can fire actions in parallel, but must try to execute exactly RATE actions per time unit (period).

>>From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right?
Yes, but we can only do it if we have RATE (or more) threads in executor. Otherwise not all (not RATE) actions will be fired at the start of each period (obviously).

>>Or did you intend to distribute the execution of the tasks evenly throughout the time period?
That's another story of which I'm thinking, but for start I prefer to not focus on the "inside period distribution strategy".

>>Given we don't have guaranteed cancellation mechanisms
Yep, we don't.
So if we have requested rate 1 action per second, duration of the action varies [1s; 2.5s], and 2 threads in executor then:
0) at period [0s; 1s) execution rate 0 (failed rate)
1) at period [1s; 2s) execution rate 0 (failed rate)
2) at period [2s; 3s) execution rate 2, because action started at 0) have completed in say 2.2s and action started at 1) have completed in 1.5s
So at the period [2s; 3s) we have rate 2 instead of 1, is this what you've called overrun? If so then it seems like such cases should also be handled by FailedRatePolicy, so a user can be notified that he should specify rate with a more coarse grained time unit (e.g. 60 actions/minute instead of 1 action/second).

On Wed, Feb 4, 2015 at 3:08 AM, David Holmes <[hidden email]> wrote:
Hi Valentin,
 
Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.
 
From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?
 
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Valentin Kovalenko
Sent: Wednesday, 4 February 2015 9:45 AM
To: [hidden email]
Subject: [concurrency-interest] Fixed Rate Executor Service

Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Valentin Kovalenko
In reply to this post by Martin Buchholz-3
Thanks! I've already found RateLimiter but unfortunately it doesn't check if the rate is actually satisfied.

On Wed, Feb 4, 2015 at 6:02 AM, Martin Buchholz <[hidden email]> wrote:

On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko <[hidden email]> wrote:
Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

David Holmes-6
In reply to this post by Valentin Kovalenko

The point is that you are governed by the potential utilization of your tasks. Even if your RATE is 100_000 you are not going to be able to (nor want to!) create 100_000 threads.
 
David
-----Original Message-----
From: Valentin Kovalenko [mailto:[hidden email]]
Sent: Wednesday, 4 February 2015 7:44 PM
To: [hidden email]; [hidden email]
Subject: Re: [concurrency-interest] Fixed Rate Executor Service

>>then you don't need RATE threads in the executor
but if rate is for example 100_000 then it's not the right approach

>>If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead)
I believe no one can tell for any action performed in Java that it takes exactly T to execute. The best what could be told is some approximation interval [t1; t2]

On Wed, Feb 4, 2015 at 4:05 AM, David Holmes <[hidden email]> wrote:
If your tasks can run in parallel (which implies they are independent and non-interfering) then you don't need RATE threads in the executor, you just need sufficient threads to execute RATE tasks within the time unit. If a task takes UNIT time to execute then you need RATE threads. If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead).
 
But simple answer: I know of no existing Executor that will do what you want.
 
David
-----Original Message-----
From: Valentin Kovalenko [mailto:[hidden email]]
Sent: Wednesday, 4 February 2015 10:51 AM
To: [hidden email]; [hidden email]
Subject: Re: [concurrency-interest] Fixed Rate Executor Service

>>the task is scheduled for release at times S, S+T, S+2T etc.
This approach is only suitable for single-thread executor, which fires actions in a sequence. If executor have more that one thread it can fire actions in parallel, but must try to execute exactly RATE actions per time unit (period).

>>From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right?
Yes, but we can only do it if we have RATE (or more) threads in executor. Otherwise not all (not RATE) actions will be fired at the start of each period (obviously).

>>Or did you intend to distribute the execution of the tasks evenly throughout the time period?
That's another story of which I'm thinking, but for start I prefer to not focus on the "inside period distribution strategy".

>>Given we don't have guaranteed cancellation mechanisms
Yep, we don't.
So if we have requested rate 1 action per second, duration of the action varies [1s; 2.5s], and 2 threads in executor then:
0) at period [0s; 1s) execution rate 0 (failed rate)
1) at period [1s; 2s) execution rate 0 (failed rate)
2) at period [2s; 3s) execution rate 2, because action started at 0) have completed in say 2.2s and action started at 1) have completed in 1.5s
So at the period [2s; 3s) we have rate 2 instead of 1, is this what you've called overrun? If so then it seems like such cases should also be handled by FailedRatePolicy, so a user can be notified that he should specify rate with a more coarse grained time unit (e.g. 60 actions/minute instead of 1 action/second).

On Wed, Feb 4, 2015 at 3:08 AM, David Holmes <[hidden email]> wrote:
Hi Valentin,
 
Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.
 
From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?
 
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Valentin Kovalenko
Sent: Wednesday, 4 February 2015 9:45 AM
To: [hidden email]
Subject: [concurrency-interest] Fixed Rate Executor Service

Hi
I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.

I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".

What I want is something like
scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),
where rate==10_000L and unit==SECONDS means 10_000 actions/second,
and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.

Does anyone know an executor which have such a functionality?



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Aleksey Shipilev-2
In reply to this post by Valentin Kovalenko
My 2c. One time I just put the TokenBucket-like limiter in front of the
regular executor, and was done with it.

Something along the lines of (GPLv2 code inside. The idea is to track
the <current-second, available-tokens> "quantum" tuple and update it in
a lock-free way):
 http://cr.openjdk.java.net/~shade/scratch/jcp/src/main/java/org/openjdk/TokenBucket.java

Such a token bucket should never over-reach the rate, and I think it can
be amended with FailedRatePolicy by detecting the current quantum is not
depleted before installing a new one.

I'm speculating Guava's RateLimiter does something like that, but from
the look over the javadoc, it seems to have a smooth refill strategy,
which may alleviate the microbursts on the quanta edges.

-Aleksey.

On 02/04/2015 12:46 PM, Valentin Kovalenko wrote:

> Thanks! I've already found RateLimiter but unfortunately it doesn't
> check if the rate is actually satisfied.
>
> On Wed, Feb 4, 2015 at 6:02 AM, Martin Buchholz <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     See also Guava RateLimiter
>     http://docs.guava-libraries.googlecode.com/git-history/master/javadoc/com/google/common/util/concurrent/RateLimiter.html
>
>     On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko
>     <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         Hi
>         I'm looking for a fixed rate executor service, and would be glad
>         if someone could give me a hint.
>
>         I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate,
>         but this method doesn't guarantees that actual rate will be not
>         less that the desired one: "If any execution of this task takes
>         longer than its period, then subsequent executions may start late".
>
>         What I want is something like
>         scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit,
>         FailedRatePolicy frp),
>         where rate==10_000L and unit==SECONDS means 10_000 actions/second,
>         and FailedRatePolicy determines what to do if during some second
>         executor failed to satisfy the specified rate (i.e. if at that
>         second it had completed 6_000 actions). Examples of
>         FailedRatePolicy: silently ignore; log a warning; throw an
>         exception. Note also that at the next second after the second
>         where rate wasn't satisfied executor doesn't try to execute
>         10_000 actions plus those actions that weren't executed, it
>         tries to execute exactly 10_000 actions during the new second.
>
>         Does anyone know an executor which have such a functionality?
>
>         _______________________________________________
>         Concurrency-interest mailing list
>         [hidden email]
>         <mailto:[hidden email]>
>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

oleksandr otenko
Why would you need current-second? Is the task that didn't return a token yet not consuming the resource any more? (Or if the concern is not the resource consumption, then what's the point of rate limiting?)

Alex

On 04/02/2015 11:11, Aleksey Shipilev wrote:
My 2c. One time I just put the TokenBucket-like limiter in front of the
regular executor, and was done with it.

Something along the lines of (GPLv2 code inside. The idea is to track
the <current-second, available-tokens> "quantum" tuple and update it in
a lock-free way):
 http://cr.openjdk.java.net/~shade/scratch/jcp/src/main/java/org/openjdk/TokenBucket.java

Such a token bucket should never over-reach the rate, and I think it can
be amended with FailedRatePolicy by detecting the current quantum is not
depleted before installing a new one.

I'm speculating Guava's RateLimiter does something like that, but from
the look over the javadoc, it seems to have a smooth refill strategy,
which may alleviate the microbursts on the quanta edges.

-Aleksey.

On 02/04/2015 12:46 PM, Valentin Kovalenko wrote:
Thanks! I've already found RateLimiter but unfortunately it doesn't
check if the rate is actually satisfied.

On Wed, Feb 4, 2015 at 6:02 AM, Martin Buchholz <[hidden email]
[hidden email]> wrote:

    See also Guava RateLimiter
    http://docs.guava-libraries.googlecode.com/git-history/master/javadoc/com/google/common/util/concurrent/RateLimiter.html

    On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko
    <[hidden email]
    [hidden email]> wrote:

        Hi
        I'm looking for a fixed rate executor service, and would be glad
        if someone could give me a hint.

        I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate,
        but this method doesn't guarantees that actual rate will be not
        less that the desired one: "If any execution of this task takes
        longer than its period, then subsequent executions may start late".

        What I want is something like
        scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit,
        FailedRatePolicy frp),
        where rate==10_000L and unit==SECONDS means 10_000 actions/second,
        and FailedRatePolicy determines what to do if during some second
        executor failed to satisfy the specified rate (i.e. if at that
        second it had completed 6_000 actions). Examples of
        FailedRatePolicy: silently ignore; log a warning; throw an
        exception. Note also that at the next second after the second
        where rate wasn't satisfied executor doesn't try to execute
        10_000 actions plus those actions that weren't executed, it
        tries to execute exactly 10_000 actions during the new second.

        Does anyone know an executor which have such a functionality?

        _______________________________________________
        Concurrency-interest mailing list
        [hidden email]
        [hidden email]
        http://cs.oswego.edu/mailman/listinfo/concurrency-interest





_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Aleksey Shipilev-2
Um... Because that's a one-sided rate limiter, not a two-sided
"semaphore" guarding a section. As far as I can see, OP does not talk
about limiting the number of concurrent tasks (that translates to
limiting the resources used at a given time).

-Aleksey.

On 02/04/2015 02:45 PM, Oleksandr Otenko wrote:

> Why would you need current-second? Is the task that didn't return a
> token yet not consuming the resource any more? (Or if the concern is not
> the resource consumption, then what's the point of rate limiting?)
>
> Alex
>
> On 04/02/2015 11:11, Aleksey Shipilev wrote:
>> My 2c. One time I just put the TokenBucket-like limiter in front of the
>> regular executor, and was done with it.
>>
>> Something along the lines of (GPLv2 code inside. The idea is to track
>> the <current-second, available-tokens> "quantum" tuple and update it in
>> a lock-free way):
>>  http://cr.openjdk.java.net/~shade/scratch/jcp/src/main/java/org/openjdk/TokenBucket.java
>>
>> Such a token bucket should never over-reach the rate, and I think it can
>> be amended with FailedRatePolicy by detecting the current quantum is not
>> depleted before installing a new one.
>>
>> I'm speculating Guava's RateLimiter does something like that, but from
>> the look over the javadoc, it seems to have a smooth refill strategy,
>> which may alleviate the microbursts on the quanta edges.
>>
>> -Aleksey.
>>
>> On 02/04/2015 12:46 PM, Valentin Kovalenko wrote:
>>> Thanks! I've already found RateLimiter but unfortunately it doesn't
>>> check if the rate is actually satisfied.
>>>
>>> On Wed, Feb 4, 2015 at 6:02 AM, Martin Buchholz <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>     See also Guava RateLimiter
>>>     http://docs.guava-libraries.googlecode.com/git-history/master/javadoc/com/google/common/util/concurrent/RateLimiter.html
>>>
>>>     On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko
>>>     <[hidden email]
>>>     <mailto:[hidden email]>> wrote:
>>>
>>>         Hi
>>>         I'm looking for a fixed rate executor service, and would be glad
>>>         if someone could give me a hint.
>>>
>>>         I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate,
>>>         but this method doesn't guarantees that actual rate will be not
>>>         less that the desired one: "If any execution of this task takes
>>>         longer than its period, then subsequent executions may start late".
>>>
>>>         What I want is something like
>>>         scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit,
>>>         FailedRatePolicy frp),
>>>         where rate==10_000L and unit==SECONDS means 10_000 actions/second,
>>>         and FailedRatePolicy determines what to do if during some second
>>>         executor failed to satisfy the specified rate (i.e. if at that
>>>         second it had completed 6_000 actions). Examples of
>>>         FailedRatePolicy: silently ignore; log a warning; throw an
>>>         exception. Note also that at the next second after the second
>>>         where rate wasn't satisfied executor doesn't try to execute
>>>         10_000 actions plus those actions that weren't executed, it
>>>         tries to execute exactly 10_000 actions during the new second.
>>>
>>>         Does anyone know an executor which have such a functionality?
>>>
>>>         _______________________________________________
>>>         Concurrency-interest mailing list
>>>         [hidden email]
>>>         <mailto:[hidden email]>
>>>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Concurrency-interest mailing list
>>> [hidden email]
>>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>
>>
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> [hidden email]
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Peter Levart
In reply to this post by David Holmes-6
What about the following strategy:

Have a submission thread submit tasks at a fixed rate (your desired rate) to a ThreadPoolExecutor with some limit on max. number of threads (using RateLimiter if you like or some other means).
Let this ThreadPoolExecutor have a SynchonousQueue.
When the Executor starts rejecting executions, it means your rate can't be satisfied with given max. number of threads.

Regards, Peter

On 02/04/2015 10:53 AM, David Holmes wrote:
The point is that you are governed by the potential utilization of your tasks. Even if your RATE is 100_000 you are not going to be able to (nor want to!) create 100_000 threads.

David
  -----Original Message-----
  From: Valentin Kovalenko [[hidden email]]
  Sent: Wednesday, 4 February 2015 7:44 PM
  To: [hidden email]; [hidden email]
  Subject: Re: [concurrency-interest] Fixed Rate Executor Service


  >>then you don't need RATE threads in the executor
  but if rate is for example 100_000 then it's not the right approach


  >>If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead)

  I believe no one can tell for any action performed in Java that it takes exactly T to execute. The best what could be told is some approximation interval [t1; t2]


  On Wed, Feb 4, 2015 at 4:05 AM, David Holmes [hidden email] wrote:

    If your tasks can run in parallel (which implies they are independent and non-interfering) then you don't need RATE threads in the executor, you just need sufficient threads to execute RATE tasks within the time unit. If a task takes UNIT time to execute then you need RATE threads. If a task takes UNIT/10 time to execute then you only need RATE/10 threads (plus allow for task management overhead).

    But simple answer: I know of no existing Executor that will do what you want.

    David
      -----Original Message-----
      From: Valentin Kovalenko [[hidden email]]
      Sent: Wednesday, 4 February 2015 10:51 AM
      To: [hidden email]; [hidden email]
      Subject: Re: [concurrency-interest] Fixed Rate Executor Service


      >>the task is scheduled for release at times S, S+T, S+2T etc.

      This approach is only suitable for single-thread executor, which fires actions in a sequence. If executor have more that one thread it can fire actions in parallel, but must try to execute exactly RATE actions per time unit (period).


      >>From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right?

      Yes, but we can only do it if we have RATE (or more) threads in executor. Otherwise not all (not RATE) actions will be fired at the start of each period (obviously).


      >>Or did you intend to distribute the execution of the tasks evenly throughout the time period?

      That's another story of which I'm thinking, but for start I prefer to not focus on the "inside period distribution strategy".



      >>Given we don't have guaranteed cancellation mechanisms

      Yep, we don't.
      So if we have requested rate 1 action per second, duration of the action varies [1s; 2.5s], and 2 threads in executor then:
      0) at period [0s; 1s) execution rate 0 (failed rate)
      1) at period [1s; 2s) execution rate 0 (failed rate)
      2) at period [2s; 3s) execution rate 2, because action started at 0) have completed in say 2.2s and action started at 1) have completed in 1.5s
      So at the period [2s; 3s) we have rate 2 instead of 1, is this what you've called overrun? If so then it seems like such cases should also be handled by FailedRatePolicy, so a user can be notified that he should specify rate with a more coarse grained time unit (e.g. 60 actions/minute instead of 1 action/second).


      On Wed, Feb 4, 2015 at 3:08 AM, David Holmes [hidden email] wrote:

        Hi Valentin,

        Normal schedule-at-fixed-rate sets up a periodic release time: S + nT where S is the start time, T is the period and n = 0,1,2 .... So the task is scheduled for release at times S, S+T, S+2T etc. The main issue here is that if one release is not complete before the next is due to start then what do you do? Given we don't have guaranteed cancellation mechanisms, and that generally you do not want two releases of the task to execute concurrently, we just go into an overrun condition with ScheduledThreadPoolExecutor and delay the next release.

        From your description it sounds like concurrent executions would be okay, so at the start of each period we would just release all RATE instances of the task - is that right? Or did you intend to distribute the execution of the tasks evenly throughout the time period?

        David
          -----Original Message-----
          From: [hidden email] [[hidden email]]On Behalf Of Valentin Kovalenko
          Sent: Wednesday, 4 February 2015 9:45 AM
          To: [hidden email]
          Subject: [concurrency-interest] Fixed Rate Executor Service


          Hi 
          I'm looking for a fixed rate executor service, and would be glad if someone could give me a hint.


          I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate, but this method doesn't guarantees that actual rate will be not less that the desired one: "If any execution of this task takes longer than its period, then subsequent executions may start late".


          What I want is something like
          scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit, FailedRatePolicy frp),

          where rate==10_000L and unit==SECONDS means 10_000 actions/second,
          and FailedRatePolicy determines what to do if during some second executor failed to satisfy the specified rate (i.e. if at that second it had completed 6_000 actions). Examples of FailedRatePolicy: silently ignore; log a warning; throw an exception. Note also that at the next second after the second where rate wasn't satisfied executor doesn't try to execute 10_000 actions plus those actions that weren't executed, it tries to execute exactly 10_000 actions during the new second.


          Does anyone know an executor which have such a functionality?






_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

oleksandr otenko
In reply to this post by Aleksey Shipilev-2
Yes, so my question then is what is the use-case for rate-limiter, the limiter of ingress of requests into the pool. Also, I suppose that when the limit is reached, the requests.....ermmm.... wait somewhere in a queue? So is the purpose of rate limiter to control contention on the executor thread pool task queue?


Alex


On 04/02/2015 12:16, Aleksey Shipilev wrote:
Um... Because that's a one-sided rate limiter, not a two-sided
"semaphore" guarding a section. As far as I can see, OP does not talk
about limiting the number of concurrent tasks (that translates to
limiting the resources used at a given time).

-Aleksey.

On 02/04/2015 02:45 PM, Oleksandr Otenko wrote:
Why would you need current-second? Is the task that didn't return a
token yet not consuming the resource any more? (Or if the concern is not
the resource consumption, then what's the point of rate limiting?)

Alex

On 04/02/2015 11:11, Aleksey Shipilev wrote:
My 2c. One time I just put the TokenBucket-like limiter in front of the
regular executor, and was done with it.

Something along the lines of (GPLv2 code inside. The idea is to track
the <current-second, available-tokens> "quantum" tuple and update it in
a lock-free way):
 http://cr.openjdk.java.net/~shade/scratch/jcp/src/main/java/org/openjdk/TokenBucket.java

Such a token bucket should never over-reach the rate, and I think it can
be amended with FailedRatePolicy by detecting the current quantum is not
depleted before installing a new one.

I'm speculating Guava's RateLimiter does something like that, but from
the look over the javadoc, it seems to have a smooth refill strategy,
which may alleviate the microbursts on the quanta edges.

-Aleksey.

On 02/04/2015 12:46 PM, Valentin Kovalenko wrote:
Thanks! I've already found RateLimiter but unfortunately it doesn't
check if the rate is actually satisfied.

On Wed, Feb 4, 2015 at 6:02 AM, Martin Buchholz <[hidden email]
[hidden email]> wrote:

    See also Guava RateLimiter
    http://docs.guava-libraries.googlecode.com/git-history/master/javadoc/com/google/common/util/concurrent/RateLimiter.html

    On Tue, Feb 3, 2015 at 3:45 PM, Valentin Kovalenko
    <[hidden email]
    [hidden email]> wrote:

        Hi
        I'm looking for a fixed rate executor service, and would be glad
        if someone could give me a hint.

        I'm aware of j.u.c.ScheduledExecutorService.scheduleAtFixedRate,
        but this method doesn't guarantees that actual rate will be not
        less that the desired one: "If any execution of this task takes
        longer than its period, then subsequent executions may start late".

        What I want is something like
        scheduleAtFixedRate(Runnable action, long rate, TimeUnit unit,
        FailedRatePolicy frp),
        where rate==10_000L and unit==SECONDS means 10_000 actions/second,
        and FailedRatePolicy determines what to do if during some second
        executor failed to satisfy the specified rate (i.e. if at that
        second it had completed 6_000 actions). Examples of
        FailedRatePolicy: silently ignore; log a warning; throw an
        exception. Note also that at the next second after the second
        where rate wasn't satisfied executor doesn't try to execute
        10_000 actions plus those actions that weren't executed, it
        tries to execute exactly 10_000 actions during the new second.

        Does anyone know an executor which have such a functionality?

        _______________________________________________
        Concurrency-interest mailing list
        [hidden email]
        [hidden email]
        http://cs.oswego.edu/mailman/listinfo/concurrency-interest





_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

      



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Valentin Kovalenko
In reply to this post by Peter Levart
At first look it seems like a good idea! :)

On Wed, Feb 4, 2015 at 3:43 PM, Peter Levart <[hidden email]> wrote:
What about the following strategy:

Have a submission thread submit tasks at a fixed rate (your desired rate) to a ThreadPoolExecutor with some limit on max. number of threads (using RateLimiter if you like or some other means).
Let this ThreadPoolExecutor have a SynchonousQueue.
When the Executor starts rejecting executions, it means your rate can't be satisfied with given max. number of threads.

Regards, Peter

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Peter Levart
On 02/04/2015 02:58 PM, Valentin Kovalenko wrote:
At first look it seems like a good idea! :)

On Wed, Feb 4, 2015 at 3:43 PM, Peter Levart [hidden email] wrote:

 What about the following strategy:

Have a submission thread submit tasks at a fixed rate (your desired rate)
to a ThreadPoolExecutor with some limit on max. number of threads (using
RateLimiter if you like or some other means).
Let this ThreadPoolExecutor have a SynchonousQueue.
When the Executor starts rejecting executions, it means your rate can't be
satisfied with given max. number of threads.

Regards, Peter


    


Here's also some meat:

public class FixedRateExecutor {

    final ThreadPoolExecutor workerExec;
    final ScheduledExecutorService schedExec;

    public FixedRateExecutor(
        int corePoolSize, int maximumPoolSize,
        long keepAliveTime, TimeUnit unit) {

        workerExec = new ThreadPoolExecutor(
            corePoolSize, maximumPoolSize,
            keepAliveTime, unit,
            new SynchronousQueue<>()
        );
        schedExec = Executors.newScheduledThreadPool(1);
    }

    public ScheduledFuture<?> scheduleAtFixedRate(
        Runnable command,
        long initialDelay, long period, TimeUnit unit,
        FailedRatePolicy failedRatePolicy) {

        return schedExec.scheduleAtFixedRate(() -> {
                try {
                    workerExec.execute(command);
                } catch (RejectedExecutionException e) {
                    failedRatePolicy.failedRate(e);
                }
            },
            initialDelay, period, unit
        );
    }

    public interface FailedRatePolicy {
        FailedRatePolicy IGNORE = e -> {};
        FailedRatePolicy THROW = e -> { throw e; };

        void failedRate(RejectedExecutionException e);
    }
}

Regards, Peter


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Peter Levart
Ah, the list and HTML formatting. Here's in plain ASCII:


public class FixedRateExecutor {

     final ThreadPoolExecutor workerExec;
     final ScheduledExecutorService schedExec;

     public FixedRateExecutor(
         int corePoolSize, int maximumPoolSize,
         long keepAliveTime, TimeUnit unit) {

         workerExec = new ThreadPoolExecutor(
             corePoolSize, maximumPoolSize,
             keepAliveTime, unit,
             new SynchronousQueue<>()
         );
         schedExec = Executors.newScheduledThreadPool(1);
     }

     public ScheduledFuture<?> scheduleAtFixedRate(
         Runnable command,
         long initialDelay, long period, TimeUnit unit,
         FailedRatePolicy failedRatePolicy) {

         return schedExec.scheduleAtFixedRate(() -> {
                 try {
                     workerExec.execute(command);
                 } catch (RejectedExecutionException e) {
                     failedRatePolicy.failedRate(e);
                 }
             },
             initialDelay, period, unit
         );
     }

     public interface FailedRatePolicy {
         FailedRatePolicy IGNORE = e -> {};
         FailedRatePolicy THROW = e -> { throw e; };

         void failedRate(RejectedExecutionException e);
     }
}


Peter

On 02/04/2015 03:01 PM, Peter Levart wrote:

> On 02/04/2015 02:58 PM, Valentin Kovalenko wrote:
>> At first look it seems like a good idea! :)
>>
>> On Wed, Feb 4, 2015 at 3:43 PM, Peter Levart <[hidden email]>
>> wrote:
>>
>>>   What about the following strategy:
>>>
>>> Have a submission thread submit tasks at a fixed rate (your desired
>>> rate)
>>> to a ThreadPoolExecutor with some limit on max. number of threads
>>> (using
>>> RateLimiter if you like or some other means).
>>> Let this ThreadPoolExecutor have a SynchonousQueue.
>>> When the Executor starts rejecting executions, it means your rate
>>> can't be
>>> satisfied with given max. number of threads.
>>>
>>> Regards, Peter
>>>
>
>
> Here's also some meat:
>
> public classFixedRateExecutor {
>
>     finalThreadPoolExecutorworkerExec;
>     finalScheduledExecutorServiceschedExec;
>
>     publicFixedRateExecutor(
>         intcorePoolSize, intmaximumPoolSize,
>         longkeepAliveTime,TimeUnit unit) {
>
>         workerExec=newThreadPoolExecutor(
>             corePoolSize,maximumPoolSize,
>             keepAliveTime,unit,
>             newSynchronousQueue<>()
>         );
>         schedExec= Executors.newScheduledThreadPool(1);
>     }
>
>     publicScheduledFuture<?>scheduleAtFixedRate(
>         Runnable command,
>         longinitialDelay, longperiod,TimeUnit unit,
>         FailedRatePolicy failedRatePolicy) {
>
>         returnschedExec.scheduleAtFixedRate(() -> {
>                 try{
>                     workerExec.execute(command);
>                 }catch(RejectedExecutionException e) {
>                     failedRatePolicy.failedRate(e);
>                 }
>             },
>             initialDelay,period,unit
>         );
>     }
>
>     public interfaceFailedRatePolicy {
>         FailedRatePolicyIGNORE= e -> {};
>         FailedRatePolicyTHROW= e -> {throwe;};
>
>         voidfailedRate(RejectedExecutionException e);
>     }
> }
>
> Regards, Peter
>
>

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

David Holmes-6
In reply to this post by Peter Levart

Peter,
 
I don't see how this applies the RATE in the way Valentin wants. 1000 per second gives an average "rate" of 1ms, but that isn't way Valentin described the requirements. Really every UNIT time units you want a task that tries to submit RATE actions within UNIT time as quickly as possible. You submit to a nicely tuned executor with the "right" number of threads and suitable queue. But then you still have the problem of detecting that not all actions completed within UNIT time. And if they didn't then for the next UNIT you have additional interference on the submission and execution of the next RATE tasks (even if you cancel the non-completed ones from the previous "period".
 
Cheers,
David
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Peter Levart
Sent: Thursday, 5 February 2015 12:02 AM
To: Valentin Kovalenko; concurrency-interest
Subject: Re: [concurrency-interest] Fixed Rate Executor Service

On 02/04/2015 02:58 PM, Valentin Kovalenko wrote:
At first look it seems like a good idea! :)

On Wed, Feb 4, 2015 at 3:43 PM, Peter Levart [hidden email] wrote:

 What about the following strategy:

Have a submission thread submit tasks at a fixed rate (your desired rate)
to a ThreadPoolExecutor with some limit on max. number of threads (using
RateLimiter if you like or some other means).
Let this ThreadPoolExecutor have a SynchonousQueue.
When the Executor starts rejecting executions, it means your rate can't be
satisfied with given max. number of threads.

Regards, Peter



Here's also some meat:

public class FixedRateExecutor {

    final ThreadPoolExecutor workerExec;
    final ScheduledExecutorService schedExec;

    public FixedRateExecutor(
        int corePoolSize, int maximumPoolSize,
        long keepAliveTime, TimeUnit unit) {

        workerExec = new ThreadPoolExecutor(
            corePoolSize, maximumPoolSize,
            keepAliveTime, unit,
            new SynchronousQueue<>()
        );
        schedExec = Executors.newScheduledThreadPool(1);
    }

    public ScheduledFuture<?> scheduleAtFixedRate(
        Runnable command,
        long initialDelay, long period, TimeUnit unit,
        FailedRatePolicy failedRatePolicy) {

        return schedExec.scheduleAtFixedRate(() -> {
                try {
                    workerExec.execute(command);
                } catch (RejectedExecutionException e) {
                    failedRatePolicy.failedRate(e);
                }
            },
            initialDelay, period, unit
        );
    }

    public interface FailedRatePolicy {
        FailedRatePolicy IGNORE = e -> {};
        FailedRatePolicy THROW = e -> { throw e; };

        void failedRate(RejectedExecutionException e);
    }
}

Regards, Peter


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Peter Levart
On 02/04/2015 09:39 PM, David Holmes wrote:
> Peter,
>
> I don't see how this applies the RATE in the way Valentin wants. 1000 per second gives an average "rate" of 1ms, but that isn't way Valentin described the requirements. Really every UNIT time units you want a task that tries to submit RATE actions within UNIT time as quickly as possible. You submit to a nicely tuned executor with the "right" number of threads and suitable queue. But then you still have the problem of detecting that not all actions completed within UNIT time. And if they didn't then for the next UNIT you have additional interference on the submission and execution of the next RATE tasks (even if you cancel the non-completed ones from the previous "period".

Valentins's requirements and explanation appeared to me as unrealistic.
As I understand, he explained the functionality of some kind of "bursty"
executor, where at the start of each UNIT (period), the execution queue
is filled with RATE tasks and at the end of period, the queue is checked
whether it is empty or not (what to do with tasks, that are still
executing is not defined).

So I thought that he actually wanted something else and he just
described it in a way he thought it might be implemented.

Valentins, do you really want a "bursty" executor?

Regards, peter

>
> Cheers,
> David
>    -----Original Message-----
>    From: [hidden email] [mailto:[hidden email]]On Behalf Of Peter Levart
>    Sent: Thursday, 5 February 2015 12:02 AM
>    To: Valentin Kovalenko; concurrency-interest
>    Subject: Re: [concurrency-interest] Fixed Rate Executor Service
>
>
>    On 02/04/2015 02:58 PM, Valentin Kovalenko wrote:
>
> At first look it seems like a good idea! :)
>
> On Wed, Feb 4, 2015 at 3:43 PM, Peter Levart <[hidden email]> wrote:
>
>   What about the following strategy:
>
> Have a submission thread submit tasks at a fixed rate (your desired rate)
> to a ThreadPoolExecutor with some limit on max. number of threads (using
> RateLimiter if you like or some other means).
> Let this ThreadPoolExecutor have a SynchonousQueue.
> When the Executor starts rejecting executions, it means your rate can't be
> satisfied with given max. number of threads.
>
> Regards, Peter
>
>
>
>    Here's also some meat:
>
>
> public class FixedRateExecutor {
>
>      final ThreadPoolExecutor workerExec;
>      final ScheduledExecutorService schedExec;
>
>      public FixedRateExecutor(
>          int corePoolSize, int maximumPoolSize,
>          long keepAliveTime, TimeUnit unit) {
>
>          workerExec = new ThreadPoolExecutor(
>              corePoolSize, maximumPoolSize,
>              keepAliveTime, unit,
>              new SynchronousQueue<>()
>          );
>          schedExec = Executors.newScheduledThreadPool(1);
>      }
>
>      public ScheduledFuture<?> scheduleAtFixedRate(
>          Runnable command,
>          long initialDelay, long period, TimeUnit unit,
>          FailedRatePolicy failedRatePolicy) {
>
>          return schedExec.scheduleAtFixedRate(() -> {
>                  try {
>                      workerExec.execute(command);
>                  } catch (RejectedExecutionException e) {
>                      failedRatePolicy.failedRate(e);
>                  }
>              },
>              initialDelay, period, unit
>          );
>      }
>
>      public interface FailedRatePolicy {
>          FailedRatePolicy IGNORE = e -> {};
>          FailedRatePolicy THROW = e -> { throw e; };
>
>          void failedRate(RejectedExecutionException e);
>      }
> }
>
> Regards, Peter
>
>

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Valentin Kovalenko
In reply to this post by Valentin Kovalenko
>>at the end of period, the queue is checked whether it is empty or not (what to do with tasks, that are still executing is not defined)
>>...if they didn't then for the next UNIT you have additional interference on the submission and execution of the next RATE tasks
Executor checks if actions are still executing, because it doesn't want tasks to be finished in the period different from the period in which actions where started. If actions aren't completed, executor uses FailedRatePolicy

Now I'll try to present another wording of my thoughts. I need something like http://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer, but if throughput can't be sustained I want to be notified. This is needed for tests in which jmeter is inconvenient/hard/impossible to use.
Actually it would be great to have such a throttling mechanism in the JMH (by Alexey Shipilev), so that we could use JMH not only to "measure time" but also in load tests.  Alexey, what do you think about this?:)

On Thu, Feb 5, 2015 at 12:46 PM, Peter Levart <[hidden email]> wrote:
On 02/04/2015 09:39 PM, David Holmes wrote:
Peter,

I don't see how this applies the RATE in the way Valentin wants. 1000 per second gives an average "rate" of 1ms, but that isn't way Valentin described the requirements. Really every UNIT time units you want a task that tries to submit RATE actions within UNIT time as quickly as possible. You submit to a nicely tuned executor with the "right" number of threads and suitable queue. But then you still have the problem of detecting that not all actions completed within UNIT time. And if they didn't then for the next UNIT you have additional interference on the submission and execution of the next RATE tasks (even if you cancel the non-completed ones from the previous "period".

Valentins's requirements and explanation appeared to me as unrealistic. As I understand, he explained the functionality of some kind of "bursty" executor, where at the start of each UNIT (period), the execution queue is filled with RATE tasks and at the end of period, the queue is checked whether it is empty or not (what to do with tasks, that are still executing is not defined).

So I thought that he actually wanted something else and he just described it in a way he thought it might be implemented.

Valentins, do you really want a "bursty" executor?

Regards, peter

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Fixed Rate Executor Service

Aleksey Shipilev-2
On 05.02.2015 15:33, Valentin Kovalenko wrote:
> Actually it would be great to have such a throttling mechanism in the
> JMH (by Alexey Shipilev), so that we could use JMH not only to "measure
> time" but also in load tests.  Alexey, what do you think about this?:)

We would gladly work with interested contributors who wish to implement
fixed-load generators in JMH. We have some experiences building such the
generators in SPECjbb2013 (it looked a bit like a token bucket limiter I
already outlined here).

-Aleksey.


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment