Single-threaded ForkJoinPool

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Single-threaded ForkJoinPool

cowwoc
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and ForkJoinPool in a deterministic manner when debugging/profiling but run full-speed in normal execution mode. I have all the moving parts nailing down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker threads getting used. I am guessing that this is caused by ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1 worker thread, no less, no more? Would you like me to file a formal feature request?

Thank you,
Gili
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Peter Levart
Hi Gili,

Have you thought of using SplittableRandom instead? Initially you create it with a predefined seed for the root task. Whenever you fork-off a task, you create for it a new instance by invoking SplittableRandom.split(). This should give you deterministic behavior regardless of how many threads you use for fork-join pool and the dynamics of execution. For example:

public class SumRandomInts extends RecursiveTask<Long> {
    final SplittableRandom rnd;
    final int count;

    public SumRandomInts(int count, SplittableRandom rnd) {
        this.rnd = rnd;
        this.count = count;
    }

    @Override
    protected Long compute() {
        if (count < 1000) {
            long sum = 0L;
            for (int i = 0; i < count; i++) {
                sum += rnd.nextInt();
            }
            return sum;
        } else {
            SumRandomInts t1 = new SumRandomInts(count / 2, rnd.split());
            SumRandomInts t2 = new SumRandomInts(count - count / 2, rnd.split());
            t1.fork();
            return t2.compute() + t1.join();
        }
    }

    public static void main(String[] args) throws Exception {
        long result = ForkJoinPool.commonPool().submit(
            new SumRandomInts(100000, new SplittableRandom(12345L))).get();
        System.out.println(result);
    }
}


Regards, Peter

On 02/06/2016 04:18 AM, cowwoc wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Viktor Klang
In reply to this post by cowwoc

What happens if you supply it with a thread factory which only allows a single thread to be alive at a time, and returns null if it already has returned a still living thread?

--
Cheers,

On Feb 6, 2016 05:19, "cowwoc" <[hidden email]> wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
In reply to this post by Peter Levart
Hi Peter,

I'm not sure I understand. Won't I end up with different random numbers depending on runtime conditions? I mean, it sounds as if SplittableRandom is only deterministic if threads invoke split() in the same order across JVM runs. What happens if task1 runs faster than task2 in one run, but the opposite is true in another run? Won't I end up with numbers?

Thanks,
Gili

On 2016-02-06 3:43 AM, Peter Levart [via JSR166 Concurrency] wrote:
Hi Gili,

Have you thought of using SplittableRandom instead? Initially you create it with a predefined seed for the root task. Whenever you fork-off a task, you create for it a new instance by invoking SplittableRandom.split(). This should give you deterministic behavior regardless of how many threads you use for fork-join pool and the dynamics of execution. For example:

public class SumRandomInts extends RecursiveTask<Long> {
    final SplittableRandom rnd;
    final int count;

    public SumRandomInts(int count, SplittableRandom rnd) {
        this.rnd = rnd;
        this.count = count;
    }

    @Override
    protected Long compute() {
        if (count < 1000) {
            long sum = 0L;
            for (int i = 0; i < count; i++) {
                sum += rnd.nextInt();
            }
            return sum;
        } else {
            SumRandomInts t1 = new SumRandomInts(count / 2, rnd.split());
            SumRandomInts t2 = new SumRandomInts(count - count / 2, rnd.split());
            t1.fork();
            return t2.compute() + t1.join();
        }
    }

    public static void main(String[] args) throws Exception {
        long result = ForkJoinPool.commonPool().submit(
            new SumRandomInts(100000, new SplittableRandom(12345L))).get();
        System.out.println(result);
    }
}


Regards, Peter

On 02/06/2016 04:18 AM, cowwoc wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13233.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
In reply to this post by Viktor Klang
Interesting idea.

By the looks of it, returning null causes ForkJoinPool to give up creating a new worker thread. If there are no active worker threads, then ForkJoinTask.get() hangs forever. Now, I wonder if it's safe to do as you mention... don't we run the risk of some sort of race conditions where the first thread shuts down at the exact same time that I reject the creation of the second thread and as a result ForkJoinTask.get() hangs forever?

Gili

On 2016-02-06 4:39 AM, Viktor Klang [via JSR166 Concurrency] wrote:

What happens if you supply it with a thread factory which only allows a single thread to be alive at a time, and returns null if it already has returned a still living thread?

--
Cheers,

On Feb 6, 2016 05:19, "cowwoc" <[hidden email]> wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13234.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Viktor Klang
Try and report back :)

On Sat, Feb 6, 2016 at 1:05 PM, cowwoc <[hidden email]> wrote:
Interesting idea.

By the looks of it, returning null causes ForkJoinPool to give up creating a new worker thread. If there are no active worker threads, then ForkJoinTask.get() hangs forever. Now, I wonder if it's safe to do as you mention... don't we run the risk of some sort of race conditions where the first thread shuts down at the exact same time that I reject the creation of the second thread and as a result ForkJoinTask.get() hangs forever?

Gili

On 2016-02-06 4:39 AM, Viktor Klang [via JSR166 Concurrency] wrote:

What happens if you supply it with a thread factory which only allows a single thread to be alive at a time, and returns null if it already has returned a still living thread?

--
Cheers,

On Feb 6, 2016 05:19, "cowwoc" <[hidden email]> wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13234.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
Normally I would, but when it comes to race conditions empirical evidence doesn't really cut it :) I am looking for a guarantee (from the specification or the implementers) that the design is meant to cover this situation.

There is nothing worse than chasing down intermittent bugs :)

Gili

On 2016-02-06 7:37 AM, Viktor Klang [via JSR166 Concurrency] wrote:
Try and report back :)

On Sat, Feb 6, 2016 at 1:05 PM, cowwoc <[hidden email]> wrote:
Interesting idea.

By the looks of it, returning null causes ForkJoinPool to give up creating a new worker thread. If there are no active worker threads, then ForkJoinTask.get() hangs forever. Now, I wonder if it's safe to do as you mention... don't we run the risk of some sort of race conditions where the first thread shuts down at the exact same time that I reject the creation of the second thread and as a result ForkJoinTask.get() hangs forever?

Gili

On 2016-02-06 4:39 AM, Viktor Klang [via JSR166 Concurrency] wrote:

What happens if you supply it with a thread factory which only allows a single thread to be alive at a time, and returns null if it already has returned a still living thread?

--
Cheers,

On Feb 6, 2016 05:19, "cowwoc" <[hidden email]> wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13234.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13237.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Viktor Klang
A guarantee would seem to imply formal verification. I'm pretty sure the FJP isn't formally verified which leaves you with either having to go with empirical evidence or faith… :)

On Sat, Feb 6, 2016 at 2:06 PM, cowwoc <[hidden email]> wrote:
Normally I would, but when it comes to race conditions empirical evidence doesn't really cut it :) I am looking for a guarantee (from the specification or the implementers) that the design is meant to cover this situation.

There is nothing worse than chasing down intermittent bugs :)

Gili

On 2016-02-06 7:37 AM, Viktor Klang [via JSR166 Concurrency] wrote:
Try and report back :)

On Sat, Feb 6, 2016 at 1:05 PM, cowwoc <[hidden email]> wrote:
Interesting idea.

By the looks of it, returning null causes ForkJoinPool to give up creating a new worker thread. If there are no active worker threads, then ForkJoinTask.get() hangs forever. Now, I wonder if it's safe to do as you mention... don't we run the risk of some sort of race conditions where the first thread shuts down at the exact same time that I reject the creation of the second thread and as a result ForkJoinTask.get() hangs forever?

Gili

On 2016-02-06 4:39 AM, Viktor Klang [via JSR166 Concurrency] wrote:

What happens if you supply it with a thread factory which only allows a single thread to be alive at a time, and returns null if it already has returned a still living thread?

--
Cheers,

On Feb 6, 2016 05:19, "cowwoc" <[hidden email]> wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13234.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13237.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Peter Levart
In reply to this post by cowwoc


On 02/06/2016 12:54 PM, cowwoc wrote:
Hi Peter,

I'm not sure I understand. Won't I end up with different random numbers depending on runtime conditions?

No. The initial seed for the root SplittableRandom instance will determine the sequences for all its descendants.

SplittableRandom r1 = new SplittableRandom(1234);
SplittableRandom r2 = r1.split();

...is conceptually equivalent to:

Random r1 = new Random(1234);
Random r2 = new Random(r1.nextLong());


Consider the following arrangement:

Task1:
Random r1 = new Random(1234);
int i11 = r1.nextInt();
int i12 = r1.nextInt();
Random r2 = new Random(r1.nextLong());
Random r3 = new Random(r1.nextLong());

Task2:
int i21 = r2.nextInt();
int i22 = r2.nextInt();

Task3:
int i31 = r3.nextInt();
int i32 = r3.nextInt();

It doesn't matter when Task2 and Task3 are executed and by which thread (as long as they are executed after Task1 which creates the r2 & r3 instances for them). Each task is using it's own private PRNG instance. If you can make the algorithm within the single task deterministic, the whole composition will be deterministic.


I mean, it sounds as if SplittableRandom is only deterministic if threads invoke split() in the same order across JVM runs. What happens if task1 runs faster than task2 in one run, but the opposite is true in another run? Won't I end up with numbers?

If each task is using its own private instance of PRNG, then other tasks can't have any effect on it, so it doesn't matter when or by which thread they are executed.

Regards, Peter


Thanks,
Gili

On 2016-02-06 3:43 AM, Peter Levart [via JSR166 Concurrency] wrote:
Hi Gili,

Have you thought of using SplittableRandom instead? Initially you create it with a predefined seed for the root task. Whenever you fork-off a task, you create for it a new instance by invoking SplittableRandom.split(). This should give you deterministic behavior regardless of how many threads you use for fork-join pool and the dynamics of execution. For example:

public class SumRandomInts extends RecursiveTask<Long> {
    final SplittableRandom rnd;
    final int count;

    public SumRandomInts(int count, SplittableRandom rnd) {
        this.rnd = rnd;
        this.count = count;
    }

    @Override
    protected Long compute() {
        if (count < 1000) {
            long sum = 0L;
            for (int i = 0; i < count; i++) {
                sum += rnd.nextInt();
            }
            return sum;
        } else {
            SumRandomInts t1 = new SumRandomInts(count / 2, rnd.split());
            SumRandomInts t2 = new SumRandomInts(count - count / 2, rnd.split());
            t1.fork();
            return t2.compute() + t1.join();
        }
    }

    public static void main(String[] args) throws Exception {
        long result = ForkJoinPool.commonPool().submit(
            new SumRandomInts(100000, new SplittableRandom(12345L))).get();
        System.out.println(result);
    }
}


Regards, Peter

On 02/06/2016 04:18 AM, cowwoc wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13233.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Jeff Hain
In reply to this post by cowwoc
Hi

>Specifically, I need to be able to an application that uses Random and
>ForkJoinPool in a deterministic manner when debugging/profiling but run
>full-speed in normal execution mode. I have all the moving parts nailing
>down except for ForkJoinPool.

You could abstract away your usage of ForkJoinPool behind a little interface,
and use a simple sequential implementation for debugging/profiling
(that's what I always do - being able to abstract away all threading issues
is priceless ;).

AFAIK ForkJoinPool is more designed for people to implement threading libraries
on top of it, than to be used directly from the domain code that you want to
debug or profile.

-Jeff


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Doug Lea
In reply to this post by Viktor Klang
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
In reply to this post by Peter Levart
Peter,

This is an excellent idea! Whoever came up with it was pretty clever :)

I still need to iron out some other aspects of deterministic behavior (e.g. making sure items are added to collections in the same order) but this is a huge step in the right direction. Thank you!

Gili

On 2016-02-06 12:58 PM, Peter Levart [via JSR166 Concurrency] wrote:


On 02/06/2016 12:54 PM, cowwoc wrote:
Hi Peter,

I'm not sure I understand. Won't I end up with different random numbers depending on runtime conditions?

No. The initial seed for the root SplittableRandom instance will determine the sequences for all its descendants.

SplittableRandom r1 = new SplittableRandom(1234);
SplittableRandom r2 = r1.split();

...is conceptually equivalent to:

Random r1 = new Random(1234);
Random r2 = new Random(r1.nextLong());


Consider the following arrangement:

Task1:
Random r1 = new Random(1234);
int i11 = r1.nextInt();
int i12 = r1.nextInt();
Random r2 = new Random(r1.nextLong());
Random r3 = new Random(r1.nextLong());

Task2:
int i21 = r2.nextInt();
int i22 = r2.nextInt();

Task3:
int i31 = r3.nextInt();
int i32 = r3.nextInt();

It doesn't matter when Task2 and Task3 are executed and by which thread (as long as they are executed after Task1 which creates the r2 & r3 instances for them). Each task is using it's own private PRNG instance. If you can make the algorithm within the single task deterministic, the whole composition will be deterministic.


I mean, it sounds as if SplittableRandom is only deterministic if threads invoke split() in the same order across JVM runs. What happens if task1 runs faster than task2 in one run, but the opposite is true in another run? Won't I end up with numbers?

If each task is using its own private instance of PRNG, then other tasks can't have any effect on it, so it doesn't matter when or by which thread they are executed.

Regards, Peter


Thanks,
Gili

On 2016-02-06 3:43 AM, Peter Levart [via JSR166 Concurrency] wrote:
Hi Gili,

Have you thought of using SplittableRandom instead? Initially you create it with a predefined seed for the root task. Whenever you fork-off a task, you create for it a new instance by invoking SplittableRandom.split(). This should give you deterministic behavior regardless of how many threads you use for fork-join pool and the dynamics of execution. For example:

public class SumRandomInts extends RecursiveTask<Long> {
    final SplittableRandom rnd;
    final int count;

    public SumRandomInts(int count, SplittableRandom rnd) {
        this.rnd = rnd;
        this.count = count;
    }

    @Override
    protected Long compute() {
        if (count < 1000) {
            long sum = 0L;
            for (int i = 0; i < count; i++) {
                sum += rnd.nextInt();
            }
            return sum;
        } else {
            SumRandomInts t1 = new SumRandomInts(count / 2, rnd.split());
            SumRandomInts t2 = new SumRandomInts(count - count / 2, rnd.split());
            t1.fork();
            return t2.compute() + t1.join();
        }
    }

    public static void main(String[] args) throws Exception {
        long result = ForkJoinPool.commonPool().submit(
            new SumRandomInts(100000, new SplittableRandom(12345L))).get();
        System.out.println(result);
    }
}


Regards, Peter

On 02/06/2016 04:18 AM, cowwoc wrote:
Hi,

Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
not, please point me to the right place.

I have a feature request for ForkJoinPool which doesn't seem to be possible
to implement without a JDK change: http://stackoverflow.com/q/34012134/14731

Specifically, I need to be able to an application that uses Random and
ForkJoinPool in a deterministic manner when debugging/profiling but run
full-speed in normal execution mode. I have all the moving parts nailing
down except for ForkJoinPool.

If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
threads getting used. I am guessing that this is caused by
ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
maybe something else is going on.

Is there a way for me to guarantee that ForkJoinThread will use exactly 1
worker thread, no less, no more? Would you like me to file a formal feature
request?

Thank you,
Gili



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13233.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



View this message in context: Re: Single-threaded ForkJoinPool
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13240.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
In reply to this post by Doug Lea
So, it turns out this solution isn't actually safe.

1. I used the following thread factory:

                ForkJoinWorkerThreadFactory factory = new ForkJoinWorkerThreadFactory()
                {
                    private WeakReference<Thread> currentWorker = new WeakReference<>(null);

                    @Override
                    public synchronized ForkJoinWorkerThread newThread(ForkJoinPool pool)
                    {
                        // If the pool already has a live thread, return null.
                        Thread thread = currentWorker.get();
                        if (thread != null && thread.isAlive())
                        {
                            System.out.println("Thread: " + thread.getName() + " is already alive, returning null.");
                            return null;
                        }
                        ForkJoinWorkerThread result = new MyForkJoinWorkerThread(pool);
                        currentWorker = new WeakReference<>(result);
                        // According to Doug Lea this will reduce the probability of short livelocks
                        Thread.yield();
                        return result;
                    }
                };

2. I started a debugging session.
3. I suspended all threads long enough for the worker thread to get flagged as idle (approximately 10 seconds).
4. I allowed all threads to continue execution.
5. Upon resuming, the worker thread shut down and at the same time the factory printed "Thread: X is already alive, returning null."
6. If I run the above scenario without suspending all threads (only suspending the "main" thread) then the worker thread shuts down, I resume execution, and a new worker thread spins up.

In other words, the proposed solution is vulnerable to a race condition.

Gili

On 2016-02-06 9:35 PM, cowwoc wrote:
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML


Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
In reply to this post by Doug Lea
I stepped through the ForkJoinPool code and saw the following execution path:
  1. ForkJoinPool.execute() -> ForkJoinPool.externalPush() -> ForkJoinPool.signalWork() -> ForkJoinPool.tryAddWorker() -> ForkJoinPool.createWorker() -> ForkJoinWorkerThreadFactory.newThread()
  2. When the factory returns null, createWorker() tries to clean up any exceptions (there are none) and all other methods return normally (as if a new thread had been created).
  3. This means that ForkJoinTask.get() will block indefinitely.

Doug is right that ForkJoinPool doesn't fail if a factory returns null, but it doesn't seem to recheck if a new worker is required unless someone adds additional tasks.

My guess is that tryAddWorker() should check if a worker was actually added as opposed to always breaking out of the loop.

Gili

On 2016-02-08 3:39 AM, cowwoc wrote:
So, it turns out this solution isn't actually safe.

1. I used the following thread factory:

                ForkJoinWorkerThreadFactory factory = new ForkJoinWorkerThreadFactory()
                {
                    private WeakReference<Thread> currentWorker = new WeakReference<>(null);

                    @Override
                    public synchronized ForkJoinWorkerThread newThread(ForkJoinPool pool)
                    {
                        // If the pool already has a live thread, return null.
                        Thread thread = currentWorker.get();
                        if (thread != null && thread.isAlive())
                        {
                            System.out.println("Thread: " + thread.getName() + " is already alive, returning null.");
                            return null;
                        }
                        ForkJoinWorkerThread result = new MyForkJoinWorkerThread(pool);
                        currentWorker = new WeakReference<>(result);
                        // According to Doug Lea this will reduce the probability of short livelocks
                        Thread.yield();
                        return result;
                    }
                };

2. I started a debugging session.
3. I suspended all threads long enough for the worker thread to get flagged as idle (approximately 10 seconds).
4. I allowed all threads to continue execution.
5. Upon resuming, the worker thread shut down and at the same time the factory printed "Thread: X is already alive, returning null."
6. If I run the above scenario without suspending all threads (only suspending the "main" thread) then the worker thread shuts down, I resume execution, and a new worker thread spins up.

In other words, the proposed solution is vulnerable to a race condition.

Gili

On 2016-02-06 9:35 PM, cowwoc wrote:
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML



Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Viktor Klang
In reply to this post by cowwoc

On Mon, Feb 8, 2016 at 8:47 AM, cowwoc <[hidden email]> wrote:
So, it turns out this solution isn't actually safe.

1. I used the following thread factory:

                ForkJoinWorkerThreadFactory factory = new ForkJoinWorkerThreadFactory()
                {
                    private WeakReference<Thread> currentWorker = new WeakReference<>(null);

                    @Override
                    public synchronized ForkJoinWorkerThread newThread(ForkJoinPool pool)
                    {
                        // If the pool already has a live thread, return null.
                        Thread thread = currentWorker.get();
                        if (thread != null && thread.isAlive())
                        {
                            System.out.println("Thread: " + thread.getName() + " is already alive, returning null.");
                            return null;
                        }
                        ForkJoinWorkerThread result = new MyForkJoinWorkerThread(pool);
                        currentWorker = new WeakReference<>(result);
                        // According to Doug Lea this will reduce the probability of short livelocks
                        Thread.yield();
                        return result;
                    }
                };

2. I started a debugging session.
3. I suspended all threads long enough for the worker thread to get flagged as idle (approximately 10 seconds).
4. I allowed all threads to continue execution.
5. Upon resuming, the worker thread shut down and at the same time the factory printed "Thread: X is already alive, returning null."
6. If I run the above scenario without suspending all threads (only suspending the "main" thread) then the worker thread shuts down, I resume execution, and a new worker thread spins up.

In other words, the proposed solution is vulnerable to a race condition.

Gili

On 2016-02-06 9:35 PM, cowwoc wrote:
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML




View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
You're still vulnerable to the same race condition:
  1. [main] Invokes ForkJoinPool.execute(task)
  2. [WorkerThread] Finishes running a task, reaches the point immediately before deregisterThread() at https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L61
  3. [main] ForkJoinPool invokes tryAddWorker(), https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65 returns null
  4. [WorkerThread] Invokes deregisterThread().

Now we have an enqueued task, no worker threads and no one is trying to spin up a new thread.

Gili

On 2016-02-08 4:53 AM, Viktor Klang wrote:

On Mon, Feb 8, 2016 at 8:47 AM, cowwoc <[hidden email]> wrote:
So, it turns out this solution isn't actually safe.

1. I used the following thread factory:

                ForkJoinWorkerThreadFactory factory = new ForkJoinWorkerThreadFactory()
                {
                    private WeakReference<Thread> currentWorker = new WeakReference<>(null);

                    @Override
                    public synchronized ForkJoinWorkerThread newThread(ForkJoinPool pool)
                    {
                        // If the pool already has a live thread, return null.
                        Thread thread = currentWorker.get();
                        if (thread != null && thread.isAlive())
                        {
                            System.out.println("Thread: " + thread.getName() + " is already alive, returning null.");
                            return null;
                        }
                        ForkJoinWorkerThread result = new MyForkJoinWorkerThread(pool);
                        currentWorker = new WeakReference<>(result);
                        // According to Doug Lea this will reduce the probability of short livelocks
                        Thread.yield();
                        return result;
                    }
                };

2. I started a debugging session.
3. I suspended all threads long enough for the worker thread to get flagged as idle (approximately 10 seconds).
4. I allowed all threads to continue execution.
5. Upon resuming, the worker thread shut down and at the same time the factory printed "Thread: X is already alive, returning null."
6. If I run the above scenario without suspending all threads (only suspending the "main" thread) then the worker thread shuts down, I resume execution, and a new worker thread spins up.

In other words, the proposed solution is vulnerable to a race condition.

Gili

On 2016-02-06 9:35 PM, cowwoc wrote:
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML




View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Viktor Klang

Haven't had time to think about this yet.

I think in general it is best to model the task in terms of dependency rather than tweaking the execution engine to get the effect you want.

--
Cheers,

On Feb 8, 2016 11:00 AM, "cowwoc" <[hidden email]> wrote:
You're still vulnerable to the same race condition:
  1. [main] Invokes ForkJoinPool.execute(task)
  2. [WorkerThread] Finishes running a task, reaches the point immediately before deregisterThread() at https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L61
  3. [main] ForkJoinPool invokes tryAddWorker(), https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65 returns null
  4. [WorkerThread] Invokes deregisterThread().

Now we have an enqueued task, no worker threads and no one is trying to spin up a new thread.

Gili

On 2016-02-08 4:53 AM, Viktor Klang wrote:

On Mon, Feb 8, 2016 at 8:47 AM, cowwoc <[hidden email]> wrote:
So, it turns out this solution isn't actually safe.

1. I used the following thread factory:

                ForkJoinWorkerThreadFactory factory = new ForkJoinWorkerThreadFactory()
                {
                    private WeakReference<Thread> currentWorker = new WeakReference<>(null);

                    @Override
                    public synchronized ForkJoinWorkerThread newThread(ForkJoinPool pool)
                    {
                        // If the pool already has a live thread, return null.
                        Thread thread = currentWorker.get();
                        if (thread != null && thread.isAlive())
                        {
                            System.out.println("Thread: " + thread.getName() + " is already alive, returning null.");
                            return null;
                        }
                        ForkJoinWorkerThread result = new MyForkJoinWorkerThread(pool);
                        currentWorker = new WeakReference<>(result);
                        // According to Doug Lea this will reduce the probability of short livelocks
                        Thread.yield();
                        return result;
                    }
                };

2. I started a debugging session.
3. I suspended all threads long enough for the worker thread to get flagged as idle (approximately 10 seconds).
4. I allowed all threads to continue execution.
5. Upon resuming, the worker thread shut down and at the same time the factory printed "Thread: X is already alive, returning null."
6. If I run the above scenario without suspending all threads (only suspending the "main" thread) then the worker thread shuts down, I resume execution, and a new worker thread spins up.

In other words, the proposed solution is vulnerable to a race condition.

Gili

On 2016-02-06 9:35 PM, cowwoc wrote:
Thanks Doug, I'll give this a try.

Thanks,
Gili

On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/06/2016 05:21 AM, Viktor Klang wrote:
> What happens if you supply it with a thread factory which only allows a single
> thread to be alive at a time, and returns null if it already has returned a
> still living thread?
>

Yes, this will work if you are positive that only one thread
is required for liveness. FJ sometimes conservatively creates
threads when it cannot itself guarantee liveness (for example,
when GC or other system load causes stalls). But it will
respond to null factory returns by rechecking, not failing.
unless a thread really is needed to maintain liveness, in which
case the program may livelock. To reduce transient near-livelock,
you might want to place a Thread.yield() call before the
"return null" in the factory.

-Doug


> --
> Cheers,
> √
>
> On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Is this the correct mailing list for discussing ForkJoinPool in JDK9? If
>     not, please point me to the right place.
>
>     I have a feature request for ForkJoinPool which doesn't seem to be possible
>     to implement without a JDK change: http://stackoverflow.com/q/34012134/14731
>
>     Specifically, I need to be able to an application that uses Random and
>     ForkJoinPool in a deterministic manner when debugging/profiling but run
>     full-speed in normal execution mode. I have all the moving parts nailing
>     down except for ForkJoinPool.
>
>     If I create ForkJoinPool with a parallelism of 1, sometimes I see two worker
>     threads getting used. I am guessing that this is caused by
>     ForkJoinTask.get() invoking ForkJoinPool.common.externalHelpComplete(), but
>     maybe something else is going on.
>
>     Is there a way for me to guarantee that ForkJoinThread will use exactly 1
>     worker thread, no less, no more? Would you like me to file a formal feature
>     request?
>
>     Thank you,
>     Gili
>
>
>
>     --
>     View this message in context:
>     http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>     Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML




View this message in context: Re: Single-threaded ForkJoinPool

Sent from the JSR166 Concurrency mailing list archive at Nabble.com.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

Doug Lea
On 02/11/2016 06:57 AM, Viktor Klang wrote:
> Haven't had time to think about this yet.

In his case, one way to force race outcome is for the factory
to join the existing single thread before returning it. This
is not in general a good practice of course...

-Doug



>
> I think in general it is best to model the task in terms of dependency rather
> than tweaking the execution engine to get the effect you want.
>
> --
> Cheers,
> √
>
> On Feb 8, 2016 11:00 AM, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     You're still vulnerable to the same race condition:
>
>      1. [main] Invokes ForkJoinPool.execute(task)
>      2. [WorkerThread] Finishes running a task, reaches the point immediately
>         before deregisterThread() at
>         https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L61
>      3. [main] ForkJoinPool invokes tryAddWorker(),
>         https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65
>         returns null
>      4. [WorkerThread] Invokes deregisterThread().
>
>     Now we have an enqueued task, no worker threads and no one is trying to spin
>     up a new thread.
>
>     Gili
>
>     On 2016-02-08 4:53 AM, Viktor Klang wrote:
>>     I was thinking something along the lines of this:
>>
>>     https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65
>>     https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L68
>>
>>     (no need for WeakReferences to Threads, synchronized etc)
>>
>>     On Mon, Feb 8, 2016 at 8:47 AM, cowwoc <[hidden email]
>>     <mailto:[hidden email]>> wrote:
>>
>>         So, it turns out this solution isn't actually safe.
>>
>>         1. I used the following thread factory:
>>
>>                         ForkJoinWorkerThreadFactory factory = new
>>         ForkJoinWorkerThreadFactory()
>>                         {
>>                             private WeakReference<Thread> currentWorker = new
>>         WeakReference<>(null);
>>
>>                             @Override
>>                             public synchronized ForkJoinWorkerThread
>>         newThread(ForkJoinPool pool)
>>                             {
>>                                 // If the pool already has a live thread,
>>         return null.
>>                                 Thread thread = currentWorker.get();
>>                                 if (thread != null && thread.isAlive())
>>                                 {
>>                                     System.out.println("Thread: " +
>>         thread.getName() + " is already alive, returning null.");
>>                                     return null;
>>                                 }
>>                                 ForkJoinWorkerThread result = new
>>         MyForkJoinWorkerThread(pool);
>>                                 currentWorker = new WeakReference<>(result);
>>                                 // According to Doug Lea this will reduce the
>>         probability of short livelocks
>>                                 Thread.yield();
>>                                 return result;
>>                             }
>>                         };
>>
>>         2. I started a debugging session.
>>         3. I suspended all threads long enough for the worker thread to get
>>         flagged as idle (approximately 10 seconds).
>>         4. I allowed all threads to continue execution.
>>         5. Upon resuming, the worker thread shut down and at the same time the
>>         factory printed "Thread: X is already alive, returning null."
>>         6. If I run the above scenario without suspending all threads (only
>>         suspending the "main" thread) then the worker thread shuts down, I
>>         resume execution, and a new worker thread spins up.
>>
>>         In other words, the proposed solution is vulnerable to a race condition.
>>
>>         Gili
>>
>>         On 2016-02-06 9:35 PM, cowwoc wrote:
>>>         Thanks Doug, I'll give this a try.
>>>
>>>         Thanks,
>>>         Gili
>>>
>>>         On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
>>>>         On 02/06/2016 05:21 AM, Viktor Klang wrote:
>>>>         > What happens if you supply it with a thread factory which only
>>>>         allows a single
>>>>         > thread to be alive at a time, and returns null if it already has
>>>>         returned a
>>>>         > still living thread?
>>>>         >
>>>>
>>>>         Yes, this will work if you are positive that only one thread
>>>>         is required for liveness. FJ sometimes conservatively creates
>>>>         threads when it cannot itself guarantee liveness (for example,
>>>>         when GC or other system load causes stalls). But it will
>>>>         respond to null factory returns by rechecking, not failing.
>>>>         unless a thread really is needed to maintain liveness, in which
>>>>         case the program may livelock. To reduce transient near-livelock,
>>>>         you might want to place a Thread.yield() call before the
>>>>         "return null" in the factory.
>>>>
>>>>         -Doug
>>>>
>>>>
>>>>         > --
>>>>         > Cheers,
>>>>         > √
>>>>         >
>>>>         > On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=0>
>>>>         > <mailto:[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=1>>> wrote:
>>>>         >
>>>>         >     Hi,
>>>>         >
>>>>         >     Is this the correct mailing list for discussing ForkJoinPool
>>>>         in JDK9? If
>>>>         >     not, please point me to the right place.
>>>>         >
>>>>         >     I have a feature request for ForkJoinPool which doesn't seem
>>>>         to be possible
>>>>         >     to implement without a JDK change:
>>>>         <http://stackoverflow.com/q/34012134/14731>http://stackoverflow.com/q/34012134/14731
>>>>         >
>>>>         >     Specifically, I need to be able to an application that uses
>>>>         Random and
>>>>         >     ForkJoinPool in a deterministic manner when
>>>>         debugging/profiling but run
>>>>         >     full-speed in normal execution mode. I have all the moving
>>>>         parts nailing
>>>>         >     down except for ForkJoinPool.
>>>>         >
>>>>         >     If I create ForkJoinPool with a parallelism of 1, sometimes I
>>>>         see two worker
>>>>         >     threads getting used. I am guessing that this is caused by
>>>>         >     ForkJoinTask.get() invoking
>>>>         ForkJoinPool.common.externalHelpComplete(), but
>>>>         >     maybe something else is going on.
>>>>         >
>>>>         >     Is there a way for me to guarantee that ForkJoinThread will
>>>>         use exactly 1
>>>>         >     worker thread, no less, no more? Would you like me to file a
>>>>         formal feature
>>>>         >     request?
>>>>         >
>>>>         >     Thank you,
>>>>         >     Gili
>>>>         >
>>>>         >
>>>>         >
>>>>         >     --
>>>>         >     View this message in context:
>>>>         >
>>>>         http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>>>>         >     Sent from the JSR166 Concurrency mailing list archive at
>>>>         Nabble.com.
>>>>         > _______________________________________________
>>>>         >     Concurrency-interest mailing list
>>>>         > [hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=2>
>>>>         <mailto:[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=3>>
>>>>         >
>>>>         <http://cs.oswego.edu/mailman/listinfo/concurrency-interest>http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>         >
>>>>         >
>>>>         >
>>>>         > _______________________________________________
>>>>         > Concurrency-interest mailing list
>>>>         > [hidden email] <http:///user/SendEmail.jtp?type=node&node=13243&i=4>
>>>>         > http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>         >
>>>>
>>>>
>>>>
>>>>         _______________________________________________
>>>>         Concurrency-interest mailing list
>>>>         [hidden email] <http:///user/SendEmail.jtp?type=node&node=13243&i=5>
>>>>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>
>>>>
>>>>         --------------------------------------------------------------------------------
>>>>         If you reply to this email, your message will be added to the
>>>>         discussion below:
>>>>         http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
>>>>
>>>>         To unsubscribe from Single-threaded ForkJoinPool, click here.
>>>>         NAML
>>>>         <http://jsr166-concurrency.10961.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>
>>
>>         --------------------------------------------------------------------------------
>>         View this message in context: Re: Single-threaded ForkJoinPool
>>         <http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13250.html>
>>
>>
>>         Sent from the JSR166 Concurrency mailing list archive
>>         <http://jsr166-concurrency.10961.n7.nabble.com/> at Nabble.com.
>>
>>         _______________________________________________
>>         Concurrency-interest mailing list
>>         [hidden email]
>>         <mailto:[hidden email]>
>>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>>
>>
>>
>>     --
>>     Cheers,
>>     √
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Single-threaded ForkJoinPool

cowwoc
Not a bad idea :) I'll give it a try.

Gili

On 2016-02-11 8:46 AM, Doug Lea [via JSR166 Concurrency] wrote:
On 02/11/2016 06:57 AM, Viktor Klang wrote:
> Haven't had time to think about this yet.

In his case, one way to force race outcome is for the factory
to join the existing single thread before returning it. This
is not in general a good practice of course...

-Doug



>
> I think in general it is best to model the task in terms of dependency rather
> than tweaking the execution engine to get the effect you want.
>
> --
> Cheers,
> √
>
> On Feb 8, 2016 11:00 AM, "cowwoc" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     You're still vulnerable to the same race condition:
>
>      1. [main] Invokes ForkJoinPool.execute(task)
>      2. [WorkerThread] Finishes running a task, reaches the point immediately
>         before deregisterThread() at
>         https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L61
>      3. [main] ForkJoinPool invokes tryAddWorker(),
>         https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65
>         returns null
>      4. [WorkerThread] Invokes deregisterThread().
>
>     Now we have an enqueued task, no worker threads and no one is trying to spin
>     up a new thread.
>
>     Gili
>
>     On 2016-02-08 4:53 AM, Viktor Klang wrote:
>>     I was thinking something along the lines of this:
>>
>>     https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L65
>>     https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala#L68
>>
>>     (no need for WeakReferences to Threads, synchronized etc)
>>
>>     On Mon, Feb 8, 2016 at 8:47 AM, cowwoc <[hidden email]
>>     <mailto:[hidden email]>> wrote:
>>
>>         So, it turns out this solution isn't actually safe.
>>
>>         1. I used the following thread factory:
>>
>>                         ForkJoinWorkerThreadFactory factory = new
>>         ForkJoinWorkerThreadFactory()
>>                         {
>>                             private WeakReference<Thread> currentWorker = new
>>         WeakReference<>(null);
>>
>>                             @Override
>>                             public synchronized ForkJoinWorkerThread
>>         newThread(ForkJoinPool pool)
>>                             {
>>                                 // If the pool already has a live thread,
>>         return null.
>>                                 Thread thread = currentWorker.get();
>>                                 if (thread != null && thread.isAlive())
>>                                 {
>>                                     System.out.println("Thread: " +
>>         thread.getName() + " is already alive, returning null.");
>>                                     return null;
>>                                 }
>>                                 ForkJoinWorkerThread result = new
>>         MyForkJoinWorkerThread(pool);
>>                                 currentWorker = new WeakReference<>(result);
>>                                 // According to Doug Lea this will reduce the
>>         probability of short livelocks
>>                                 Thread.yield();
>>                                 return result;
>>                             }
>>                         };
>>
>>         2. I started a debugging session.
>>         3. I suspended all threads long enough for the worker thread to get
>>         flagged as idle (approximately 10 seconds).
>>         4. I allowed all threads to continue execution.
>>         5. Upon resuming, the worker thread shut down and at the same time the
>>         factory printed "Thread: X is already alive, returning null."
>>         6. If I run the above scenario without suspending all threads (only
>>         suspending the "main" thread) then the worker thread shuts down, I
>>         resume execution, and a new worker thread spins up.
>>
>>         In other words, the proposed solution is vulnerable to a race condition.
>>
>>         Gili
>>
>>         On 2016-02-06 9:35 PM, cowwoc wrote:
>>>         Thanks Doug, I'll give this a try.
>>>
>>>         Thanks,
>>>         Gili
>>>
>>>         On 2016-02-06 3:02 PM, Doug Lea [via JSR166 Concurrency] wrote:
>>>>         On 02/06/2016 05:21 AM, Viktor Klang wrote:
>>>>         > What happens if you supply it with a thread factory which only
>>>>         allows a single
>>>>         > thread to be alive at a time, and returns null if it already has
>>>>         returned a
>>>>         > still living thread?
>>>>         >
>>>>
>>>>         Yes, this will work if you are positive that only one thread
>>>>         is required for liveness. FJ sometimes conservatively creates
>>>>         threads when it cannot itself guarantee liveness (for example,
>>>>         when GC or other system load causes stalls). But it will
>>>>         respond to null factory returns by rechecking, not failing.
>>>>         unless a thread really is needed to maintain liveness, in which
>>>>         case the program may livelock. To reduce transient near-livelock,
>>>>         you might want to place a Thread.yield() call before the
>>>>         "return null" in the factory.
>>>>
>>>>         -Doug
>>>>
>>>>
>>>>         > --
>>>>         > Cheers,
>>>>         > √
>>>>         >
>>>>         > On Feb 6, 2016 05:19, "cowwoc" <[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=0>
>>>>         > <mailto:[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=1>>> wrote:
>>>>         >
>>>>         >     Hi,
>>>>         >
>>>>         >     Is this the correct mailing list for discussing ForkJoinPool
>>>>         in JDK9? If
>>>>         >     not, please point me to the right place.
>>>>         >
>>>>         >     I have a feature request for ForkJoinPool which doesn't seem
>>>>         to be possible
>>>>         >     to implement without a JDK change:
>>>>         <http://stackoverflow.com/q/34012134/14731>http://stackoverflow.com/q/34012134/14731
>>>>         >
>>>>         >     Specifically, I need to be able to an application that uses
>>>>         Random and
>>>>         >     ForkJoinPool in a deterministic manner when
>>>>         debugging/profiling but run
>>>>         >     full-speed in normal execution mode. I have all the moving
>>>>         parts nailing
>>>>         >     down except for ForkJoinPool.
>>>>         >
>>>>         >     If I create ForkJoinPool with a parallelism of 1, sometimes I
>>>>         see two worker
>>>>         >     threads getting used. I am guessing that this is caused by
>>>>         >     ForkJoinTask.get() invoking
>>>>         ForkJoinPool.common.externalHelpComplete(), but
>>>>         >     maybe something else is going on.
>>>>         >
>>>>         >     Is there a way for me to guarantee that ForkJoinThread will
>>>>         use exactly 1
>>>>         >     worker thread, no less, no more? Would you like me to file a
>>>>         formal feature
>>>>         >     request?
>>>>         >
>>>>         >     Thank you,
>>>>         >     Gili
>>>>         >
>>>>         >
>>>>         >
>>>>         >     --
>>>>         >     View this message in context:
>>>>         >
>>>>         http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232.html
>>>>         >     Sent from the JSR166 Concurrency mailing list archive at
>>>>         Nabble.com.
>>>>         > _______________________________________________
>>>>         >     Concurrency-interest mailing list
>>>>         > [hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=2>
>>>>         <mailto:[hidden email]
>>>>         <http:///user/SendEmail.jtp?type=node&node=13243&i=3>>
>>>>         >
>>>>         <http://cs.oswego.edu/mailman/listinfo/concurrency-interest>http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>         >
>>>>         >
>>>>         >
>>>>         > _______________________________________________
>>>>         > Concurrency-interest mailing list
>>>>         > [hidden email] <http:///user/SendEmail.jtp?type=node&node=13243&i=4>
>>>>         > http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>         >
>>>>
>>>>
>>>>
>>>>         _______________________________________________
>>>>         Concurrency-interest mailing list
>>>>         [hidden email] <http:///user/SendEmail.jtp?type=node&node=13243&i=5>
>>>>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>>
>>>>
>>>>         --------------------------------------------------------------------------------
>>>>         If you reply to this email, your message will be added to the
>>>>         discussion below:
>>>>         http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13243.html
>>>>
>>>>         To unsubscribe from Single-threaded ForkJoinPool, click here.
>>>>         NAML
>>>>         <http://jsr166-concurrency.10961.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>
>>
>>         --------------------------------------------------------------------------------
>>         View this message in context: Re: Single-threaded ForkJoinPool
>>         <http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13250.html>
>>
>>
>>         Sent from the JSR166 Concurrency mailing list archive
>>         <http://jsr166-concurrency.10961.n7.nabble.com/> at Nabble.com.
>>
>>         _______________________________________________
>>         Concurrency-interest mailing list
>>         [hidden email]
>>         <mailto:[hidden email]>
>>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>>
>>
>>
>>     --
>>     Cheers,
>>     √
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



If you reply to this email, your message will be added to the discussion below:
http://jsr166-concurrency.10961.n7.nabble.com/Single-threaded-ForkJoinPool-tp13232p13260.html
To unsubscribe from Single-threaded ForkJoinPool, click here.
NAML

12