Re: Concurrency-interest Digest, Vol 162, Issue 3

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: Concurrency-interest Digest, Vol 162, Issue 3

JSR166 Concurrency mailing list
Hi Jonas

$ java --version
java 10.0.2 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

$ uname -a
Darwin dstuebe 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64

$ java -classpath out/production/efstress com.upserve.NestedParallel
starting
*** application hangs ***

Running jstack should dump a thread stack including a section like:

Found one Java-level deadlock:
=============================
"outer pool1":
  waiting to lock monitor 0x00007fbf02006800 (object 0x00000006cfa288d0, a java.util.concurrent.ConcurrentHashMap$Node),
  which is held by "outer pool2"
"outer pool2":
  waiting to lock monitor 0x00007fbefe11ac00 (object 0x00000006cfaac488, a java.util.concurrent.ConcurrentHashMap$Node),
  which is held by "outer pool1"

Please accept my apologies for the code structure. I did my best to synthesize a simple repro case that demonstrates the issue and allows experimentation with different implementations for the inner and outer loop.

Best

David


On Tue, Aug 7, 2018 at 12:00 PM <[hidden email]> wrote:
Send Concurrency-interest mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://cs.oswego.edu/mailman/listinfo/concurrency-interest
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Concurrency-interest digest..."


Today's Topics:

   1. Re: CHM.compute restrictions (Jonas Konrad)
   2. Re: CHM.compute restrictions (David Stuebe)


----------------------------------------------------------------------

Message: 1
Date: Tue, 7 Aug 2018 17:13:24 +0200
From: Jonas Konrad <[hidden email]>
To: [hidden email]
Subject: Re: [concurrency-interest] CHM.compute restrictions
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=utf-8; format=flowed

How often is this supposed to deadlock? And in what environment? Because
I can't get it to. Also, the code is *very* confusing.

- Jonas

On 08/07/2018 04:39 PM, David Stuebe via Concurrency-interest wrote:
> Hi Doug, Concurrency Interest
>
> Sorry I lost the subject when I reposted.
>
> I understand that updating other keys or recursively initializing the
> same key is illegal in a CHM.compute. I don't understand how this
> example code could be recursive though?
>
> It seems to be a defect in parallel stream. Using a for loop to submit
> the tasks in parallel for the out loop has no issue. These should be
> effectively the same. There is some interaction between the outer
> parallel stream that I don't understand.
>
> I don't think Stream.parallel is expected to result in stream operators
> executing recursively on stream elements?
>
> Thanks
>
> On Mon, Aug 6, 2018 at 12:01 PM
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Send Concurrency-interest mailing list submissions to
>     [hidden email]
>     <mailto:[hidden email]>
>
>     To subscribe or unsubscribe via the World Wide Web, visit
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>     or, via email, send a message with subject or body 'help' to
>     [hidden email]
>     <mailto:[hidden email]>
>
>     You can reach the person managing the list at
>     [hidden email]
>     <mailto:[hidden email]>
>
>     When replying, please edit your Subject line so it is more specific
>     than "Re: Contents of Concurrency-interest digest..."
>
>
>     Today's Topics:
>
>         1. (no subject) (David Stuebe)
>         2. Re: CHM.compute restrictions (was: no     subject) (Doug Lea)
>
>
>     ----------------------------------------------------------------------
>
>     Message: 1
>     Date: Mon, 6 Aug 2018 10:10:33 -0400
>     From: David Stuebe <[hidden email]
>     <mailto:[hidden email]>>
>     To: "[hidden email]
>     <mailto:[hidden email]>"
>              <[hidden email]
>     <mailto:[hidden email]>>
>     Subject: [concurrency-interest] (no subject)
>     Message-ID:
>             
>     <CAJh0pqEVuzteLqiijAsFiVMFm1XL+KO7o12MzH5wu=[hidden email]
>     <mailto:[hidden email]>>
>     Content-Type: text/plain; charset="utf-8"
>
>     Hey folks
>
>     Attempted to post this last week while I was joining the list. I
>     think it
>     bounced. Please ignore if it is a repeat. I have not seen any
>     responses and
>     I am sad.
>
>     I found some great discussion of a previous issue* with nested
>     Stream.parallel operations here and hoping I might find some answers
>     to a
>     related question.
>     *
>     http://cs.oswego.edu/pipermail/concurrency-interest/2014-May/012652.html
>
>     I am using a ConcurrentHashMap.compute operation in an outer
>     Stream.parallel forEach operation. A library used in the compute method
>     also uses Stream.parallel.
>
>     I have written a test that illustrates the issue and explores different
>     implementations in 215 lines of code.
>     https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-nestedparallel-java
>
>     The code deadlocks with the following stack trace:
>     https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-stacktrace-txt
>
>     I do not understand why line 87 (the compute block) appears to be called
>     recursively leading to deadlock when I use Stream.parallel for the outer
>     loop.
>
>     Best
>
>     David
>     -------------- next part --------------
>     An HTML attachment was scrubbed...
>     URL:
>     <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20180806/543a8fb9/attachment-0001.html>
>
>     ------------------------------
>
>     Message: 2
>     Date: Mon, 6 Aug 2018 10:31:55 -0400
>     From: Doug Lea <[hidden email] <mailto:[hidden email]>>
>     To: [hidden email]
>     <mailto:[hidden email]>
>     Subject: Re: [concurrency-interest] CHM.compute restrictions (was: no
>              subject)
>     Message-ID: <[hidden email]
>     <mailto:[hidden email]>>
>     Content-Type: text/plain; charset=utf-8
>
>     On 08/06/2018 10:10 AM, David Stuebe via Concurrency-interest wrote:
>      > Hey folks
>      >
>      > Attempted to post this last week while I was joining the list.
>
>     To reduce spam, non-member posts are silently dropped. Sorry.
>
>      >
>      > I am using a ConcurrentHashMap.compute operation in an outer
>      > Stream.parallel forEach operation. A library used in the compute
>     method
>      > also uses Stream.parallel.
>      >
>      > I have written a test that illustrates the issue and explores
>     different
>      > implementations in 215 lines of code.
>      >
>     https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-nestedparallel-java
>      >
>      > The code deadlocks with the following stack trace:
>      >
>     https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-stacktrace-txt
>      >
>      > I do not understand why line 87 (the compute block) appears to be
>     called
>      > recursively leading to deadlock when I use Stream.parallel for
>     the outer
>      > loop.
>
>     A recursive CHM.compute call appears to invoked while trying to
>     initialize the contents of an element in the same map, which is
>     disallowed in general, but sometimes works anyway. In most other cases,
>     CHM successfully detects this and throws an exception, but it cannot
>     catch all of them.
>
>     -Doug
>
>
>
>
>
>
>     ------------------------------
>
>     Subject: Digest Footer
>
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>     ------------------------------
>
>     End of Concurrency-interest Digest, Vol 162, Issue 1
>     ****************************************************
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>


------------------------------

Message: 2
Date: Tue, 7 Aug 2018 11:32:05 -0400
From: David Stuebe <[hidden email]>
To: Peter Levart <[hidden email]>
Cc: [hidden email]
Subject: Re: [concurrency-interest] CHM.compute restrictions
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi Peter, Doug

That is very interesting!

I am not sure I fully grok your explanation, but it did suggest changing
the inner task pool to be a fixed thread pool rather than a forkjoin pool.
This appears to alleviate the issue!

More experiments to follow.

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20180807/19501b59/attachment-0001.html>

------------------------------

Subject: Digest Footer

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


------------------------------

End of Concurrency-interest Digest, Vol 162, Issue 3
****************************************************

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest