Is Reference.reachabilityFence() needed in Reference constructor?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Is Reference.reachabilityFence() needed in Reference constructor?

Peter Levart
Hi,

I have a question about a use-case for Reference.reachabilityFence().

While reviewing code of new Cleaner API that is being proposed on
core-libs-dev, the following java.lang.Reference constructor caught my eye:

     Reference(T referent, ReferenceQueue<? super T> queue) {
         this.referent = referent;
         this.queue = (queue == null) ? ReferenceQueue.NULL : queue;
     }


Say it is being used like that:

ReferenceQueue<Object> queue = ...;

WeakReference<Object> wr = new WeakReference(new Object(), queue);

Is it possible that the newly constructed Object is found
weakly-reachable before the 'queue' is assigned to the Reference.queue
field or is there something in the JVM that prevents this happening
(like all the constructor parameters are reachable at least until the
constructor is finished)? Might the following be needed or not:

     Reference(T referent, ReferenceQueue<? super T> queue) {
         this.referent = referent;
         this.queue = (queue == null) ? ReferenceQueue.NULL : queue;
         reachabilityFence(referent);
     }


Regards, Peter

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Andrew Haley
On 10/21/2015 01:34 PM, Peter Levart wrote:
> Is it possible that the newly constructed Object is found
> weakly-reachable before the 'queue' is assigned to the Reference.queue
> field or is there something in the JVM that prevents this happening
> (like all the constructor parameters are reachable at least until the
> constructor is finished)?

No, there's nothing like that.

I think you're right: from the point where the reference is assigned
to the referent field of the WeakReference it is only reachable by
traversing a weak reference, therefore it is weakly reachable.

Adding the reachabilityFence would not hurt anything, and if it did
make a difference to generated code it'd eliminate a bug.

There may be a few more of these.

Andrew.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Aleksey Shipilev-2
In reply to this post by Peter Levart
Hi,

On 10/21/2015 03:34 PM, Peter Levart wrote:

> Say it is being used like that:
>
> ReferenceQueue<Object> queue = ...;
>
> WeakReference<Object> wr = new WeakReference(new Object(), queue);
>
> Is it possible that the newly constructed Object is found
> weakly-reachable before the 'queue' is assigned to the Reference.queue
> field or is there something in the JVM that prevents this happening
> (like all the constructor parameters are reachable at least until the
> constructor is finished)?
I think this is largely dependent on VM implementation. IIRC, Hotspot
does not treat constructors differently from regular methods w.r.t.
reachability analysis.

But, I suppose when you are storing the incoming argument in the
instance field, then a story gets simpler, since you now "switch" the
reachability via the argument (which is obviously alive up to the last
use) for the reachability via $this itself.

The Reference case is a bit special here. If we were to store the
referent in the "ordinary" instance, then we are changing one strong ref
for another strong one. But for Reference, we are changing strong ref
for a weak ref, and so we need to make sure there is a strong ref until
the end of the method, with reachabilityFence.

In other words, this seems to be needed, but only for Reference.referent:

> Might the following be needed or not:
>
>     Reference(T referent, ReferenceQueue<? super T> queue) {
>         this.referent = referent;
>         this.queue = (queue == null) ? ReferenceQueue.NULL : queue;
>         reachabilityFence(referent);
>     }

Of course, this does not tell another (scary) part of the story, what if
the Reference itself is not discovered as strong ref by GC, e.g. when it
isn't published on heap, or scalarized by compiler, etc.
reachabilityFence, as currently implemented, extends the "liveness"
scope of the local variable, but does not convey anything special to the
GC/runtime otherwise.

Thanks,
-Aleksey


P.S. Tried to reproduce the failure in jcstress case to no avail. Maybe
somebody else is more lucky:

@JCStressTest
@Outcome(id = "[true]", expect = Expect.ACCEPTABLE, desc = "TBD")
@State
public class ReferenceNew {

    final ReferenceQueue<Object> refq = new ReferenceQueue<>();
    WeakReference<Object> wr;

    @Actor
    public void producer() {
        wr = new WeakReference<>(new Object(), refq);
    }

    @Actor
    public void consumer(BooleanResult1 r)  {
        try {
            for (int c = 0; c < 10; c++) {
                Reference<?> ref = refq.remove(100);
                if (ref != null) {
                    r.r1 = true;
                    return;
                }
            }
            r.r1 = false;
        } catch (Throwable e) {
            r.r1 = false;
        }
    }

    static {
        Executors.newScheduledThreadPool(1, r -> {
            Thread t = new Thread(r);
            t.setDaemon(true);
            return t;
        }).scheduleAtFixedRate(
                System::gc, 10L, 10L, TimeUnit.MILLISECONDS
        );
    }

}



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Andrew Haley
On 10/21/2015 04:44 PM, Aleksey Shipilev wrote:
> Of course, this does not tell another (scary) part of the story, what if
> the Reference itself is not discovered as strong ref by GC, e.g. when it
> isn't published on heap, or scalarized by compiler, etc.
> reachabilityFence, as currently implemented, extends the "liveness"
> scope of the local variable, but does not convey anything special to the
> GC/runtime otherwise.

If the Reference itself is not reachable from a strong root then the
Reference is dead so we don't care what happens to it.  But if the
Reference is put on a reachable ReferenceQueue, then it's fine.

Andrew.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Aleksey Shipilev-2
On 10/21/2015 06:53 PM, Andrew Haley wrote:

> On 10/21/2015 04:44 PM, Aleksey Shipilev wrote:
>> Of course, this does not tell another (scary) part of the story, what if
>> the Reference itself is not discovered as strong ref by GC, e.g. when it
>> isn't published on heap, or scalarized by compiler, etc.
>> reachabilityFence, as currently implemented, extends the "liveness"
>> scope of the local variable, but does not convey anything special to the
>> GC/runtime otherwise.
>
> If the Reference itself is not reachable from a strong root then the
> Reference is dead so we don't care what happens to it.  But if the
> Reference is put on a reachable ReferenceQueue, then it's fine.
Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref), or those references are dead (e.g. the
weakref is buried somewhere deep in garbage subgraph, and purged on
sweep), then everything goes awry.

But the thing is, a WeakReference is put on ReferenceQueue by the GC
itself. In the object graph, ReferenceQueue does not normally reference
weakrefs back. Before enqueueing the reference GC has to first discover
that WeakReference -- but from where? In other words, "registering" on a
reachable RefQueue does not make weakref to be reachable.

Case in point:

public class WhereIsWaldo {

    public static void main(String... args) throws Exception {
        for (int c = 0; true; c++) {
            new WhereIsWaldo().work(c);
        }
    }

    final ReferenceQueue<Object> rq = new ReferenceQueue<>();

    void work(int id) throws Exception {
        hideWaldo(id);
        findWaldo(id);
    }

    WeakReference<Object> gwr;

    void hideWaldo(int id) {
        // Does not work, d'uh
        new WeakReference<>(new Object(), rq);

        // Does not work either :(
        WeakReference<Object> wr =
           new WeakReference<>(new Object(), rq);

        // This also does not help... once you leave the method,
        // the weakref is gone.
        Reference.reachabilityFence(wr);

        // This one helps, makes Waldo reachable
        // gwr = wr;
    }

    void findWaldo(int id) throws Exception {
        while (Thread.currentThread().isAlive()) {
            System.gc();
            Reference ref = rq.remove(1000);
            if (ref != null) {
                return;
            } else {
                System.out.println("Where's Waldo #" + id + "?");
            }
        }
    }
}


Thanks,
-Aleksey


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

thurstonn
This post was updated on .
In reply to this post by Peter Levart
But the referent arg (i.e. its oop is of course) is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Aleksey Shipilev-2
In reply to this post by thurstonn
No, they aren't. That's a common misconception.

https://docs.oracle.com/javase/specs/jls/se8/html/jls-12.html#jls-12.6.1:

"Optimizing transformations of a program can be designed that reduce the
number of objects that are reachable to be less than those which would
naively be considered reachable. For example, a Java compiler or code
generator may choose to set a variable or parameter that will no longer
be used to null to cause the storage for such an object to be
potentially reclaimable sooner."


Thanks,
-Aleksey

On 10/21/2015 11:29 PM, thurstonn wrote:

> But the referent arg is on the stack (since the constructor is)
> Aren't all variables on the stack strongly reachable?
>
>
>
> --
> View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
> Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Gregg Wonderly-3
In reply to this post by thurstonn

On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.

Gregg Wonderly


On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich
That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can

That's what this thread is about.

On Wed, Oct 21, 2015 at 6:10 PM, Gregg Wonderly <[hidden email]> wrote:

On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.

Gregg Wonderly


On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich
In reply to this post by Aleksey Shipilev-2
Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref)

If you look at escape analysis code (http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/a60bd3d34158/src/share/vm/opto/escape.cpp#l803), Reference and subclasses get special treatment and marked as GlobalEscape.

On Wed, Oct 21, 2015 at 1:08 PM, Aleksey Shipilev <[hidden email]> wrote:
On 10/21/2015 06:53 PM, Andrew Haley wrote:
> On 10/21/2015 04:44 PM, Aleksey Shipilev wrote:
>> Of course, this does not tell another (scary) part of the story, what if
>> the Reference itself is not discovered as strong ref by GC, e.g. when it
>> isn't published on heap, or scalarized by compiler, etc.
>> reachabilityFence, as currently implemented, extends the "liveness"
>> scope of the local variable, but does not convey anything special to the
>> GC/runtime otherwise.
>
> If the Reference itself is not reachable from a strong root then the
> Reference is dead so we don't care what happens to it.  But if the
> Reference is put on a reachable ReferenceQueue, then it's fine.

Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref), or those references are dead (e.g. the
weakref is buried somewhere deep in garbage subgraph, and purged on
sweep), then everything goes awry.

But the thing is, a WeakReference is put on ReferenceQueue by the GC
itself. In the object graph, ReferenceQueue does not normally reference
weakrefs back. Before enqueueing the reference GC has to first discover
that WeakReference -- but from where? In other words, "registering" on a
reachable RefQueue does not make weakref to be reachable.

Case in point:

public class WhereIsWaldo {

    public static void main(String... args) throws Exception {
        for (int c = 0; true; c++) {
            new WhereIsWaldo().work(c);
        }
    }

    final ReferenceQueue<Object> rq = new ReferenceQueue<>();

    void work(int id) throws Exception {
        hideWaldo(id);
        findWaldo(id);
    }

    WeakReference<Object> gwr;

    void hideWaldo(int id) {
        // Does not work, d'uh
        new WeakReference<>(new Object(), rq);

        // Does not work either :(
        WeakReference<Object> wr =
           new WeakReference<>(new Object(), rq);

        // This also does not help... once you leave the method,
        // the weakref is gone.
        Reference.reachabilityFence(wr);

        // This one helps, makes Waldo reachable
        // gwr = wr;
    }

    void findWaldo(int id) throws Exception {
        while (Thread.currentThread().isAlive()) {
            System.gc();
            Reference ref = rq.remove(1000);
            if (ref != null) {
                return;
            } else {
                System.out.println("Where's Waldo #" + id + "?");
            }
        }
    }
}


Thanks,
-Aleksey



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

David Holmes-6
In reply to this post by Gregg Wonderly-3

All fields hold strong references, not just static fields. It is only locals/stack-variables that don’t guarantee reachability.

 

David

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gregg Wonderly
Sent: Thursday, October 22, 2015 8:10 AM
To: Vitaly Davidovich
Cc: thurstonn; [hidden email]
Subject: Re: [concurrency-interest] Is Reference.reachabilityFence() needed in Reference constructor?

 

 

On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:

 

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

 

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

 

That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.

 

Gregg Wonderly



 

On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich
In reply to this post by Aleksey Shipilev-2
This is also the case on the CLR, and it produces "interesting" scenarios such as finalizer running concurrent to an instance method: http://blogs.msdn.com/b/oldnewthing/archive/2010/08/10/10048149.aspx.  In .NET, there's a GC.KeepAlive(obj) method to extend liveness manually for cases where it's needed.


On Wed, Oct 21, 2015 at 6:04 PM, Aleksey Shipilev <[hidden email]> wrote:
No, they aren't. That's a common misconception.

https://docs.oracle.com/javase/specs/jls/se8/html/jls-12.html#jls-12.6.1:

"Optimizing transformations of a program can be designed that reduce the
number of objects that are reachable to be less than those which would
naively be considered reachable. For example, a Java compiler or code
generator may choose to set a variable or parameter that will no longer
be used to null to cause the storage for such an object to be
potentially reclaimable sooner."


Thanks,
-Aleksey

On 10/21/2015 11:29 PM, thurstonn wrote:
> But the referent arg is on the stack (since the constructor is)
> Aren't all variables on the stack strongly reachable?
>
>
>
> --
> View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
> Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich
In reply to this post by David Holmes-6
All fields hold strong references, not just static fields. It is only locals/stack-variables that don’t guarantee reachability.

I'm not sure that's a useful distinction.  Fields hold a strong reference only if the enclosing class is reachable.  Likewise, stack/register references are strong if they're deemed live by the runtime (interpreter or compiler).  Static fields are nothing special other than belonging to a ClassLoader (if the ClassLoader is unreachable, so are those static fields).  It's turtles all the way down :).

On Wed, Oct 21, 2015 at 6:54 PM, David Holmes <[hidden email]> wrote:

All fields hold strong references, not just static fields. It is only locals/stack-variables that don’t guarantee reachability.

 

David

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gregg Wonderly
Sent: Thursday, October 22, 2015 8:10 AM
To: Vitaly Davidovich
Cc: thurstonn; [hidden email]
Subject: Re: [concurrency-interest] Is Reference.reachabilityFence() needed in Reference constructor?

 

 

On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:

 

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

 

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

 

That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.

 

Gregg Wonderly



 

On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Gregg Wonderly-3
In reply to this post by Gregg Wonderly-3
Yes David, but the object holding a strong reference can not guarantee that itself is strongly referenced.  So, the reference graph could result in the reference never being queued if any part of the graph is not strongly referenced.  We’ve gone through this before on this list with a use of PhantomReference that had exactly this problem.  I had to establish a complete strong reference graph to make the class work in production and that made it impossible to create a “utility” class because the use had to have very explicit reference semantics.

Gregg

On Oct 21, 2015, at 5:54 PM, David Holmes <[hidden email]> wrote:

All fields hold strong references, not just static fields. It is only locals/stack-variables that don’t guarantee reachability.
 
David
 
From: [hidden email] [[hidden email]] On Behalf Of Gregg Wonderly
Sent: Thursday, October 22, 2015 8:10 AM
To: Vitaly Davidovich
Cc: thurstonn; [hidden email]
Subject: Re: [concurrency-interest] Is Reference.reachabilityFence() needed in Reference constructor?
 
 
On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:
 
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?
 
No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.
 
That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.
 
Gregg Wonderly


 
On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:
But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
 
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

David M. Lloyd-3
In reply to this post by Vitaly Davidovich
In fact this exact scenario has been observed in OpenJDK as well, hence
the reason for having reachabilityFence() in the first place.

On 10/21/2015 05:32 PM, Vitaly Davidovich wrote:

> This is also the case on the CLR, and it produces "interesting"
> scenarios such as finalizer running concurrent to an instance method:
> http://blogs.msdn.com/b/oldnewthing/archive/2010/08/10/10048149.aspx.
> In .NET, there's a GC.KeepAlive(obj) method to extend liveness manually
> for cases where it's needed.
>
>
> On Wed, Oct 21, 2015 at 6:04 PM, Aleksey Shipilev
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     No, they aren't. That's a common misconception.
>
>     https://docs.oracle.com/javase/specs/jls/se8/html/jls-12.html#jls-12.6.1:
>
>     "Optimizing transformations of a program can be designed that reduce the
>     number of objects that are reachable to be less than those which would
>     naively be considered reachable. For example, a Java compiler or code
>     generator may choose to set a variable or parameter that will no longer
>     be used to null to cause the storage for such an object to be
>     potentially reclaimable sooner."
>
>
>     Thanks,
>     -Aleksey
>
>     On 10/21/2015 11:29 PM, thurstonn wrote:
>     > But the referent arg is on the stack (since the constructor is)
>     > Aren't all variables on the stack strongly reachable?
>     >
>     >
>     >
>     > --
>     > View this message in context:http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
>     > Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
>     > _______________________________________________
>     > Concurrency-interest mailing list
>     >[hidden email]
>     <mailto:[hidden email]>
>     >http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>     >
>
>
>
>     _______________________________________________
>     Concurrency-interest mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>

--
- DML
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

David Holmes-6
In reply to this post by Gregg Wonderly-3

Yes but the point seems somewhat moot. If object A knows it has a strong reference to object B and wants to make a weak-reference to B, then it hardly matters whether B is already not strongly reachable.

 

The whole ReachabilityFence is really just a local fix to say “if this object is already strongly reachable, don’t let it become less reachable due to this piece of code”.

 

David

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gregg Wonderly
Sent: Thursday, October 22, 2015 9:12 AM
To: [hidden email]
Cc: thurstonn; [hidden email]
Subject: Re: [concurrency-interest] Is Reference.reachabilityFence() needed in Reference constructor?

 

Yes David, but the object holding a strong reference can not guarantee that itself is strongly referenced.  So, the reference graph could result in the reference never being queued if any part of the graph is not strongly referenced.  We’ve gone through this before on this list with a use of PhantomReference that had exactly this problem.  I had to establish a complete strong reference graph to make the class work in production and that made it impossible to create a “utility” class because the use had to have very explicit reference semantics.

 

Gregg

 

On Oct 21, 2015, at 5:54 PM, David Holmes <[hidden email]> wrote:

 

All fields hold strong references, not just static fields. It is only locals/stack-variables that don’t guarantee reachability.

 

David

 

From: [hidden email] [[hidden email]] On Behalf Of Gregg Wonderly
Sent: Thursday, October 22, 2015 8:10 AM
To: Vitaly Davidovich
Cc: thurstonn; [hidden email]
Subject: Re: [concurrency-interest] Is Reference.reachabilityFence() needed in Reference constructor?

 

 

On Oct 21, 2015, at 4:44 PM, Vitaly Davidovich <[hidden email]> wrote:

 

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?

 

No.  The stacks are scanned and oopmaps are inspected to find live objects, which then serve as the root of a reachability graph, but mere presence of an object on the stack does not mean it's reachable for GC purposes.

 

That seems highly problematic for WeakReference and company to be usable.  How can any object using those APIs guarantee that the object itself is strongly reachable?  Doesn’t seem like they can.  Strong references can only ever be guaranteed by using a static reference, which makes for non-static environments to be really difficult to “develop” in.

 

Gregg Wonderly




 

On Wed, Oct 21, 2015 at 4:29 PM, thurstonn <[hidden email]> wrote:

But the referent arg is on the stack (since the constructor is)
Aren't all variables on the stack strongly reachable?



--
View this message in context: http://jsr166-concurrency.10961.n7.nabble.com/Is-Reference-reachabilityFence-needed-in-Reference-constructor-tp12819p12824.html
Sent from the JSR166 Concurrency mailing list archive at Nabble.com.
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

 


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Peter Levart
In reply to this post by Vitaly Davidovich
Hi,

Thanks for the insight.

I'd like to note that I'm not concerned about the reachability of the Reference object itself (program code takes care of that - for example in Cleaner API it hooks Reference objects into a doubly-linked list exactly for them to stay reachable until discovered by GC and enqueued by ReferenceHandler thread into the associated ReferenceQueue). I'm merely interested in the reachability of the referent until the Reference (and any possible Reference subclass constructor) fully constructs the Reference instance.

The situation where this would manifest as a malfunction is improbable (as the referent is usually passed to other parts of code too and is reachable from elsewhere for some time). But in case it isn't, the following is possible:

    Reference(T referent, ReferenceQueue<? super T> queue) {
        this.referent = referent;
        // - safepoint with GC happens, 'referent' is found weakly-reachable, 'this' Reference is hooked on the pending chain
        // - ReferenceHandler thread unhooks 'this' from pending chain and tries to enqueue it, but this.queue is still null
        // - BANG! NPE in Referencehandler thread which terminates it!
        this.queue = (queue == null) ? ReferenceQueue.NULL : queue;
    }

A pre-requisite for above scenario is a safepoint between "this.referent = referent" and "this.queue = ...". In interpreter, safepoint check is after every bytecode instruction right? But in Interpreter, I think arguments are also kept on the stack and are therefore reachable at least until the end of constructor. How probable is a safepoint between those two statements in JIT-ed code? Maybe the Reference constructor itself is not in danger because it is very trivial, but what about subclasses that must do their own part of initialization too (for example Cleaner API discussed on core-libs-dev)?

Regards, Peter

On 10/22/2015 12:25 AM, Vitaly Davidovich wrote:
Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref)

If you look at escape analysis code (http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/a60bd3d34158/src/share/vm/opto/escape.cpp#l803), Reference and subclasses get special treatment and marked as GlobalEscape.

On Wed, Oct 21, 2015 at 1:08 PM, Aleksey Shipilev <[hidden email]> wrote:
On 10/21/2015 06:53 PM, Andrew Haley wrote:
> On 10/21/2015 04:44 PM, Aleksey Shipilev wrote:
>> Of course, this does not tell another (scary) part of the story, what if
>> the Reference itself is not discovered as strong ref by GC, e.g. when it
>> isn't published on heap, or scalarized by compiler, etc.
>> reachabilityFence, as currently implemented, extends the "liveness"
>> scope of the local variable, but does not convey anything special to the
>> GC/runtime otherwise.
>
> If the Reference itself is not reachable from a strong root then the
> Reference is dead so we don't care what happens to it.  But if the
> Reference is put on a reachable ReferenceQueue, then it's fine.

Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref), or those references are dead (e.g. the
weakref is buried somewhere deep in garbage subgraph, and purged on
sweep), then everything goes awry.

But the thing is, a WeakReference is put on ReferenceQueue by the GC
itself. In the object graph, ReferenceQueue does not normally reference
weakrefs back. Before enqueueing the reference GC has to first discover
that WeakReference -- but from where? In other words, "registering" on a
reachable RefQueue does not make weakref to be reachable.

Case in point:

public class WhereIsWaldo {

    public static void main(String... args) throws Exception {
        for (int c = 0; true; c++) {
            new WhereIsWaldo().work(c);
        }
    }

    final ReferenceQueue<Object> rq = new ReferenceQueue<>();

    void work(int id) throws Exception {
        hideWaldo(id);
        findWaldo(id);
    }

    WeakReference<Object> gwr;

    void hideWaldo(int id) {
        // Does not work, d'uh
        new WeakReference<>(new Object(), rq);

        // Does not work either :(
        WeakReference<Object> wr =
           new WeakReference<>(new Object(), rq);

        // This also does not help... once you leave the method,
        // the weakref is gone.
        Reference.reachabilityFence(wr);

        // This one helps, makes Waldo reachable
        // gwr = wr;
    }

    void findWaldo(int id) throws Exception {
        while (Thread.currentThread().isAlive()) {
            System.gc();
            Reference ref = rq.remove(1000);
            if (ref != null) {
                return;
            } else {
                System.out.println("Where's Waldo #" + id + "?");
            }
        }
    }
}


Thanks,
-Aleksey




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

Vitaly Davidovich

It's unlikely there would be a safepoint between those two statements in compiled code.  Polls are typically placed at method exit (modulo polls inserted for some types of loops, but that's irrelevant to this instance).  With inlining, the poll distance is likely to increase as well.  However, there's nothing stopping compiler from scheduling these two statements sufficiently apart, particularly since there's no data or control flow dependence between them, and ending up with a safepoint in between.  It's unlikely probably, but possible.  The bottom line is this code needs manual lifetime extension facility.

sent from my phone

On Oct 22, 2015 7:22 AM, "Peter Levart" <[hidden email]> wrote:
Hi,

Thanks for the insight.

I'd like to note that I'm not concerned about the reachability of the Reference object itself (program code takes care of that - for example in Cleaner API it hooks Reference objects into a doubly-linked list exactly for them to stay reachable until discovered by GC and enqueued by ReferenceHandler thread into the associated ReferenceQueue). I'm merely interested in the reachability of the referent until the Reference (and any possible Reference subclass constructor) fully constructs the Reference instance.

The situation where this would manifest as a malfunction is improbable (as the referent is usually passed to other parts of code too and is reachable from elsewhere for some time). But in case it isn't, the following is possible:

    Reference(T referent, ReferenceQueue<? super T> queue) {
        this.referent = referent;
        // - safepoint with GC happens, 'referent' is found weakly-reachable, 'this' Reference is hooked on the pending chain
        // - ReferenceHandler thread unhooks 'this' from pending chain and tries to enqueue it, but this.queue is still null
        // - BANG! NPE in Referencehandler thread which terminates it!
        this.queue = (queue == null) ? ReferenceQueue.NULL : queue;
    }

A pre-requisite for above scenario is a safepoint between "this.referent = referent" and "this.queue = ...". In interpreter, safepoint check is after every bytecode instruction right? But in Interpreter, I think arguments are also kept on the stack and are therefore reachable at least until the end of constructor. How probable is a safepoint between those two statements in JIT-ed code? Maybe the Reference constructor itself is not in danger because it is very trivial, but what about subclasses that must do their own part of initialization too (for example Cleaner API discussed on core-libs-dev)?

Regards, Peter

On 10/22/2015 12:25 AM, Vitaly Davidovich wrote:
Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref)

If you look at escape analysis code (http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/a60bd3d34158/src/share/vm/opto/escape.cpp#l803), Reference and subclasses get special treatment and marked as GlobalEscape.

On Wed, Oct 21, 2015 at 1:08 PM, Aleksey Shipilev <[hidden email]> wrote:
On 10/21/2015 06:53 PM, Andrew Haley wrote:
> On 10/21/2015 04:44 PM, Aleksey Shipilev wrote:
>> Of course, this does not tell another (scary) part of the story, what if
>> the Reference itself is not discovered as strong ref by GC, e.g. when it
>> isn't published on heap, or scalarized by compiler, etc.
>> reachabilityFence, as currently implemented, extends the "liveness"
>> scope of the local variable, but does not convey anything special to the
>> GC/runtime otherwise.
>
> If the Reference itself is not reachable from a strong root then the
> Reference is dead so we don't care what happens to it.  But if the
> Reference is put on a reachable ReferenceQueue, then it's fine.

Yes, if runtime stripes that instance before it is exposed anywhere
(e.g. scalarizes the weakref), or those references are dead (e.g. the
weakref is buried somewhere deep in garbage subgraph, and purged on
sweep), then everything goes awry.

But the thing is, a WeakReference is put on ReferenceQueue by the GC
itself. In the object graph, ReferenceQueue does not normally reference
weakrefs back. Before enqueueing the reference GC has to first discover
that WeakReference -- but from where? In other words, "registering" on a
reachable RefQueue does not make weakref to be reachable.

Case in point:

public class WhereIsWaldo {

    public static void main(String... args) throws Exception {
        for (int c = 0; true; c++) {
            new WhereIsWaldo().work(c);
        }
    }

    final ReferenceQueue<Object> rq = new ReferenceQueue<>();

    void work(int id) throws Exception {
        hideWaldo(id);
        findWaldo(id);
    }

    WeakReference<Object> gwr;

    void hideWaldo(int id) {
        // Does not work, d'uh
        new WeakReference<>(new Object(), rq);

        // Does not work either :(
        WeakReference<Object> wr =
           new WeakReference<>(new Object(), rq);

        // This also does not help... once you leave the method,
        // the weakref is gone.
        Reference.reachabilityFence(wr);

        // This one helps, makes Waldo reachable
        // gwr = wr;
    }

    void findWaldo(int id) throws Exception {
        while (Thread.currentThread().isAlive()) {
            System.gc();
            Reference ref = rq.remove(1000);
            if (ref != null) {
                return;
            } else {
                System.out.println("Where's Waldo #" + id + "?");
            }
        }
    }
}


Thanks,
-Aleksey




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Is Reference.reachabilityFence() needed in Reference constructor?

thurstonn
In reply to this post by Aleksey Shipilev-2
Well, you learn something new every day.

I wrote some simple tests to confirm Aleksey's claim, and they do.

In fact, Peter's original code could be modified to an even more surprising:


Object obj = new Object();
WeakReference<?> wr = new WeakReference(obj)

. . .
assert null != wr.get() : "This can fail"

I might suggest that the doc for reference package is not ideal:

"An object is strongly reachable if it can be reached by some thread without traversing any reference objects. A newly-created object is strongly reachable by the thread that created it."

Depending on how you parse it, it may not be wrong, but is misleading
12