map over j.u.c.CompletableFuture with strict threading semantics

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

map over j.u.c.CompletableFuture with strict threading semantics

Dimitar Georgiev
Sorry if this has already been discussed.

I need to implement the following function:

public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
upstream, Function<A, B> f);

It's pretty apparent what the function does. However, it has the
requirement that f() be always executed in the thread where upstream
is completed. (That is, if upstream is completed non-exceptionally,
since if it was not, f will not be called so there is no threading
semantics concerning f)

Is this possible with j.u.c.CompletableFuture?

Regards, Dimitar
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Viktor Klang
Hi Dimitar,

In general this is not possible, nor desirable, since completion is likely to happen *before* a transformation is added.

However, there is nothing technical which prevents you to create an implementation of CompletionStage which stores a reference to the Executor to be used, and runs transformations which are applied to it on that Executor unless some other Executor is specified.

--
Cheers,


On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]> wrote:
Sorry if this has already been discussed.

I need to implement the following function:

public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
upstream, Function<A, B> f);

It's pretty apparent what the function does. However, it has the
requirement that f() be always executed in the thread where upstream
is completed. (That is, if upstream is completed non-exceptionally,
since if it was not, f will not be called so there is no threading
semantics concerning f)

Is this possible with j.u.c.CompletableFuture?

Regards, Dimitar
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Dimitar Georgiev
Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...






On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:

> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Viktor Klang
Dimitar,

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

Pseudocode (since I'm having lunch):
enum Trigger { On; }
CompletableFuture<X> upstream = …
CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

YMMV,




On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:
Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...






On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Millies, Sebastian

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,


Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Viktor Klang
Hi Sebastian,

On Sat, Oct 21, 2017 at 2:01 PM, Millies, Sebastian <[hidden email]> wrote:

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?


Sorry, on my way to the airport, and I might have misunderstood your question, but think about "Which thread adds the transformations" vs "Which thread executes the transformations". (There needs to be a happens-before relationship between "all transformations already added" and "intended executor executes the value which produces the first result"—so that all the transformations are piggybacked on that executor and not the thread which adds the transformation.)

Cheers,
 

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,


Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Millies, Sebastian

I see. What I meant was, why do you need to establish that happens-before relationship in such a complicated way if all you want to do is to execute a transformation in some given executor known in advance? Instead, can’t you just as well supply that executor to one of the async methods in CF in each of the downstream transformations?

 

From: Viktor Klang [mailto:[hidden email]]
Sent: Saturday, October 21, 2017 6:06 PM
To: Millies, Sebastian
Cc: Dimitar Georgiev; concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Hi Sebastian,

 

On Sat, Oct 21, 2017 at 2:01 PM, Millies, Sebastian <[hidden email]> wrote:

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?

 

Sorry, on my way to the airport, and I might have misunderstood your question, but think about "Which thread adds the transformations" vs "Which thread executes the transformations". (There needs to be a happens-before relationship between "all transformations already added" and "intended executor executes the value which produces the first result"—so that all the transformations are piggybacked on that executor and not the thread which adds the transformation.)

 

Cheers,

 

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,

 

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com



 

--

Cheers,


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Viktor Klang
You can of course put the executor explicitly everywhere you want, but that wasn't the OPs question. 😊

--
Cheers,

On Oct 22, 2017 14:08, "Millies, Sebastian" <[hidden email]> wrote:

I see. What I meant was, why do you need to establish that happens-before relationship in such a complicated way if all you want to do is to execute a transformation in some given executor known in advance? Instead, can’t you just as well supply that executor to one of the async methods in CF in each of the downstream transformations?

 

From: Viktor Klang [mailto:[hidden email]]
Sent: Saturday, October 21, 2017 6:06 PM
To: Millies, Sebastian
Cc: Dimitar Georgiev; concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Hi Sebastian,

 

On Sat, Oct 21, 2017 at 2:01 PM, Millies, Sebastian <[hidden email]> wrote:

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?

 

Sorry, on my way to the airport, and I might have misunderstood your question, but think about "Which thread adds the transformations" vs "Which thread executes the transformations". (There needs to be a happens-before relationship between "all transformations already added" and "intended executor executes the value which produces the first result"—so that all the transformations are piggybacked on that executor and not the thread which adds the transformation.)

 

Cheers,

 

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,

 

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com



 

--

Cheers,


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Dimitar Georgiev
The point is to have an API where forks only ever happen explicitly, i.e. when the programmer returns a future that will complete in a new thread via flatMap, or when explicitly invoking a fork() operator.

In all other cases map() and flatMap() will run in upstream's thread.

This model of threading is, I think, roughly equivalent to scalaz.concurrent.Task, if you are familiar with that. 

 I will push a testcase sometime this week to show more explicitly what I expect, and try the trick Viktor suggested. Thanks everyone! 

On 22 October 2017 at 15:31, Viktor Klang <[hidden email]> wrote:
You can of course put the executor explicitly everywhere you want, but that wasn't the OPs question. 😊

--
Cheers,

On Oct 22, 2017 14:08, "Millies, Sebastian" <[hidden email]> wrote:

I see. What I meant was, why do you need to establish that happens-before relationship in such a complicated way if all you want to do is to execute a transformation in some given executor known in advance? Instead, can’t you just as well supply that executor to one of the async methods in CF in each of the downstream transformations?

 

From: Viktor Klang [mailto:[hidden email]]
Sent: Saturday, October 21, 2017 6:06 PM
To: Millies, Sebastian
Cc: Dimitar Georgiev; concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Hi Sebastian,

 

On Sat, Oct 21, 2017 at 2:01 PM, Millies, Sebastian <[hidden email]> wrote:

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?

 

Sorry, on my way to the airport, and I might have misunderstood your question, but think about "Which thread adds the transformations" vs "Which thread executes the transformations". (There needs to be a happens-before relationship between "all transformations already added" and "intended executor executes the value which produces the first result"—so that all the transformations are piggybacked on that executor and not the thread which adds the transformation.)

 

Cheers,

 

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,

 

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com



 

--

Cheers,



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: map over j.u.c.CompletableFuture with strict threading semantics

Viktor Klang
Dimitar,

Worth noting is that Task is a completely different construct. (Not saying Task is not useful, just saying it is a different construct.)

Also, for context, Scala Futures does not support "run it on the completing thread OR the attaching thread, whichever happens first during execution", as explained here by Havoc Pennington: https://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchronous/

Most times running something on the completing thread is a questionable idea since now it is difficult to reason about which Executor performs what workloads, and when it is safe to shut down the ExecutorService from a lifecycle/ownership perspective as it is being used to execute unknown workloads.

--
Cheers,

On Oct 22, 2017 18:13, "Dimitar Georgiev" <[hidden email]> wrote:
The point is to have an API where forks only ever happen explicitly, i.e. when the programmer returns a future that will complete in a new thread via flatMap, or when explicitly invoking a fork() operator.

In all other cases map() and flatMap() will run in upstream's thread.

This model of threading is, I think, roughly equivalent to scalaz.concurrent.Task, if you are familiar with that. 

 I will push a testcase sometime this week to show more explicitly what I expect, and try the trick Viktor suggested. Thanks everyone! 

On 22 October 2017 at 15:31, Viktor Klang <[hidden email]> wrote:
You can of course put the executor explicitly everywhere you want, but that wasn't the OPs question. 😊

--
Cheers,

On Oct 22, 2017 14:08, "Millies, Sebastian" <[hidden email]> wrote:

I see. What I meant was, why do you need to establish that happens-before relationship in such a complicated way if all you want to do is to execute a transformation in some given executor known in advance? Instead, can’t you just as well supply that executor to one of the async methods in CF in each of the downstream transformations?

 

From: Viktor Klang [mailto:[hidden email]]
Sent: Saturday, October 21, 2017 6:06 PM
To: Millies, Sebastian
Cc: Dimitar Georgiev; concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Hi Sebastian,

 

On Sat, Oct 21, 2017 at 2:01 PM, Millies, Sebastian <[hidden email]> wrote:

Hi,

 

I’m sorry to be so dense, but can you explain what the difference is to simply upstream.thenApplyAsync(f, YOUR_INTENDED_EXECUTOR) ?

 

Sorry, on my way to the airport, and I might have misunderstood your question, but think about "Which thread adds the transformations" vs "Which thread executes the transformations". (There needs to be a happens-before relationship between "all transformations already added" and "intended executor executes the value which produces the first result"—so that all the transformations are piggybacked on that executor and not the thread which adds the transformation.)

 

Cheers,

 

 

n  SebaSTIAN

 

From: Concurrency-interest [mailto:[hidden email]] On Behalf Of Viktor Klang
Sent: Friday, October 20, 2017 8:06 PM
To: Dimitar Georgiev
Cc: concurrency-interest
Subject: Re: [concurrency-interest] map over j.u.c.CompletableFuture with strict threading semantics

 

Dimitar,

 

Just a note: I realized that you can probably get to what you want by controlling injecting a "valve"—CompletableFuture, and only complete that once you want the transformations to be applied:

 

Pseudocode (since I'm having lunch):

enum Trigger { On; }

CompletableFuture<X> upstream = …

CompletableFuture<Trigger> upstreamTrigger = new CompletableFuture<Trigger>()

 

CompletableFuture<Y> result = upstream.thenCombineAsync(upstreamTrigger, (x, discard) -> x, YOUR_INTENDED_EXECUTOR).thenApply(…).thenCompose(…)…

 

upstreamTrigger.complete(Trigger.On); // this will trigger the execution of the transformations on YOUR_INTENDED_EXECUTOR regardless of what transformations are added to `result` as long as they HAPPEN-BEFORE the execution of this line.

 

YMMV,

 

 

 

 

On Fri, Oct 20, 2017 at 5:32 PM, Dimitar Georgiev <[hidden email]> wrote:

Thanks Viktor! I had not thought of it this way, and you are absolutely right.

I hope it is okay to expand the discussion a little bit and go a bit
off-topic from j.u.c. Viktor probably already sees where this is
going. Next is flatMap:

public static <A, B> CompletableFuture<B> flatMap(CompletableFuture<A>
future, Function<A, CompletableFuture<B>> f);

Semantics my app requires here: if the result of f() is completed on a
new threading context, I want subsequent map / flatMap operations to
use that context. If not (if the result of f() is pure, i.e. of the
form new CompletableFuture<>().complete(value)), continue in the same
threading context.

 Viktor, your remark leads me to think there is no sane way to achieve
this semantics with an eager future implementation such as
CompletableFuture. I would need something lazy which is a description,
creating the thing be referentially transparent, and
submission/execution be decoupled from creating the thing.

To finish the off-topic with the actual question:
- Does such a thing exist in the Java ecosystem at all? I know it does
in Scala, but I need a Java alternative.

P.S. I found a couple of months ago https://github.com/traneio/future.
A claim is made that the threading semantics are what I just
described, but it is an eager future. I will look into the
implementation in the coming days; I expect it to be something along
the "workaround" Viktor described. Myself, I don't feel I am smart
enough or have the testing and formal verification tools to go down
that path...







On 20 October 2017 at 17:52, Viktor Klang <[hidden email]> wrote:
> Hi Dimitar,
>
> In general this is not possible, nor desirable, since completion is likely
> to happen *before* a transformation is added.
>
> However, there is nothing technical which prevents you to create an
> implementation of CompletionStage which stores a reference to the Executor
> to be used, and runs transformations which are applied to it on that
> Executor unless some other Executor is specified.
>
> --
> Cheers,
> √
>
>
> On Oct 20, 2017 09:41, "Dimitar Georgiev" <[hidden email]>
> wrote:
>
> Sorry if this has already been discussed.
>
> I need to implement the following function:
>
> public static <A, B> CompletableFuture<B> map(CompletableFuture<A>
> upstream, Function<A, B> f);
>
> It's pretty apparent what the function does. However, it has the
> requirement that f() be always executed in the thread where upstream
> is completed. (That is, if upstream is completed non-exceptionally,
> since if it was not, f will not be called so there is no threading
> semantics concerning f)
>
> Is this possible with j.u.c.CompletableFuture?
>
> Regards, Dimitar
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>



 

--

Cheers,

 

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com



 

--

Cheers,




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest