Quantcast

Persist data across CompletableFutures

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Persist data across CompletableFutures

insider
I feel that there should be some standard way of persisting data across single chain of CompletableFutures (something similar to ThreadLocal only with different scope). I believe C# has this as System.Threading.AsyncLocal<T>

What's your thoughts on this?

--
Best regards,
Nikolay

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Persist data across CompletableFutures

Per Mildner

On 14 Jan 2017, at 06:07, insider <[hidden email]> wrote:

I feel that there should be some standard way of persisting data across single chain of CompletableFutures (something similar to ThreadLocal only with different scope). I believe C# has this as System.Threading.AsyncLocal<T>

What's your thoughts on this?

I asked a related question, with a proof-of-concept implementation, last year, see <http://cs.oswego.edu/pipermail/concurrency-interest/2016-August/015323.html>. This did not get much feedback, unfortunately.

We use something like this, and it seems to work well. The main, huge, drawback is that our implementation can not pass the “task-tree-local variables” to tasks created by the Java 8 concurrent streams etc..

I would love to see something like this implemented in a way that makes it work with all the built-in fork/join features in Java 8.

Regards,


--
Best regards,
Nikolay
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

Per Mildner                                     [hidden email]
Swedish Institute of Computer Science


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Persist data across CompletableFutures

Viktor Klang
Hey,

I apologize in advance if this comes across as negative, I promised myself to not complain in 2017 so I hope you see this as a bit of a "tales from the trenches" report. :)

It's possible to build a feature like that with Scala Futures since the `ExecutionContext` (think Executor) has a `prepare()`-method[1] which can be used to store away thread-local state to be restored once the Runnable is actually executed.

However, there are a couple of concerns:

* What happens to forks? (tasks split into multiple subtasks)
   – because without immutable or coordinated data in those "locals" Here Be Dragons
* What happens to joins? (tasks coalesced into fewer tasks)
   – because which of the values will you use? What if they have both been modified? Or just are different?
* How do you reason about memory consumption when you can have possibly millions of async tasks being queued up? What's the visibility into this "on the side" storage?
* What's the implication for nested execution? (i.e. if some Executor executes the code synchronously)
* A single Executors' failure to support this feature means that things just stop working and it might be really hard to know why, especially for libraries that have embedded, opaque, thread pools.


TL;DR:

In my experience the value of said feature doesn't really carry its weight since it tends to lead to non-deterministic failure cases with very limited opportunity for the developer to figure out what went wrong.

I'd love to see alternate proposals for solving the "transient context" problem, but I believe the appropriate solution is true continuations on the JVM. *fingers crossed*

1: Note that I deprecated that method for 2.12.0 since it is extremely tricky for developers of custom Executors to remember to use it, and use it in the right places the right number of times.


On Sat, Jan 14, 2017 at 10:53 AM, Per Mildner <[hidden email]> wrote:

On 14 Jan 2017, at 06:07, insider <[hidden email]> wrote:

I feel that there should be some standard way of persisting data across single chain of CompletableFutures (something similar to ThreadLocal only with different scope). I believe C# has this as System.Threading.AsyncLocal<T>

What's your thoughts on this?

I asked a related question, with a proof-of-concept implementation, last year, see <http://cs.oswego.edu/pipermail/concurrency-interest/2016-August/015323.html>. This did not get much feedback, unfortunately.

We use something like this, and it seems to work well. The main, huge, drawback is that our implementation can not pass the “task-tree-local variables” to tasks created by the Java 8 concurrent streams etc..

I would love to see something like this implemented in a way that makes it work with all the built-in fork/join features in Java 8.

Regards,


--
Best regards,
Nikolay
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

Per Mildner                                     [hidden email]
Swedish Institute of Computer Science


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest




--
Cheers,

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Loading...