Thread local contexts and CompletableFutures

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Thread local contexts and CompletableFutures

James Roper
Hi All,

This is my first post to this list, so let me first introduce myself.  I work for Typesafe as the tech lead on Play Framework.

Play Framework is an asynchronous web framework that supports both Scala and Java applications.  For Java, we currently have our own Promise API which is heavily used by users of our framework.  This is backed by the Scala Future API, and uses it's own SAM interfaces for passing functions so it's lambda ready.  However, we would very much like to use the CompletableFuture API instead, as this will allow seamless interoperability with other Java libraries that use this API.  Right now we are working out a plan for how we can move towards it (given that for a time, we still need to support Java 6 and 7).

However there is an important feature that we would need for the CompletableFuture API to be useful to us.  Thread locals are very commonly used in Java frameworks.  In particular, standard Java APIs such as JPA use them to store transaction context, they're used for storing a security context in JEE, and almost anything that works directly with classloaders, including using reflection or any sort of byte code generation, must use the context classloader in most deployment environments, in particular in a servlet container.  Play also makes heavy use of thread locals when using the Java API, particularly for holding request context, when using JPA, and the context classloader is also very important with many of the popular Java third party libraries that we use.

As far as I can see, and correct me if I'm wrong, there is nothing in the CompletableFuture API that addresses this, or allows it to be addressed.  This will seriously inhibit the usefulness of CompletableFutures in many typical deployment scenarios, most obviously in a JEE container, and also where I am most concerned, in Play Framework.

In contrast, our existing Promise API can easily address this, because Scala's ExecutionContext (roughly equivalent to the ExecutorService that gets passed in with callbacks to the CompletableFuture API) has a prepare method that returns a new (or the same) execution context.  APIs that accept callbacks to be executed in a given execution context then invoke the prepare method when the callback is submitted, which allows a thread local aware ExecutionContext to capture the current state, and set it in the callbacks thread when the callback is executed.  This feature I think is far more important to Java code than it is to Scala code though, since Java frameworks typically make much more use of thread locals and reflection.

This allows a transparent way of ensuring that callbacks execute in the right thread context.  The management of this context is then also managed by the same resource manager that manages the threads themselves, and this seems like the right place for it to be.

So has this concern been discussed or addressed before?  Any thoughts?  Could ExecutorService have a similar method added to it?  Or would it make sense to have another interface that sat above ExecutorService that managed this?

One way that it could managed is by convention in user code, instead of user code having an ExecutorService that it passed to CompletableFuture methods, it could have a factory/builder class that produced an ExecutorService, and the user would be required to invoke this themselves each time they passed it, and not hold references to the produced ExecutorService.  From a syntax point of view the overhead is not too high, but I personally don't think it's a good idea to force a convention like this.

Regards,

James Roper

--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Thread local contexts and CompletableFutures

Doug Lea
On 05/02/13 03:36, James Roper wrote:

>
> However there is an important feature that we would need for the
> CompletableFuture API to be useful to us.  Thread locals are very commonly used
> in Java frameworks.
>
> As far as I can see, and correct me if I'm wrong, there is nothing in the
> CompletableFuture API that addresses this, or allows it to be addressed.
>
> In contrast, our existing Promise API can easily address this, because Scala's
> ExecutionContext (roughly equivalent to the ExecutorService that gets passed in
> with callbacks to the CompletableFuture API) has a prepare method that returns a
> new (or the same) execution context.

This seems possible using the current API.
All async completion methods have a form accepting an Executor,
that could be some form of your ExecutionContext.
You might want to layer on something ensuring this.
Additionally, if it is possible for direct (non-async) execution
to also use a form of ExecutionContext, then you'd need
to use the async form in these cases as well, but supply
a version of ExecutionContext that directly executes after
establishing context. (in other words, there's nothing about
the "async" methods that strictly requires asynchrony.)
It's conceivable that there is some case not handled by these
options, but I don't see any offhand.

-Doug

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Thread local contexts and CompletableFutures

James Roper
On Thu, May 2, 2013 at 8:54 PM, Doug Lea <[hidden email]> wrote:
On 05/02/13 03:36, James Roper wrote:

However there is an important feature that we would need for the
CompletableFuture API to be useful to us.  Thread locals are very commonly used
in Java frameworks.

As far as I can see, and correct me if I'm wrong, there is nothing in the
CompletableFuture API that addresses this, or allows it to be addressed.

In contrast, our existing Promise API can easily address this, because Scala's
ExecutionContext (roughly equivalent to the ExecutorService that gets passed in
with callbacks to the CompletableFuture API) has a prepare method that returns a
new (or the same) execution context.

This seems possible using the current API.
All async completion methods have a form accepting an Executor,
that could be some form of your ExecutionContext.
You might want to layer on something ensuring this.
Additionally, if it is possible for direct (non-async) execution
to also use a form of ExecutionContext, then you'd need
to use the async form in these cases as well, but supply
a version of ExecutionContext that directly executes after
establishing context. (in other words, there's nothing about
the "async" methods that strictly requires asynchrony.)
It's conceivable that there is some case not handled by these
options, but I don't see any offhand.

Yes, the executor can definitely should hold the context, but the difficulty is how it gets the context.  If the client code is responsible for instantiating the context, then that's the answer, but ideally, client code would just be able to use an executor provided by whatever container they are in.  For example, currently this wouldn't be possible:

@Inject
public MyService(Executor executor, AsyncDao dao) {
  this.e = executor;
  this.dao = dao;
}

@TransactionAttribute(REQUIRED)
public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., executor);
}

Since nothing interacts with the executor in the original thread, and so it can't capture the context.  On the other hand, this would be feasible:

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., StaticContext.currentExecutor());
}

though has the disadvantage of being static.  The final possibility would be:

@Inject
public MyService(Context context, AsyncDao dao) {
  this.context = context;
  this.dao = dao;
}

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., context.currentExecutor());
}

This solution is not too bad, it's just not as nice since it's not transparent, and requires the user to understand that they must never cache the current executor.

Anyway, I've possibly answered my own questions here, but do any of the above seem to be what you would expect to become a typical usage pattern inside containers that use thread locals?  Eg would this become the way things are done in a possible future asynchronous JEE spec with asynchronous database access?  The important thing that I want to ensure here is that whatever we do in Play framework will be idiomatic for a Java developer, but as yet there are no idioms because this stuff is new.

Cheers,

James
 


-Doug


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Thread local contexts and CompletableFutures

Martin Buchholz-3
What concrete additions to CompletableFuture might you suggest that couldn't be easily done in a layered library?  The current API is conceptually rather simple.


On Thu, May 2, 2013 at 3:34 PM, James Roper <[hidden email]> wrote:
On Thu, May 2, 2013 at 8:54 PM, Doug Lea <[hidden email]> wrote:
On 05/02/13 03:36, James Roper wrote:

However there is an important feature that we would need for the
CompletableFuture API to be useful to us.  Thread locals are very commonly used
in Java frameworks.

As far as I can see, and correct me if I'm wrong, there is nothing in the
CompletableFuture API that addresses this, or allows it to be addressed.

In contrast, our existing Promise API can easily address this, because Scala's
ExecutionContext (roughly equivalent to the ExecutorService that gets passed in
with callbacks to the CompletableFuture API) has a prepare method that returns a
new (or the same) execution context.

This seems possible using the current API.
All async completion methods have a form accepting an Executor,
that could be some form of your ExecutionContext.
You might want to layer on something ensuring this.
Additionally, if it is possible for direct (non-async) execution
to also use a form of ExecutionContext, then you'd need
to use the async form in these cases as well, but supply
a version of ExecutionContext that directly executes after
establishing context. (in other words, there's nothing about
the "async" methods that strictly requires asynchrony.)
It's conceivable that there is some case not handled by these
options, but I don't see any offhand.

Yes, the executor can definitely should hold the context, but the difficulty is how it gets the context.  If the client code is responsible for instantiating the context, then that's the answer, but ideally, client code would just be able to use an executor provided by whatever container they are in.  For example, currently this wouldn't be possible:

@Inject
public MyService(Executor executor, AsyncDao dao) {
  this.e = executor;
  this.dao = dao;
}

@TransactionAttribute(REQUIRED)
public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., executor);
}

Since nothing interacts with the executor in the original thread, and so it can't capture the context.  On the other hand, this would be feasible:

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., StaticContext.currentExecutor());
}

though has the disadvantage of being static.  The final possibility would be:

@Inject
public MyService(Context context, AsyncDao dao) {
  this.context = context;
  this.dao = dao;
}

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., context.currentExecutor());
}

This solution is not too bad, it's just not as nice since it's not transparent, and requires the user to understand that they must never cache the current executor.

Anyway, I've possibly answered my own questions here, but do any of the above seem to be what you would expect to become a typical usage pattern inside containers that use thread locals?  Eg would this become the way things are done in a possible future asynchronous JEE spec with asynchronous database access?  The important thing that I want to ensure here is that whatever we do in Play framework will be idiomatic for a Java developer, but as yet there are no idioms because this stuff is new.

Cheers,

James
 


-Doug


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Thread local contexts and CompletableFutures

James Roper
On Fri, May 3, 2013 at 1:58 PM, Martin Buchholz <[hidden email]> wrote:
What concrete additions to CompletableFuture might you suggest that couldn't be easily done in a layered library?  The current API is conceptually rather simple.

Are you suggesting that we layer something on top of CompletableFuture to do this?  That won't do, since the API that created the CompletableFuture won't know anything about the context that the callbacks passed to it need.

Consider this code:

HttpClient httpClient = ...
DatabaseClient databaseClient = ...

CompletableFuture<Response> httpResponse = httpClient.makeRequest("http://foo");
httpResponse.thenComposeAsync(response -> databaseClient.makeQuery(response.getBody));

The callback passed to thenComposeAsync needs to be run with the transaction context setup.  But this is on a CompletableFuture returned by the HttpClient, what does HttpClient know about transaction contexts?  It can't wrap the future in one that will handle this, since it doesn't nor should it know anything about the contexts of the callbacks passed to it.  Layering on top of CompletableFuture is definitely not an option.

I wouldn't suggest changing the CompletableFuture API.  What I would suggest is adding something like:

Excecutor captureContext();

To java.util.concurrent.Executor, and having every call that accepted an executor in the CompletableFuture API call it.  Then layering can be transparently done on top of the Executor API to handle this.  Though, this would mean introducing the concept of context to executor, which currently doesn't exist.



On Thu, May 2, 2013 at 3:34 PM, James Roper <[hidden email]> wrote:
On Thu, May 2, 2013 at 8:54 PM, Doug Lea <[hidden email]> wrote:
On 05/02/13 03:36, James Roper wrote:

However there is an important feature that we would need for the
CompletableFuture API to be useful to us.  Thread locals are very commonly used
in Java frameworks.

As far as I can see, and correct me if I'm wrong, there is nothing in the
CompletableFuture API that addresses this, or allows it to be addressed.

In contrast, our existing Promise API can easily address this, because Scala's
ExecutionContext (roughly equivalent to the ExecutorService that gets passed in
with callbacks to the CompletableFuture API) has a prepare method that returns a
new (or the same) execution context.

This seems possible using the current API.
All async completion methods have a form accepting an Executor,
that could be some form of your ExecutionContext.
You might want to layer on something ensuring this.
Additionally, if it is possible for direct (non-async) execution
to also use a form of ExecutionContext, then you'd need
to use the async form in these cases as well, but supply
a version of ExecutionContext that directly executes after
establishing context. (in other words, there's nothing about
the "async" methods that strictly requires asynchrony.)
It's conceivable that there is some case not handled by these
options, but I don't see any offhand.

Yes, the executor can definitely should hold the context, but the difficulty is how it gets the context.  If the client code is responsible for instantiating the context, then that's the answer, but ideally, client code would just be able to use an executor provided by whatever container they are in.  For example, currently this wouldn't be possible:

@Inject
public MyService(Executor executor, AsyncDao dao) {
  this.e = executor;
  this.dao = dao;
}

@TransactionAttribute(REQUIRED)
public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., executor);
}

Since nothing interacts with the executor in the original thread, and so it can't capture the context.  On the other hand, this would be feasible:

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., StaticContext.currentExecutor());
}

though has the disadvantage of being static.  The final possibility would be:

@Inject
public MyService(Context context, AsyncDao dao) {
  this.context = context;
  this.dao = dao;
}

public CompletableFuture<Foo> getFoo(long id) {
  return dao.get(id).thenApplyAsync(...., context.currentExecutor());
}

This solution is not too bad, it's just not as nice since it's not transparent, and requires the user to understand that they must never cache the current executor.

Anyway, I've possibly answered my own questions here, but do any of the above seem to be what you would expect to become a typical usage pattern inside containers that use thread locals?  Eg would this become the way things are done in a possible future asynchronous JEE spec with asynchronous database access?  The important thing that I want to ensure here is that whatever we do in Play framework will be idiomatic for a Java developer, but as yet there are no idioms because this stuff is new.

Cheers,

James
 


-Doug


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest





--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Thread local contexts and CompletableFutures

deanhiller
This post has NOT been accepted by the mailing list yet.
I am also running into this issue.  I come from the world of twitter Future.scala and Local.scala.  I believe the jdk could benefit hugely from a Local.java that CompleteableFuture knows about.  The CompleteableFuture api would remain the same.

The thing that breaks is stuff like logback's and log4j's MDC.  Previously when everything was synchronous, one would use the MDC like this(and it was backed by a ThreadLocal)

  1. begin of every request.... MDC.put("requestId", someId)
  2. log.info("request url="+url);
  3. log.info("request complete");

The output of the logs was something like

  INFO {time} requestId524 request url=/path
  INFO {time} requestId524 request complete

Now in scala, all we do is modify the MDC to use Local instead of ThreadLocal and with twitter Future, this logging still works without users having to 'remember' to do anything special.  The MDC was a great way to put attach a request id to every log.info statement so the developer didn't have to remember to do it(we tried that way once and people always forget to add it).

Sooo, how to implemet 1, 2, 3 above in a CompleteableFuture environment 'without the user having to remember to use special future.thenApply(someMethod, somespecialExecutor) every time.  At twitter, no one 'has to remember anything and it just works'

I think we really need the same thing in java personally and it would require modifying the internals of CompleteableFuture (and not needing to change the api as far as I can tell).