Contract of java.io.OutputStream.write(byte b[])

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Contract of java.io.OutputStream.write(byte b[])

Elias Ross-5

I know this isn't really the forum for asking this, but I've spent a
number of hours looking into a definite yes or no...

I wanted to know if OutputStream.write(byte b[]) was considered atomic.
If two threads are writing to the same file, would the output from
OS.write(...) overlap with another OS.write(...)?

I eventually downloaded the Java source to answer this question.  The
JavaDoc did not specify.  As it turns out, the answer is "no" -- which
is something that you'd only be able to determine by looking at the C
source code for the native calls.

The JDK interpretation of write() is slightly different than the UNIX
one, which looks like this:

       ssize_t write(int fd, const void *buf, size_t count);

so it makes sense that a Java write() call may have to do more than one
system write() call.

I wonder if there some better way that "concurrent" and "atomic" methods
can be documented in the JDK?  It's also very helpful as a user to know
if I should be locking on the OutputStream itself or create my own
locks.


_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Joe Bowbeer
New annotations have been created for the purpose of documenting which lock guards which state.

Watch this space for @GuardedBy and friends:

  http://jcip.net/

In the near term, I recommend providing your own locks unless you're intimate with the details of the specific OutputStream subclass implementation (on your specific platform).

  synchronized (outLock) {
    out.write(buffer);
  }

If is often the case that the outputstream object can serve as this lock.

In other cases, no explicit locking is needed.  The write method is synchronized in the ByteArrayOutputStream implementation I'm looking at now..

--
Joe Bowbeer ~ http://joebowbeer.thruhere.net



On 5/31/06, Elias Ross <[hidden email]> wrote:

I know this isn't really the forum for asking this, but I've spent a
number of hours looking into a definite yes or no...

I wanted to know if OutputStream.write(byte b[]) was considered atomic.
If two threads are writing to the same file, would the output from
OS.write(...) overlap with another OS.write(...)?

I eventually downloaded the Java source to answer this question.  The
JavaDoc did not specify.  As it turns out, the answer is "no" -- which
is something that you'd only be able to determine by looking at the C
source code for the native calls.

The JDK interpretation of write() is slightly different than the UNIX
one, which looks like this:

       ssize_t write(int fd, const void *buf, size_t count);

so it makes sense that a Java write() call may have to do more than one
system write() call.

I wonder if there some better way that "concurrent" and "atomic" methods
can be documented in the JDK?  It's also very helpful as a user to know
if I should be locking on the OutputStream itself or create my own
locks.




_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

wpugh
In reply to this post by Elias Ross-5
The fact that many of the implementations of OutputStream are  
synchronized was
a pretty horrible design mistake.

If you have two different threads trying to push bytes into an output  
stream, I'm hard pressed to
imagine situations in which allowing the threads to non-
deterministically interleave is a good idea.

If you are creating your own implementation of OutputStream, I would  
recommending avoiding any
use of synchronization. It serves no purpose, reduces performance,  
and I don't believe the class
contract (or Sun's TCK tests) require it.

        Bill Pugh


On May 31, 2006, at 6:33 PM, Elias Ross wrote:

>
> I know this isn't really the forum for asking this, but I've spent a
> number of hours looking into a definite yes or no...
>
> I wanted to know if OutputStream.write(byte b[]) was considered  
> atomic.
> If two threads are writing to the same file, would the output from
> OS.write(...) overlap with another OS.write(...)?
>
> I eventually downloaded the Java source to answer this question.  The
> JavaDoc did not specify.  As it turns out, the answer is "no" -- which
> is something that you'd only be able to determine by looking at the C
> source code for the native calls.
>
> The JDK interpretation of write() is slightly different than the UNIX
> one, which looks like this:
>
>        ssize_t write(int fd, const void *buf, size_t count);
>
> so it makes sense that a Java write() call may have to do more than  
> one
> system write() call.
>
> I wonder if there some better way that "concurrent" and "atomic"  
> methods
> can be documented in the JDK?  It's also very helpful as a user to  
> know
> if I should be locking on the OutputStream itself or create my own
> locks.
>
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Gregg Wonderly-2


Bill Pugh wrote:
> If you have two different threads trying to push bytes into an output  
> stream, I'm hard pressed to
> imagine situations in which allowing the threads to non-
> deterministically interleave is a good idea.

A logging stream is one place where letting the writes happen in any order is
probably okay.  You do want the entire byte[] to go out in one piece, so
synchronizing on the write call is an easy way to do this.

I've counted on the synchronized OutputStream behavior in more than one place,
without second thought that its presense in an API's Javadoc doesn't necessarily
make it a requirement of all implementations extending or using that API.

Humm....

Gregg Wonderly
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Dhanji R. Prasanna
In reply to this post by wpugh
On 6/1/06, Bill Pugh <[hidden email]> wrote:

> The fact that many of the implementations of OutputStream are
> synchronized was
> a pretty horrible design mistake.
>
> If you have two different threads trying to push bytes into an output
> stream, I'm hard pressed to
> imagine situations in which allowing the threads to non-
> deterministically interleave is a good idea.
>
> If you are creating your own implementation of OutputStream, I would
> recommending avoiding any
> use of synchronization. It serves no purpose, reduces performance,
> and I don't believe the class
> contract (or Sun's TCK tests) require it.

that sounds quite sensible.
I would also suggest using wait() and notify() for deterministic
synchronization at the producer level (user of the outputstream) as
opposed to a synchronized block unless the latter is more appropriate
(optimistic stream writes).
In any case isnt it better practise to synchronize on a buffer and
allow only one thread access to the underlying outputstream?

>
>         Bill Pugh
>
>
> On May 31, 2006, at 6:33 PM, Elias Ross wrote:
>
> >
> > I know this isn't really the forum for asking this, but I've spent a
> > number of hours looking into a definite yes or no...
> >
> > I wanted to know if OutputStream.write(byte b[]) was considered
> > atomic.
> > If two threads are writing to the same file, would the output from
> > OS.write(...) overlap with another OS.write(...)?
> >
> > I eventually downloaded the Java source to answer this question.  The
> > JavaDoc did not specify.  As it turns out, the answer is "no" -- which
> > is something that you'd only be able to determine by looking at the C
> > source code for the native calls.
> >
> > The JDK interpretation of write() is slightly different than the UNIX
> > one, which looks like this:
> >
> >        ssize_t write(int fd, const void *buf, size_t count);
> >
> > so it makes sense that a Java write() call may have to do more than
> > one
> > system write() call.
> >
> > I wonder if there some better way that "concurrent" and "atomic"
> > methods
> > can be documented in the JDK?  It's also very helpful as a user to
> > know
> > if I should be locking on the OutputStream itself or create my own
> > locks.
> >
> >
> > _______________________________________________
> > Concurrency-interest mailing list
> > [hidden email]
> > http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
>
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Joe Bowbeer
At the risk of drifting away from the original question, in which
locking of the output stream was the stated goal... see comments
below.

On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:

> I would also suggest using wait() and notify() for deterministic
> synchronization at the producer level (user of the outputstream)
> as opposed to a synchronized block unless the latter is more
> appropriate (optimistic stream writes).

Can you explain further?

While I don't advocate upgrading every "synchronized" to some form of
java.util.concurrent.locks.Lock:

  New lock classes improve on synchronized -- but don't replace it
  http://www-128.ibm.com/developerworks/java/library/j-jtp10264/

I *do* advocate stamping-out all explicit obj.wait()s and
obj.notify()s.  Given all the tools in java.util.concurrent, there's
got to be something better already implemented.


If clean code is the goal, my preference is to avoid explicit locking
in favor or queueing.

In other words, producers insert their byte[]s onto a thread-safe
queue, while an outputter thread pulls byte[]s from the queue and
writes them to the stream.  This separation of concerns results in
very clean code. Furthermore, several flavors of queue have already
been implemented in java.util.concurrent.

--Joe


On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:

> On 6/1/06, Bill Pugh <[hidden email]> wrote:
> > The fact that many of the implementations of OutputStream are
> > synchronized was
> > a pretty horrible design mistake.
> >
> > If you have two different threads trying to push bytes into an output
> > stream, I'm hard pressed to
> > imagine situations in which allowing the threads to non-
> > deterministically interleave is a good idea.
> >
> > If you are creating your own implementation of OutputStream, I would
> > recommending avoiding any
> > use of synchronization. It serves no purpose, reduces performance,
> > and I don't believe the class
> > contract (or Sun's TCK tests) require it.
>
> that sounds quite sensible.
> I would also suggest using wait() and notify() for deterministic
> synchronization at the producer level (user of the outputstream) as
> opposed to a synchronized block unless the latter is more appropriate
> (optimistic stream writes).
> In any case isnt it better practise to synchronize on a buffer and
> allow only one thread access to the underlying outputstream?
>
> >
> >         Bill Pugh
> >
> >
> > On May 31, 2006, at 6:33 PM, Elias Ross wrote:
> >
> > >
> > > I know this isn't really the forum for asking this, but I've spent a
> > > number of hours looking into a definite yes or no...
> > >
> > > I wanted to know if OutputStream.write(byte b[]) was considered
> > > atomic.
> > > If two threads are writing to the same file, would the output from
> > > OS.write(...) overlap with another OS.write(...)?
> > >
> > > I eventually downloaded the Java source to answer this question.  The
> > > JavaDoc did not specify.  As it turns out, the answer is "no" -- which
> > > is something that you'd only be able to determine by looking at the C
> > > source code for the native calls.
> > >
> > > The JDK interpretation of write() is slightly different than the UNIX
> > > one, which looks like this:
> > >
> > >        ssize_t write(int fd, const void *buf, size_t count);
> > >
> > > so it makes sense that a Java write() call may have to do more than
> > > one
> > > system write() call.
> > >
> > > I wonder if there some better way that "concurrent" and "atomic"
> > > methods
> > > can be documented in the JDK?  It's also very helpful as a user to
> > > know
> > > if I should be locking on the OutputStream itself or create my own
> > > locks.
> > >
> >
>
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Dhanji R. Prasanna
On 6/1/06, Joe Bowbeer <[hidden email]> wrote:

> At the risk of drifting away from the original question, in which
> locking of the output stream was the stated goal... see comments
> below.
>
> On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:
>
> > I would also suggest using wait() and notify() for deterministic
> > synchronization at the producer level (user of the outputstream)
> > as opposed to a synchronized block unless the latter is more
> > appropriate (optimistic stream writes).
>
> Can you explain further?
>
> While I don't advocate upgrading every "synchronized" to some form of
> java.util.concurrent.locks.Lock:
>

wait(), notify() offers a simple way of ordering behavior in a
producer-consumer couple. That's all I meant. synchronized() will
yield non-deterministic behavior with regard to the order of the
queue/buffer as has been noted elsewhere in this thread.

>   New lock classes improve on synchronized -- but don't replace it
>   http://www-128.ibm.com/developerworks/java/library/j-jtp10264/
>
> I *do* advocate stamping-out all explicit obj.wait()s and
> obj.notify()s.  Given all the tools in java.util.concurrent, there's
> got to be something better already implemented.
>

Correct me if I'm wrong but to my understanding
java.util.concurrent.locks is an extended, more flexible toolkit for
concurrency (for situations such as releasing locks in non-lexical
strictness) and not a replacement for the language basics, whether
synchronized or wait/notify.
The 1.5 javadoc for java.util.concurrent.locks.Lock says something to
this effect.

Or am I totally on the wrong trail to oregon (the one that goes via
mexico ;)--are you suggesting deprecation of wait() and notify()
altogether?

>
> If clean code is the goal, my preference is to avoid explicit locking
> in favor or queueing.
>

I completely agree with this. Explicit locking should be hidden by the
api with threadsafe/concurrent queues. At the risk of drifting
(along), my point above is for academic or other areas where
threadsafe queues may not be feasible (like soft realtime apps).

> In other words, producers insert their byte[]s onto a thread-safe
> queue, while an outputter thread pulls byte[]s from the queue and
> writes them to the stream.  This separation of concerns results in
> very clean code. Furthermore, several flavors of queue have already
> been implemented in java.util.concurrent.
>
> --Joe
>
>
> On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:
> > On 6/1/06, Bill Pugh <[hidden email]> wrote:
> > > The fact that many of the implementations of OutputStream are
> > > synchronized was
> > > a pretty horrible design mistake.
> > >
> > > If you have two different threads trying to push bytes into an output
> > > stream, I'm hard pressed to
> > > imagine situations in which allowing the threads to non-
> > > deterministically interleave is a good idea.
> > >
> > > If you are creating your own implementation of OutputStream, I would
> > > recommending avoiding any
> > > use of synchronization. It serves no purpose, reduces performance,
> > > and I don't believe the class
> > > contract (or Sun's TCK tests) require it.
> >
> > that sounds quite sensible.
> > I would also suggest using wait() and notify() for deterministic
> > synchronization at the producer level (user of the outputstream) as
> > opposed to a synchronized block unless the latter is more appropriate
> > (optimistic stream writes).
> > In any case isnt it better practise to synchronize on a buffer and
> > allow only one thread access to the underlying outputstream?
> >
> > >
> > >         Bill Pugh
> > >
> > >
> > > On May 31, 2006, at 6:33 PM, Elias Ross wrote:
> > >
> > > >
> > > > I know this isn't really the forum for asking this, but I've spent a
> > > > number of hours looking into a definite yes or no...
> > > >
> > > > I wanted to know if OutputStream.write(byte b[]) was considered
> > > > atomic.
> > > > If two threads are writing to the same file, would the output from
> > > > OS.write(...) overlap with another OS.write(...)?
> > > >
> > > > I eventually downloaded the Java source to answer this question.  The
> > > > JavaDoc did not specify.  As it turns out, the answer is "no" -- which
> > > > is something that you'd only be able to determine by looking at the C
> > > > source code for the native calls.
> > > >
> > > > The JDK interpretation of write() is slightly different than the UNIX
> > > > one, which looks like this:
> > > >
> > > >        ssize_t write(int fd, const void *buf, size_t count);
> > > >
> > > > so it makes sense that a Java write() call may have to do more than
> > > > one
> > > > system write() call.
> > > >
> > > > I wonder if there some better way that "concurrent" and "atomic"
> > > > methods
> > > > can be documented in the JDK?  It's also very helpful as a user to
> > > > know
> > > > if I should be locking on the OutputStream itself or create my own
> > > > locks.
> > > >
> > >
> >
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
>
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: Contract of java.io.OutputStream.write(byte b[])

Joe Bowbeer
On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:

> are you suggesting deprecation of wait() and notify() altogether?

I wouldn't say "deprecate".  What I would say is that I advise against
their use :-)

There's very little need for anyone to write code that calls these
low-level methods, and when they do there are almost always bugs.
(Show me a wait and a notify and I'll show you a bug -- it almost
never fails!)

I would prefer that a higher level construct were used, such as a
Queue or Future or even an explicit Condition:

http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/locks/Condition.html


On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:

> On 6/1/06, Joe Bowbeer <[hidden email]> wrote:
> > At the risk of drifting away from the original question, in which
> > locking of the output stream was the stated goal... see comments
> > below.
> >
> > On 5/31/06, Dhanji R. Prasanna <[hidden email]> wrote:
> >
> > > I would also suggest using wait() and notify() for deterministic
> > > synchronization at the producer level (user of the outputstream)
> > > as opposed to a synchronized block unless the latter is more
> > > appropriate (optimistic stream writes).
> >
> > Can you explain further?
> >
> > While I don't advocate upgrading every "synchronized" to some form of
> > java.util.concurrent.locks.Lock:
> >
>
> wait(), notify() offers a simple way of ordering behavior in a
> producer-consumer couple. That's all I meant. synchronized() will
> yield non-deterministic behavior with regard to the order of the
> queue/buffer as has been noted elsewhere in this thread.
>
> >   New lock classes improve on synchronized -- but don't replace it
> >   http://www-128.ibm.com/developerworks/java/library/j-jtp10264/
> >
> > I *do* advocate stamping-out all explicit obj.wait()s and
> > obj.notify()s.  Given all the tools in java.util.concurrent, there's
> > got to be something better already implemented.
> >
>
> Correct me if I'm wrong but to my understanding
> java.util.concurrent.locks is an extended, more flexible toolkit for
> concurrency (for situations such as releasing locks in non-lexical
> strictness) and not a replacement for the language basics, whether
> synchronized or wait/notify.
> The 1.5 javadoc for java.util.concurrent.locks.Lock says something to
> this effect.
>
> Or am I totally on the wrong trail to oregon (the one that goes via
> mexico ;)--are you suggesting deprecation of wait() and notify()
> altogether?
>
> >
> > If clean code is the goal, my preference is to avoid explicit locking
> > in favor or queueing.
> >
>
> I completely agree with this. Explicit locking should be hidden by the
> api with threadsafe/concurrent queues. At the risk of drifting
> (along), my point above is for academic or other areas where
> threadsafe queues may not be feasible (like soft realtime apps).
>
> > In other words, producers insert their byte[]s onto a thread-safe
> > queue, while an outputter thread pulls byte[]s from the queue and
> > writes them to the stream.  This separation of concerns results in
> > very clean code. Furthermore, several flavors of queue have already
> > been implemented in java.util.concurrent.
> >
> > --Joe
> >
_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest