unit testing concurrency code.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

unit testing concurrency code.

Peter Veentjer - Anchor Men
How do you (unit) tests concurrency code? I'm experienced
with JUnit, but I haven't found a good extension for junit.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: unit testing concurrency code.

Brian Goetz
Like the old joke about how porcupines mate -- very carefully.

Testing concurrent code is an extension of testing regular code.  First,
you must have good tests for functionality, and be able to test as many
of your classes invariants as possible.  The trick is then trying to
generate as many random interleavings of operations as you can, without
the test framework introducing timing artifacts that will prevent
certain interleavings from being tested.

Our book, Java Concurrency in Practice, due out by the end of the year,
will cover some of this.

Peter Veentjer - Anchor Men wrote:
> How do you (unit) tests concurrency code? I'm experienced
> with JUnit, but I haven't found a good extension for junit.
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: unit testing concurrency code.

Craig Mattocks
In reply to this post by Peter Veentjer - Anchor Men
On Tue, 06 Sep 2005 14:42:35 Brian Goetz <[hidden email]> wrote:

 > Our book, Java Concurrency in Practice, due out by the end of the  
year,
 > will cover some of this.

Great - I've been searching for a book like this!  Seems you can  
already pre-order it from Amazon:

http://www.amazon.com/exec/obidos/tg/detail/-/0321349601/ 
qid=1126238399/sr=8-1/ref=pd_bbs_1/002-8907356-9580856?
v=glance&s=books&n=507846

Please post an announcement when it actually ships.

Does it contain the famous "baboons crossing a bridge" (platooning)  
problem?  :-)

Craig

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: unit testing concurrency code.

Beton, Richard
In reply to this post by Brian Goetz
Brian Goetz wrote:

 > Testing concurrent code is an extension of testing regular code.  
 > First, you must have good tests for functionality, and be able to test
 > as many of your classes invariants as possible.  The trick is then
 > trying to generate as many random interleavings of operations as you
 > can, without the test framework introducing timing artifacts that will
 > prevent certain interleavings from being tested.


Surely this overlooks something: there are four kinds of dynamic failure
(deadlock, livelock, starvation and race) that you cannot prove are
absent simply by testing alone.  You can prove they are *present* if
your testing finds them, but you can't prove they are *absent*.

This is quite important if you are writing concurrency classes for other
people to use.  You cannot write tests with enough coverage to handle
every end-user's use cases.  Simply covering a certain subset of dynamic
behaviour and then passing such classes as "tested" may be a bit too
optimistic and misleading.

I'd like to draw your attention to Peter Welch's posting ("CSP, the
pi-calculus and CPA-2005") and the work he and others have done using
CSP as the basis for establishing thread reliability (establishing the
absence of deadlock, livelock, starvation and race).  My own experience
is limited to being a JCSP user, rather than having much theoretical
skill.  There are some straightforward design rules that guarantee
deadlock freedom for example.  This makes JCSP a simple strategy to
apply to practical usage.

Regards,
Rick









--

Visit our website at www.roke.co.uk

Roke Manor Research Ltd, Roke Manor, Romsey, Hampshire SO51 0ZN, UK.

The information contained in this e-mail and any attachments is proprietary to
Roke Manor Research Ltd and must not be passed to any third party without
permission. This communication is for information only and shall not create or
change any contractual relationship.

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Reply | Threaded
Open this post in threaded view
|

Re: unit testing concurrency code.

Joshua Bloch-2
Nor can you reliably determine that a program works merely by proving
it correct.  You can only prove correctness with respect to some
model, which may itself be flawed.  Also your proof may be faulty.
Some combination of testing and proof works best.  Also it's important
to show your program to other experts.  Two (or more) heads are better
than one.

        Josh

On 9/20/05, Rick Beton <[hidden email]> wrote:

> Brian Goetz wrote:
>
>  > Testing concurrent code is an extension of testing regular code.
>  > First, you must have good tests for functionality, and be able to test
>  > as many of your classes invariants as possible.  The trick is then
>  > trying to generate as many random interleavings of operations as you
>  > can, without the test framework introducing timing artifacts that will
>  > prevent certain interleavings from being tested.
>
>
> Surely this overlooks something: there are four kinds of dynamic failure
> (deadlock, livelock, starvation and race) that you cannot prove are
> absent simply by testing alone.  You can prove they are *present* if
> your testing finds them, but you can't prove they are *absent*.
>
> This is quite important if you are writing concurrency classes for other
> people to use.  You cannot write tests with enough coverage to handle
> every end-user's use cases.  Simply covering a certain subset of dynamic
> behaviour and then passing such classes as "tested" may be a bit too
> optimistic and misleading.
>
> I'd like to draw your attention to Peter Welch's posting ("CSP, the
> pi-calculus and CPA-2005") and the work he and others have done using
> CSP as the basis for establishing thread reliability (establishing the
> absence of deadlock, livelock, starvation and race).  My own experience
> is limited to being a JCSP user, rather than having much theoretical
> skill.  There are some straightforward design rules that guarantee
> deadlock freedom for example.  This makes JCSP a simple strategy to
> apply to practical usage.
>
> Regards,
> Rick
>
>
>
>
>
>
>
>
>
> --
>
> Visit our website at www.roke.co.uk
>
> Roke Manor Research Ltd, Roke Manor, Romsey, Hampshire SO51 0ZN, UK.
>
> The information contained in this e-mail and any attachments is proprietary to
> Roke Manor Research Ltd and must not be passed to any third party without
> permission. This communication is for information only and shall not create or
> change any contractual relationship.
>
> _______________________________________________
> Concurrency-interest mailing list
> [hidden email]
> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
>

_______________________________________________
Concurrency-interest mailing list
[hidden email]
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest