[K42-discussion] Memory consistency across block/unblock

[Bryan S Rosenburg]

I'm worried that we've been ignoring memory consistency issues in one of 
our common (and recommended) programming patterns.  We use the following 
pattern all over the place:

Thread A:
        sharedVariable.waiter = Scheduler::GetCurThread();
        while (sharedVariable.waiter != Scheduler::NullThreadID) {
                Scheduler::Block();
        }

Thread B:
        tmpThrd = sharedVariable.waiter;
        sharedVariable.waiter = Scheduler::NullThreadID;
        Scheduler::Unblock(tmpThrd);

By itself, this code is okay.  But I think in many cases thread B will 
have made some state change before unblocking thread A, and thread A will 
expect to see the change.  For the change to be seen correctly, thread B 
must execute an appropriate memory barrier before nulling 
sharedVariable.waiter, and thread A must execute a matching barrier after 
the while-loop.  I don't think we've been careful about inserting such 
barriers.  The problem can only occur if thread A doesn't actually block, 
because the Block/Unblock paths include context-synchronizing operations, 
but we can in no way count on thread A always blocking.

I'm afraid this calls for a systematic examination of all our 
thread-unblocking paths.  A quick egrep says there are about 70 of them.

- Bryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://ozlabs.org/pipermail/k42-discussion/attachments/20050929/51f53549/attachment.htm