Saturday, February 12, 2011

Argument-based methods synchronization in Python

I've been implementing yet another client for a REST-like service and I've spotted a problem I typically haven't faced while working with REST services. I needed to modify some resource -- meaning issuing a PUT request on resource /api/resource/id -- for which changes didn't apply immediately but it rather went into 'building' state or something like that. I had a method like modify_bar(self, resource_id) and things worked great for a single thread, but caused problems with multiple active threads. Obviously, I checked that resource_id is ready for operations, but in the middle of the end of the check and starting of actual work another thread could get in and cause troubles.

Let's check sample code:



And its output:


(14:46) novel@nov-testing2:~/sync %> ./sync.py
working on bar id = 1
working on bar id = 1
working on bar id = 1
done with bar id = 1
done with bar id = 1
done with bar id = 1
(14:47) novel@nov-testing2:~/sync %>


But the result I wanted to achieve:


(14:46) novel@nov-testing2:~/sync %> ./sync.py
working on bar id = 1
done with bar id = 1
working on bar id = 1
done with bar id = 1
working on bar id = 1
done with bar id = 1
(14:47) novel@nov-testing2:~/sync %>


I've been thinking how to do it without class redesign and API changes and after some time came up with a synchronization solution for methods based on their arguments.

Implementation looks like this:



Basically, we keep a list of ids of objects we're working with. When we want to start working with some object we check the list first, if its id in the list it means it's currently being worked on and we have to wait, otherwise we push the id to the list and start working.

And the result is:


(15:09) novel@nov-testing2:~/sync %> ./sync2.py
working on bar id = 1
done with bar id = 1
working on bar id = 1
done with bar id = 1
working on bar id = 1
done with bar id = 1
(15:10) novel@nov-testing2:~/sync %>


For the sake of experiment let's start another thread with different bar_id value to make sure it's not blocked:



And the output now:


(15:13) novel@nov-testing2:~/sync %> ./sync2.py
working on bar id = 1
working on bar id = 0
done with bar id = 1
working on bar id = 1
done with bar id = 0
done with bar id = 1
working on bar id = 1
done with bar id = 1
working on bar id = 1
done with bar id = 1
(15:14) novel@nov-testing2:~/sync %>


As you can see, things are now working as expected.

No comments:

Post a Comment