lockfile
— Platform-independent file locking¶
Warning
This package is deprecated. It is highly preferred that instead of
using this code base that instead fasteners or oslo.concurrency is
used instead. For any questions or comments or further help needed
please email openstack-dev and prefix your email subject
with [oslo][pylockfile]
(for a faster response).
Note
This package is pre-release software. Between versions 0.8 and 0.9 it was changed from a module to a package. It is quite possible that the API and implementation will change again in important ways as people test it and provide feedback and bug fixes. In particular, if the mkdir-based locking scheme is sufficient for both Windows and Unix platforms, the link-based scheme may be deleted so that only a single locking scheme is used, providing cross-platform lockfile cooperation.
Note
The implementation uses the with statement, both in the tests and in the
main code, so will only work out-of-the-box with Python 2.5 or later.
However, the use of the with statement is minimal, so if you apply the
patch in the included 2.4.diff file you can use it with Python 2.4. It’s
possible that it will work in Python 2.3 with that patch applied as well,
though the doctest code relies on APIs new in 2.4, so will have to be
rewritten somewhat to allow testing on 2.3. As they say, patches welcome.
;-)
The lockfile
package exports a LockFile
class which provides
a simple API for locking files. Unlike the Windows msvcrt.locking()
function, the Unix fcntl.flock()
, fcntl.lockf()
and the
deprecated posixfile
module, the API is identical across both Unix
(including Linux and Mac) and Windows platforms. The lock mechanism relies
on the atomic nature of the link()
(on Unix) and mkdir()
(On
Windows) system calls. It also contains several lock-method-specific
modules: lockfile.linklockfile
, lockfile.mkdirlockfile
, and
lockfile.sqlitelockfile
, each one exporting a single class. For
backwards compatibility with versions before 0.9 the LinkFileLock
,
MkdirFileLock
and SQLiteFileLock
objects are exposed as
attributes of the top-level lockfile package, though this use was deprecated
starting with version 0.9 and will be removed in version 1.0.
Note
The current implementation uses os.link()
on Unix, but since that
function is unavailable on Windows it uses os.mkdir()
there. At
this point it’s not clear that using the os.mkdir()
method would be
insufficient on Unix systems. If it proves to be adequate on Unix then
the implementation could be simplified and truly cross-platform locking
would be possible.
Note
The current implementation doesn’t provide for shared vs. exclusive locks. It should be possible for multiple reader processes to hold the lock at the same time.
The module defines the following exceptions:
- exception lockfile.Error¶
This is the base class for all exceptions raised by the
LockFile
class.
- exception lockfile.LockError¶
This is the base class for all exceptions raised when attempting to lock a file.
- exception lockfile.UnlockError¶
This is the base class for all exceptions raised when attempting to unlock a file.
- exception lockfile.LockTimeout¶
This exception is raised if the
LockFile.acquire()
method is called with a timeout which expires before an existing lock is released.
- exception lockfile.AlreadyLocked¶
This exception is raised if the
LockFile.acquire()
detects a file is already locked when in non-blocking mode.
- exception lockfile.LockFailed¶
This exception is raised if the
LockFile.acquire()
detects some other condition (such as a non-writable directory) which prevents it from creating its lock file.
- exception lockfile.NotLocked¶
This exception is raised if the file is not locked when
LockFile.release()
is called.
- exception lockfile.NotMyLock¶
This exception is raised if the file is locked by another thread or process when
LockFile.release()
is called.
The following classes are provided:
- class linklockfile.LinkLockFile(path, threaded=True)¶
This class uses the
link(2)()
system call as the basic lock mechanism. path is an object in the file system to be locked. It need not exist, but its directory must exist and be writable at the time theacquire()
andrelease()
methods are called. threaded is optional, but when set toTrue
locks will be distinguished between threads in the same process.
- class symlinklockfile.SymlinkLockFile(path, threaded=True)¶
This class uses the
symlink(2)()
system call as the basic lock mechanism. The parameters have the same meaning and constraints as for theLinkLockFile
class.
- class mkdirlockfile.MkdirLockFile(path, threaded=True)¶
This class uses the
mkdir(2)()
system call as the basic lock mechanism. The parameters have the same meaning and constraints as for theLinkLockFile
class.
- class sqlitelockfile.SQLiteLockFile(path, threaded=True)¶
This class uses the
sqlite3
module to implement the lock mechanism. The parameters have the same meaning as for theLinkLockFile
class.
- class lockfile.LockBase(path, threaded=True)¶
This is the base class for all concrete implementations and is available at the lockfile package level so programmers can implement other locking schemes.
- lockfile.locked(path, timeout=None)¶
This function provides a decorator which insures the decorated function is always called with the lock held.
By default, the LockFile
object refers to the
mkdirlockfile.MkdirLockFile
class on Windows. On all other
platforms it refers to the linklockfile.LinkLockFile
class.
When locking a file the linklockfile.LinkLockFile
class creates a
uniquely named hard link to an empty lock file. That hard link contains the
hostname, process id, and if locks between threads are distinguished, the
thread identifier. For example, if you want to lock access to a file named
“README”, the lock file is named “README.lock”. With per-thread locks
enabled the hard link is named HOSTNAME-THREADID-PID. With only per-process
locks enabled the hard link is named HOSTNAME–PID.
When using the mkdirlockfile.MkdirLockFile
class the lock file is a
directory. Referring to the example above, README.lock will be a directory
and HOSTNAME-THREADID-PID will be an empty file within that directory.
See also
- Module
msvcrt
Provides the
locking()
function, the standard Windows way of locking (parts of) a file.- Module
posixfile
The deprecated (since Python 1.5) way of locking files on Posix systems.
- Module
fcntl
Provides the current best way to lock files on Unix systems (
lockf()
andflock()
).
LockFile Objects¶
LockFile
objects support the context manager protocol used by the
statement:with statement. The timeout option is not supported when used in
this fashion. While support for timeouts could be implemented, there is no
support for handling the eventual Timeout
exceptions raised by the
__enter__()
method, so you would have to protect the with statement with
a try statement. The resulting construct would not be any simpler than just
using a try statement in the first place.
LockFile
has the following user-visible methods:
- LockFile.acquire(timeout=None)¶
Lock the file associated with the
LockFile
object. If the timeout is omitted orNone
the caller will block until the file is unlocked by the object currently holding the lock. If the timeout is zero or a negative number theAlreadyLocked
exception will be raised if the file is currently locked by another process or thread. If the timeout is positive, the caller will block for that many seconds waiting for the lock to be released. If the lock is not released within that period theLockTimeout
exception will be raised.
- LockFile.release()¶
Unlock the file associated with the
LockFile
object. If the file is not currently locked, theNotLocked
exception is raised. If the file is locked by another thread or process theNotMyLock
exception is raised.
- lockfile.is_locked()¶
Return the status of the lock on the current file. If any process or thread (including the current one) is locking the file,
True
is returned, otherwiseFalse
is returned.
- lockfile.break_lock()¶
If the file is currently locked, break it.
- lockfile.i_am_locking()¶
Returns true if the caller holds the lock.
Examples¶
This example is the “hello world” for the lockfile
package:
from lockfile import LockFile
lock = LockFile("/some/file/or/other")
with lock:
print lock.path, 'is locked.'
To use this with Python 2.4, you can execute:
from lockfile import LockFile
lock = LockFile("/some/file/or/other")
lock.acquire()
print lock.path, 'is locked.'
lock.release()
If you don’t want to wait forever, you might try:
from lockfile import LockFile
lock = LockFile("/some/file/or/other")
while not lock.i_am_locking():
try:
lock.acquire(timeout=60) # wait up to 60 seconds
except LockTimeout:
lock.break_lock()
lock.acquire()
print "I locked", lock.path
lock.release()
You can also insure that a lock is always held when appropriately decorated functions are called:
from lockfile import locked
@locked("/tmp/mylock")
def func(a, b):
return a + b
Other Libraries¶
The idea of implementing advisory locking with a standard API is not new
with lockfile
. There are a number of other libraries available:
locknix - http://pypi.python.org/pypi/locknix - Unix only
mx.MiscLockFile - from Marc André Lemburg, part of the mx.Base distribution - cross-platform.
Twisted - http://twistedmatrix.com/trac/browser/trunk/twisted/python/lockfile.py
zc.lockfile - http://pypi.python.org/pypi/zc.lockfile
Contacting the Author¶
If you encounter any problems with lockfile
, would like help or want to
submit a patch, check http://launchpad.net/pylockfile