flufl.lock-2.4.1/0000775000175000017500000000000012614427341014043 5ustar barrybarry00000000000000flufl.lock-2.4.1/MANIFEST.in0000664000175000017500000000013112614426402015571 0ustar barrybarry00000000000000include *.py MANIFEST.in *.ini global-include *.txt *.rst exclude .gitignore prune build flufl.lock-2.4.1/flufl.lock.egg-info/0000775000175000017500000000000012614427341017574 5ustar barrybarry00000000000000flufl.lock-2.4.1/flufl.lock.egg-info/SOURCES.txt0000664000175000017500000000106712614427341021464 0ustar barrybarry00000000000000MANIFEST.in README.rst setup.cfg setup.py setup_helpers.py template.py tox.ini flufl/__init__.py flufl.lock.egg-info/PKG-INFO flufl.lock.egg-info/SOURCES.txt flufl.lock.egg-info/dependency_links.txt flufl.lock.egg-info/namespace_packages.txt flufl.lock.egg-info/top_level.txt flufl/lock/NEWS.rst flufl/lock/README.rst flufl/lock/__init__.py flufl/lock/_lockfile.py flufl/lock/conf.py flufl/lock/docs/__init__.py flufl/lock/docs/using.rst flufl/lock/tests/__init__.py flufl/lock/tests/subproc.py flufl/lock/tests/test_documentation.py flufl/lock/tests/test_lockfile.pyflufl.lock-2.4.1/flufl.lock.egg-info/top_level.txt0000664000175000017500000000000612614427341022322 0ustar barrybarry00000000000000flufl flufl.lock-2.4.1/flufl.lock.egg-info/dependency_links.txt0000664000175000017500000000000112614427341023642 0ustar barrybarry00000000000000 flufl.lock-2.4.1/flufl.lock.egg-info/namespace_packages.txt0000664000175000017500000000000612614427341024123 0ustar barrybarry00000000000000flufl flufl.lock-2.4.1/flufl.lock.egg-info/PKG-INFO0000664000175000017500000001461712614427341020702 0ustar barrybarry00000000000000Metadata-Version: 1.1 Name: flufl.lock Version: 2.4.1 Summary: NFS-safe file locking with timeouts for POSIX systems. Home-page: http://launchpad.net/flufl.lock Author: Barry Warsaw Author-email: barry@python.org License: LGPLv3 Download-URL: https://pypi.python.org/pypi/flufl.lock Description: ========== flufl.lock ========== NFS-safe file locking with timeouts for POSIX systems. The `flufl.lock` library provides an NFS-safe file-based locking algorithm influenced by the GNU/Linux `open(2)` manpage, under the description of the `O_EXCL` option. [...] O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. The assumption made here is that there will be no *outside interference*, e.g. no agent external to this code will ever `link()` to the specific lock files used. Lock objects support lock-breaking so that you can't wedge a process forever. This is especially helpful in a web environment, but may not be appropriate for all applications. Locks have a *lifetime*, which is the maximum length of time the process expects to retain the lock. It is important to pick a good number here because other processes will not break an existing lock until the expected lifetime has expired. Too long and other processes will hang; too short and you'll end up trampling on existing process locks -- and possibly corrupting data. In a distributed (NFS) environment, you also need to make sure that your clocks are properly synchronized. Author ====== `flufl.lock` is Copyright (C) 2007-2015 Barry Warsaw Licensed under the terms of the GNU Lesser General Public License, version 3 or later. See the COPYING.LESSER file for details. Project details =============== * Project home: https://gitlab.com/warsaw/flufl.lock * Report bugs at: https://gitlab.com/warsaw/flufl.lock/issues * Code hosting: git@gitlab.com:warsaw/flufl.lock.git * Documentation: http://flufllock.readthedocs.org/ =================== NEWS for flufl.lock =================== 2.4.1 (2015-10-29) ================== * Fix the MANIFEST.in so that tox.ini is included in the sdist. 2.4 (2015-10-10) ================ * Drop Python 2.6 compatibility. * Add Python 3.5 compatibility. 2.3.1 (2014-09-26) ================== * Include MANIFEST.in in the sdist tarball, otherwise the Debian package won't built correctly. 2.3 (2014-09-25) ================ * Fix documentation bug. (LP: #1026403) * Catch ESTALE along with ENOENT, as NFS servers are supposed to (but don't always) throw ESTALE instead of ENOENT. (LP: #977999) * Purge all references to `distribute`. (LP: #1263794) 2.2.1 (2012-04-19) ================== * Add classifiers to setup.py and make the long description more compatible with the Cheeseshop. * Other changes to make the Cheeseshop page look nicer. (LP: #680136) * setup_helper.py version 2.1. 2.2 (2012-01-19) ================ * Support Python 3 without the use of 2to3. * Make the documentation clear that the `flufl.test.subproc` functions are not part of the public API. (LP: #838338) * Fix claim file format in `take_possession()`. (LP: #872096) * Provide a new API for dealing with possible additional unexpected errnos while trying to read the lock file. These can happen in some NFS environments. If you want to retry the read, set the lock file's `retry_errnos` property to a sequence of errnos. If one of those errnos occurs, the read is unconditionally (and infinitely) retried. `retry_errnos` is a property which must be set to a sequence; it has a getter and a deleter too. (LP: #882261) 2.1.1 (2011-08-20) ================== * Fixed TypeError in .lock() method due to race condition in _releasetime property. Found by Stephen A. Goss. (LP: #827052) 2.1 (2010-12-22) ================ * Added lock.details. 2.0.2 (2010-12-19) ================== * Small adjustment to doctest. 2.0.1 (2010-11-27) ================== * Add missing exception to __all__. 2.0 (2010-11-26) ================ * Package renamed to flufl.lock. Earlier ======= Try `bzr log lp:flufl.lock` for details. Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+) Classifier: Operating System :: POSIX Classifier: Operating System :: MacOS :: MacOS X Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Software Development :: Libraries Classifier: Topic :: Software Development :: Libraries :: Python Modules flufl.lock-2.4.1/setup.py0000664000175000017500000000405112606257340015556 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . from setup_helpers import ( description, get_version, long_description, require_python) from setuptools import setup, find_packages require_python(0x20700f0) __version__ = get_version('flufl/lock/__init__.py') setup( name='flufl.lock', version=__version__, namespace_packages=['flufl'], packages=find_packages(), include_package_data=True, maintainer='Barry Warsaw', maintainer_email='barry@python.org', description=description('README.rst'), long_description=long_description('README.rst', 'flufl/lock/NEWS.rst'), license='LGPLv3', url='http://launchpad.net/flufl.lock', download_url='https://pypi.python.org/pypi/flufl.lock', test_suite='flufl.lock.tests', classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: ' 'GNU Lesser General Public License v3 or later (LGPLv3+)', 'Operating System :: POSIX', 'Operating System :: MacOS :: MacOS X', 'Programming Language :: Python', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Topic :: Software Development :: Libraries', 'Topic :: Software Development :: Libraries :: Python Modules', ] ) flufl.lock-2.4.1/setup.cfg0000664000175000017500000000022112614427341015657 0ustar barrybarry00000000000000[build_sphinx] source_dir = flufl/lock [upload_docs] upload_dir = build/sphinx/html [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 flufl.lock-2.4.1/setup_helpers.py0000664000175000017500000001210612606257442017303 0ustar barrybarry00000000000000# Copyright (C) 2009-2015 Barry A. Warsaw # # This file is part of setup_helpers.py # # setup_helpers.py is free software: you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by the # Free Software Foundation, version 3 of the License. # # setup_helpers.py is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with setup_helpers.py. If not, see . """setup.py helper functions.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ 'description', 'find_doctests', 'get_version', 'long_description', 'require_python', ] import os import re import sys DEFAULT_VERSION_RE = re.compile( r'(?P\d+\.\d+(?:\.\d+)?(?:(?:a|b|rc)\d+)?)') EMPTYSTRING = '' __version__ = '2.3' def require_python(minimum): """Require at least a minimum Python version. The version number is expressed in terms of `sys.hexversion`. E.g. to require a minimum of Python 2.6, use:: >>> require_python(0x206000f0) :param minimum: Minimum Python version supported. :type minimum: integer """ if sys.hexversion < minimum: hversion = hex(minimum)[2:] if len(hversion) % 2 != 0: hversion = '0' + hversion split = list(hversion) parts = [] while split: parts.append(int(''.join((split.pop(0), split.pop(0))), 16)) major, minor, micro, release = parts if release == 0xf0: print('Python {0}.{1}.{2} or better is required'.format( major, minor, micro)) else: print('Python {0}.{1}.{2} ({3}) or better is required'.format( major, minor, micro, hex(release)[2:])) sys.exit(1) def get_version(filename, pattern=None): """Extract the __version__ from a file without importing it. While you could get the __version__ by importing the module, the very act of importing can cause unintended consequences. For example, Distribute's automatic 2to3 support will break. Instead, this searches the file for a line that starts with __version__, and extract the version number by regular expression matching. By default, two or three dot-separated digits are recognized, but by passing a pattern parameter, you can recognize just about anything. Use the `version` group name to specify the match group. :param filename: The name of the file to search. :type filename: string :param pattern: Optional alternative regular expression pattern to use. :type pattern: string :return: The version that was extracted. :rtype: string """ if pattern is None: cre = DEFAULT_VERSION_RE else: cre = re.compile(pattern) with open(filename) as fp: for line in fp: if line.startswith('__version__'): mo = cre.search(line) assert mo, 'No valid __version__ string found' return mo.group('version') raise AssertionError('No __version__ assignment found') def find_doctests(start='.', extension='.rst'): """Find separate-file doctests in the package. This is useful for Distribute's automatic 2to3 conversion support. The `setup()` keyword argument `convert_2to3_doctests` requires file names, which may be difficult to track automatically as you add new doctests. :param start: Directory to start searching in (default is cwd) :type start: string :param extension: Doctest file extension (default is .txt) :type extension: string :return: The doctest files found. :rtype: list """ doctests = [] for dirpath, dirnames, filenames in os.walk(start): doctests.extend(os.path.join(dirpath, filename) for filename in filenames if filename.endswith(extension)) return doctests def long_description(*filenames): """Provide a long description.""" res = [''] for filename in filenames: with open(filename) as fp: for line in fp: res.append(' ' + line) res.append('') res.append('\n') return EMPTYSTRING.join(res) def description(filename): """Provide a short description.""" # This ends up in the Summary header for PKG-INFO and it should be a # one-liner. It will get rendered on the package page just below the # package version header but above the long_description, which ironically # gets stuff into the Description header. It should not include reST, so # pick out the first single line after the double header. with open(filename) as fp: for lineno, line in enumerate(fp): if lineno < 3: continue line = line.strip() if len(line) > 0: return line flufl.lock-2.4.1/flufl/0000775000175000017500000000000012614427341015153 5ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/0000775000175000017500000000000012614427341016103 5ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/docs/0000775000175000017500000000000012614427341017033 5ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/docs/using.rst0000664000175000017500000001362412476403672020727 0ustar barrybarry00000000000000============================ Using the flufl.lock library ============================ The ``flufl.lock`` package provides NFS-safe file locking with timeouts for POSIX systems. The implementation is influenced by the GNU/Linux `open(2)`_ manpage, under the description of the ``O_EXCL`` option: [...] O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. The assumption made here is that there will be no *outside interference*, e.g. no agent external to this code will ever ``link()`` to the specific lock files used. Lock objects support lock-breaking so that you can't wedge a process forever. This is especially helpful in a web environment, but may not be appropriate for all applications. Locks have a *lifetime*, which is the maximum length of time the process expects to retain the lock. It is important to pick a good number here because other processes will not break an existing lock until the expected lifetime has expired. Too long and other processes will hang; too short and you'll end up trampling on existing process locks -- and possibly corrupting data. In a distributed (NFS) environment, you also need to make sure that your clocks are properly synchronized. Creating a lock =============== To create a lock, you must first instantiate a ``Lock`` object, specifying the path to a file that will be used to synchronize the lock. This file should not exist. :: # This function comes from the test infrastructure. >>> filename = temporary_lockfile() >>> from flufl.lock import Lock >>> lock = Lock(filename) >>> lock Locks have a default lifetime... >>> lock.lifetime datetime.timedelta(0, 15) ...which you can change. >>> from datetime import timedelta >>> lock.lifetime = timedelta(seconds=30) >>> lock.lifetime datetime.timedelta(0, 30) >>> lock.lifetime = timedelta(seconds=15) You can ask whether the lock is acquired or not. >>> lock.is_locked False Acquiring the lock is easy if no other process has already acquired it. >>> lock.lock() >>> lock.is_locked True Once you have the lock, it's easy to release it. >>> lock.unlock() >>> lock.is_locked False It is an error to attempt to acquire the lock more than once in the same process. :: >>> from flufl.lock import AlreadyLockedError >>> lock.lock() >>> try: ... lock.lock() ... except AlreadyLockedError as error: ... print(error) We already had the lock >>> lock.unlock() Lock objects also support the context manager protocol. >>> lock.is_locked False >>> with lock: ... lock.is_locked True >>> lock.is_locked False Lock acquisition blocks ======================= Trying to lock the file when the lock is unavailable (because another process has already acquired it), the lock call will block. *Note:* the `_acquire()` function is not part of the public API. :: >>> from flufl.lock.tests.subproc import _acquire >>> import time >>> t0 = time.time() >>> _acquire(filename, timedelta(seconds=5)) >>> lock.lock() >>> t1 = time.time() >>> lock.unlock() >>> t1 - t0 > 4 True Refreshing a lock ================= A process can *refresh* a lock if it realizes that it needs to hold the lock for a little longer. You cannot refresh an unlocked lock. >>> from flufl.lock import NotLockedError >>> try: ... lock.refresh() ... except NotLockedError as error: ... print(error) >> from datetime import datetime >>> lock.lifetime = timedelta(seconds=2) >>> lock.lock() >>> lock.is_locked True After the current lifetime expires, the lock is stolen from the parent process even if the parent never unlocks it. *Note:* the `_waitfor()` function is not part of the public API. >>> from flufl.lock.tests.subproc import _waitfor >>> t_broken = _waitfor(filename, lock.lifetime) >>> t_broken < 5 True >>> lock.is_locked False However, if the process holding the lock refreshes it, it will hold it can hold it for as long as it needs. >>> lock.lock() >>> lock.refresh(timedelta(seconds=5)) >>> t_broken = _waitfor(filename, lock.lifetime) >>> t_broken > 3 True >>> lock.is_locked False Lock details ============ Lock files are written with unique contents that can be queried for information about the host name the lock was acquired on, the id of the process that acquired the lock, and the path to the lock file. >>> import os, socket >>> lock.lock() >>> hostname, pid, lockfile = lock.details >>> hostname == socket.getfqdn() True >>> pid == os.getpid() True >>> lockfile == filename True >>> lock.unlock() Even if another process has acquired the lock, the details can be queried. >>> _acquire(filename, timedelta(seconds=3)) >>> lock.is_locked False >>> hostname, pid, lockfile = lock.details >>> hostname == socket.getfqdn() True >>> pid == os.getpid() False >>> lockfile == filename True However, if no process has acquired the lock, the details are unavailable. >>> lock.lock() >>> lock.unlock() >>> try: ... lock.details ... except NotLockedError as error: ... print(error) Details are unavailable .. _`open(2)`: http://manpages.ubuntu.com/manpages/dapper/en/man2/open.2.html flufl.lock-2.4.1/flufl/lock/docs/__init__.py0000664000175000017500000000000012476403672021141 0ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/NEWS.rst0000664000175000017500000000415612614426522017417 0ustar barrybarry00000000000000=================== NEWS for flufl.lock =================== 2.4.1 (2015-10-29) ================== * Fix the MANIFEST.in so that tox.ini is included in the sdist. 2.4 (2015-10-10) ================ * Drop Python 2.6 compatibility. * Add Python 3.5 compatibility. 2.3.1 (2014-09-26) ================== * Include MANIFEST.in in the sdist tarball, otherwise the Debian package won't built correctly. 2.3 (2014-09-25) ================ * Fix documentation bug. (LP: #1026403) * Catch ESTALE along with ENOENT, as NFS servers are supposed to (but don't always) throw ESTALE instead of ENOENT. (LP: #977999) * Purge all references to `distribute`. (LP: #1263794) 2.2.1 (2012-04-19) ================== * Add classifiers to setup.py and make the long description more compatible with the Cheeseshop. * Other changes to make the Cheeseshop page look nicer. (LP: #680136) * setup_helper.py version 2.1. 2.2 (2012-01-19) ================ * Support Python 3 without the use of 2to3. * Make the documentation clear that the `flufl.test.subproc` functions are not part of the public API. (LP: #838338) * Fix claim file format in `take_possession()`. (LP: #872096) * Provide a new API for dealing with possible additional unexpected errnos while trying to read the lock file. These can happen in some NFS environments. If you want to retry the read, set the lock file's `retry_errnos` property to a sequence of errnos. If one of those errnos occurs, the read is unconditionally (and infinitely) retried. `retry_errnos` is a property which must be set to a sequence; it has a getter and a deleter too. (LP: #882261) 2.1.1 (2011-08-20) ================== * Fixed TypeError in .lock() method due to race condition in _releasetime property. Found by Stephen A. Goss. (LP: #827052) 2.1 (2010-12-22) ================ * Added lock.details. 2.0.2 (2010-12-19) ================== * Small adjustment to doctest. 2.0.1 (2010-11-27) ================== * Add missing exception to __all__. 2.0 (2010-11-26) ================ * Package renamed to flufl.lock. Earlier ======= Try `bzr log lp:flufl.lock` for details. flufl.lock-2.4.1/flufl/lock/__init__.py0000664000175000017500000000206612614426454020224 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, version 3 of the License. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . """Package init.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ 'AlreadyLockedError', 'Lock', 'LockError', 'NotLockedError', 'TimeOutError', '__version__', ] __version__ = '2.4.1' # Public API. from flufl.lock._lockfile import ( AlreadyLockedError, Lock, LockError, NotLockedError, TimeOutError) flufl.lock-2.4.1/flufl/lock/_lockfile.py0000664000175000017500000005251112476403672020417 0ustar barrybarry00000000000000# Copyright (C) 2007-2015 Barry A. Warsaw # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. """Portable, NFS-safe file locking with timeouts for POSIX systems. This code implements an NFS-safe file-based locking algorithm influenced by the GNU/Linux open(2) manpage, under the description of the O_EXCL option: [...] O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. The assumption made here is that there will be no 'outside interference', e.g. no agent external to this code will ever link() to the specific lock files used. Lock objects support lock-breaking so that you can't wedge a process forever. This is especially helpful in a web environment, but may not be appropriate for all applications. Locks have a 'lifetime', which is the maximum length of time the process expects to retain the lock. It is important to pick a good number here because other processes will not break an existing lock until the expected lifetime has expired. Too long and other processes will hang; too short and you'll end up trampling on existing process locks -- and possibly corrupting data. In a distributed (NFS) environment, you also need to make sure that your clocks are properly synchronized. """ from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ 'LockError', 'AlreadyLockedError', 'NotLockedError', 'Lock', ] import os import sys import time import errno import random import socket import logging import datetime DEFAULT_LOCK_LIFETIME = datetime.timedelta(seconds=15) # Allowable a bit of clock skew. CLOCK_SLOP = datetime.timedelta(seconds=10) try: MAXINT = sys.maxint except AttributeError: # Python 3. MAXINT = sys.maxsize # Details separator; also used in calculating the claim file path. Lock files # should not include this character. SEP = '|' # LP: #977999 - catch both ENOENT and ESTALE. The latter is what an NFS # server should return, but some Linux versions return ENOENT. ERRORS = (errno.ENOENT, errno.ESTALE) log = logging.getLogger('flufl.lock') # Install a null handler to avoid warnings when applications don't set their # own flufl.lock logger. See http://docs.python.org/library/logging.html try: from logging import NullHandler except ImportError: # Python < 2.7. class NullHandler(logging.Handler): def emit(self, record): pass logging.getLogger('flufl.lock').addHandler(NullHandler()) # Exceptions that can be raised by this module class LockError(Exception): """Base class for all exceptions in this module.""" class AlreadyLockedError(LockError): """An attempt is made to lock an already locked object.""" class NotLockedError(LockError): """An attempt is made to unlock an object that isn't locked.""" class TimeOutError(LockError): """The timeout interval elapsed before the lock succeeded.""" class Lock: """A portable way to lock resources by way of the file system.""" def __init__(self, lockfile, lifetime=None): """Create the resource lock using the given file name and lifetime. Each process laying claim to this resource lock will create their own temporary lock file based on the path specified. An optional lifetime is the length of time that the process expects to hold the lock. :param lockfile: The full path to the lock file. :param lifetime: The expected maximum lifetime of the lock, as a timedelta. Defaults to 15 seconds. """ if lifetime is None: lifetime = DEFAULT_LOCK_LIFETIME self._lockfile = lockfile self._lifetime = lifetime self._set_claimfile() # For transferring ownership across a fork. self._owned = True # For extending the set of NFS errnos that are retried in _read(). self._retry_errnos = [] def __repr__(self): return '<%s %s [%s: %s] pid=%s at %#xx>' % ( self.__class__.__name__, self._lockfile, ('locked' if self.is_locked else 'unlocked'), self._lifetime, os.getpid(), id(self)) @property def details(self): """Details as read from the lock file. :return: A 3-tuple of hostname, process id, file name. :rtype: (str, int, str) :raises NotLockedError: if the lock is not acquired. """ try: with open(self._lockfile) as fp: filename = fp.read().strip() except IOError as error: if error.errno in ERRORS: raise NotLockedError('Details are unavailable') raise # Rearrange for signature. try: lockfile, hostname, pid, random_ignored = filename.split(SEP) except ValueError: raise NotLockedError('Details are unavailable') return hostname, int(pid), lockfile @property def lifetime(self): return self._lifetime @lifetime.setter def lifetime(self, lifetime): self._lifetime = lifetime def refresh(self, lifetime=None, unconditionally=False): """Refreshes the lifetime of a locked file. Use this if you realize that you need to keep a resource locked longer than you thought. :param lifetime: If given, this sets the lock's new lifetime. This must be a datetime.timedelta. :param unconditionally: When False (the default), a `NotLockedError` is raised if an unlocked lock is refreshed. :raises NotLockedError: if the lock is not set, unless optional `unconditionally` flag is set to True. """ if lifetime is not None: self._lifetime = lifetime # Do we have the lock? As a side effect, this refreshes the lock! if not self.is_locked and not unconditionally: raise NotLockedError('{0}: {1}'.format(repr(self), self._read())) def lock(self, timeout=None): """Acquire the lock. This blocks until the lock is acquired unless optional timeout is not None, in which case a `TimeOutError` is raised when the timeout expires without lock acquisition. :param timeout: A datetime.timedelta indicating approximately how long the lock acquisition attempt should be made. None (the default) means keep trying forever. :raises AlreadyLockedError: if the lock is already acquired. :raises TimeOutError: if `timeout` is not None and the indicated time interval expires without a lock acquisition. """ if timeout is not None: timeout_time = datetime.datetime.now() + timeout # Make sure the claim file exists, and that its contents are current. self._write() # XXX This next call can fail with an EPERM. I have no idea why, but # I'm nervous about wrapping this in a try/except. It seems to be a # very rare occurrence, only happens from cron, and has only(?) been # observed on Solaris 2.6. self._touch() log.debug('laying claim: {0}'.format(self._lockfile)) # For quieting the logging output loopcount = -1 while True: loopcount += 1 # Create the hard link and test for exactly 2 links to the file. try: os.link(self._claimfile, self._lockfile) # If we got here, we know we know we got the lock, and never # had it before, so we're done. Just touch it again for the # fun of it. log.debug('got the lock: {0}'.format(self._lockfile)) self._touch() break except OSError as error: # The link failed for some reason, possibly because someone # else already has the lock (i.e. we got an EEXIST), or for # some other bizarre reason. if error.errno in ERRORS: # XXX in some Linux environments, it is possible to get an # ENOENT, which is truly strange, because this means that # self._claimfile didn't exist at the time of the # os.link(), but self._write() is supposed to guarantee # that this happens! I don't honestly know why this # happens -- possibly due to weird caching file systems? # -- but for now we just say we didn't acquire the lock # and try again next time. pass elif error.errno != errno.EEXIST: # Something very bizarre happened. Clean up our state and # pass the error on up. log.exception('unexpected link') os.unlink(self._claimfile) raise elif self._linkcount != 2: # Somebody's messin' with us! Log this, and try again # later. XXX should we raise an exception? log.error('unexpected linkcount: {0:d}'.format( self._linkcount)) elif self._read() == self._claimfile: # It was us that already had the link. log.debug('already locked: {0}'.format(self._lockfile)) raise AlreadyLockedError('We already had the lock') # otherwise, someone else has the lock pass # We did not acquire the lock, because someone else already has # it. Have we timed out in our quest for the lock? if timeout is not None and timeout_time < datetime.datetime.now(): os.unlink(self._claimfile) log.error('timed out') raise TimeOutError('Could not acquire the lock') # Okay, we haven't timed out, but we didn't get the lock. Let's # find if the lock lifetime has expired. Cache the release time # to avoid race conditions. (LP: #827052) release_time = self._releasetime if (release_time != -1 and datetime.datetime.now() > release_time + CLOCK_SLOP): # Yes, so break the lock. self._break() log.error('lifetime has expired, breaking') # Okay, someone else has the lock, our claim hasn't timed out yet, # and the expected lock lifetime hasn't expired yet either. So # let's wait a while for the owner of the lock to give it up. elif not loopcount % 100: log.debug('waiting for claim: {0}'.format(self._lockfile)) self._sleep() def unlock(self, unconditionally=False): """Release the lock. :param unconditionally: When False (the default), a `NotLockedError` is raised if this is called on an unlocked lock. :raises NotLockedError: if we don't own the lock, either because of unbalanced unlock calls, or because the lock was stolen out from under us, unless optional `unconditionally` is True. """ is_locked = self.is_locked if not is_locked and not unconditionally: raise NotLockedError('Already unlocked') # If we owned the lock, remove the lockfile, relinquishing the lock. if is_locked: try: os.unlink(self._lockfile) except OSError as error: if error.errno not in ERRORS: raise # Remove our claim file. try: os.unlink(self._claimfile) except OSError as error: if error.errno not in ERRORS: raise log.debug('unlocked: {0}'.format(self._lockfile)) @property def is_locked(self): """True if we own the lock, False if we do not. Checking the status of the lock resets the lock's lifetime, which helps avoid race conditions during the lock status test. """ # Discourage breaking the lock for a while. try: self._touch() except OSError as error: if error.errno == errno.EPERM: # We can't touch the file because we're not the owner. I # don't see how we can own the lock if we're not the owner. return False else: raise # XXX Can the link count ever be > 2? if self._linkcount != 2: return False return self._read() == self._claimfile def finalize(self): """Unconditionally unlock the file.""" log.debug('finalize: {0}'.format(self._lockfile)) self.unlock(unconditionally=True) def __del__(self): log.debug('__del__: {0}'.format(self._lockfile)) if self._owned: self.finalize() # Python 2.5 context manager protocol support. def __enter__(self): self.lock() return self def __exit__(self, exc_type, exc_val, exc_tb): self.unlock() # Don't suppress any exception that might have occurred. return False def transfer_to(self, pid): """Transfer ownership of the lock to another process. Use this only if you're transfering ownership to a child process across a fork. Use at your own risk, but it should be race-condition safe. transfer_to() is called in the parent, passing in the pid of the child. take_possession() is called in the child, and blocks until the parent has transferred possession to the child. disown() is used to set the 'owned' flag to False, and it is a disgusting wart necessary to make forced lock acquisition work. :( :param pid: The process id of the child process that will take possession of the lock. """ # First touch it so it won't get broken while we're fiddling about. self._touch() # Find out current claim's file name winner = self._read() # Now twiddle ours to the given pid. self._set_claimfile(pid) # Create a hard link from the global lock file to the claim file. # This actually does things in reverse order of normal operation # because we know that lockfile exists, and claimfile better not! os.link(self._lockfile, self._claimfile) # Now update the lock file to contain a reference to the new owner self._write() # Toggle off our ownership of the file so we don't try to finalize it # in our __del__() self._owned = False # Unlink the old winner, completing the transfer. os.unlink(winner) # And do some sanity checks assert self._linkcount == 2, ( 'Unexpected link count: wanted 2, got {0:d}'.format( self._linkcount)) assert self.is_locked, 'Expected to be locked' log.debug('transferred the lock: {0}'.format(self._lockfile)) def take_possession(self): """Take possession of a lock from another process. See `transfer_to()` for more information. """ self._set_claimfile() # Wait until the linkcount is 2, indicating the parent has completed # the transfer. while self._linkcount != 2 or self._read() != self._claimfile: time.sleep(0.25) log.debug('took possession of the lock: {0}'.format(self._lockfile)) def disown(self): """Disown this lock. See `transfer_to()` for more information. """ self._owned = False def _set_claimfile(self, pid=None): """Set the _claimfile private variable.""" if pid is None: pid = os.getpid() # Calculate a hard link file name that will be used to lay claim to # the lock. We need to watch out for two Lock objects in the same # process pointing to the same lock file. Without this, if you lock # lf1 and do not lock lf2, lf2.locked() will still return True. self._claimfile = SEP.join(( self._lockfile, socket.getfqdn(), str(pid), str(random.randint(0, MAXINT)), )) def _write(self): """Write our claim file's name to the claim file.""" # Make sure it's group writable fp = open(self._claimfile, 'w') try: fp.write(self._claimfile) finally: fp.close() @property def retry_errnos(self): """The set of errno values that cause a read retry.""" return self._retry_errnos[:] @retry_errnos.setter def retry_errnos(self, errnos): self._retry_errnos = [] self._retry_errnos.extend(errnos) @retry_errnos.deleter def retry_errnos(self): self._retry_errnos = [] def _read(self): """Read the contents of our lock file. :return: The contents of the lock file or None if the lock file does not exist. """ while True: try: with open(self._lockfile) as fp: return fp.read() except EnvironmentError as error: if error.errno in self._retry_errnos: self._sleep() elif error.errno not in ERRORS: raise else: return None def _touch(self, filename=None): """Touch the claim file into the future. :param filename: If given, the file to touch, otherwise our claim file is touched. """ expiration_date = datetime.datetime.now() + self._lifetime t = time.mktime(expiration_date.timetuple()) try: # XXX We probably don't need to modify atime, but this is easier. os.utime(filename or self._claimfile, (t, t)) except OSError as error: if error.errno not in ERRORS: raise @property def _releasetime(self): """The time when the lock should be released. :return: The mtime of the file, which is when the lock should be released, or -1 if the lockfile doesn't exist. """ try: return datetime.datetime.fromtimestamp( os.stat(self._lockfile).st_mtime) except OSError as error: if error.errno in ERRORS: return -1 raise @property def _linkcount(self): """The number of hard links to the lock file. :return: the number of hard links to the lock file, or -1 if the lock file doesn't exist. """ try: return os.stat(self._lockfile).st_nlink except OSError as error: if error.errno in ERRORS: return -1 raise def _break(self): """Break the lock.""" # First, touch the lock file. This reduces but does not eliminate the # chance for a race condition during breaking. Two processes could # both pass the test for lock expiry in lock() before one of them gets # to touch the lock file. This shouldn't be too bad because all # they'll do in that function is delete the lock files, not claim the # lock, and we can be defensive for ENOENTs here. # # Touching the lock could fail if the process breaking the lock and # the process that claimed the lock have different owners. Don't do # that. try: self._touch(self._lockfile) except OSError as error: if error.errno != errno.EPERM: raise # Get the name of the old winner's temp file. winner = self._read() # Remove the global lockfile, which actually breaks the lock. try: os.unlink(self._lockfile) except OSError as error: if error.errno not in ERRORS: raise # Try to remove the old winner's claim file, since we're assuming the # winner process has hung or died. Don't worry too much if we can't # unlink their claim file -- this doesn't wreck the locking algorithm, # but will leave claim file turds laying around, a minor inconvenience. try: if winner: os.unlink(winner) except OSError as error: if error.errno not in ERRORS: raise def _sleep(self): """Snooze for a random amount of time.""" interval = random.random() * 2.0 + 0.01 time.sleep(interval) flufl.lock-2.4.1/flufl/lock/conf.py0000664000175000017500000001537112476403672017420 0ustar barrybarry00000000000000# -*- coding: utf-8 -*- # # flufl.lock documentation build configuration file, created by # sphinx-quickstart on Thu Jan 7 18:41:30 2010. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. from __future__ import print_function import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.append(os.path.abspath('.')) # -- General configuration ----------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc'] # Add any paths that contain templates here, relative to this directory. templates_path = ['../../_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8' # The master toctree document. master_doc = 'README' # General information about the project. project = 'flufl.lock' copyright = '2004-2015, Barry A. Warsaw' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # from flufl.lock import __version__ # The short X.Y version. version = __version__ # The full version, including alpha/beta/rc tags. release = __version__ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of documents that shouldn't be included in the build. #unused_docs = [] # List of directories, relative to source directory, that shouldn't be searched # for source files. exclude_trees = ['_build', 'build', 'flufl.lock.egg-info', 'distribute-0.6.10'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['../../_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_use_modindex = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = '' # Output file base name for HTML help builder. htmlhelp_basename = 'flufllockdoc' # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('README.rst', 'flufllock.tex', 'flufl.lock Documentation', 'Barry A. Warsaw', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_use_modindex = True import errno def index_html(): cwd = os.getcwd() try: os.chdir('build/sphinx/html') try: os.unlink('index.html') except OSError as error: if error.errno != errno.ENOENT: raise os.symlink('README.html', 'index.html') print('index.html -> README.html') finally: os.chdir(cwd) import atexit atexit.register(index_html) flufl.lock-2.4.1/flufl/lock/README.rst0000664000175000017500000000367512527515322017605 0ustar barrybarry00000000000000================================== flufl.lock - An NFS-safe file lock ================================== This package is called ``flufl.lock``. It is an NFS-safe file-based lock with timeouts for POSIX systems. Requirements ============ ``flufl.lock`` requires Python 2.6 or newer, and is compatible with Python 3. Documentation ============= A `simple guide`_ to using the library is available within this package, in the form of doctests. Project details =============== * Project home: https://gitlab.com/warsaw/flufl.lock * Report bugs at: https://gitlab.com/warsaw/flufl.lock/issues * Code hosting: git@gitlab.com:warsaw/flufl.lock.git * Documentation: http://flufllock.readthedocs.org/ You can install it with ``pip``:: % pip install flufl.lock You can grab the latest development copy of the code using git. The master repository is hosted on GitLab. If you have git installed, you can grab your own branch of the code like this:: $ git clone git@gitlab.com:warsaw/flufl.lock.git You may contact the author via barry@python.org. Copyright ========= Copyright (C) 2004-2015 Barry A. Warsaw This file is part of flufl.lock. flufl.lock is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. flufl.lock is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with flufl.lock. If not, see . Table of Contents ================= .. toctree:: :glob: docs/using NEWS .. _`simple guide`: docs/using.html .. _`virtualenv`: http://www.virtualenv.org/en/latest/index.html flufl.lock-2.4.1/flufl/lock/tests/0000775000175000017500000000000012614427341017245 5ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/tests/test_lockfile.py0000664000175000017500000000605612476403672022464 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, version 3 of the License. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . """Testing other aspects of the implementation and API.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ ] import os import errno import tempfile import unittest try: # Python 3 import builtins except ImportError: # Python 2 import __builtin__ as builtins from flufl.lock._lockfile import Lock, NotLockedError class TestableEnvironmentError(EnvironmentError): def __init__(self, errno): super(TestableEnvironmentError, self).__init__() self.errno = errno EMOCKEDFAILURE = 99 EOTHERMOCKEDFAILURE = 98 class ErrnoRetryTests(unittest.TestCase): def setUp(self): self._builtin_open = builtins.open self._failure_countdown = None self._retry_count = None self._errno = EMOCKEDFAILURE fd, self._lockfile = tempfile.mkstemp('.lck') os.close(fd) self._lock = Lock(self._lockfile) def tearDown(self): self._disable() try: self._lock.unlock() except NotLockedError: pass try: os.remove(self._lockfile) except OSError as error: if error.errno != errno.ENOENT: raise def _enable(self): builtins.open = self._testable_open self._failure_countdown = 3 self._retry_count = 0 def _disable(self): builtins.open = self._builtin_open def _testable_open(self, *args, **kws): if self._failure_countdown <= 0: return self._builtin_open(*args, **kws) self._failure_countdown -= 1 self._retry_count += 1 raise TestableEnvironmentError(self._errno) def test_retry_errno_api(self): self.assertEqual(self._lock.retry_errnos, []) self._lock.retry_errnos = [EMOCKEDFAILURE, EOTHERMOCKEDFAILURE] self.assertEqual(self._lock.retry_errnos, [EMOCKEDFAILURE, EOTHERMOCKEDFAILURE]) del self._lock.retry_errnos self.assertEqual(self._lock.retry_errnos, []) def test_retries(self): # Test that _read() will retry when a given errno is encountered. self._lock.lock() self._lock.retry_errnos = [self._errno] self._enable() self.assertTrue(self._lock.is_locked) # The _read() trigged by the .is_locked call should have been retried. self.assertEqual(self._retry_count, 3) flufl.lock-2.4.1/flufl/lock/tests/subproc.py0000664000175000017500000000405612476403672021310 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, version 3 of the License. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . """A helper for setting up subprocess locks.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ '_acquire', '_waitfor', ] import time import multiprocessing from flufl.lock import Lock def child_locker(filename, lifetime, queue): # First, acquire the file lock. with Lock(filename, lifetime): # Now inform the parent that we've acquired the lock. queue.put(True) # Keep the file lock for a while. time.sleep(lifetime.seconds - 1) def _acquire(filename, lifetime=None): """Acquire the named lock file in a subprocess.""" queue = multiprocessing.Queue() proc = multiprocessing.Process(target=child_locker, args=(filename, lifetime, queue)) proc.start() while not queue.get(): time.sleep(0.1) def child_waitfor(filename, lifetime, queue): t0 = time.time() # Try to acquire the lock. with Lock(filename, lifetime): # Tell the parent how long it took to acquire the lock. queue.put(time.time() - t0) def _waitfor(filename, lifetime): """Fire off a child that waits for a lock.""" queue = multiprocessing.Queue() proc = multiprocessing.Process(target=child_waitfor, args=(filename, lifetime, queue)) proc.start() time = queue.get() return time flufl.lock-2.4.1/flufl/lock/tests/__init__.py0000664000175000017500000000000012476403672021353 0ustar barrybarry00000000000000flufl.lock-2.4.1/flufl/lock/tests/test_documentation.py0000664000175000017500000001024712476403672023542 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . """Test harness for doctests.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ 'additional_tests', ] import os import errno import atexit import doctest import logging import tempfile import unittest from datetime import timedelta from io import StringIO from pkg_resources import ( resource_filename, resource_exists, resource_listdir, cleanup_resources) COMMASPACE = ', ' DOT = '.' DOCTEST_FLAGS = ( doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE | doctest.REPORT_NDIFF) # For logging debugging. log_stream = StringIO() def stop(): """Call into pdb.set_trace()""" # Do the import here so that you get the wacky special hacked pdb instead # of Python's normal pdb. import pdb pdb.set_trace() def make_temporary_lockfile(testobj): """Make a temporary lock file for the tests.""" def lockfile_creator(): fd, testobj._lockfile = tempfile.mkstemp() os.close(fd) os.remove(testobj._lockfile) return testobj._lockfile return lockfile_creator def setup(testobj): """Test setup.""" # Truncate the log. log_stream.truncate() # Note that the module has a default built-in *clock slop* of 10 seconds # to handle differences in machine clocks. Since this test is happening on # the same machine, we can bump the slop down to a more reasonable number. from flufl.lock import _lockfile testobj._slop = _lockfile.CLOCK_SLOP _lockfile.CLOCK_SLOP = timedelta(seconds=0) # Make sure future statements in our doctests match the Python code. When # run with 2to3, the future import gets removed and these names are not # defined. try: testobj.globs['absolute_import'] = absolute_import testobj.globs['print_function'] = print_function testobj.globs['unicode_literals'] = unicode_literals except NameError: pass testobj.globs['temporary_lockfile'] = make_temporary_lockfile(testobj) testobj.globs['log_stream'] = log_stream testobj.globs['stop'] = stop def teardown(testobj): """Test teardown.""" # Restore the original clock slop. from flufl.lock import _lockfile _lockfile.CLOCK_SLOP = testobj._slop try: os.remove(testobj._lockfile) except OSError as error: if error.errno != errno.ENOENT: raise except AttributeError: # lockfile_creator() was never called. pass def additional_tests(): "Run the doc tests (README.rst and docs/*, if any exist)" # Initialize logging for testing purposes. logging.basicConfig(stream=log_stream, level=logging.DEBUG, datefmt='%b %d %H:%M:%S %Y', format='%(asctime)s (%(process)d) %(message)s', ) doctest_files = [ os.path.abspath(resource_filename('flufl.lock', 'README.rst'))] if resource_exists('flufl.lock', 'docs'): for name in resource_listdir('flufl.lock', 'docs'): if name.endswith('.rst'): doctest_files.append( os.path.abspath( resource_filename('flufl.lock', 'docs/%s' % name))) kwargs = dict(module_relative=False, optionflags=DOCTEST_FLAGS, setUp=setup, tearDown=teardown, ) atexit.register(cleanup_resources) return unittest.TestSuite(( doctest.DocFileSuite(*doctest_files, **kwargs))) flufl.lock-2.4.1/flufl/__init__.py0000664000175000017500000000156312476403672017300 0ustar barrybarry00000000000000# Copyright (C) 2004-2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 of the License. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public # License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . # this is a namespace package try: import pkg_resources pkg_resources.declare_namespace(__name__) except ImportError: import pkgutil __path__ = pkgutil.extend_path(__path__, __name__) flufl.lock-2.4.1/tox.ini0000664000175000017500000000011212606257330015350 0ustar barrybarry00000000000000[tox] envlist = py27,py34,py35 [testenv] commands = python setup.py test flufl.lock-2.4.1/PKG-INFO0000664000175000017500000001461712614427341015151 0ustar barrybarry00000000000000Metadata-Version: 1.1 Name: flufl.lock Version: 2.4.1 Summary: NFS-safe file locking with timeouts for POSIX systems. Home-page: http://launchpad.net/flufl.lock Author: Barry Warsaw Author-email: barry@python.org License: LGPLv3 Download-URL: https://pypi.python.org/pypi/flufl.lock Description: ========== flufl.lock ========== NFS-safe file locking with timeouts for POSIX systems. The `flufl.lock` library provides an NFS-safe file-based locking algorithm influenced by the GNU/Linux `open(2)` manpage, under the description of the `O_EXCL` option. [...] O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. The assumption made here is that there will be no *outside interference*, e.g. no agent external to this code will ever `link()` to the specific lock files used. Lock objects support lock-breaking so that you can't wedge a process forever. This is especially helpful in a web environment, but may not be appropriate for all applications. Locks have a *lifetime*, which is the maximum length of time the process expects to retain the lock. It is important to pick a good number here because other processes will not break an existing lock until the expected lifetime has expired. Too long and other processes will hang; too short and you'll end up trampling on existing process locks -- and possibly corrupting data. In a distributed (NFS) environment, you also need to make sure that your clocks are properly synchronized. Author ====== `flufl.lock` is Copyright (C) 2007-2015 Barry Warsaw Licensed under the terms of the GNU Lesser General Public License, version 3 or later. See the COPYING.LESSER file for details. Project details =============== * Project home: https://gitlab.com/warsaw/flufl.lock * Report bugs at: https://gitlab.com/warsaw/flufl.lock/issues * Code hosting: git@gitlab.com:warsaw/flufl.lock.git * Documentation: http://flufllock.readthedocs.org/ =================== NEWS for flufl.lock =================== 2.4.1 (2015-10-29) ================== * Fix the MANIFEST.in so that tox.ini is included in the sdist. 2.4 (2015-10-10) ================ * Drop Python 2.6 compatibility. * Add Python 3.5 compatibility. 2.3.1 (2014-09-26) ================== * Include MANIFEST.in in the sdist tarball, otherwise the Debian package won't built correctly. 2.3 (2014-09-25) ================ * Fix documentation bug. (LP: #1026403) * Catch ESTALE along with ENOENT, as NFS servers are supposed to (but don't always) throw ESTALE instead of ENOENT. (LP: #977999) * Purge all references to `distribute`. (LP: #1263794) 2.2.1 (2012-04-19) ================== * Add classifiers to setup.py and make the long description more compatible with the Cheeseshop. * Other changes to make the Cheeseshop page look nicer. (LP: #680136) * setup_helper.py version 2.1. 2.2 (2012-01-19) ================ * Support Python 3 without the use of 2to3. * Make the documentation clear that the `flufl.test.subproc` functions are not part of the public API. (LP: #838338) * Fix claim file format in `take_possession()`. (LP: #872096) * Provide a new API for dealing with possible additional unexpected errnos while trying to read the lock file. These can happen in some NFS environments. If you want to retry the read, set the lock file's `retry_errnos` property to a sequence of errnos. If one of those errnos occurs, the read is unconditionally (and infinitely) retried. `retry_errnos` is a property which must be set to a sequence; it has a getter and a deleter too. (LP: #882261) 2.1.1 (2011-08-20) ================== * Fixed TypeError in .lock() method due to race condition in _releasetime property. Found by Stephen A. Goss. (LP: #827052) 2.1 (2010-12-22) ================ * Added lock.details. 2.0.2 (2010-12-19) ================== * Small adjustment to doctest. 2.0.1 (2010-11-27) ================== * Add missing exception to __all__. 2.0 (2010-11-26) ================ * Package renamed to flufl.lock. Earlier ======= Try `bzr log lp:flufl.lock` for details. Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+) Classifier: Operating System :: POSIX Classifier: Operating System :: MacOS :: MacOS X Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Software Development :: Libraries Classifier: Topic :: Software Development :: Libraries :: Python Modules flufl.lock-2.4.1/template.py0000664000175000017500000000145712476403672016246 0ustar barrybarry00000000000000# Copyright (C) 2015 Barry A. Warsaw # # This file is part of flufl.lock. # # flufl.lock is free software: you can redistribute it and/or modify it under # the terms of the GNU Lesser General Public License as published by the Free # Software Foundation, version 3 of the License. # # flufl.lock is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License # for more details. # # You should have received a copy of the GNU Lesser General Public License # along with flufl.lock. If not, see . """Module contents.""" from __future__ import absolute_import, print_function, unicode_literals __metaclass__ = type __all__ = [ ] flufl.lock-2.4.1/README.rst0000664000175000017500000000406212527515265015542 0ustar barrybarry00000000000000========== flufl.lock ========== NFS-safe file locking with timeouts for POSIX systems. The `flufl.lock` library provides an NFS-safe file-based locking algorithm influenced by the GNU/Linux `open(2)` manpage, under the description of the `O_EXCL` option. [...] O_EXCL is broken on NFS file systems, programs which rely on it for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. The assumption made here is that there will be no *outside interference*, e.g. no agent external to this code will ever `link()` to the specific lock files used. Lock objects support lock-breaking so that you can't wedge a process forever. This is especially helpful in a web environment, but may not be appropriate for all applications. Locks have a *lifetime*, which is the maximum length of time the process expects to retain the lock. It is important to pick a good number here because other processes will not break an existing lock until the expected lifetime has expired. Too long and other processes will hang; too short and you'll end up trampling on existing process locks -- and possibly corrupting data. In a distributed (NFS) environment, you also need to make sure that your clocks are properly synchronized. Author ====== `flufl.lock` is Copyright (C) 2007-2015 Barry Warsaw Licensed under the terms of the GNU Lesser General Public License, version 3 or later. See the COPYING.LESSER file for details. Project details =============== * Project home: https://gitlab.com/warsaw/flufl.lock * Report bugs at: https://gitlab.com/warsaw/flufl.lock/issues * Code hosting: git@gitlab.com:warsaw/flufl.lock.git * Documentation: http://flufllock.readthedocs.org/