txfixtures-0.2.6/000077500000000000000000000000001303764413000137455ustar00rootroot00000000000000txfixtures-0.2.6/AUTHORS000066400000000000000000000002731303764413000150170ustar00rootroot00000000000000Free Ekanayaka Free Ekanayaka Gavin Panella Manish Tomar Martin Pool txfixtures-0.2.6/ChangeLog000066400000000000000000000027631303764413000155270ustar00rootroot00000000000000CHANGES ======= 0.2.6 ----- * Fix test_no_expected_port_timeout failing if port 9999 is used 0.2.5 ----- * Disable journaling in MongoDB * Use the default 15 seconds timeout in MongoDB tests 0.2.4 ----- * Set minUptime correctly in ServiceIntegrationTest.test_non_executable_command 0.2.3 ----- * Fix flaky ServiceIntegrationTest.test_hung test 0.2.2 ----- * In Python 3, pass mongodb arguments as bytes not unicode * Add link to documentation in README (#3) 0.2.1 ----- * Exclude .travis.yml from release files * Add MongoDB fixture * Add Phantomjs fixture * Add Service fixture (#3) 0.1.5 ----- * Add travis and tox configuration * Switch to PBR * [r=cjwatson] Add basic sphinx documentation * [r=allenap] Attach the log using addDetail() instead of as an exception message * [r=bigjools] Remove spurious copy of get_pid_from_file() from txfixtures.tachandler * Use super() to call up * Attach the log using addDetail() instead of as an exception message * Update ignores * Fix lint * Remove get_pid_from_file() from tachandler.py 0.1.4 ----- * Release 0.1.4 * twistd can't be invoked as a module; explain some more failure cases 0.1.3 ----- * Add more pypi metadata * Don't use PYTHONWARNINGS which only exists in python>=2.7 * rm dead code * Don't use relative imports, so we can cope with Python2.4 * Rename TacTestSetup to TacTestFixture 0.1.1 ----- * Declare Python dependencies * Fix Trove licence classification * Add distutils setup.py * txfixtures split out from lp and all tests are passing txfixtures-0.2.6/MANIFEST.in000066400000000000000000000000451303764413000155020ustar00rootroot00000000000000exclude .travis.yml exclude .mailmap txfixtures-0.2.6/Makefile000066400000000000000000000005451303764413000154110ustar00rootroot00000000000000PYTHON ?= python COVERAGE ?= $(PYTHON)-coverage SOURCE = txfixtures OMIT = $(SOURCE)/osutils.py,$(SOURCE)/tachandler.py check: rm -f .coverage $(COVERAGE) run --omit=$(OMIT) --source=$(SOURCE) -m testtools.run discover $(COVERAGE) report -m --fail-under=100 check-doc: $(MAKE) -C doc doctest html: $(MAKE) -C doc html .PHONY: check check-doc html txfixtures-0.2.6/PKG-INFO000066400000000000000000000021241303764413000150410ustar00rootroot00000000000000Metadata-Version: 1.1 Name: txfixtures Version: 0.2.6 Summary: Treat Twisted applications as Python test fixtures Home-page: https://launchpad.net/txfixtures Author: Martin Pool Author-email: mbp@canonical.com License: GPLv3 Description: Twisted integration with Python Testfixtures ============================================ txfixtures hooks into the testtools 'test fixture' interface, so that you can write tests that rely on having an external Twisted daemon. Documentation: http://txfixtures.readthedocs.io/ See: - https://launchpad.net/txfixtures - https://launchpad.net/testtools Licence: GPLv3 Platform: POSIX Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU General Public License (GPL) Classifier: Operating System :: POSIX Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Topic :: Software Development :: Quality Assurance Classifier: Topic :: Software Development :: Testing txfixtures-0.2.6/README.rst000066400000000000000000000005631303764413000154400ustar00rootroot00000000000000Twisted integration with Python Testfixtures ============================================ txfixtures hooks into the testtools 'test fixture' interface, so that you can write tests that rely on having an external Twisted daemon. Documentation: http://txfixtures.readthedocs.io/ See: - https://launchpad.net/txfixtures - https://launchpad.net/testtools Licence: GPLv3 txfixtures-0.2.6/doc/000077500000000000000000000000001303764413000145125ustar00rootroot00000000000000txfixtures-0.2.6/doc/Makefile000066400000000000000000000170201303764413000161520ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " applehelp to make an Apple Help Book" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" @echo " coverage to run coverage check of the documentation (if enabled)" .PHONY: clean clean: rm -rf $(BUILDDIR)/* .PHONY: html html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." .PHONY: dirhtml dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." .PHONY: singlehtml singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." .PHONY: pickle pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." .PHONY: json json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." .PHONY: htmlhelp htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." .PHONY: qthelp qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/txreactorfixture.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/txreactorfixture.qhc" .PHONY: applehelp applehelp: $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp @echo @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." @echo "N.B. You won't be able to view it unless you put it in" \ "~/Library/Documentation/Help or install it in your application" \ "bundle." .PHONY: devhelp devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/txreactorfixture" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/txreactorfixture" @echo "# devhelp" .PHONY: epub epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." .PHONY: latex latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." .PHONY: latexpdf latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." .PHONY: latexpdfja latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." .PHONY: text text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." .PHONY: man man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." .PHONY: texinfo texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." .PHONY: info info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." .PHONY: gettext gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." .PHONY: changes changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." .PHONY: linkcheck linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." .PHONY: doctest doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." .PHONY: coverage coverage: $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage @echo "Testing of coverage in the sources finished, look at the " \ "results in $(BUILDDIR)/coverage/python.txt." .PHONY: xml xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." .PHONY: pseudoxml pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." txfixtures-0.2.6/doc/api.rst000066400000000000000000000006431303764413000160200ustar00rootroot00000000000000API documentation ================= Generated reference documentation for all the public functionality of txfixtures. .. toctree:: :maxdepth: 2 txfixtures.reactor ------------------ .. automodule:: txfixtures.reactor :members: txfixtures.service ------------------ .. automodule:: txfixtures.service :members: txfixtures.phantomjs -------------------- .. automodule:: txfixtures.phantomjs :members: txfixtures-0.2.6/doc/conf.py000066400000000000000000000227461303764413000160240ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # txfixtures documentation build configuration file, created by # sphinx-quickstart on Wed Nov 9 19:33:32 2016. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('..')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.viewcode', 'sphinx.ext.intersphinx', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = 'txfixtures' copyright = '2016, Free Ekanayaka' author = 'Free Ekanayaka' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. from txfixtures import __version__ version = __version__ # The full version, including alpha/beta/rc tags. release = __version__ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None default_role = 'obj' # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = True # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (relative to this directory) to use as a favicon of # the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Language to be used for generating the HTML full-text search index. # Sphinx supports the following languages: # 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja' # 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr' #html_search_language = 'en' # A dictionary with options for the search language support, empty by default. # Now only 'ja' uses this config value #html_search_options = {'type': 'default'} # The name of a javascript file (relative to the configuration directory) that # implements a search results scorer. If empty, the default will be used. #html_search_scorer = 'scorer.js' # Output file base name for HTML help builder. htmlhelp_basename = 'txfixturesdoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', # Latex figure (float) alignment #'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'txfixtures.tex', 'txfixtures Documentation', 'Free Ekanayaka', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'txfixtures', 'txfixtures Documentation', [author], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'txfixtures', 'txfixtures Documentation', author, 'txfixtures', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False intersphinx_mapping = { 'python': ('https://docs.python.org/3.5', None), 'twisted': ('http://twistedmatrix.com/documents/current/api', None), 'testtools': ('https://testtools.readthedocs.io/en/latest/', None), } autoclass_content = 'both' txfixtures-0.2.6/doc/index.rst000066400000000000000000000014441303764413000163560ustar00rootroot00000000000000.. txreactorfixture documentation master file, created by sphinx-quickstart on Wed Nov 9 19:33:32 2016. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Twisted integration with Python Testfixtures ============================================ txfixtures hooks into the testtools `test fixture`_ interface, so that you can write tests that rely on having an external Twisted daemon. See: * https://launchpad.net/txfixtures * https://launchpad.net/testtools Contents: .. toctree:: :maxdepth: 1 reactor service phantomjs api Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` .. _`test fixture`: http://testtools.readthedocs.io/en/latest/for-test-authors.html#fixtures txfixtures-0.2.6/doc/phantomjs.rst000066400000000000000000000024751303764413000172570ustar00rootroot00000000000000Setup a phantomjs Selenium driver ================================= The :class:`~txfixtures.phantomjs.PhantomJS` fixture starts a phantomjs_ service in the background and exposes it via its `webdriver` attribute, which can then be used by test cases for Selenium_-based assertions: .. doctest:: >>> from fixtures import FakeLogger >>> from testtools import TestCase >>> from txfixtures import Reactor, Service, PhantomJS >>> TWIST_COMMAND = "twistd -n web".split(" ") >>> class HTTPServerTest(TestCase): ... ... def setUp(self): ... super().setUp() ... self.logger = self.useFixture(FakeLogger()) ... self.useFixture(Reactor()) ... ... # Create a sample web server ... self.service = Service(TWIST_COMMAND) ... self.service.expectPort(8080) ... self.useFixture(self.service) ... ... self.phantomjs = self.useFixture(PhantomJS()) ... ... def test_home_page(self): ... self.phantomjs.webdriver.get("http://localhost:8080") ... self.assertEqual("Twisted Web Demo", self.phantomjs.webdriver.title) >>> test = HTTPServerTest(methodName="test_home_page") >>> test.run().wasSuccessful() True .. _phantomjs: http://phantomjs.org .. _Selenium: http://selenium-python.readthedocs.io/ txfixtures-0.2.6/doc/reactor.rst000066400000000000000000000034351303764413000167100ustar00rootroot00000000000000Run asynchronous code from test cases ===================================== The :class:`~txfixtures.reactor.Reactor` fixture can be used to drive asynchronous Twisted code from a regular synchronous Python :class:`~testtools.TestCase`. The approach differs from trial_ or `testtools twisted support`_: instead of starting the reactor in the main thread and letting it spin for a while waiting for the :class:`~twisted.internet.defer.Deferred` returned by the test to fire, this fixture will keep the reactor running in a background thread until cleanup. When used with testresources_'s :class:`FixtureResource` and :class:`OptimisingTestSuite`, this fixture makes it possible to have full control and monitoring over long-running processes that should be up for the whole test suite run, and maybe produce output useful for the test itself. The typical use case is integration testing. .. doctest:: >>> from testtools import TestCase >>> from twisted.internet import reactor >>> from twisted.internet.threads import blockingCallFromThread >>> from twisted.internet.utils import getProcessOutput >>> from txfixtures import Reactor >>> class TestUsingAsyncAPIs(TestCase): ... ... def setUp(self): ... super().setUp() ... self.useFixture(Reactor()) ... ... def test_uptime(self): ... out = blockingCallFromThread(reactor, getProcessOutput, b"uptime") ... self.assertIn("load average", out.decode("utf-8")) ... >>> test = TestUsingAsyncAPIs(methodName="test_uptime") >>> test.run().wasSuccessful() True .. _testresources: https://pypi.python.org/pypi/testresources .. _`testtools twisted support`: https://testtools.readthedocs.io/en/latest/twisted-support.html .. _trial: http://twistedmatrix.com/trac/wiki/TwistedTrial txfixtures-0.2.6/doc/service.rst000066400000000000000000000064631303764413000167150ustar00rootroot00000000000000Spawn, control and monitor test services ======================================== The :class:`~txfixtures.service.Service` fixture can be used to spawn a background service process (for instance a web application), and leave it running for the duration of the test suite (see :ref:`testresources-integration`). It supports real-time streaming of the service standard output to Python's :py:mod:`logging` system. Spawn a simple service fixture listening to a port -------------------------------------------------- Let's create a test that spawns a dummy HTTP server that listens to port 8080: .. doctest:: >>> import socket >>> from testtools import TestCase >>> from txfixtures import Reactor, Service >>> HTTP_SERVER = "python3 -m http.server 8080".split(" ") >>> class HTTPServerTest(TestCase): ... ... def setUp(self): ... super().setUp() ... self.useFixture(Reactor()) ... ... # Create a service fixture that will spawn the HTTP server ... # and wait for it to listen to port 8080. ... self.service = Service(HTTP_SERVER) ... self.service.expectPort(8080) ... ... self.useFixture(self.service) ... ... def test_connect(self): ... connection = socket.socket() ... connection.connect(("127.0.0.1", 8080)) ... self.assertEqual(connection.getsockname()[0], "127.0.0.1") >>> test = HTTPServerTest(methodName="test_connect") >>> test.run().wasSuccessful() True Forward standard output to the Python logging system ----------------------------------------------------- Let's spawn a simple HTTP server and have its standard output forwarded to the Python logging system: .. doctest:: >>> import requests >>> from fixtures import FakeLogger >>> TWIST_COMMAND = "twistd -n web".split(" ") # This format string will be used to build a regular expression to parse # each output line of the service, and map it to a Python LogRecord. A # sample output line from the twistd web command looks like: # # 2016-11-17T22:18:36+0000 [-] Site starting on 8080 # >>> TWIST_FORMAT = "{Y}-{m}-{d}T{H}:{M}:{S}\+0000 \[{name}\] {message}" # This output string will be used as a "marker" indicating that the service # has initialized, and should shortly start listening to the expected port (if # one was given). The fixture.setUp() method will intercept this marker and # then wait for the service to actually open the port. >>> TWIST_OUTPUT = "Site starting on 8080" >>> class TwistedWebTest(TestCase): ... ... def setUp(self): ... super().setUp() ... self.logger = self.useFixture(FakeLogger()) ... self.useFixture(Reactor()) ... self.service = Service(TWIST_COMMAND) ... self.service.setOutputFormat(TWIST_FORMAT) ... self.service.expectOutput(TWIST_OUTPUT) ... self.service.expectPort(8080) ... self.useFixture(self.service) ... ... def test_request(self): ... response = requests.get("http://localhost:8080") ... self.assertEqual(200, response.status_code) ... self.assertIn('"GET / HTTP/1.1" 200', self.logger.output) ... >>> test = TwistedWebTest(methodName="test_request") >>> test.run().wasSuccessful() True txfixtures-0.2.6/requirements.txt000066400000000000000000000000511303764413000172250ustar00rootroot00000000000000extras fixtures testtools twisted psutil txfixtures-0.2.6/setup.cfg000066400000000000000000000013661303764413000155740ustar00rootroot00000000000000[metadata] name = txfixtures summary = Treat Twisted applications as Python test fixtures home-page = https://launchpad.net/txfixtures description-file = README.rst author = Martin Pool author-email = mbp@canonical.com license = GPLv3 platform = POSIX classifier = Intended Audience :: Developers License :: OSI Approved :: GNU General Public License (GPL) Operating System :: POSIX Programming Language :: Python Programming Language :: Python :: 2 Topic :: Software Development :: Quality Assurance Topic :: Software Development :: Testing [extras] test = coverage retrying systemfixtures>=0.6.2 doc = sphinx phantomjs = selenium mongodb = pymongo [bdist_wheel] universal = 1 [egg_info] tag_svn_revision = 0 tag_build = tag_date = 0 txfixtures-0.2.6/setup.py000077500000000000000000000002241303764413000154600ustar00rootroot00000000000000#!/usr/bin/env python """distutils metadata/installer for txfixtures""" import setuptools setuptools.setup(setup_requires=['pbr>=1.3'], pbr=True) txfixtures-0.2.6/tests/000077500000000000000000000000001303764413000151075ustar00rootroot00000000000000txfixtures-0.2.6/tests/__init__.py000066400000000000000000000000001303764413000172060ustar00rootroot00000000000000txfixtures-0.2.6/tests/cannotlisten.tac000066400000000000000000000015621303764413000203050ustar00rootroot00000000000000# Copyright 2009 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """ This TAC is used for the TacTestSetupTestCase.test_couldNotListenTac test case in test_tachandler.py. It fails with a CannotListenError. """ __metaclass__ = type from twisted.application import ( internet, service, ) from twisted.internet import protocol application = service.Application('CannotListen') serviceCollection = service.IServiceCollection(application) # We almost certainly can't listen on port 1 (usually it requires root # permissions), so this should fail. internet.TCPServer(1, protocol.Factory()).setServiceParent(serviceCollection) # Just in case we can, try listening on port 1 *again*. This will fail. internet.TCPServer(1, protocol.Factory()).setServiceParent(serviceCollection) # vim: ft=python txfixtures-0.2.6/tests/okay.tac000066400000000000000000000013051303764413000165420ustar00rootroot00000000000000# Copyright 2011 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """ This TAC is used for the TacTestSetupTestCase.test_pidForNotRunningProcess test case in test_tachandler.py. It simply starts up correctly, listening on a port that should typically be free. """ __metaclass__ = type from twisted.application import service from twisted.application import ( internet, service, ) from twisted.internet import protocol application = service.Application('Okay') serviceCollection = service.IServiceCollection(application) internet.TCPServer(9876, protocol.Factory()).setServiceParent(serviceCollection) # vim: ft=python txfixtures-0.2.6/tests/test_mongodb.py000066400000000000000000000012021303764413000201400ustar00rootroot00000000000000from testtools import TestCase from fixtures import FakeLogger from txfixtures.reactor import Reactor from txfixtures.mongodb import MongoDB class MongoDBIntegrationTest(TestCase): def setUp(self): super(MongoDBIntegrationTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.useFixture(Reactor()) self.mongodb = MongoDB() def test_client(self): """ After setUp is run, the service is fully ready and the client connected. """ self.useFixture(self.mongodb) info = self.mongodb.client.server_info() self.assertIsNotNone(info) txfixtures-0.2.6/tests/test_phantomjs.py000066400000000000000000000016431303764413000205270ustar00rootroot00000000000000from testtools import TestCase from fixtures import FakeLogger from txfixtures.reactor import Reactor from txfixtures.service import Service from txfixtures.phantomjs import PhantomJS class PhantomJSIntegrationTest(TestCase): def setUp(self): super(PhantomJSIntegrationTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.useFixture(Reactor()) # Setup a local web server to test the WebDriver server = Service(["twist", "web"], timeout=5) server.expectOutput("Starting reactor...") server.expectPort(8080) self.useFixture(server) self.fixture = PhantomJS(timeout=5) def test_webdriver(self): """After setUp is run, the service is fully ready.""" self.useFixture(self.fixture) self.fixture.webdriver.get("http://localhost:8080") self.assertEqual("Twisted Web Demo", self.fixture.webdriver.title) txfixtures-0.2.6/tests/test_reactor.py000066400000000000000000000261331303764413000201640ustar00rootroot00000000000000import logging import threading from unittest import skipIf from six import b from six.moves.queue import Queue from testtools import ( TestCase, try_import, ) from testtools.monkey import MonkeyPatcher from fixtures import FakeLogger from twisted.internet import reactor from twisted.internet.utils import getProcessOutput from txfixtures._twisted.threading import CallFromThreadTimeout from txfixtures.reactor import Reactor TIMEOUT = 5 asyncio = try_import("asyncio") AsyncioSelectorReactor = try_import( "twisted.internet.asyncioreactor.AsyncioSelectorReactor") class ReactorPatcher(MonkeyPatcher): """Monkey patch reactor methods to simulate various failure scenarios.""" def __init__(self): super(ReactorPatcher, self).__init__() self._originalMainLoop = reactor.mainLoop self._originalCallFromThread = reactor.callFromThread self.add_patch(reactor, "mainLoop", self._mainLoop) self.add_patch(reactor, "callFromThread", self._callFromThread) self.crashingDo = None self.crashingNotify = None self.crashingAbruptly = False self.hangingDo = None self.callFromThreadTimeout = None def scheduleCrash(self, abruptly=False): """ When the reactor is run, it will hang until a value is put into the `crashingDo` queue, and then crash. :param abruptly: If True, then crash badly by simply exiting the thread, without even calling reactor.crash(). """ self.crashingDo = Queue() self.crashingNotify = Queue() self.crashingAbruptly = abruptly return self.crashingDo def scheduleHang(self): """ When the reactor is run, hang until a value is put into the `hangingDo` queue. """ self.hangingDo = Queue() return self.hangingDo def scheduleCallFromThreadTimeout(self, function): """ When the given function is called as argument via callFromThread, make it timeout. """ self.callFromThreadTimeout = function def restore(self): """Restore the original reactor methods.""" super(ReactorPatcher, self).restore() logging.info("Restoring reactor") if self.hangingDo: self.hangingDo.put(None) if self.crashingDo: self.crashingDo.put(None) reactor.crash() def _mainLoop(self): if self.hangingDo: self._waitAndHang() raise SystemExit(0) elif self.crashingDo: self._waitAndCrash() raise SystemExit(0) else: logging.info("Starting main loop") self._originalMainLoop() def _waitAndHang(self): logging.info("Hanging reactor") if self.crashingDo: self._waitAndCrash() logging.info("Waiting for hang queue") self.hangingDo.get(timeout=TIMEOUT) logging.info("Resuming hung main loop") self.hangingDo = None def _waitAndCrash(self): logging.info("Waiting for crash queue") self.crashingDo.get(timeout=TIMEOUT) abruptely = " abruptely" if self.crashingAbruptly else "" logging.info("Crashing main loop%s", abruptely) if not self.crashingAbruptly: reactor.crash() # Notify that we have successfully crashed self.crashingNotify.put(None) self.crashingDo = None def _callFromThread(self, f, *args, **kwargs): # We assume here that the only potential caller of the # reactor.callFromThread API is the interruptableCallFromThread # function defined in txfixtures._twisted.threading (since there's # no other direct or indirect use of reactor.callFromThread in the # code under test). # # The arguments that interruptableCallFromThread passes to # reactor.callFromThread are 2: # # - the queue to use for timing out the call # - the function to call in the main thread # # Here we check if the function argument matches the function that # we want to timeout. if args[1] == self.callFromThreadTimeout: logging.info("Trigger callFromThread timeout") def timeout(timeout=None): raise CallFromThreadTimeout() args[0].get = timeout return elif self.hangingDo or self.crashingDo: # Pretend we succeeded logging.info("Pretend callFromThread succeeded") args[0].put(None) return logging.info("Use original callFromThread") self._originalCallFromThread(f, *args, **kwargs) class ReactorIntegrationTest(TestCase): def setUp(self): super(ReactorIntegrationTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.patcher = ReactorPatcher() self.patcher.patch() self.fixture = Reactor() self.addCleanup(self._cleanup) def _cleanup(self): # Make sure that the thread has terminated and that the # reactor is back to a clean state. self.patcher.restore() if self.fixture.thread: logging.info("Waiting for thread to terminate") self.fixture.thread.join(timeout=TIMEOUT) assert not self.fixture.thread.isAlive(), "Thread did not stop" def test_reactor_running(self): """After setUp is run, the reactor is spinning.""" self.useFixture(self.fixture) self.assertTrue(reactor.running) @skipIf(not AsyncioSelectorReactor, "asyncio reactor not available") def test_asyncio_reactor(self): """It's possible to start a custom reactor, like the asyncio one.""" eventloop = asyncio.new_event_loop() self.fixture.reactor = AsyncioSelectorReactor(eventloop=eventloop) self.useFixture(self.fixture) self.assertTrue(self.fixture.reactor.running) # The asyncio loop is actually running ready = Queue() eventloop.call_soon_threadsafe(ready.put, None) self.assertIsNone(ready.get(timeout=TIMEOUT)) def test_separate_thread(self): """The reactor runs in a separate thread.""" self.useFixture(self.fixture) # Figure the number of active threads, excluding the twisted thread # pool. threads = [] for thread in threading.enumerate(): if thread.name.startswith("PoolThread-twisted.internet.reactor"): continue threads.append(thread) self.assertEqual(2, len(threads)) def test_call(self): """The call() method is a convenience around blockingFromThread.""" self.useFixture(self.fixture) output = self.fixture.call(TIMEOUT, getProcessOutput, b("uptime")) self.assertIn(b("load average"), output) def test_reset_thread_and_reactor_died(self): """ The reset() method creates a new thread if the initial one has died. """ self.useFixture(self.fixture) self.fixture.call(TIMEOUT, reactor.crash) self.fixture.thread.join(timeout=TIMEOUT) self.assertFalse(self.fixture.thread.isAlive()) self.fixture.reset() self.assertTrue(reactor.running) self.assertIn( "Twisted reactor thread died, trying to recover", self.logger.output) def test_reset_thread_died_but_reactor_is_running(self): """ If the reactor crashes badly and is left in a bad state (e.g. running), the fixtures tries a best-effort clean up. """ self.patcher.scheduleCrash(abruptly=True) self.useFixture(self.fixture) self.patcher.crashingDo.put(None) # At this point the thread should be dead and the reactor broken self.fixture.thread.join(TIMEOUT) self.assertFalse(self.fixture.thread.isAlive()) self.assertTrue(reactor.running) self.fixture.reset() self.assertIn( "Twisted reactor thread died, trying to recover", self.logger.output) self.assertIn( "Twisted reactor has broken state, trying to reset", self.logger.output) # Things should be back to normality self.assertTrue(self.fixture.thread.isAlive(), "Thread did not resume") self.assertTrue(reactor.running, "Reactor did not recover") def test_reset_thread_alive_but_reactor_is_not_running(self): """ The reset() method bails out if the thread is alive but the reactor doesn't appear to be running. """ self.patcher.scheduleHang() self.patcher.scheduleCrash() self.fixture.setUp() self.patcher.crashingDo.put(None) self.patcher.crashingNotify.get(timeout=TIMEOUT) # At this point the thread should be alive and the reactor broken self.assertTrue(self.fixture.thread.isAlive()) self.assertFalse(reactor.running) error = self.assertRaises(RuntimeError, self.fixture.reset) self.assertEqual("Hung reactor thread detected", str(error)) def test_cleanup_stops_thread_and_reactor(self): """After cleanUp is run, the reactor is stopped.""" self.fixture.setUp() self.fixture.cleanUp() self.assertFalse(self.fixture.thread.isAlive()) self.assertFalse(reactor.running) def test_cleanup_thread_not_alive(self): """ If the thread is not alive, the cleanup phase is essentially a no-op. """ self.fixture.setUp() self.fixture.call(TIMEOUT, reactor.crash) self.fixture.thread.join(TIMEOUT) self.fixture.cleanUp() # There's only the entry about starting the thread, since upon cleanup # nothing was running. self.assertNotIn( "Stopping Twisted reactor and wait for its thread", self.logger.output) self.assertNotIn( "Twisted reactor has broken state, trying to reset", self.logger.output) def test_cleanup_hung_thread(self): """ If cleanUp() detects a hung thread with no reactor running, an error is raised. """ self.patcher.scheduleHang() self.patcher.scheduleCrash() self.fixture.setUp() self.patcher.crashingDo.put(None) self.patcher.crashingNotify.get(timeout=TIMEOUT) # At this point the thread should be alive and the reactor stopped self.assertTrue(self.fixture.thread.isAlive()) self.assertFalse(reactor.running) error = self.assertRaises(RuntimeError, self.fixture.cleanUp) self.assertEqual("Hung reactor thread detected", str(error)) def test_cleanup_hung_reactor(self): """ If cleanUp() can't stop the reactor, an error is raised. """ self.patcher.scheduleHang() self.patcher.scheduleCallFromThreadTimeout(reactor.crash) self.fixture.setUp() # At this point the thread should be alive and the reactor running self.assertTrue(self.fixture.thread.isAlive()) self.assertTrue(reactor.running) error = self.assertRaises(RuntimeError, self.fixture.cleanUp) self.assertEqual("Could not stop the reactor", str(error)) txfixtures-0.2.6/tests/test_service.py000066400000000000000000000210251303764413000201600ustar00rootroot00000000000000import os import signal import socket from testtools import TestCase from testtools.twistedsupport import AsynchronousDeferredRunTest from fixtures import ( FakeLogger, MultipleExceptions, TempDir ) from systemfixtures import FakeExecutable from twisted.internet import reactor from twisted.internet.defer import ( TimeoutError, inlineCallbacks, ) from twisted.internet.error import ( ProcessTerminated, ProcessDone, ) from txfixtures.reactor import Reactor from txfixtures.service import ( Service, ServiceProtocol, ) class ServiceIntegrationTest(TestCase): def setUp(self): super(ServiceIntegrationTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.useFixture(Reactor()) self.script = self.useFixture(FakeExecutable()) self.fixture = Service([self.script.path.encode("utf-8")], timeout=1) def test_service_ready(self): """After setUp is run, the service is fully ready.""" self.script.out("hello") self.script.listen() self.script.sleep(2) self.fixture.expectOutput("hello") self.fixture.expectPort(self.script.port) self.useFixture(self.fixture) def test_unknown_command(self): """If an unknown command is given, setUp raises an error.""" self.fixture.command = [b"/foobar"] error = self.assertRaises(MultipleExceptions, self.fixture.setUp) self.assertIsInstance(error.args[0][1], ProcessTerminated) self.assertIn("No such file or directory", self.logger.output) def test_non_executable_command(self): """If the given command is not executable, setUp raises an error.""" executable = self.useFixture(TempDir()).join("foobar") with open(executable, "w") as fd: fd.write("") self.fixture.command = [executable.encode("utf-8")] self.fixture.protocol.minUptime = 2.5 error = self.assertRaises(MultipleExceptions, self.fixture.setUp) self.assertIsInstance(error.args[0][1], ProcessTerminated) def test_hung(self): """ If the given command doesn't terminate with SIGTERM, it's SIGKILL'ed. """ self.script.hang() self.fixture.protocol.timeout = 0.2 self.fixture.expectOutput("hanging") self.fixture.setUp() self.fixture.cleanUp() self.assertIn( "Service process didn't terminate, trying to kill it", self.logger.output) class ServiceProtocolIntegrationTest(TestCase): run_tests_with = AsynchronousDeferredRunTest.make_factory(timeout=5) def setUp(self): super(ServiceProtocolIntegrationTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.protocol = ServiceProtocol() self.process = None self.script = self.useFixture(FakeExecutable()) def tearDown(self): super(ServiceProtocolIntegrationTest, self).tearDown() if self.process and self.process.pid: os.kill(self.process.pid, signal.SIGKILL) return self.protocol.terminated @inlineCallbacks def test_ready(self): """ The `ready` deferred fires when the service is ready. """ self.script.sleep(1) self.process = reactor.spawnProcess(self.protocol, self.script.path) yield self.protocol.ready self.assertIn("Service process ready", self.logger.output) @inlineCallbacks def test_ready_with_expected_output(self): """ If an expected output is provided, the 'ready' deferred fires only when that output gets emitted. """ self.script.out("hello") self.script.sleep(1) self.protocol.expectedOutput = "hello" self.process = reactor.spawnProcess(self.protocol, self.script.path) yield self.protocol.ready self.assertIn("hello", self.logger.output) @inlineCallbacks def test_ready_with_expected_port(self): """ If an expected output is provided, the 'ready' deferred fires only when that output gets emitted. """ self.script.listen() self.script.sleep(1) self.protocol.expectedPort = self.script.port self.process = reactor.spawnProcess(self.protocol, self.script.path) yield self.protocol.ready sock = socket.socket() sock.connect(("localhost", self.script.port)) self.addCleanup(sock.close) self.assertIn("Service opened port", self.logger.output) @inlineCallbacks def test_ready_with_expected_port_retry(self): """ If the service takes a bit to listen to the expected port, the protocol will retry. """ self.script.sleep(1) self.script.listen() self.script.sleep(1) self.protocol.expectedPort = self.script.port self.process = reactor.spawnProcess(self.protocol, self.script.path) yield self.protocol.ready sock = socket.socket() sock.connect(("localhost", self.script.port)) self.addCleanup(sock.close) self.assertIn("Service port probe failed", self.logger.output) self.assertIn("Service opened port", self.logger.output) @inlineCallbacks def test_no_min_uptime(self): """If the service doesn't stay up for minUpTime, an error is raised.""" # Spawn a non-existing process, which will make os.execvp fail, # triggering ServiceProtocol.processExited almost immediately. self.process = reactor.spawnProcess(self.protocol, b"/foo/bar") try: yield self.protocol.ready except ProcessTerminated as error: self.assertEqual(1, error.exitCode) else: self.fail( "The 'ready' deferred did not errback, while we were expecting" "an error, due to the process not staying up for at least 0.1" "seconds") @inlineCallbacks def test_no_expected_output_exit(self): """ If the process exits while we're waiting for it to emit the expected output, the 'ready' deferred fires with an error. """ self.script.sleep(0.2) self.protocol.expectedOutput = "hello" self.process = reactor.spawnProcess(self.protocol, self.script.path) try: yield self.protocol.ready except ProcessDone as error: self.assertEqual(0, error.exitCode) else: self.fail("The 'ready' deferred did not errback") @inlineCallbacks def test_no_expected_output_timeout(self): """ If the process doesn't emit the expected output, the 'ready' deferred doesn't fire. """ self.script.sleep(1) self.protocol.expectedOutput = "hello" self.protocol.ready.addTimeout(0.2, reactor) self.process = reactor.spawnProcess(self.protocol, self.script.path) try: yield self.protocol.ready except TimeoutError as error: self.assertEqual(0.2, error.args[0]) self.assertNotIn("hello", self.logger.output) else: self.fail("The 'ready' deferred did not timeout while waiting" "for output") @inlineCallbacks def test_no_expected_port_timeout(self): """ If the process doesn't listen to the expected port, the 'ready' deferred doesn't fire. """ self.script.sleep(1) # Find an unused port sock = socket.socket() sock.bind(("localhost", 0)) self.addCleanup(sock.close) _, self.protocol.expectedPort = sock.getsockname() self.protocol.ready.addTimeout(0.2, reactor) self.process = reactor.spawnProcess(self.protocol, self.script.path) try: yield self.protocol.ready except TimeoutError as error: self.assertEqual(0.2, error.args[0]) else: self.fail( "The 'ready' deferred did not timeout while waiting for" "the process to listen to the expected port") @inlineCallbacks def test_no_expected_port_exit(self): """ If the process exits while we're waiting for it to open the expected port, the 'ready' deferred fires with an error. """ self.script.sleep(0.2) self.protocol.expectedPort = 9999 self.process = reactor.spawnProcess(self.protocol, self.script.path) try: yield self.protocol.ready except ProcessDone as error: self.assertEqual(0, error.exitCode) else: self.fail("The 'ready' deferred did not errback") txfixtures-0.2.6/tests/test_tachandler.py000066400000000000000000000112121303764413000206220ustar00rootroot00000000000000# Copyright 2009-2011 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for txfixtures.tachandler""" __metaclass__ = type import os from os.path import ( dirname, exists, join, ) import subprocess import warnings from fixtures import TempDir import testtools from testtools.matchers import ( Matcher, Mismatch, Not, ) from twisted.scripts import twistd from retrying import retry from txfixtures.tachandler import ( TacException, TacTestFixture, ) from txfixtures.osutils import ( get_pid_from_file, ) class SimpleTac(TacTestFixture): def __init__(self, name, tempdir, port): super(SimpleTac, self).__init__() self.name, self.tempdir = name, tempdir self.port = port def setUp(self): # The TWISTD_SCRIPT environment variable gets typically # set by tox (see tox.ini). super(SimpleTac, self).setUp( twistd_script=os.environ.get("TWISTD_SCRIPT")) @property def root(self): return dirname(__file__) @property def tacfile(self): return join(self.root, '%s.tac' % self.name) @property def pidfile(self): return join(self.tempdir, '%s.pid' % self.name) @property def logfile(self): return join(self.tempdir, '%s.log' % self.name) @property def daemon_port(self): return self.port def setUpRoot(self): pass class IsRunning(Matcher): """Ensures the `TacTestFixture`'s process is running.""" def match(self, fixture): pid = get_pid_from_file(fixture.pidfile) if pid is None or not exists("/proc/%d" % pid): return Mismatch("Fixture %r is not running." % fixture) def __str__(self): return self.__class__.__name__ class TacTestFixtureTestCase(testtools.TestCase): """Some tests for the error handling of TacTestFixture.""" def test_okay(self): """TacTestFixture sets up and runs a simple service.""" tempdir = self.useFixture(TempDir()).path fixture = SimpleTac("okay", tempdir, 9876) # Fire up the fixture, capturing warnings. with warnings.catch_warnings(record=True) as warnings_log: with fixture: self.assertThat(fixture, IsRunning()) self.assertThat(fixture, Not(IsRunning())) # No warnings are emitted. self.assertEqual([], warnings_log) def test_missingTac(self): """TacTestFixture raises TacException if the tacfile doesn't exist""" fixture = SimpleTac("missing", "/file/does/not/exist", 0) try: self.assertRaises(TacException, fixture.setUp) self.assertThat(fixture, Not(IsRunning())) finally: fixture.cleanUp() def test_couldNotListenTac(self): """If the tac fails due to not being able to listen on the needed port, TacTestFixture will fail. """ tempdir = self.useFixture(TempDir()).path fixture = SimpleTac("cannotlisten", tempdir, 1) # Since the process might take a small while to shutdown, we'll # retry a few times. retryingAssertThat = retry( stop_max_attempt_number=10, wait_fixed=100)(self.assertThat) try: self.assertRaises(TacException, fixture.setUp) retryingAssertThat(fixture, Not(IsRunning())) finally: fixture.cleanUp() def test_stalePidFile(self): """TacTestFixture complains about stale pid files.""" tempdir = self.useFixture(TempDir()).path fixture = SimpleTac("okay", tempdir, 9876) # Run a short-lived process with the intention of using its pid in the # next step. Linux uses pids sequentially (from the information I've # been able to discover) so this approach is safe as long as we don't # delay until pids wrap... which should be a very long time unless the # machine is seriously busy. process = subprocess.Popen("true") process.wait() # Put the (now bogus) pid in the pid file. with open(fixture.pidfile, "w") as pidfile: pidfile.write(str(process.pid)) # Fire up the fixture, capturing warnings. with warnings.catch_warnings(record=True) as warnings_log: try: self.assertRaises(TacException, fixture.setUp) self.assertThat(fixture, Not(IsRunning())) finally: fixture.cleanUp() # One deprecation warning is emitted. self.assertEqual(1, len(warnings_log)) self.assertIs(UserWarning, warnings_log[0].category) txfixtures-0.2.6/tox.ini000066400000000000000000000006641303764413000152660ustar00rootroot00000000000000[tox] envlist = py27,py35,doc skipsdist = True [tox:travis] 2.7 = py27 3.5 = py35,doc [testenv] usedevelop = True install_command = pip install -U {opts} {packages} deps = .[test,phantomjs,mongodb] whitelist_externals = make phantomjs setenv = TWISTD_SCRIPT={envbindir}/twistd COVERAGE=coverage commands = make check [testenv:doc] basepython = python3.5 deps = .[doc,test,phantomjs] commands = make check-doc txfixtures-0.2.6/txfixtures.egg-info/000077500000000000000000000000001303764413000176645ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures.egg-info/PKG-INFO000066400000000000000000000021241303764413000207600ustar00rootroot00000000000000Metadata-Version: 1.1 Name: txfixtures Version: 0.2.6 Summary: Treat Twisted applications as Python test fixtures Home-page: https://launchpad.net/txfixtures Author: Martin Pool Author-email: mbp@canonical.com License: GPLv3 Description: Twisted integration with Python Testfixtures ============================================ txfixtures hooks into the testtools 'test fixture' interface, so that you can write tests that rely on having an external Twisted daemon. Documentation: http://txfixtures.readthedocs.io/ See: - https://launchpad.net/txfixtures - https://launchpad.net/testtools Licence: GPLv3 Platform: POSIX Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU General Public License (GPL) Classifier: Operating System :: POSIX Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Topic :: Software Development :: Quality Assurance Classifier: Topic :: Software Development :: Testing txfixtures-0.2.6/txfixtures.egg-info/SOURCES.txt000066400000000000000000000021111303764413000215430ustar00rootroot00000000000000AUTHORS ChangeLog MANIFEST.in Makefile README.rst requirements.txt setup.cfg setup.py tox.ini doc/Makefile doc/api.rst doc/conf.py doc/index.rst doc/phantomjs.rst doc/reactor.rst doc/service.rst tests/__init__.py tests/cannotlisten.tac tests/okay.tac tests/test_mongodb.py tests/test_phantomjs.py tests/test_reactor.py tests/test_service.py tests/test_tachandler.py txfixtures/__init__.py txfixtures/mongodb.py txfixtures/osutils.py txfixtures/phantomjs.py txfixtures/reactor.py txfixtures/service.py txfixtures/tachandler.py txfixtures.egg-info/PKG-INFO txfixtures.egg-info/SOURCES.txt txfixtures.egg-info/dependency_links.txt txfixtures.egg-info/not-zip-safe txfixtures.egg-info/pbr.json txfixtures.egg-info/requires.txt txfixtures.egg-info/top_level.txt txfixtures/_twisted/__init__.py txfixtures/_twisted/testing.py txfixtures/_twisted/threading.py txfixtures/_twisted/tests/__init__.py txfixtures/_twisted/tests/test_threading.py txfixtures/tests/__init__.py txfixtures/tests/test_mongodb.py txfixtures/tests/test_phantomjs.py txfixtures/tests/test_reactor.py txfixtures/tests/test_service.pytxfixtures-0.2.6/txfixtures.egg-info/dependency_links.txt000066400000000000000000000000011303764413000237320ustar00rootroot00000000000000 txfixtures-0.2.6/txfixtures.egg-info/not-zip-safe000066400000000000000000000000011303764413000221120ustar00rootroot00000000000000 txfixtures-0.2.6/txfixtures.egg-info/pbr.json000066400000000000000000000000561303764413000213430ustar00rootroot00000000000000{"is_release": true, "git_version": "b0215e4"}txfixtures-0.2.6/txfixtures.egg-info/requires.txt000066400000000000000000000002201303764413000222560ustar00rootroot00000000000000extras fixtures testtools twisted psutil [doc] sphinx [mongodb] pymongo [phantomjs] selenium [test] coverage retrying systemfixtures>=0.6.2 txfixtures-0.2.6/txfixtures.egg-info/top_level.txt000066400000000000000000000000131303764413000224100ustar00rootroot00000000000000txfixtures txfixtures-0.2.6/txfixtures/000077500000000000000000000000001303764413000161725ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/__init__.py000066400000000000000000000007651303764413000203130ustar00rootroot00000000000000from pbr.version import VersionInfo from extras import try_import from .reactor import ( Reactor, ) from .service import Service # Since the PhantomJS fixure requires the Selenium Python package, we make # the import fail gracefully if it's not installed. PhantomJS = try_import("txfixtures.phantomjs.PhantomJS") __all__ = [ "Reactor", "Service", "PhantomJS", ] _v = VersionInfo("txfixtures").semantic_version() __version__ = _v.release_string() version_info = _v.version_tuple() txfixtures-0.2.6/txfixtures/_twisted/000077500000000000000000000000001303764413000200145ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/_twisted/__init__.py000066400000000000000000000000001303764413000221130ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/_twisted/testing.py000066400000000000000000000057741303764413000220600ustar00rootroot00000000000000"""Extensions to Twisted's stock testing helpers.""" import signal from twisted.test.proto_helpers import MemoryReactorClock from twisted.internet.error import ProcessTerminated from twisted.internet._baseprocess import BaseProcess from twisted.python.failure import Failure EXPECTED_SIGNALS = (signal.SIGTERM, signal.SIGTERM) class ThreadedMemoryReactorClock(MemoryReactorClock): """Extend Twisted's test reactor with more reactor-level features. :ivar async: A flag indicating whether callFromThread calls should be executed synchronously as soon as callFromThread is called. """ def __init__(self): super(ThreadedMemoryReactorClock, self).__init__() self.async = False self.process = MemoryProcess() self._internalReaders = set() def run(self, installSignalHandlers=False): super(ThreadedMemoryReactorClock, self).run() self.installSignalHandlers = installSignalHandlers self.running = True def crash(self): super(ThreadedMemoryReactorClock, self).crash() self.running = False def callFromThread(self, f, *args, **kwargs): if self.async: return f(*args, **kwargs) # Spin the timer until there are no delayed calls left, or until the # limit is reached. limit = 10 count = 0 while self.getDelayedCalls() and count < limit: call = self.getDelayedCalls()[0] self.advance(call.getTime() - self.seconds()) count += 1 def addReader(self, reader): reader.install = lambda: setattr(reader, "installed", True) super(ThreadedMemoryReactorClock, self).addReader(reader) def spawnProcess(self, processProtocol, executable, args=(), env={}, path=None, uid=None, gid=None, usePTY=0, childFDs=None): self.process.args = args self.process.proto = processProtocol self.process.pid = 123 self.process.proto.makeConnection(self.process) if self.process.data is not None: self.process.proto.outReceived(self.process.data) return self.process def connectTCP(self, host, port, factory, timeout=30, bindAddress=None): protocol = factory.buildProtocol(None) protocol.connectionMade() class MemoryProcess(BaseProcess): """Simulate a real :class:`~twisted.internet.process.Process`.""" def __init__(self): super(MemoryProcess, self).__init__(None) self.signals = EXPECTED_SIGNALS self.signalled = True # These are expected by the base class self.data = None self.executable = None def signalProcess(self, signalID): assert signalID in self.signals, "Unexpected signal: %s" % signalID self.signalled = True self.pid = None self.processEnded(signalID) def _getReason(self, status): status = signal = None if self.signalled: signal = status return Failure(ProcessTerminated(status=status, signal=signal)) txfixtures-0.2.6/txfixtures/_twisted/tests/000077500000000000000000000000001303764413000211565ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/_twisted/tests/__init__.py000066400000000000000000000000001303764413000232550ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/_twisted/tests/test_threading.py000066400000000000000000000024431303764413000245370ustar00rootroot00000000000000from six import b from testtools import TestCase from twisted.internet import reactor from twisted.internet.defer import ( Deferred, succeed, fail, ) from txfixtures._twisted.threading import ( CallFromThreadTimeout, interruptableCallFromThread, ) from txfixtures.reactor import Reactor class InterruptableCallFromThreadTest(TestCase): def setUp(self): super(InterruptableCallFromThreadTest, self).setUp() self.useFixture(Reactor()) def test_success(self): """ If the async call executed in the thread succeeds, the result is returned. """ self.assertEqual( "hello", interruptableCallFromThread(reactor, 1, lambda: succeed("hello"))) def test_fail(self): """ If the async call executed in the thread fails, an exception is raised. """ self.assertRaises( RuntimeError, interruptableCallFromThread, reactor, 1, lambda: fail(RuntimeError("boom"))) def test_timeout(self): """ If the async call takes more than the given timeout to execuluted, an error is raised. """ self.assertRaises( CallFromThreadTimeout, interruptableCallFromThread, reactor, 0.1, lambda: Deferred()) txfixtures-0.2.6/txfixtures/_twisted/threading.py000066400000000000000000000017371303764413000223430ustar00rootroot00000000000000"""Extensions to stock Twisted code.""" from six.moves.queue import ( Queue, Empty, ) from twisted.internet.defer import maybeDeferred from twisted.python.failure import Failure class CallFromThreadTimeout(Exception): """Raised when interruptableCallFromThread times out.""" def interruptableCallFromThread(reactor, timeout, f, *a, **kw): """An interruptable version of Twisted's blockingCallFromThread. This function has all arguments and semantics of the original one, plus a new 'timeout' argument that will make the call fail after the given amount of seconds. """ queue = Queue() def _callFromThread(queue, f): result = maybeDeferred(f, *a, **kw) result.addBoth(queue.put) reactor.callFromThread(_callFromThread, queue, f) try: result = queue.get(timeout=timeout) except Empty: raise CallFromThreadTimeout() if isinstance(result, Failure): result.raiseException() return result txfixtures-0.2.6/txfixtures/mongodb.py000066400000000000000000000026511303764413000201750ustar00rootroot00000000000000import pymongo from fixtures import TempDir from txfixtures.service import Service FORMAT = ( "{Y}-{m}-{d}T{H}:{M}:{S}\.{msecs}\+0000 {levelname} " "[A-Z]+ +\[{name}\] {message}") class MongoDB(Service): """Start and stop a `mongodb` process in the background. """ def __init__(self, mongod=b"mongod", args=(), **kwargs): command = [mongod] + list(args) super(MongoDB, self).__init__(command, **kwargs) self.expectOutput("waiting for connections on port") self.setOutputFormat(FORMAT) self.setClientKwargs() def setClientKwargs(self, **kwargs): """Additional keyword arguments to pass to the client.""" self.clientKwargs = kwargs @property def port(self): return self.protocol.expectedPort def _setUp(self): self.expectPort(self.allocatePort()) self._dbPath = self.useFixture(TempDir()) super(MongoDB, self)._setUp() uri = "mongodb://localhost:%d" % self.port self.client = pymongo.MongoClient(uri, **self.clientKwargs) self.addCleanup(self.client.close) # XXX Workaround pymongo leaving threads around. self.addCleanup(pymongo.periodic_executor._shutdown_executors) @property def _args(self): return self.command[:] + [ b"--port=%d" % self.port, b"--dbpath=%s" % self._dbPath.path.encode("utf-8"), b"--nojournal", ] txfixtures-0.2.6/txfixtures/osutils.py000066400000000000000000000070561303764413000202560ustar00rootroot00000000000000# Copyright 2009-2011 Canonical Ltd. This software is licensed under the # GNU General Public License version 3. """General os utilities useful for txfxtures.""" import errno import os import os.path from signal import ( SIGKILL, SIGTERM, ) import socket import time def _kill_may_race(pid, signal_number): """Kill a pid accepting that it may not exist.""" try: os.kill(pid, signal_number) except OSError as e: if e.errno in (errno.ESRCH, errno.ECHILD): # Process has already been killed. return # Some other issue (e.g. different user owns it) raise def get_pid_from_file(pidfile_path): """Retrieve the PID from the given file, if it exists, None otherwise.""" if not os.path.exists(pidfile_path): return None # Get the pid. with open(pidfile_path, 'r') as fd: pid = fd.read().split()[0] try: pid = int(pid) except ValueError: # pidfile contains rubbish return None return pid def two_stage_kill(pid, poll_interval=0.1, num_polls=50): """Kill process 'pid' with SIGTERM. If it doesn't die, SIGKILL it. :param pid: The pid of the process to kill. :param poll_interval: The polling interval used to check if the process is still around. :param num_polls: The number of polls to do before doing a SIGKILL. """ # Kill the process. _kill_may_race(pid, SIGTERM) # Poll until the process has ended. for i in range(num_polls): try: # Reap the child process and get its return value. If it's not # gone yet, continue. new_pid, result = os.waitpid(pid, os.WNOHANG) if new_pid: return result time.sleep(poll_interval) except OSError as e: if e.errno in (errno.ESRCH, errno.ECHILD): # Raised if the process is gone by the time we try to get the # return value. return # The process is still around, so terminate it violently. _kill_may_race(pid, SIGKILL) def kill_by_pidfile(pidfile_path, poll_interval=0.1, num_polls=50): """Kill a process identified by the pid stored in a file. The pid file is removed from disk. """ try: pid = get_pid_from_file(pidfile_path) if pid is None: return two_stage_kill(pid, poll_interval, num_polls) finally: remove_if_exists(pidfile_path) def remove_if_exists(path): """Remove the given file if it exists.""" try: os.remove(path) except OSError as e: if e.errno != errno.ENOENT: raise def until_no_eintr(retries, function, *args, **kwargs): """Run 'function' until it doesn't raise EINTR errors. :param retries: The maximum number of times to try running 'function'. :param function: The function to run. :param *args: Arguments passed to the function. :param **kwargs: Keyword arguments passed to the function. :return: The return value of 'function'. """ if not retries: return for i in range(retries): try: return function(*args, **kwargs) except (IOError, OSError) as e: if e.errno == errno.EINTR: continue raise except socket.error as e: # In Python 2.6 we can use IOError instead. It also has # reason.errno but we might be using 2.5 here so use the # index hack. if e[0] == errno.EINTR: continue raise else: raise txfixtures-0.2.6/txfixtures/phantomjs.py000066400000000000000000000027221303764413000205520ustar00rootroot00000000000000from selenium.webdriver.common.desired_capabilities import DesiredCapabilities from selenium.webdriver.remote import webdriver from fixtures import TempDir from txfixtures.service import Service FORMAT = ( "\[{levelname} +- +{Y}-{m}-{d}T{H}:{M}:{S}\.{msecs}Z\] {name} - {message}") class PhantomJS(Service): """Start and stop a `phantomjs` process in the background. """ def __init__(self, phantomjs="phantomjs", args=(), **kwargs): command = [phantomjs] + list(args) super(PhantomJS, self).__init__(command, **kwargs) #: Desired capabilities that will be passed to the webdriver. self.desiredCapabilities = DesiredCapabilities.PHANTOMJS #: A WebDriver instance pointing to the phantomjs process spawned #: by this fixture. self.webdriver = None self.expectOutput("running on port") self.setOutputFormat(FORMAT) def _setUp(self): self.expectPort(self.allocatePort()) self._cookies = self.useFixture(TempDir()).join("phantomjs.cookies") super(PhantomJS, self)._setUp() url = "http://localhost:%d/wd/hub" % self.protocol.expectedPort self.webdriver = webdriver.WebDriver( command_executor=url, desired_capabilities=self.desiredCapabilities) @property def _args(self): return self.command[:] + [ "--webdriver=%d" % self.protocol.expectedPort, "--cookies-file=%s" % self._cookies, ] txfixtures-0.2.6/txfixtures/reactor.py000066400000000000000000000200211303764413000201760ustar00rootroot00000000000000import sys import signal import logging import threading from six.moves.queue import Queue from fixtures import Fixture from twisted.internet import reactor as defaultTwistedReactor from twisted.internet.posixbase import _SIGCHLDWaker from txfixtures._twisted.threading import ( CallFromThreadTimeout, interruptableCallFromThread, ) TIMEOUT = 5 class Reactor(Fixture): """A fixture to run the Twisted reactor in a separate thread. This fixture will spawn a new thread in the test process and run the Twisted reactor in it. Test code can then invoke asynchronous APIs by using :func:`~twisted.internet.threads.blockingCallFromThread`. """ def __init__(self, reactor=None, timeout=TIMEOUT): """ :param reactor: The Twisted reactor to run. :param timeout: Raise an exception if the reactor or the thread is it runs in doesn't start (at setUp time) or doesn't stop (at cleanUp time) in this amount of seconds. :ivar thread: The `~threading.Thread` that the reactor runs in. """ super(Reactor, self).__init__() self.reactor = reactor or defaultTwistedReactor self.timeout = timeout self.thread = None def call(self, timeout, f, *a, **kw): """ Convenience around `~twisted.internet.threads.blockingCallFromThread`, with timeout support. The function `f` will be invoked in the reactor's thread with the given arguments and keyword arguments. If `f` returns a Deferred, the calling code will block until it has fired. :return: The value returned by `f` or the value fired by the Deferred it returned. If `f` traces back or the Deferred it returned errbacks, the relevant exception will be propagated to the caller of this method. :raises CallFromThreadTimeout: If `timeout` seconds have elapsed and the Deferred returned by `f` hasn't fired yet. """ return interruptableCallFromThread(self.reactor, timeout, f, *a, **kw) def reset(self): """Make sure that the reactor is still running. If the reactor and its thread have died, this method will try to recover them by creating a new thread and starting the reactor again. """ if not self.thread.isAlive(): # The thread died, let's try our best to recover. logging.warning("Twisted reactor thread died, trying to recover") self._stop() # Resets the reactor in case it's in a broken state. self._start() else: # The thread is still running, make sure the reactor as well. self._assertReactorRunning() def _setUp(self): logging.info("Starting Twisted reactor in a separate thread") self._start() def _start(self): ready = Queue() # Will be put None as soon as the reactor starts self.reactor.callWhenRunning(ready.put, None) self.thread = threading.Thread( target=self.reactor.run, # Don't let the reactor try to install signal handlers, since they # can only be installed from the main thread (we'll do it by hand # just below). kwargs=dict(installSignalHandlers=False), ) self.addCleanup(self._stop) # Run in daemon mode. No matter what happens, when the test process # exists we don't want to hang waitng for the reactor thread to # terminate. self.thread.daemon = True self.thread.start() # Wait for the reactor to actually start and double check it's spinning ready.get(timeout=self.timeout) assert self.reactor.running, "Could not start the reactor" # Add the SIGCHLD waker as reactor reader. This needs to run in the # reactor thread as it's not thread-safe. The SIGCHLD waker will # react to SIGCHLD signals by writing to a dummy pipe, which will # wake up epoll() calls. self.call(1, self._addSIGCHLDWaker) # Install the actual signal hander (this needs to happen in the main # thread). self.reactor._childWaker.install() # Handle SIGINT (ctrl-c) and SIGTERM. This mimics the regular Twisted # code in _SignalReactorMixin._handleSignals (which can't be called # from a non-main thread). signal.signal(signal.SIGINT, self._handleSigInt) signal.signal(signal.SIGTERM, self._handleSigTerm) logging.info("Reactor started") def _stop(self): if self.thread.isAlive(): # The thread is running, let's attempt a clean shutdown. logging.info("Stopping Twisted reactor and wait for its thread") # Assert that the reactor is still running, because, if not, it # means that it's basically hung, and there's nothing we can # do to stop it (we're in a different thread here). self._assertReactorRunning() # Use reactor.crash(), since calling reactor.stop() makes it # impossible to re-start it. try: self.call(self.timeout, self.reactor.crash) except CallFromThreadTimeout: raise RuntimeError("Could not stop the reactor") # The thread should exit almost immediately, try to wait a bit, and # fail if it doesn't. self.thread.join(timeout=self.timeout) if self.thread.isAlive(): raise RuntimeError("Could not stop the reactor thread") elif self.reactor.running: # If the thread is dead but the reactor is still "running", it # probably means that the thread crashed badly, let's clean up # the reactor's state as much as we can and hope for the best. # It's thread-safe to invoke crash() from here since the reactor # thread isn't running anymore. logging.warning( "Twisted reactor has broken state, trying to reset it") self.reactor.crash() logging.info("Reactor stopped") def _assertReactorRunning(self): """Check if self.reactor is still running. This method is called by _stop() and _reset() in case the reactor's thread is still runnning. It will make sure that the reactor is still running as well, or raise an exception otherwise (since in that situation the thread is basically hung and there's nothing we can do for recovering). """ if not self.reactor.running: raise RuntimeError("Hung reactor thread detected") def _addSIGCHLDWaker(self): """Add a `_SIGNCHLDWaker` to wake up the reactor when a child exits.""" self.reactor._childWaker = _SIGCHLDWaker(self.reactor) self.reactor._internalReaders.add(self.reactor._childWaker) self.reactor.addReader(self.reactor._childWaker) # TODO: the signal handling code below is not tested, probably the best way # would be to have an integration test that spawns a separate test # process and send signals to it (using subunit.IsolatedTestCase?). def _handleSigInt(self, *args): # pragma: no cover """ Called when a SIGINT signal is received (for example user hit ctrl-c). """ self.reactor.sigInt(*args) self._maybeFixReactorThreadRace() signal.default_int_handler() def _handleSigTerm(self, *args): # pragma: no cover """ Called when a SIGTERM signal is received. """ self.reactor.sigTerm(*args) self._maybeFixReactorThreadRace() raise sys.exit(args[0]) def _maybeFixReactorThreadRace(self): # pragma: no cover # XXX For some obscure reason, this is needed in order to have the # reactor properly wait for the shutdown sequence. It's probably # a race between this thread and the reactor thread. Needs # investigation. spin = Queue() self.reactor.callFromThread(self.reactor.callLater, 0, spin.put, None) spin.get(timeout=self.timeout) txfixtures-0.2.6/txfixtures/service.py000066400000000000000000000467741303764413000202260ustar00rootroot00000000000000import logging import os import re import signal import socket import logging from datetime import datetime from psutil import Process from fixtures import Fixture from twisted.internet import reactor as defaultTwistedReactor from twisted.internet.protocol import ( Factory, Protocol, ProcessProtocol, ) from twisted.internet.defer import ( Deferred, inlineCallbacks, ) from twisted.internet.task import LoopingCall from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.internet.error import ( ConnectionRefusedError, ConnectingCancelledError, ) from twisted.protocols.basic import LineOnlyReceiver from txfixtures._twisted.threading import interruptableCallFromThread TIMEOUT = 15 # Some processes (like mongodb) use an abbreviated code for level names. We # keep a mapping for transparently convert between them and standard Python # level names. SHORT_LEVELS = { "C": "CRITICAL", "E": "ERROR", "W": "WARNING", "I": "INFO", "D": "DEBUG", } class Service(Fixture): """Spawn, control and monitor a background service.""" def __init__(self, command, reactor=None, timeout=TIMEOUT, env=None): super(Service, self).__init__() self.command = command self.env = _encodeDictValues(env or os.environ.copy()) parser = ServiceOutputParser(self._executable) # XXX Set the reactor as private, since the public 'reactor' attribute # is typically a Reactor fixture, set by testresources as # dependency. if reactor is None: reactor = defaultTwistedReactor self._reactor = reactor self.protocol = ServiceProtocol( reactor=self._reactor, parser=parser, timeout=timeout) self._eventTriggerID = None def reset(self): if self.protocol.terminated.called: raise RuntimeError("Service died") def expectOutput(self, data): self.protocol.expectedOutput = data def expectPort(self, port): self.protocol.expectedPort = port def setOutputFormat(self, outFormat): self.protocol.parser.pattern = outFormat def allocatePort(self): """Allocate an unused port. This method can be used by subclasses to allocate a random ports for the service they spawn. There is a small race condition here (between the time we allocate the port, and the time it actually gets used), but for the purposes for which this method gets used it isn't a problem in practice. """ sock = socket.socket() try: sock.bind(("localhost", 0)) _, port = sock.getsockname() return port finally: sock.close() def _setUp(self): logging.info("Spawning service process %s", self.command) self.addCleanup(self._stop) self._callFromThread(self._start) @property def _executable(self): return self.command[0] @property def _args(self): return self.command @property def _name(self): return os.path.basename(self._executable) @inlineCallbacks def _start(self): self._reactor.spawnProcess( self.protocol, self._executable, args=self._args, env=self.env) # This cleanup handler will be triggered in case of SIGTERM and SIGINT, # when the reactor will initiate an unexpected shutdown sequence. self._eventTriggerID = self._reactor.addSystemEventTrigger( "before", "shutdown", self._terminateProcess) yield self.protocol.ready def _stop(self): logging.info("Stopping service process %s", self.command) try: self._callFromThread(self._terminateProcess) except: if self.protocol.transport.pid: # In case something goes wrong let's try our best to not leave # running processes around. logging.info( "Service process didn't terminate, trying to kill it") process = Process(self.protocol.transport.pid) process.kill() process.wait(timeout=1) def _callFromThread(self, f): # Set an additional timeout for the callFromThread call itself. We # want this timeout to be greater than the 'ready' deferred timeout # set in _start(), so if the reactor thread is hung or dies we still # properly timeout. timeout = self.protocol.timeout + 1 interruptableCallFromThread(self._reactor, timeout, f) @inlineCallbacks def _terminateProcess(self): if self._eventTriggerID: # Clear the shutdown event trigger, since we're going to cleanup # normally. self._reactor.removeSystemEventTrigger(self._eventTriggerID) if self.protocol.transport.pid: logging.info("Sending SIGTERM to service process '%s'", self._name) self.protocol.transport.signalProcess(signal.SIGTERM) logging.info("Waiting for service process to terminate") yield self.protocol.terminated class ServiceProtocol(ProcessProtocol): """Start and stop a background service process. This :class:`~twisted.internet.protocol.ProcessProtocol` manages the start up and termination phases of a background service process. The process is considered 'running' when it has stayed up for at least 0.1 seconds (or any other non default value which `minUptime` is set too), and optionally when it has emitted a certain string and/or it has started listening to a certain port. """ #: The service process must stay up at least this amount of seconds, before #: it's considered running. This allows to catch common issues like the #: service process executable not being in PATH or not being executable. minUptime = 0.1 def __init__(self, reactor=None, parser=None, timeout=TIMEOUT): self.reactor = reactor or defaultTwistedReactor self.parser = parser or ServiceOutputParser("") #: Maximum amount of seconds to wait for the service to be ready. After #: that, the 'ready' deferred will errback with a TimeoutError. self.timeout = timeout #: Optional text that we expect the process to emit in standard output #: before we consider it ready. self.expectedOutput = None #: Optional port number that we expect the service process to listen, #: before we consider it ready. self.expectedPort = None #: Deferred that will fire when the service is considered ready, i.e. #: it has stayed up for at least minUptime seconds, has produced the #: expected output (if any), and is listening to the expected port (if #: any). Upon cancellation, any waiting activity will be stopped. self.ready = Deferred(lambda _: self._stopWaitingForReady()) #: Deferred that will fire when the service has fully terminated, i.e. #: it has exited and we parent process have read any outstanding data #: in the pipes and have closed them. self.terminated = Deferred() # Delayed call that gets started right after the process has been # spawned. Its purpose is to make the protocol "sleep" for a minUptime # seconds (typically 0.1 seconds): if the process exits before this # little time has elapsed, an error gets raised. self._minUptimeCall = None # Deferred that will be fired when the process emits the expected # output (if any). self._expectedOutputReady = Deferred() # A LoopingCall instance that will periodically try to open the port # that the process is supposed to start listening to. self._probePortLoop = None # A connector as returned by TCP4ClientEndpoint.connect() that can be # used to abort an ongoing connection attempt as performed by the # port probe loop. self._probePortAttempt = None def connectionMade(self): # Called (indirectly) by `spawnProcess` after the `os.fork` call has # succeeded. logging.info("Service process spawned") self.ready.addTimeout(self.timeout, self.reactor) # The twisted.protocols.basic.LineOnlyReceiver class expects to know # when the transport is disconnecting. self.disconnecting = False # Let's see if the process stays running for at least self._minUptimeCall = self.reactor.callLater( self.minUptime, self._minUptimeElapsed) if self.expectedOutput: # From this point on, be prepared to receive the expected output at # any time. self.parser.whenLineContains( self.expectedOutput, self._expectedOutputReceived) else: # There's no output we expect, so we fire this Deferred right away. # When _minUptimeElapsed will be called, the callback that gets # attached to this Deferred will fire synchronously. self._expectedOutputReady.callback(None) self.parser.makeConnection(self) def outReceived(self, data): # Called when we receive data from the standard output of the service. self.parser.dataReceived(data) errReceived = outReceived def processExited(self, reason): # Called when the service process exited. logging.info("Service process exited: %s", reason.getErrorMessage()) # If we did not reach the 'ready' state yet, the let's fire the 'ready' # Deferred with an error. if not self.ready.called: self._stopWaitingForReady(reason) def processEnded(self, reason): # Called when the process has been reaped. logging.info("Service process reaped") self.terminated.callback(None) def _minUptimeElapsed(self): """ Called if the process didn't exit in the first `minUptime` seconds after having been spawned. """ logging.info("Service process alive for %.1f seconds", self.minUptime) # Now wait for the expected output and then start polling the port # we expect the service to listen to (if there's no expected output # and/or no expected port, these deferreds will fire synchronously). if self.expectedPort: self._expectedOutputReady.addCallback(self._startProbePortLoop) self._expectedOutputReady.addCallback(self._maybeFireReady) def _expectedOutputReceived(self): """ Called after `_minUptimeElapsed` and the service process has emitted the expected output string. """ # Let's fire the relevant deferred, so we can move forward to polling # the expected port, or declaring the service as ready (if there's no # expected port). logging.info("Service process emitted '%s'", self.expectedOutput) self._expectedOutputReady.callback(None) @inlineCallbacks def _startProbePortLoop(self, _): """ Called when the service process has stayed up for at least `minUptime` seconds and it has emitted the expected output string (or there was no expected output string at all). """ self._probePortLoop = LoopingCall(self._probePort) self._probePortLoop.clock = self.reactor # The LoopingCall.start() method returns a deferred that will fire # when the loop stops, i.e. when we successfully probe the port. yield self._probePortLoop.start(0.1) @inlineCallbacks def _probePort(self): """Perform a single attempt to connect to the expected port. If the probe succeeds the probe loop will be stoped. If the probe fails with a connection error, we'll just return gracefully (we'll be invoked again at the next loop iteration). """ logging.info("Polling service port '%s'", self.expectedPort) endpoint = TCP4ClientEndpoint( self.reactor, "localhost", self.expectedPort) try: factory = Factory() factory.protocol = Protocol self._probePortAttempt = endpoint.connect(factory) yield self._probePortAttempt except ConnectionRefusedError as error: logging.info("Service port probe failed: %s", error) except ConnectingCancelledError as error: # This happens if _stopWaitingForReady gets called while we are # waiting for the enpoint connect() to succeed or fail. logging.info("Service port probe cancelled: %s", error) else: if self._probePortLoop.running: self._probePortLoop.stop() logging.info("Service opened port %d", self.expectedPort) finally: self._probePortAttempt = None def _maybeFireReady(self, result): """Fire the 'ready' deferred, unless we're aborting the startup. If the startup sequence is aborting (either because the `ready` deferred was cancelled by user code, or because the process died and `processExited` was called), this will just be a no-op, as we rely on the aborting code to errback the `ready` deferred. """ if not self.disconnecting: logging.info("Service process ready") self.ready.callback(result) def _stopWaitingForReady(self, reason=None): """ Stop any delayed call or activity associated with the initial waiting for the service to be ready. If `reason` is passed, the `ready` deferred will errback with the given failure. """ # This will prevent the ServiceOutputParser protocol from firing any # further lineReceived event, so we don't fire _expectedOutputReady. # # It will also prevent _maybeFireReady from firing the 'ready' # deferred, since we want to do it ourselves with the given reason (if # any). self.disconnecting = True message = None if self._minUptimeCall.active(): self._minUptimeCall.cancel() message = "minimum uptime not yet elapsed" elif self.expectedOutput and not self._expectedOutputReady.called: message = "expected output not yet received" elif self.expectedPort: if self._probePortAttempt: self._probePortAttempt.cancel() self._probePortLoop.stop() message = "expected port not yet open" # We can safely assume that one of the conditions above is holding, # because otherwise the 'ready' deferred would have already fired. In # any case let's put an explicit assertion here for good measure. assert message, "Unexpected protocol state while cancelling wait" logging.info( "Give up waiting for the service to be ready: %s", message) if reason: self.ready.callback(reason) class ServiceOutputParser(LineOnlyReceiver): """ Parse the standard output stream of a service and forward it to the Python logging system. The stream is assumed to be a UTF-8 sequence of lines each delimited by a (configurable) delimiter character. Each received line is tested against the RegEx pattern provided in the constructor. If a match is found, a :class:`~logging.LogRecord` is built using the information from the groups of the match object, otherwise default values will be used. The record is then passed to the :class:`~logging.Logger` provided in the constructor. Match objects that result from the RegEx pattern are supposed to provide groups named after the substitutions below. """ #: The delimiter character identifying the end of a line. delimiter = b"\n" #: Substitutions for commonly used groups in line match patterns. For #: example, this allows you to use "{Y}-{m}-{S}" as pattern snippet, as # opposed to an explicit "(?P\d{4})-(?P\d{2})-(?P\d{2})". substitutions = { "Y": "(?P\d{4})", "m": "(?P\d{2})", "d": "(?P\d{2})", "H": "(?P\d{2})", "M": "(?P\d{2})", "S": "(?P\d{2})", "msecs": "(?P\d{3})", "levelname": "(?P[a-zA-Z]+)", "name": "(?P.+)", "message": "(?P.+)", } def __init__(self, service, logger=None, pattern=None): """ :param service: A string identifying the service whose output is being parsed. It will be attached as 'service' attribute to all log records emitted. """ self.service = service self.pattern = pattern or "{message}" self.logger = logger or logging.getLogger("") self._callbacks = {} def whenLineContains(self, text, callback): """Fire the given callback when a line contains the given text. The callback will be fired only once when and if a match is found. """ self._callbacks[text] = callback def lineReceived(self, line): """Foward the received line to the Python logging system.""" message = line.decode("utf-8") params = { "levelname": "NOTSET", "levelno": 0, "msg": message, "processName": self.service, } match = re.match(self.pattern.format(**self.substitutions), message) if match: params.update(self._getLogRecordParamsForMatch(match)) record = logging.makeLogRecord(params) self.logger.handle(record) for text in list(self._callbacks.keys()): if text in record.msg: self._callbacks.pop(text)() def lineLengthExceeded(self, line): """Simply truncate the line.""" self.lineReceived(line[:self.MAX_LENGTH]) def _getLogRecordParamsForMatch(self, match): """ Use the given `match` regex object to create a dict of parameters to be passed to `logging.makeLogRecord`. This method will try to use all the information extracted by the match. If some of it is missing or incomplete, it will be discarded. """ groups = _filterNoneValues(match.groupdict()) params = { "name": groups.get("name"), "msg": groups.get("message"), } if "levelname" in groups: levelname = groups["levelname"].upper() if len(levelname) == 1: levelname = SHORT_LEVELS.get(levelname, "INFO") params["levelname"] = levelname params["levelno"] = logging.getLevelName(params["levelname"]) # Only set creation time if all date-related groups are there. if set(groups.keys()).issuperset({"Y", "m", "d", "H", "M", "S"}): params["created"] = float(datetime( int(groups["Y"]), int(groups["m"]), int(groups["d"]), int(groups["H"]), int(groups["M"]), int(groups["S"]), ).strftime("%s")) if "msecs" in groups: params["msecs"] = float(groups["msecs"]) return params def _filterNoneValues(d): """ Return a dict which is the same as `d`, except for keys with None values, which get discarded. """ return dict([(k, v) for k, v in d.items() if v is not None]) def _encodeDictValues(d): """ Return a dict whose unicode values get UTF-8 encoded to bytes. """ return dict( [(_maybeEncode(k), _maybeEncode(v)) for k, v in d.items() if v is not None]) def _maybeEncode(x): """ If x is a string, encode it to bytes using UTF-8. """ if isinstance(x, str): x = x.encode("utf-8") return x txfixtures-0.2.6/txfixtures/tachandler.py000066400000000000000000000155751303764413000206660ustar00rootroot00000000000000# Copyright 2009-2011 Canonical Ltd. This software is licensed under the # GNU General Public License version 3. """Test harness for TAC (Twisted Application Configuration) files.""" __metaclass__ = type __all__ = [ 'TacException', 'TacTestFixture', ] import errno import os import socket import subprocess import sys import time import warnings from fixtures import Fixture from testtools.content import content_from_file from txfixtures.osutils import ( get_pid_from_file, kill_by_pidfile, two_stage_kill, until_no_eintr, ) class TacException(Exception): """Error raised by TacTestSetup.""" class TacTestFixture(Fixture): """Setup an TAC file as daemon for use by functional tests. You must override setUpRoot to set up a root directory for the daemon. You may override _hasDaemonStarted, typically by calling _isPortListening, to tell how long to wait before the daemon is available. """ _proc = None def setUp(self, spew=False, umask=None, python_path=None, twistd_script=None): """Initialize a new TacTestFixture fixture. :param python_path: If set, run twistd under this Python interpreter. :param twistd_script: If set, run this twistd script rather than the system default. Must be provided if python_path is given. """ super(TacTestFixture, self).setUp() if get_pid_from_file(self.pidfile): # An attempt to run while there was an existing live helper # was made. Note that this races with helpers which use unique # roots, so when moving/eliminating this code check subclasses # for workarounds and remove those too. pid = get_pid_from_file(self.pidfile) warnings.warn("Attempt to start Tachandler %r with an existing " "instance (%d) running in %s." % ( self.tacfile, pid, self.pidfile), UserWarning, stacklevel=2) two_stage_kill(pid) # If the pid file still exists, it may indicate that the process # respawned itself, or that two processes were started (race?) and # one is still running while the other has ended, or the process # was killed but it didn't remove the pid file (bug), or the # machine was hard-rebooted and the pid file was not cleaned up # (bug again). In other words, it's not safe to assume that a # stale pid file is safe to delete without human intervention. stale_pid = get_pid_from_file(self.pidfile) if stale_pid: raise TacException( "Could not kill stale process %s from %s." % ( stale_pid, self.pidfile,)) self.setUpRoot() if python_path is None: python_path = sys.executable if twistd_script is None: twistd_script = '/usr/bin/twistd' args = [python_path, '-Wignore::DeprecationWarning', twistd_script, '-o', '-y', self.tacfile, '--pidfile', self.pidfile, '--logfile', self.logfile] if spew: args.append('--spew') if umask is not None: args.extend(('--umask', umask)) # 2010-04-26, Salgado, http://pad.lv/570246: Deprecation warnings # in Twisted are not our problem. They also aren't easy to suppress, # and cause test failures due to spurious stderr output. Just shut # the whole bloody mess up. # Run twistd, and raise an error if the return value is non-zero or # stdout/stderr are written to. self._proc = subprocess.Popen( args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, ) self.addCleanup(self.killTac) stdout = until_no_eintr(10, self._proc.stdout.read) if stdout: raise TacException('Error running %s: unclean stdout/err: %s' % (args, stdout)) rv = self._proc.wait() # twistd will normally fork off into the background with the # originally-spawned process exiting 0. if rv != 0: raise TacException('Error %d running %s' % (rv, args)) self.addDetail(self.logfile, content_from_file(self.logfile)) self._waitForDaemonStartup() def _hasDaemonStarted(self): """Has the daemon started? """ return self._isPortListening('localhost', self.daemon_port) def _isPortListening(self, host, port): """True if a tcp port is accepting connections. This can be used by subclasses overriding _hasDaemonStarted, if they want to check the port is up rather than for the contents of the log file. """ try: s = socket.socket() s.settimeout(2.0) s.connect((host, port)) s.close() return True except socket.error as e: if e.errno == errno.ECONNREFUSED: return False else: raise def _waitForDaemonStartup(self): """ Wait for the daemon to fully start. Times out after 20 seconds. If that happens, the log file content will be included in the exception message for debugging purpose. :raises TacException: Timeout. """ # Watch the log file for readyservice.LOG_MAGIC to signal that startup # has completed. now = time.time() deadline = now + 20 while now < deadline and not self._hasDaemonStarted(): time.sleep(0.1) now = time.time() if now >= deadline: raise TacException('Unable to start %s.' % self.tacfile) def tearDown(self): # For compatibility - migrate to cleanUp. self.cleanUp() def killTac(self): """Kill the TAC file if it is running.""" pidfile = self.pidfile kill_by_pidfile(pidfile) if self._proc: # Close the pipe self._proc.stdout.close() def sendSignal(self, sig): """Send the given signal to the tac process.""" pid = get_pid_from_file(self.pidfile) if pid is None: return os.kill(pid, sig) def setUpRoot(self): """Override this. This should be able to cope with the root already existing, because it will be left behind after each test in case it's needed to diagnose a test failure (e.g. log files might contain helpful tracebacks). """ raise NotImplementedError @property def root(self): raise NotImplementedError @property def tacfile(self): raise NotImplementedError @property def pidfile(self): raise NotImplementedError @property def logfile(self): raise NotImplementedError @property def daemon_port(self): raise NotImplementedError txfixtures-0.2.6/txfixtures/tests/000077500000000000000000000000001303764413000173345ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/tests/__init__.py000066400000000000000000000000001303764413000214330ustar00rootroot00000000000000txfixtures-0.2.6/txfixtures/tests/test_mongodb.py000066400000000000000000000035361303764413000224010ustar00rootroot00000000000000import pymongo from testtools import TestCase from testtools.matchers import ( StartsWith, DirExists, ) from fixtures import FakeLogger from txfixtures._twisted.testing import ThreadedMemoryReactorClock from txfixtures.mongodb import MongoDB OUT = ( b"2016-11-30T10:35:25.476+0000 I CONTROL [init] MongoDB starting", b"2016-11-30T10:35:25.948+0000 I NETWORK [init] waiting for connections on port 666", b"", ) class MongoDBTest(TestCase): def setUp(self): super(MongoDBTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.reactor = ThreadedMemoryReactorClock() self.fixture = MongoDB(reactor=self.reactor) def test_setup(self): """ The fixture passes port and dbpath as extra arguments, and configure the output format to match mongodb's one. """ self.reactor.process.data = b"\n".join(OUT) client = [] class FakeMongoClient(object): def __init__(self, endpoint): client.append(endpoint) def close(self): client.append("close") self.patch(self.fixture, "allocatePort", lambda: 666) self.patch(pymongo, "MongoClient", FakeMongoClient) self.fixture.setUp() executable, arg1, arg2, arg3 = self.reactor.process.args self.assertEqual(b"mongod", executable) self.assertEqual(b"--port=666", arg1) self.assertEqual(["mongodb://localhost:666"], client) self.assertThat(arg2, StartsWith(b"--dbpath=")) self.assertThat(arg2.split(b"=")[1], DirExists()) self.assertEqual(b"--nojournal", arg3) self.assertIn( "waiting for connections on port 666", self.logger.output.split("\n")) self.fixture.cleanUp() self.assertEqual(["mongodb://localhost:666", "close"], client) txfixtures-0.2.6/txfixtures/tests/test_phantomjs.py000066400000000000000000000027601303764413000227550ustar00rootroot00000000000000import os from testtools import TestCase from testtools.matchers import ( StartsWith, DirExists, ) from selenium.webdriver.remote import webdriver from fixtures import FakeLogger from txfixtures._twisted.testing import ThreadedMemoryReactorClock from txfixtures.phantomjs import PhantomJS OUT = ( b"[INFO - 2016-11-17T09:01:38.591Z] GhostDriver - ", b"Main - running on port 666", b"" ) class PhantomJSTest(TestCase): def setUp(self): super(PhantomJSTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.reactor = ThreadedMemoryReactorClock() self.fixture = PhantomJS(reactor=self.reactor) def test_setup(self): """ The fixture passes port and cookies paths as extra argument, and configure the output format to match phantomjs' one. """ self.reactor.process.data = b"\n".join(OUT) class FakeWebDriver(object): def __init__(self, **kwargs): pass self.patch(self.fixture, "allocatePort", lambda: 666) self.patch(webdriver, "WebDriver", FakeWebDriver) self.fixture.setUp() executable, arg1, arg2 = self.reactor.process.args self.assertEqual("phantomjs", executable) self.assertEqual("--webdriver=666", arg1) self.assertThat(arg2, StartsWith("--cookies-file=")) self.assertThat(os.path.dirname(arg2.split("=")[1]), DirExists()) self.assertIn("running on port 666", self.logger.output) txfixtures-0.2.6/txfixtures/tests/test_reactor.py000066400000000000000000000105461303764413000224120ustar00rootroot00000000000000from testtools import TestCase from fixtures import FakeLogger from systemfixtures import FakeThreads from twisted.internet.defer import succeed from twisted.internet.posixbase import _SIGCHLDWaker from txfixtures._twisted.testing import ThreadedMemoryReactorClock from txfixtures.reactor import Reactor class ReactorTest(TestCase): def setUp(self): super(ReactorTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.reactor = ThreadedMemoryReactorClock() self.fixture = Reactor(reactor=self.reactor, timeout=0) self.threads = self.useFixture(FakeThreads()) def test_install_sigchld_waker(self): """ At setup time, a reactor waker for the SIGCHLD signal is installed. """ self.fixture.setUp() [reader] = list(self.reactor.readers) self.assertIsInstance(reader, _SIGCHLDWaker) self.assertTrue(reader.installed) def test_call(self): """The call() method is a convenience around blockingFromThread.""" output = self.fixture.call(0, lambda: succeed("hello")) self.assertEqual("hello", output) def test_reset_thread_and_reactor_died(self): """ The reset() method creates a new thread if the initial one has died. """ self.fixture.setUp() self.fixture.reset() self.assertIn( "Twisted reactor thread died, trying to recover", self.logger.output) def test_reset_thread_died_but_reactor_is_running(self): """ If the reactor crashes badly and is left in a bad state (e.g. running), the fixtures tries a best-effort clean up. """ self.fixture.setUp() self.reactor.running = True self.fixture.reset() self.assertTrue(self.reactor.hasCrashed) self.assertIn( "Twisted reactor thread died, trying to recover", self.logger.output) self.assertIn( "Twisted reactor has broken state, trying to reset", self.logger.output) def test_reset_hung_thread(self): """ The reset() method bails out if the thread is alive but the reactor doesn't appear to be running. """ self.fixture.setUp() self.fixture.thread.alive = True self.reactor.running = False error = self.assertRaises(RuntimeError, self.fixture.reset) self.assertEqual("Hung reactor thread detected", str(error)) def test_cleanup_stops_thread_and_reactor(self): """After cleanUp is run, the reactor is stopped.""" self.fixture.setUp() self.fixture.cleanUp() self.assertFalse(self.fixture.thread.isAlive()) self.assertFalse(self.fixture.reactor.running) def test_cleanup_thread_not_alive(self): """ If the thread is not alive, the cleanup phase is essentially a no-op. """ self.fixture.setUp() self.reactor.stop() self.fixture.thread.alive = False self.fixture.cleanUp() # There's only the entry about starting the thread, since upon cleanup # nothing was running. self.assertIn( "Starting Twisted reactor in a separate thread", self.logger.output) def test_cleanup_hung_thread(self): """ If cleanUp() detects a hung thread with no reactor running, an error is raised. """ self.fixture.setUp() self.fixture.thread.alive = True self.reactor.running = False error = self.assertRaises(RuntimeError, self.fixture.cleanUp) self.assertEqual("Hung reactor thread detected", str(error)) def test_cleanup_hung_reactor(self): """ If cleanUp() can't stop the reactor, an error is raised. """ self.fixture.setUp() self.reactor.async = True self.fixture.thread.alive = True error = self.assertRaises(RuntimeError, self.fixture.cleanUp) self.assertEqual("Could not stop the reactor", str(error)) def test_cleanup_thread_does_not_die(self): """ If cleanUp() can't stop the thread, an error is raised. """ self.fixture.setUp() self.fixture.thread.hang = True self.fixture.thread.alive = True error = self.assertRaises(RuntimeError, self.fixture.cleanUp) self.assertEqual("Could not stop the reactor thread", str(error)) txfixtures-0.2.6/txfixtures/tests/test_service.py000066400000000000000000000361371303764413000224170ustar00rootroot00000000000000import logging from datetime import datetime from logging.handlers import BufferingHandler from testtools import TestCase from testtools.matchers import ( Is, IsInstance, MatchesStructure, ) from testtools.twistedsupport import ( succeeded, failed, has_no_result, ) from fixtures import ( FakeLogger, LogHandler, MultipleExceptions, ) from twisted.python.failure import Failure from twisted.internet.error import ( ProcessTerminated, ConnectionRefusedError, ) from twisted.internet.defer import ( Deferred, CancelledError, TimeoutError, ) from twisted.test.proto_helpers import ( StringTransport, MemoryReactorClock, ) from txfixtures._twisted.testing import ( ThreadedMemoryReactorClock, MemoryProcess, ) from txfixtures.service import ( Service, ServiceProtocol, ServiceOutputParser, ) class ServiceTest(TestCase): def setUp(self): super(ServiceTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.reactor = ThreadedMemoryReactorClock() self.fixture = Service(["foo"], reactor=self.reactor) def test_setup_start_process(self): """ The fixture spawns the given process at setup time, the waits for it to be started. """ self.fixture.setUp() self.assertEqual(["foo"], self.reactor.process.args) self.assertIn("Service process ready", self.logger.output) def test_expect_output(self): """ It's possible to specify a string that should match the service's standard output, before the service is considered ready and setup completes. """ self.reactor.process.data = b"hi\n" self.fixture.expectOutput("hi") self.fixture.setUp() self.assertIn("Service process emitted 'hi'", self.logger.output) def test_expect_output_timeout(self): """ If the expected out is not received within the given timeout, an error is raised. """ self.fixture.expectOutput("hi") error = self.assertRaises(MultipleExceptions, self.fixture.setUp) self.assertIs(error.args[0][0], TimeoutError) self.assertNotIn("Service process ready", self.logger.output) def test_cleanup_stop_process(self): """ The fixture stops the given process at cleanup time, then waits for it to terminate. """ self.fixture.setUp() self.fixture.cleanUp() self.assertIn("Service process reaped", self.logger.output) def test_set_output_format(self): """ It's possible to specify an output format to, that will be used to generate a regular expression for parsing the service's output and emit the relevant logging record. """ records = [] logging.getLogger().handlers[0].handle = records.append self.reactor.process.data = b"2016-11-14 08:59:41.400 INFO my-app hi\n" fmt = "{Y}-{m}-{d} {H}:{M}:{S}.{msecs} {levelname} {name} {message}" self.fixture.setOutputFormat(fmt) self.fixture.setUp() for record in records: if record.processName != "MainProcess": break self.assertEqual("foo", record.processName) self.assertEqual("INFO", record.levelname) self.assertEqual("my-app", record.name) self.assertEqual("hi", record.msg) def test_expect_port(self): """ It's possible to specify a port that the service process should start listening to, before the it's considered ready and setup completes. """ self.fixture.expectPort(666) self.fixture.setUp() self.assertIn("Service opened port 666", self.logger.output) def test_reset(self): """ The reset() method is a no-op if the service is still running. """ self.fixture.setUp() self.assertIsNone(self.fixture.reset()) def test_reset_service_died(self): """ The reset() method raises an error if the service died. """ self.fixture.setUp() self.reactor.process.processEnded(Failure(Exception("boom"))) error = self.assertRaises(RuntimeError, self.fixture.reset) self.assertEqual("Service died", str(error)) class ServiceProtocolTest(TestCase): def setUp(self): super(ServiceProtocolTest, self).setUp() self.logger = self.useFixture(FakeLogger()) self.reactor = MemoryReactorClock() self.process = MemoryProcess() self.protocol = ServiceProtocol(reactor=self.reactor) self.process.proto = self.protocol def test_fork(self): """ When the connection is made it means that we sucessfully forked the service process, so we start waiting a bit to see if it stays running or exits shortly. """ self.protocol.makeConnection(self.process) [call1, call2] = self.reactor.getDelayedCalls() self.assertEqual(call1.time, self.protocol.minUptime) self.assertEqual(call2.time, self.protocol.timeout) self.assertIn("Service process spawned", self.logger.output) def test_min_uptime(self): """ If the process stays running for at least minUptime seconds, the 'ready' Deferred gets fired. """ self.protocol.makeConnection(self.process) self.reactor.advance(0.1) self.assertThat(self.protocol.ready, succeeded(Is(None))) self.assertIn( "Service process alive for 0.1 seconds", self.logger.output) def test_expected_output(self): """ If some expected output is required, the 'ready' deferred fires only when such output has been received. """ self.protocol.expectedOutput = "hello" self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.assertThat(self.protocol.ready, has_no_result()) self.protocol.outReceived(b"hello world!\n") self.assertThat(self.protocol.ready, succeeded(Is(None))) self.assertIn("Service process emitted 'hello'", self.logger.output) def test_expected_port(self): """ If some expected port is required, the 'ready' deferred fires only when such port has been opened. """ self.protocol.expectedPort = 1234 self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.assertThat(self.protocol.ready, has_no_result()) factory = self.reactor.tcpClients[0][2] factory.buildProtocol(None).connectionMade() self.assertThat(self.protocol.ready, succeeded(Is(None))) self.assertIn("Service opened port 1234", self.logger.output) def test_expected_port_probe_failed(self): """ If probing for the expected port fails, the probe will be retried. """ self.protocol.expectedPort = 1234 self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.assertThat(self.protocol.ready, has_no_result()) factory = self.reactor.tcpClients[0][2] factory.clientConnectionFailed(None, ConnectionRefusedError()) self.assertIn("Service port probe failed", self.logger.output) self.reactor.advance(0.1) factory = self.reactor.tcpClients[1][2] factory.buildProtocol(None).connectionMade() self.assertThat(self.protocol.ready, succeeded(Is(None))) self.assertIn("Service opened port 1234", self.logger.output) def test_process_dies_shortly_after_fork(self): """ If the service process exists right after having been spawned (for example the executable was not found), the 'ready' Deferred fires with an errback. """ self.protocol.makeConnection(self.process) error = ProcessTerminated(exitCode=1, signal=None) self.protocol.processExited(Failure(error)) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=Is(error)))) def test_cancel_while_waiting_for_uptime(self): """ If the 'ready' deferred gets cancelled while still waiting for the minumum uptime, a proper message is emitted. """ self.protocol.makeConnection(self.process) self.protocol.ready.cancel() self.assertIn( "minimum uptime not yet elapsed", self.logger.output) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=IsInstance(CancelledError)))) def test_process_dies_while_waiting_expected_output(self): """ If the service process exists while waiting for the expected output, the 'ready' Deferred fires with an errback. """ self.protocol.expectedOutput = "hello" self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) error = ProcessTerminated(exitCode=1, signal=None) self.protocol.processExited(Failure(error)) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=Is(error)))) # Further input received on the file descriptor will be discarded self.protocol.ready = Deferred() # pretend that we didn't get fired self.protocol.outReceived(b"hello world!\n") self.assertThat(self.protocol.ready, has_no_result()) def test_timeout_while_waiting_expected_output(self): """ If the timeout elapses while waiting for the expected output, the 'ready' Deferred fires with an errback. """ self.protocol.expectedOutput = "hello" self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.reactor.advance(self.protocol.timeout) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=IsInstance(TimeoutError)))) self.assertIn( "expected output not yet received", self.logger.output) def test_process_dies_while_probing_port(self): """ If the service process exists while waiting for the expected port to, be open, the 'ready' Deferred fires with an errback. """ self.protocol.expectedPort = 1234 self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) error = ProcessTerminated(exitCode=1, signal=None) self.protocol.processExited(Failure(error)) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=Is(error)))) # No further probe will happen self.reactor.advance(0.1) self.assertEqual(1, len(self.reactor.tcpClients)) def test_timeout_while_probing_port(self): """ If the service process doesn't listen to the expected port within the, timeout, 'ready' Deferred fires with an errback. """ self.protocol.expectedPort = 1234 self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.reactor.advance(self.protocol.timeout) self.assertThat( self.protocol.ready, failed(MatchesStructure(value=IsInstance(TimeoutError)))) self.assertIn( "expected port not yet open", self.logger.output) def test_cancel_ready(self): """ If the `ready` deferred gets cancelled, the protocol will stop doing anything related to waiting for the service to be ready. """ self.protocol.makeConnection(self.process) self.protocol.ready.cancel() self.assertThat( self.protocol.ready, failed(MatchesStructure(value=IsInstance(CancelledError)))) self.assertEqual(0, len(self.reactor.getDelayedCalls())) def test_terminated(self): """ When the process is fully terminated, the 'terminated' deferred gets fired. """ self.protocol.makeConnection(self.process) self.reactor.advance(self.protocol.minUptime) self.protocol.transport.processEnded(0) self.assertThat(self.protocol.terminated, succeeded(Is(None))) class ServiceOutputParserTest(TestCase): def setUp(self): super(ServiceOutputParserTest, self).setUp() self.transport = StringTransport() self.handler = BufferingHandler(2) self.useFixture(LogHandler(self.handler)) self.parser = ServiceOutputParser("my-app") self.parser.makeConnection(self.transport) def test_full_match(self): """ If a line matches the given pattern, a log record is created using the values extracted from the match dictionary. """ self.parser.pattern = ( "{Y}-{m}-{d} {H}:{M}:{S}.{msecs} {levelname} {name} {message}") line = b"2016-11-14 08:59:41.400 INFO logger hi\n" self.parser.dataReceived(line) [record] = self.handler.buffer self.assertEqual("INFO", record.levelname) self.assertEqual("logger", record.name) self.assertEqual("hi", record.msg) self.assertEqual(1479113981, record.created) self.assertEqual(400, record.msecs) self.assertEqual("my-app", record.processName) def test_partial_match(self): """ If a line only partially matches the given pattern, missing record values will be left alone. """ self.parser.pattern = "{Y}-{m}-{d}( {H}:{M}:{S})? {name} {message}" self.parser.dataReceived(b"2016-11-14 logger hi\n") [record] = self.handler.buffer self.assertEqual("NOTSET", record.levelname) self.assertEqual(0, record.levelno) self.assertEqual("logger", record.name) self.assertEqual("hi", record.msg) date = datetime.utcfromtimestamp(record.created) self.assertNotEqual( ("2016", "11", "14"), (date.year, date.month, date.day)) def test_no_match(self): """ If a line doesn't match the given pattern, a log record is created with a message that equals the whole line. """ self.parser.pattern = "{Y}-{m}-{d} {H}:{M}:{S} {message}" self.parser.dataReceived(b"hello world!\n") [record] = self.handler.buffer self.assertEqual("NOTSET", record.levelname) self.assertEqual(0, record.levelno) self.assertIsNone(record.name) self.assertEqual("hello world!", record.msg) def test_truncated(self): """ If a line exceeds `MAX_LENGTH`, it will be truncated. """ self.parser.MAX_LENGTH = 5 self.parser.dataReceived(b"hello world!\n") [record] = self.handler.buffer self.assertEqual("hello", record.msg) def test_when_line_contains(self): """ It's possible to set a callback that will be fired when a line contains a certain text. """ tokens = [None, None] self.parser.whenLineContains("hello", tokens.pop) self.parser.dataReceived(b"hello world!\n") self.assertEqual([None], tokens) # If a further match is found, the callback is *not* fired. self.parser.dataReceived(b"hello world!\n") self.assertEqual([None], tokens)