pax_global_header00006660000000000000000000000064130464241510014512gustar00rootroot0000000000000052 comment=99bade0888464cef7f12ba4964309b9fa9c9da59 ldif3-3.2.2/000077500000000000000000000000001304642415100125175ustar00rootroot00000000000000ldif3-3.2.2/.gitignore000066400000000000000000000000551304642415100145070ustar00rootroot00000000000000*.pyc .env .tox .cover .coverage docs/_build ldif3-3.2.2/.travis.yml000066400000000000000000000001721304642415100146300ustar00rootroot00000000000000language: python python: - "2.7" install: - "pip install tox" script: tox os: - linux notifications: email: false ldif3-3.2.2/CHANGES.rst000066400000000000000000000052311304642415100143220ustar00rootroot000000000000003.2.2 (2017-02-07) ------------------ - Fix detection of unsafe strings in ``unparse`` (See `#7 `_) 3.2.1 (2016-12-27) ------------------ - Ignore non-unicode characters in "dn" in non-strict mode. (Fixes `#5 `_) 3.2.0 (2016-06-03) ------------------ - Overhaule the unicode support to also support binary data (e.g. images) encoded in LDIF. You can now pass an encoding to the parser which will be used to decode values. If decoding failes, a bytestring will be returned. If you pass an encoding of ``None``, the parser will not try to do any conversion and return bytes directly. This change should be completely backwards compatible, as the parser now gracefully handles a case where it crashed previously. (See `#4 `_) 3.1.1 (2015-09-20) ------------------ - Allow empty values for attributes. 3.1.0 (2015-07-09) ------------------ This is mostly a reaction to `python-ldap 2.4.20 `_. - Restore support for ``records_read`` as well as adding ``line_counter`` and ``byte_counter`` that were introduced in python-ldap 2.4.20. - Stricter order checking of ``dn:``. - Remove partial support for parsing change records. A more complete implementation based on improvements made in python-ldap may be included later. But for now, I don't have the time. **Breaking change**: ``LDIFParser.parse()`` now yields ``dn, entry`` rather than ``dn, changetype, entry``. 3.0.2 (2015-06-22) ------------------ - Include documentation source and changelog in source distribution. (Thanks to Michael Fladischer) - Add LICENSE file 3.0.1 (2015-05-22) ------------------ - Use OrderedDict for entries. 3.0.0 (2015-05-22) ------------------ This is the first version of a fork of the ``ldif`` module from `python-ldap `_. For any changes before that, see the documentation over there. The last version before the fork was 2.4.15. The changes introduced with this version are: - Dropped support for python < 2.7. - Added support for python 3, including unicode support. - All deprecated functions (``CreateLDIF``, ``ParseLDIF``) were removed. - ``LDIFCopy`` and ``LDIFRecordList`` were removed. - ``LDIFParser.handle()`` was removed. Instead, ``LDIFParser.parse()`` yields the records. - ``LDIFParser`` has now a ``strict`` option that defaults to ``True`` for backwards-compatibility. If set to ``False``, recoverable parse errors will produce log warnings rather than exceptions. ldif3-3.2.2/LICENSE000066400000000000000000000024701304642415100135270ustar00rootroot00000000000000Copyright (c) 2015, Tobias Bengfort All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ldif3-3.2.2/MANIFEST.in000066400000000000000000000000771304642415100142610ustar00rootroot00000000000000include CHANGES.rst recursive-include docs Makefile *.rst *.py ldif3-3.2.2/README.rst000066400000000000000000000026521304642415100142130ustar00rootroot00000000000000ldif3 - generate and parse LDIF data (see `RFC 2849`_). This is a fork of the ``ldif`` module from `python-ldap`_ with python3/unicode support. See the first entry in CHANGES.rst for a more complete list of differences. Usage ----- Parse LDIF from a file (or ``BytesIO``):: from ldif3 import LDIFParser from pprint import pprint parser = LDIFParser(open('data.ldif', 'rb')) for dn, entry in parser.parse(): print('got entry record: %s' % dn) pprint(record) Write LDIF to a file (or ``BytesIO``):: from ldif3 import LDIFWriter writer = LDIFWriter(open('data.ldif', 'wb')) writer.unparse('mail=alice@example.com', { 'cn': ['Alice Alison'], 'mail': ['alice@example.com'], 'objectclass': ['top', 'person'], }) Unicode support --------------- The stream object that is passed to parser or writer must be an ascii byte stream. The spec allows to include arbitrary data in base64 encoding or via URL. There is no way of knowing the encoding of this data. To handle this, there are two modes: By default, the ``LDIFParser`` will try to interpret all values as UTF-8 and leave only the ones that fail to decode as bytes. But you can also pass an ``encoding`` of ``None`` to the constructor, in which case the parser will not try to do any conversion and return bytes directly. .. _RFC 2849: https://tools.ietf.org/html/rfc2849 .. _python-ldap: http://www.python-ldap.org/ ldif3-3.2.2/docs/000077500000000000000000000000001304642415100134475ustar00rootroot00000000000000ldif3-3.2.2/docs/Makefile000066400000000000000000000151261304642415100151140ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/2.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/2.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/2" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/2" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." ldif3-3.2.2/docs/conf.py000066400000000000000000000007071304642415100147520ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys import os import subprocess current_dir = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, os.path.dirname(current_dir)) def get_meta(key): cmd = ['python', '../setup.py', '--' + key] return subprocess.check_output(cmd).rstrip() extensions = [ 'sphinx.ext.autodoc', ] master_doc = 'index' project = get_meta('name') copyright = u'2015, ' + get_meta('author') version = get_meta('version') ldif3-3.2.2/docs/index.rst000066400000000000000000000002671304642415100153150ustar00rootroot00000000000000ldif3 +++++ Introduction ============ .. include:: ../README.rst API reference ============= .. automodule:: ldif3 :members: Changelog ========= .. include:: ../CHANGES.rst ldif3-3.2.2/ldif3.py000066400000000000000000000314051304642415100140750ustar00rootroot00000000000000"""ldif3 - generate and parse LDIF data (see RFC 2849).""" from __future__ import unicode_literals import base64 import re import logging from collections import OrderedDict try: # pragma: nocover from urlparse import urlparse from urllib import urlopen except ImportError: # pragma: nocover from urllib.parse import urlparse from urllib.request import urlopen __version__ = '3.2.2' __all__ = [ # constants 'LDIF_PATTERN', # classes 'LDIFWriter', 'LDIFParser', ] log = logging.getLogger('ldif3') ATTRTYPE_PATTERN = r'[\w;.-]+(;[\w_-]+)*' ATTRVALUE_PATTERN = r'(([^,]|\\,)+|".*?")' ATTR_PATTERN = ATTRTYPE_PATTERN + r'[ ]*=[ ]*' + ATTRVALUE_PATTERN RDN_PATTERN = ATTR_PATTERN + r'([ ]*\+[ ]*' + ATTR_PATTERN + r')*[ ]*' DN_PATTERN = RDN_PATTERN + r'([ ]*,[ ]*' + RDN_PATTERN + r')*[ ]*' DN_REGEX = re.compile('^%s$' % DN_PATTERN) LDIF_PATTERN = ('^((dn(:|::) %(DN_PATTERN)s)|(%(ATTRTYPE_PATTERN)' 's(:|::) .*)$)+' % vars()) MOD_OPS = ['add', 'delete', 'replace'] CHANGE_TYPES = ['add', 'delete', 'modify', 'modrdn'] def is_dn(s): """Return True if s is a LDAP DN.""" if s == '': return True rm = DN_REGEX.match(s) return rm is not None and rm.group(0) == s UNSAFE_STRING_PATTERN = ( '(^[^\x01-\x09\x0b-\x0c\x0e-\x1f\x21-\x39\x3b\x3d-\x7f]' '|[^\x01-\x09\x0b-\x0c\x0e-\x7f])') UNSAFE_STRING_RE = re.compile(UNSAFE_STRING_PATTERN) def lower(l): """Return a list with the lowercased items of l.""" return [i.lower() for i in l or []] class LDIFWriter(object): """Write LDIF entry or change records to file object. :type output_file: file-like object in binary mode :param output_file: File for output :type base64_attrs: List[string] :param base64_attrs: List of attribute types to be base64-encoded in any case :type cols: int :param cols: Specifies how many columns a line may have before it is folded into many lines :type line_sep: bytearray :param line_sep: line separator :type encoding: string :param encoding: Encoding to use for converting values to bytes. Note that the spec requires the dn field to be UTF-8 encoded, so it does not really make sense to use anything else. Default: ``'utf8'``. """ def __init__( self, output_file, base64_attrs=[], cols=76, line_sep=b'\n', encoding='utf8'): self._output_file = output_file self._base64_attrs = lower(base64_attrs) self._cols = cols self._line_sep = line_sep self._encoding = encoding self.records_written = 0 #: number of records that have been written def _fold_line(self, line): """Write string line as one or more folded lines.""" if len(line) <= self._cols: self._output_file.write(line) self._output_file.write(self._line_sep) else: pos = self._cols self._output_file.write(line[0:self._cols]) self._output_file.write(self._line_sep) while pos < len(line): self._output_file.write(b' ') end = min(len(line), pos + self._cols - 1) self._output_file.write(line[pos:end]) self._output_file.write(self._line_sep) pos = end def _needs_base64_encoding(self, attr_type, attr_value): """Return True if attr_value has to be base-64 encoded. This is the case because of special chars or because attr_type is in self._base64_attrs """ return attr_type.lower() in self._base64_attrs or \ isinstance(attr_value, bytes) or \ UNSAFE_STRING_RE.search(attr_value) is not None def _unparse_attr(self, attr_type, attr_value): """Write a single attribute type/value pair.""" if self._needs_base64_encoding(attr_type, attr_value): if not isinstance(attr_value, bytes): attr_value = attr_value.encode(self._encoding) encoded = base64.encodestring(attr_value)\ .replace(b'\n', b'')\ .decode('ascii') line = ':: '.join([attr_type, encoded]) else: line = ': '.join([attr_type, attr_value]) self._fold_line(line.encode('ascii')) def _unparse_entry_record(self, entry): """ :type entry: Dict[string, List[string]] :param entry: Dictionary holding an entry """ for attr_type in sorted(entry.keys()): for attr_value in entry[attr_type]: self._unparse_attr(attr_type, attr_value) def _unparse_changetype(self, mod_len): """Detect and write the changetype.""" if mod_len == 2: changetype = 'add' elif mod_len == 3: changetype = 'modify' else: raise ValueError("modlist item of wrong length") self._unparse_attr('changetype', changetype) def _unparse_change_record(self, modlist): """ :type modlist: List[Tuple] :param modlist: List of additions (2-tuple) or modifications (3-tuple) """ mod_len = len(modlist[0]) self._unparse_changetype(mod_len) for mod in modlist: if len(mod) != mod_len: raise ValueError("Subsequent modlist item of wrong length") if mod_len == 2: mod_type, mod_vals = mod elif mod_len == 3: mod_op, mod_type, mod_vals = mod self._unparse_attr(MOD_OPS[mod_op], mod_type) for mod_val in mod_vals: self._unparse_attr(mod_type, mod_val) if mod_len == 3: self._output_file.write(b'-' + self._line_sep) def unparse(self, dn, record): """Write an entry or change record to the output file. :type dn: string :param dn: distinguished name :type record: Union[Dict[string, List[string]], List[Tuple]] :param record: Either a dictionary holding an entry or a list of additions (2-tuple) or modifications (3-tuple). """ self._unparse_attr('dn', dn) if isinstance(record, dict): self._unparse_entry_record(record) elif isinstance(record, list): self._unparse_change_record(record) else: raise ValueError("Argument record must be dictionary or list") self._output_file.write(self._line_sep) self.records_written += 1 class LDIFParser(object): """Read LDIF entry or change records from file object. :type input_file: file-like object in binary mode :param input_file: file to read the LDIF input from :type ignored_attr_types: List[string] :param ignored_attr_types: List of attribute types that will be ignored :type process_url_schemes: List[bytearray] :param process_url_schemes: List of URL schemes to process with urllib. An empty list turns off all URL processing and the attribute is ignored completely. :type line_sep: bytearray :param line_sep: line separator :type encoding: string :param encoding: Encoding to use for converting values to unicode strings. If decoding failes, the raw bytestring will be used instead. You can also pass ``None`` which will skip decoding and always produce bytestrings. Note that this only applies to entry values. ``dn`` and entry keys will always be unicode strings. :type strict: boolean :param strict: If set to ``False``, recoverable parse errors will produce log warnings rather than exceptions. """ def _strip_line_sep(self, s): """Strip trailing line separators from s, but no other whitespaces.""" if s[-2:] == b'\r\n': return s[:-2] elif s[-1:] == b'\n': return s[:-1] else: return s def __init__( self, input_file, ignored_attr_types=[], process_url_schemes=[], line_sep=b'\n', encoding='utf8', strict=True): self._input_file = input_file self._process_url_schemes = lower(process_url_schemes) self._ignored_attr_types = lower(ignored_attr_types) self._line_sep = line_sep self._encoding = encoding self._strict = strict self.line_counter = 0 #: number of lines that have been read self.byte_counter = 0 #: number of bytes that have been read self.records_read = 0 #: number of records that have been read def _iter_unfolded_lines(self): """Iter input unfoled lines. Skip comments.""" line = self._input_file.readline() while line: self.line_counter += 1 self.byte_counter += len(line) line = self._strip_line_sep(line) nextline = self._input_file.readline() while nextline and nextline[:1] == b' ': line += self._strip_line_sep(nextline)[1:] nextline = self._input_file.readline() if not line.startswith(b'#'): yield line line = nextline def _iter_blocks(self): """Iter input lines in blocks separated by blank lines.""" lines = [] for line in self._iter_unfolded_lines(): if line: lines.append(line) elif lines: self.records_read += 1 yield lines lines = [] if lines: self.records_read += 1 yield lines def _decode_value(self, attr_type, attr_value): if attr_type == u'dn': try: return attr_type, attr_value.decode('utf8') except UnicodeError as err: self._error(err) return attr_type, attr_value.decode('utf8', 'ignore') elif self._encoding is not None: try: return attr_type, attr_value.decode(self._encoding) except UnicodeError: pass return attr_type, attr_value def _parse_attr(self, line): """Parse a single attribute type/value pair.""" colon_pos = line.index(b':') attr_type = line[0:colon_pos].decode('ascii') if line[colon_pos:].startswith(b'::'): attr_value = base64.decodestring(line[colon_pos + 2:]) elif line[colon_pos:].startswith(b':<'): url = line[colon_pos + 2:].strip() attr_value = b'' if self._process_url_schemes: u = urlparse(url) if u[0] in self._process_url_schemes: attr_value = urlopen(url.decode('ascii')).read() else: attr_value = line[colon_pos + 1:].strip() return self._decode_value(attr_type, attr_value) def _error(self, msg): if self._strict: raise ValueError(msg) else: log.warning(msg) def _check_dn(self, dn, attr_value): """Check dn attribute for issues.""" if dn is not None: self._error('Two lines starting with dn: in one record.') if not is_dn(attr_value): self._error('No valid string-representation of ' 'distinguished name %s.' % attr_value) def _check_changetype(self, dn, changetype, attr_value): """Check changetype attribute for issues.""" if dn is None: self._error('Read changetype: before getting valid dn: line.') if changetype is not None: self._error('Two lines starting with changetype: in one record.') if attr_value not in CHANGE_TYPES: self._error('changetype value %s is invalid.' % attr_value) def _parse_entry_record(self, lines): """Parse a single entry record from a list of lines.""" dn = None entry = OrderedDict() for line in lines: attr_type, attr_value = self._parse_attr(line) if attr_type == 'dn': self._check_dn(dn, attr_value) dn = attr_value elif attr_type == 'version' and dn is None: pass # version = 1 else: if dn is None: self._error('First line of record does not start ' 'with "dn:": %s' % attr_type) if attr_value is not None and \ attr_type.lower() not in self._ignored_attr_types: if attr_type in entry: entry[attr_type].append(attr_value) else: entry[attr_type] = [attr_value] return dn, entry def parse(self): """Iterate LDIF entry records. :rtype: Iterator[Tuple[string, Dict]] :return: (dn, entry) """ for block in self._iter_blocks(): yield self._parse_entry_record(block) ldif3-3.2.2/setup.cfg000066400000000000000000000003061304642415100143370ustar00rootroot00000000000000[nosetests] all-modules=1 with-coverage=1 cover-package=ldif3 cover-erase=1 cover-branches=1 cover-html=1 cover-html-dir=.cover [flake8] exclude=.git,.tox,.env,build,dist,setup.py ignore=E127,E128 ldif3-3.2.2/setup.py000066400000000000000000000020471304642415100142340ustar00rootroot00000000000000#!/usr/bin/env python import os import re from setuptools import setup DIRNAME = os.path.abspath(os.path.dirname(__file__)) rel = lambda *parts: os.path.abspath(os.path.join(DIRNAME, *parts)) README = open(rel('README.rst')).read() MAIN = open(rel('ldif3.py')).read() VERSION = re.search("__version__ = '([^']+)'", MAIN).group(1) NAME = re.search('^"""(.*) - (.*)"""', MAIN).group(1) DESCRIPTION = re.search('^"""(.*) - (.*)"""', MAIN).group(2) setup( name=NAME, version=VERSION, description=DESCRIPTION, long_description=README, url='https://github.com/xi/ldif3', author='Tobias Bengfort', author_email='tobias.bengfort@posteo.de', py_modules=['ldif3'], license='BSD', classifiers=[ 'Development Status :: 4 - Beta', 'Operating System :: OS Independent', 'Programming Language :: Python', 'License :: OSI Approved :: BSD License', 'Intended Audience :: Developers', 'Topic :: System :: Systems Administration :: ' 'Authentication/Directory :: LDAP', ]) ldif3-3.2.2/tests.py000066400000000000000000000303521304642415100142360ustar00rootroot00000000000000# -*- encoding: utf8 -*- from __future__ import unicode_literals import unittest try: from unittest import mock except ImportError: import mock from io import BytesIO import ldif3 BYTES = b"""version: 1 dn: cn=Alice Alison, mail=alicealison@example.com objectclass: top objectclass: person objectclass: organizationalPerson cn: Alison Alison mail: alicealison@example.com modifytimestamp: 4a463e9a # another person dn: mail=foobar@example.org objectclass: top objectclass: person mail: foobar@example.org modifytimestamp: 4a463e9a """ BYTES_SPACE = b'\n\n'.join([block + b'\n' for block in BYTES.split(b'\n\n')]) BYTES_OUT = b"""dn: cn=Alice Alison,mail=alicealison@example.com cn: Alison Alison mail: alicealison@example.com modifytimestamp: 4a463e9a objectclass: top objectclass: person objectclass: organizationalPerson dn: mail=foobar@example.org mail: foobar@example.org modifytimestamp: 4a463e9a objectclass: top objectclass: person """ BYTES_EMPTY_ATTR_VALUE = b"""dn: uid=foo123,dc=ws1,dc=webhosting,o=eim uid: foo123 domainname: foo.bar homeDirectory: /foo/bar.local aliases: aliases: foo.bar """ LINES = [ b'version: 1', b'dn: cn=Alice Alison,mail=alicealison@example.com', b'objectclass: top', b'objectclass: person', b'objectclass: organizationalPerson', b'cn: Alison Alison', b'mail: alicealison@example.com', b'modifytimestamp: 4a463e9a', b'', b'dn: mail=foobar@example.org', b'objectclass: top', b'objectclass: person', b'mail: foobar@example.org', b'modifytimestamp: 4a463e9a', ] BLOCKS = [[ b'version: 1', b'dn: cn=Alice Alison,mail=alicealison@example.com', b'objectclass: top', b'objectclass: person', b'objectclass: organizationalPerson', b'cn: Alison Alison', b'mail: alicealison@example.com', b'modifytimestamp: 4a463e9a', ], [ b'dn: mail=foobar@example.org', b'objectclass: top', b'objectclass: person', b'mail: foobar@example.org', b'modifytimestamp: 4a463e9a', ]] DNS = [ 'cn=Alice Alison,mail=alicealison@example.com', 'mail=foobar@example.org' ] CHANGETYPES = [None, None] RECORDS = [{ 'cn': ['Alison Alison'], 'mail': ['alicealison@example.com'], 'modifytimestamp': ['4a463e9a'], 'objectclass': ['top', 'person', 'organizationalPerson'], }, { 'mail': ['foobar@example.org'], 'modifytimestamp': ['4a463e9a'], 'objectclass': ['top', 'person'], }] URL = b'https://tools.ietf.org/rfc/rfc2849.txt' URL_CONTENT = 'The LDAP Data Interchange Format (LDIF)' class TestUnsafeString(unittest.TestCase): unsafe_chars = ['\0', '\n', '\r'] unsafe_chars_init = unsafe_chars + [' ', ':', '<'] def _test_all(self, unsafes, fn): for i in range(128): # TODO: test range(255) try: match = ldif3.UNSAFE_STRING_RE.search(fn(i)) if i <= 127 and chr(i) not in unsafes: self.assertIsNone(match) else: self.assertIsNotNone(match) except AssertionError: print(i) raise def test_unsafe_chars(self): self._test_all(self.unsafe_chars, lambda i: 'a%s' % chr(i)) def test_unsafe_chars_init(self): self._test_all(self.unsafe_chars_init, lambda i: '%s' % chr(i)) def test_example(self): s = 'cn=Alice, Alison,mail=Alice.Alison@example.com' self.assertIsNone(ldif3.UNSAFE_STRING_RE.search(s)) def test_trailing_newline(self): self.assertIsNotNone(ldif3.UNSAFE_STRING_RE.search('asd\n')) class TestLower(unittest.TestCase): def test_happy(self): self.assertEqual(ldif3.lower(['ASD', 'HuHu']), ['asd', 'huhu']) def test_falsy(self): self.assertEqual(ldif3.lower(None), []) def test_dict(self): self.assertEqual(ldif3.lower({'Foo': 'bar'}), ['foo']) def test_set(self): self.assertEqual(ldif3.lower(set(['FOo'])), ['foo']) class TestIsDn(unittest.TestCase): def test_happy(self): pass # TODO class TestLDIFParser(unittest.TestCase): def setUp(self): self.stream = BytesIO(BYTES) self.p = ldif3.LDIFParser(self.stream) def test_strip_line_sep(self): self.assertEqual(self.p._strip_line_sep(b'asd \n'), b'asd ') self.assertEqual(self.p._strip_line_sep(b'asd\t\n'), b'asd\t') self.assertEqual(self.p._strip_line_sep(b'asd\r\n'), b'asd') self.assertEqual(self.p._strip_line_sep(b'asd\r\t\n'), b'asd\r\t') self.assertEqual(self.p._strip_line_sep(b'asd\n\r'), b'asd\n\r') self.assertEqual(self.p._strip_line_sep(b'asd'), b'asd') self.assertEqual(self.p._strip_line_sep(b' asd '), b' asd ') def test_iter_unfolded_lines(self): self.assertEqual(list(self.p._iter_unfolded_lines()), LINES) def test_iter_blocks(self): self.assertEqual(list(self.p._iter_blocks()), BLOCKS) def test_iter_blocks_with_additional_spaces(self): self.stream = BytesIO(BYTES_SPACE) self.p = ldif3.LDIFParser(self.stream) self.assertEqual(list(self.p._iter_blocks()), BLOCKS) def _test_error(self, fn): self.p._strict = True with self.assertRaises(ValueError): fn() with mock.patch('ldif3.log.warning') as warning: self.p._strict = False fn() assert warning.called def test_check_dn_not_none(self): self._test_error(lambda: self.p._check_dn('some dn', 'mail=alicealison@example.com')) def test_check_dn_invalid(self): self._test_error(lambda: self.p._check_dn(None, 'invalid')) def test_check_dn_happy(self): self.p._check_dn(None, 'mail=alicealison@example.com') def test_check_changetype_dn_none(self): self._test_error(lambda: self.p._check_changetype(None, None, 'add')) def test_check_changetype_not_none(self): self._test_error(lambda: self.p._check_changetype('some dn', 'some changetype', 'add')) def test_check_changetype_invalid(self): self._test_error(lambda: self.p._check_changetype('some dn', None, 'invalid')) def test_check_changetype_happy(self): self.p._check_changetype('some dn', None, 'add') def test_parse_attr_base64(self): attr_type, attr_value = self.p._parse_attr(b'foo:: YQpiCmM=\n') self.assertEqual(attr_type, 'foo') self.assertEqual(attr_value, 'a\nb\nc') def test_parse_attr_url(self): self.p._process_url_schemes = [b'https'] attr_type, attr_value = self.p._parse_attr(b'foo:< ' + URL + b'\n') self.assertIn(URL_CONTENT, attr_value) def test_parse_attr_url_all_ignored(self): attr_type, attr_value = self.p._parse_attr(b'foo:< ' + URL + b'\n') self.assertEqual(attr_value, '') def test_parse_attr_url_this_ignored(self): self.p._process_url_schemes = [b'file'] attr_type, attr_value = self.p._parse_attr(b'foo:< ' + URL + b'\n') self.assertEqual(attr_value, '') def test_parse_attr_dn_non_utf8(self): def run(): attr = ( b'dn: \x75\x69\x64\x3d\x6b\x6f\xb3\x6f\x62' b'\x69\x7a\x6e\x65\x73\x75\x40\x77\n' ) attr_type, attr_value = self.p._parse_attr(attr) self.assertEqual(attr_type, 'dn') self.assertEqual(attr_value, 'uid=koobiznesu@w') self._test_error(run) def test_parse(self): items = list(self.p.parse()) for i, item in enumerate(items): dn, record = item self.assertEqual(dn, DNS[i]) self.assertEqual(record, RECORDS[i]) def test_parse_binary(self): self.stream = BytesIO(b'dn: cn=Bjorn J Jensen\n' b'jpegPhoto:: 8PLz\nfoo: bar') self.p = ldif3.LDIFParser(self.stream) items = list(self.p.parse()) self.assertEqual(items, [( u'cn=Bjorn J Jensen', { u'jpegPhoto': [b'\xf0\xf2\xf3'], u'foo': [u'bar'], } )]) def test_parse_binary_raw(self): self.stream = BytesIO(b'dn: cn=Bjorn J Jensen\n' b'jpegPhoto:: 8PLz\nfoo: bar') self.p = ldif3.LDIFParser(self.stream, encoding=None) items = list(self.p.parse()) self.assertEqual(items, [( 'cn=Bjorn J Jensen', { u'jpegPhoto': [b'\xf0\xf2\xf3'], u'foo': [b'bar'], } )]) class TestLDIFParserEmptyAttrValue(unittest.TestCase): def setUp(self): self.stream = BytesIO(BYTES_EMPTY_ATTR_VALUE) self.p = ldif3.LDIFParser(self.stream) def test_parse(self): list(self.p.parse()) def test_parse_value(self): dn, record = list(self.p.parse())[0] self.assertEqual(record['aliases'], ['', 'foo.bar']) class TestLDIFWriter(unittest.TestCase): def setUp(self): self.stream = BytesIO() self.w = ldif3.LDIFWriter(self.stream) def test_fold_line_10_n(self): self.w._cols = 10 self.w._line_sep = b'\n' self.w._fold_line(b'abcdefghijklmnopqrstuvwxyz') folded = b'abcdefghij\n klmnopqrs\n tuvwxyz\n' self.assertEqual(self.stream.getvalue(), folded) def test_fold_line_12_underscore(self): self.w._cols = 12 self.w._line_sep = b'__' self.w._fold_line(b'abcdefghijklmnopqrstuvwxyz') folded = b'abcdefghijkl__ mnopqrstuvw__ xyz__' self.assertEqual(self.stream.getvalue(), folded) def test_fold_line_oneline(self): self.w._cols = 100 self.w._line_sep = b'\n' self.w._fold_line(b'abcdefghijklmnopqrstuvwxyz') folded = b'abcdefghijklmnopqrstuvwxyz\n' self.assertEqual(self.stream.getvalue(), folded) def test_needs_base64_encoding_forced(self): self.w._base64_attrs = ['attr_type'] result = self.w._needs_base64_encoding('attr_type', 'attr_value') self.assertTrue(result) def test_needs_base64_encoding_not_safe(self): result = self.w._needs_base64_encoding('attr_type', '\r') self.assertTrue(result) def test_needs_base64_encoding_safe(self): result = self.w._needs_base64_encoding('attr_type', 'abcABC123_+') self.assertFalse(result) def test_unparse_attr_base64(self): self.w._unparse_attr('foo', 'a\nb\nc') value = self.stream.getvalue() self.assertEqual(value, b'foo:: YQpiCmM=\n') def test_unparse_entry_record(self): self.w._unparse_entry_record(RECORDS[0]) value = self.stream.getvalue() self.assertEqual(value, ( b'cn: Alison Alison\n' b'mail: alicealison@example.com\n' b'modifytimestamp: 4a463e9a\n' b'objectclass: top\n' b'objectclass: person\n' b'objectclass: organizationalPerson\n')) def test_unparse_changetype_add(self): self.w._unparse_changetype(2) value = self.stream.getvalue() self.assertEqual(value, b'changetype: add\n') def test_unparse_changetype_modify(self): self.w._unparse_changetype(3) value = self.stream.getvalue() self.assertEqual(value, b'changetype: modify\n') def test_unparse_changetype_other(self): with self.assertRaises(ValueError): self.w._unparse_changetype(4) with self.assertRaises(ValueError): self.w._unparse_changetype(1) def test_unparse(self): for i, record in enumerate(RECORDS): self.w.unparse(DNS[i], record) value = self.stream.getvalue() self.assertEqual(value, BYTES_OUT) def test_unparse_fail(self): with self.assertRaises(ValueError): self.w.unparse(DNS[0], 'foo') def test_unparse_binary(self): self.w.unparse(u'cn=Bjorn J Jensen', {u'jpegPhoto': [b'\xf0\xf2\xf3']}) value = self.stream.getvalue() self.assertEqual(value, b'dn: cn=Bjorn J Jensen\njpegPhoto:: 8PLz\n\n') def test_unparse_unicode_dn(self): self.w.unparse(u'cn=Björn J Jensen', {u'foo': [u'bar']}) value = self.stream.getvalue() self.assertEqual(value, b'dn:: Y249QmrDtnJuIEogSmVuc2Vu\nfoo: bar\n\n') def test_unparse_uniqode(self): self.w.unparse("o=x", {'test': [u'日本語']}) value = self.stream.getvalue() self.assertEqual(value, b'dn: o=x\ntest:: 5pel5pys6Kqe\n\n') ldif3-3.2.2/tox.ini000066400000000000000000000005651304642415100140400ustar00rootroot00000000000000# Tox (http://tox.testrun.org/) is a tool for running tests # in multiple virtualenvs. This configuration file will run the # test suite on all supported python versions. To use it, "pip install tox" # and then run "tox" from this directory. [tox] envlist = py27, py34, pypy [testenv] commands = flake8 nosetests deps = nose coverage flake8 mock