tqdm-4.11.2/0000775000175000017500000000000013041514547013324 5ustar cr45hincr45hin00000000000000tqdm-4.11.2/MANIFEST.in0000664000175000017500000000043113041134651015052 0ustar cr45hincr45hin00000000000000# Misc include .coveragerc include CONTRIBUTING.md include LICENCE include logo.png # include images/logo.gif include Makefile include README.rst include RELEASE include tox.ini # Test suite recursive-include tqdm/tests *.py # Examples/Documentation recursive-include examples * tqdm-4.11.2/setup.cfg0000664000175000017500000000013013041514547015137 0ustar cr45hincr45hin00000000000000[bdist_wheel] universal = 1 [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 tqdm-4.11.2/tqdm.egg-info/0000775000175000017500000000000013041514547015763 5ustar cr45hincr45hin00000000000000tqdm-4.11.2/tqdm.egg-info/top_level.txt0000664000175000017500000000000513041514547020510 0ustar cr45hincr45hin00000000000000tqdm tqdm-4.11.2/tqdm.egg-info/dependency_links.txt0000664000175000017500000000000113041514547022031 0ustar cr45hincr45hin00000000000000 tqdm-4.11.2/tqdm.egg-info/SOURCES.txt0000664000175000017500000000166613041514547017660 0ustar cr45hincr45hin00000000000000.coveragerc CONTRIBUTING.md LICENCE MANIFEST.in Makefile README.rst logo.png setup.cfg setup.py tox.ini examples/.git examples/7zx.py examples/include_no_requirements.py examples/pandas_progress_apply.py examples/simple_examples.py examples/tqdm_wget.py examples/progressbar/__init__.py examples/progressbar/__init__.pyc examples/progressbar/compat.py examples/progressbar/compat.pyc examples/progressbar/progressbar.py examples/progressbar/progressbar.pyc examples/progressbar/widgets.py examples/progressbar/widgets.pyc tqdm/__init__.py tqdm/__main__.py tqdm/_main.py tqdm/_tqdm.py tqdm/_tqdm_gui.py tqdm/_tqdm_notebook.py tqdm/_tqdm_pandas.py tqdm/_utils.py tqdm/_version.py tqdm.egg-info/PKG-INFO tqdm.egg-info/SOURCES.txt tqdm.egg-info/dependency_links.txt tqdm.egg-info/entry_points.txt tqdm.egg-info/top_level.txt tqdm/tests/tests_main.py tqdm/tests/tests_pandas.py tqdm/tests/tests_perf.py tqdm/tests/tests_tqdm.py tqdm/tests/tests_version.pytqdm-4.11.2/tqdm.egg-info/entry_points.txt0000664000175000017500000000005213041514547021256 0ustar cr45hincr45hin00000000000000[console_scripts] tqdm = tqdm._main:main tqdm-4.11.2/tqdm.egg-info/PKG-INFO0000664000175000017500000010432113041514547017061 0ustar cr45hincr45hin00000000000000Metadata-Version: 1.1 Name: tqdm Version: 4.11.2 Summary: Fast, Extensible Progress Meter Home-page: https://github.com/tqdm/tqdm Author: tqdm developers Author-email: python.tqdm@gmail.com License: MPLv2.0, MIT Licences Description: |Logo| tqdm ==== |PyPI-Status| |PyPI-Versions| |Conda-Forge-Status| |Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |DOI-URI| |LICENCE| ``tqdm`` means "progress" in Arabic (taqadum, تقدّم) and an abbreviation for "I love you so much" in Spanish (te quiero demasiado). Instantly make your loops show a smart progress meter - just wrap any iterable with ``tqdm(iterable)``, and you're done! .. code:: python from tqdm import tqdm for i in tqdm(range(10000)): ... ``76%|████████████████████████████         | 7568/10000 [00:33<00:10, 229.00it/s]`` ``trange(N)`` can be also used as a convenient shortcut for ``tqdm(xrange(N))``. |Screenshot| REPL: `ptpython `__ It can also be executed as a module with pipes: .. code:: sh $ seq 9999999 | tqdm --unit_scale | wc -l 10.0Mit [00:02, 3.58Mit/s] 9999999 $ 7z a -bd -r backup.7z docs/ | grep Compressing | \ tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] Overhead is low -- about 60ns per iteration (80ns with ``tqdm_gui``), and is unit tested against performance regression. By comparison, the well established `ProgressBar `__ has an 800ns/iter overhead. In addition to its low overhead, ``tqdm`` uses smart algorithms to predict the remaining time and to skip unnecessary iteration displays, which allows for a negligible overhead in most cases. ``tqdm`` works on any platform (Linux, Windows, Mac, FreeBSD, Solaris/SunOS), in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks. ``tqdm`` does not require any library (not even curses!) to run, just a vanilla Python interpreter will do and an environment supporting ``carriage return \r`` and ``line feed \n`` control characters. ------------------------------------------ .. contents:: Table of contents :backlinks: top :local: Installation ------------ Latest PyPI stable release ~~~~~~~~~~~~~~~~~~~~~~~~~~ |PyPI-Status| .. code:: sh pip install tqdm Latest development release on github ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |GitHub-Status| |GitHub-Stars| |GitHub-Forks| Pull and install in the current directory: .. code:: sh pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm Latest Conda release ~~~~~~~~~~~~~~~~~~~~ |Conda-Forge-Status| .. code:: sh conda install -c conda-forge tqdm Changelog --------- The list of all changes is available either on GitHub's Releases: |GitHub-Status| or on crawlers such as `allmychanges.com `_. Usage ----- ``tqdm`` is very versatile and can be used in a number of ways. The three main ones are given below. Iterable-based ~~~~~~~~~~~~~~ Wrap ``tqdm()`` around any iterable: .. code:: python text = "" for char in tqdm(["a", "b", "c", "d"]): text = text + char ``trange(i)`` is a special optimised instance of ``tqdm(range(i))``: .. code:: python for i in trange(100): pass Instantiation outside of the loop allows for manual control over ``tqdm()``: .. code:: python pbar = tqdm(["a", "b", "c", "d"]) for char in pbar: pbar.set_description("Processing %s" % char) Manual ~~~~~~ Manual control on ``tqdm()`` updates by using a ``with`` statement: .. code:: python with tqdm(total=100) as pbar: for i in range(10): pbar.update(10) If the optional variable ``total`` (or an iterable with ``len()``) is provided, predictive stats are displayed. ``with`` is also optional (you can just assign ``tqdm()`` to a variable, but in this case don't forget to ``del`` or ``close()`` at the end: .. code:: python pbar = tqdm(total=100) for i in range(10): pbar.update(10) pbar.close() Module ~~~~~~ Perhaps the most wonderful use of ``tqdm`` is in a script or on the command line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass through all ``stdin`` to ``stdout`` while printing progress to ``stderr``. The example below demonstrated counting the number of lines in all python files in the current directory, with timing information included. .. code:: sh $ time find . -name '*.py' -exec cat \{} \; | wc -l 857365 real 0m3.458s user 0m0.274s sys 0m3.325s $ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 857366it [00:03, 246471.31it/s] 857365 real 0m3.585s user 0m0.862s sys 0m3.358s Note that the usual arguments for ``tqdm`` can also be specified. .. code:: sh $ find . -name '*.py' -exec cat \{} \; | tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|███████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s] Backing up a large directory? .. code:: sh $ 7z a -bd -r backup.7z docs/ | grep Compressing | tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] FAQ and Known Issues -------------------- The most common issues relate to excessive output on multiple lines, instead of a neat one-line progress bar. - Consoles in general: require support for carriage return (``CR``, ``\r``). - Nested progress bars: * Consoles in general: require support for moving cursors up to the previous line. For example, `IDLE `__, `ConEmu `__ and `PyCharm `__ (also `here `__ and `here `__) lack full support. * Windows: additionally may require the python module ``colorama``. - Wrapping enumerated iterables: use ``enumerate(tqdm(...))`` instead of ``tqdm(enumerate(...))`. The same applies to ``numpy.ndenumerate``. This is because enumerate functions tend to hide the length of iterables. ``tqdm`` does not. - Wrapping zipped iterables has similar issues due to internal optimisations. ``tqdm(zip(a, b))`` should be replaced with ``zip(tqdm(a), b)`` or even ``zip(tqdm(a), tqdm(b))`` If you come across any other difficulties, browse/open issues `here `__. Documentation ------------- |PyPI-Versions| |README-Hits| (Since 19 May 2016) .. code:: python class tqdm(object): """ Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested. """ def __init__(self, iterable=None, desc=None, total=None, leave=True, file=sys.stderr, ncols=None, mininterval=0.1, maxinterval=10.0, miniters=None, ascii=None, disable=False, unit='it', unit_scale=False, dynamic_ncols=False, smoothing=0.3, bar_format=None, initial=0, position=None, postfix=None): Parameters ~~~~~~~~~~ * iterable : iterable, optional Iterable to decorate with a progressbar. Leave blank to manually manage the updates. * desc : str, optional Prefix for the progressbar. * total : int, optional The number of expected iterations. If (default: None), len(iterable) is used if possible. As a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If ``gui`` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int(9e9). * leave : bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. * file : ``io.TextIOWrapper`` or ``io.StringIO``, optional Specifies where to output the progress messages [default: sys.stderr]. Uses ``file.write(str)`` and ``file.flush()`` methods. * ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). * mininterval : float, optional Minimum progress display update interval, in seconds [default: 0.1]. * maxinterval : float, optional Maximum progress display update interval, in seconds [default: 10]. Automatically adjusts ``miniters`` to correspond to ``mininterval`` after long display update lag. Only works if ``dynamic_miniters`` or monitor thread is enabled. * miniters : int, optional Minimum progress display update interval, in iterations. If 0 and ``dynamic_miniters``, will automatically adjust to equal ``mininterval`` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and ``mininterval`` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. * ascii : bool, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters ``1-9 #``. * disable : bool, optional Whether to disable the entire progressbar wrapper [default: False]. * unit : str, optional String that will be used to define the unit of each iteration [default: it]. * unit_scale : bool, optional If set, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. * dynamic_ncols : bool, optional If set, constantly alters ``ncols`` to the environment (allowing for window resizes) [default: False]. * smoothing : float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. * bar_format : str, optional Specify a custom bar string formatting. May impact performance. If unspecified, will use '{l_bar}{bar}{r_bar}', where l_bar is '{desc}{percentage:3.0f}%|' and r_bar is '| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. * initial : int, optional The initial counter value. Useful when restarting a progress bar [default: 0]. * position : int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). * postfix : dict, optional Specify additional stats to display at the end of the bar. Extra CLI Options ~~~~~~~~~~~~~~~~~ * delim : chr, optional Delimiting character [default: '\n']. Use '\0' for null. N.B.: on Windows systems, Python converts '\n' to '\r\n'. * buf_size : int, optional String buffer size in bytes [default: 256] used when ``delim`` is specified. * bytes : bool, optional If true, will count bytes and ignore ``delim``. Returns ~~~~~~~ * out : decorated iterator. .. code:: python def update(self, n=1): """ Manually update the progress bar, useful for streams such as reading files. E.g.: >>> t = tqdm(total=filesize) # Initialise >>> for current_buffer in stream: ... ... ... t.update(len(current_buffer)) >>> t.close() The last line is highly recommended, but possibly not necessary if ``t.update()`` will be called in such a way that ``filesize`` will be exactly reached and printed. Parameters ---------- n : int Increment to add to the internal counter of iterations [default: 1]. """ def close(self): """ Cleanup and (if leave=False) close the progressbar. """ def clear(self): """ Clear current bar display """ def refresh(self): """ Force refresh the display of this bar """ def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm (without overlap with bars) """ def set_description(self, desc=None): """ Set/modify description of the progress bar. """ def set_postfix(self, **kwargs): """ Set/modify postfix (additional stats) with automatic formatting based on datatype. """ def trange(*args, **kwargs): """ A shortcut for tqdm(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ class tqdm_gui(tqdm): """ Experimental GUI version of tqdm! """ def tgrange(*args, **kwargs): """ Experimental GUI version of trange! """ class tqdm_notebook(tqdm): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ def tnrange(*args, **kwargs): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ Examples and Advanced Usage --------------------------- - See the `examples `__ folder; - import the module and run ``help()``, or - consult the `wiki `__. - this has an `excellent article `__ on how to make a **great** progressbar. Description and additional stats ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Custom information can be displayed and updated dynamically on ``tqdm`` bars with the ``desc`` and ``postfix`` arguments: .. code:: python from tqdm import trange from random import random, randint from time import sleep t = trange(100) for i in t: # Description will be displayed on the left t.set_description('GEN %i' % i) # Postfix will be displayed on the right, and will format automatically # based on argument's datatype t.set_postfix(loss=random(), gen=randint(1,999), str='h', lst=[1, 2]) sleep(0.1) Nested progress bars ~~~~~~~~~~~~~~~~~~~~ ``tqdm`` supports nested progress bars. Here's an example: .. code:: python from tqdm import trange from time import sleep for i in trange(10, desc='1st loop'): for j in trange(5, desc='2nd loop', leave=False): for k in trange(100, desc='3nd loop'): sleep(0.01) On Windows `colorama `__ will be used if available to produce a beautiful nested display. For manual control over positioning (e.g. for multi-threaded use), you may specify `position=n` where `n=0` for the outermost bar, `n=1` for the next, and so on. Hooks and callbacks ~~~~~~~~~~~~~~~~~~~ ``tqdm`` can easily support callbacks/hooks and manual updates. Here's an example with ``urllib``: **urllib.urlretrieve documentation** | [...] | If present, the hook function will be called once | on establishment of the network connection and once after each block read | thereafter. The hook will be passed three arguments; a count of blocks | transferred so far, a block size in bytes, and the total size of the file. | [...] .. code:: python import urllib from tqdm import tqdm def my_hook(t): """ Wraps tqdm instance. Don't forget to close() or __exit__() the tqdm instance once you're done with it (easiest using `with` syntax). Example ------- >>> with tqdm(...) as t: ... reporthook = my_hook(t) ... urllib.urlretrieve(..., reporthook=reporthook) """ last_b = [0] def inner(b=1, bsize=1, tsize=None): """ b : int, optional Number of blocks just transferred [default: 1]. bsize : int, optional Size of each block (in tqdm units) [default: 1]. tsize : int, optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: t.total = tsize t.update((b - last_b[0]) * bsize) last_b[0] = b return inner eg_link = 'http://www.doc.ic.ac.uk/~cod11/matryoshka.zip' with tqdm(unit='B', unit_scale=True, miniters=1, desc=eg_link.split('/')[-1]) as t: # all optional kwargs urllib.urlretrieve(eg_link, filename='/dev/null', reporthook=my_hook(t), data=None) It is recommend to use ``miniters=1`` whenever there is potentially large differences in iteration speed (e.g. downloading a file over a patchy connection). Pandas Integration ~~~~~~~~~~~~~~~~~~ Due to popular demand we've added support for ``pandas`` -- here's an example for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``: .. code:: python import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.) tqdm.pandas(desc="my bar!") # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2) In case you're interested in how this works (and how to modify it for your own callbacks), see the `examples `__ folder or import the module and run ``help()``. IPython/Jupyter Integration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ IPython/Jupyter is supported via the `tqdm_notebook` submodule: .. code:: python from tqdm import tnrange, tqdm_notebook from time import sleep for i in tnrange(10, desc='1st loop'): for j in tqdm_notebook(xrange(100), desc='2nd loop'): sleep(0.01) In addition to `tqdm` features, the submodule provides a native Jupyter widget (compatible with IPython v1-v4 and Jupyter), fully working nested bars and color hints (blue: normal, green: completed, red: error/interrupt, light blue: no ETA); as demonstrated below. |Screenshot-Jupyter1| |Screenshot-Jupyter2| |Screenshot-Jupyter3| Writing messages ~~~~~~~~~~~~~~~~ Since ``tqdm`` uses a simple printing mechanism to display progress bars, you should not write any message in the terminal using ``print()``. To write messages in the terminal without any collision with ``tqdm`` bar display, a ``.write()`` method is provided: .. code:: python from tqdm import tqdm, trange from time import sleep bar = trange(10) for i in bar: # Print using tqdm class method .write() sleep(0.1) if not (i % 3): tqdm.write("Done task %i" % i) # Can also use bar.write() By default, this will print to standard output ``sys.stdout``. but you can specify any file-like object using the ``file`` argument. For example, this can be used to redirect the messages writing to a log file or class. Redirecting writing ~~~~~~~~~~~~~~~~~~~ If using a library that can print messages to the console, editing the library by replacing ``print()`` with ``tqdm.write()`` may not be desirable. In that case, redirecting ``sys.stdout`` to ``tqdm.write()`` is an option. To redirect ``sys.stdout``, create a file-like class that will write any input string to ``tqdm.write()``, and supply the arguments ``file=sys.stdout, dynamic_ncols=True``. A reusable canonical example is given below: .. code:: python from time import sleep import contextlib import sys from tqdm import tqdm class DummyTqdmFile(object): """Dummy file-like that will write to tqdm""" file = None def __init__(self, file): self.file = file def write(self, x): # Avoid print() second call (useless \n) if len(x.rstrip()) > 0: tqdm.write(x, file=self.file) @contextlib.contextmanager def stdout_redirect_to_tqdm(): save_stdout = sys.stdout try: sys.stdout = DummyTqdmFile(sys.stdout) yield save_stdout # Relay exceptions except Exception as exc: raise exc # Always restore sys.stdout if necessary finally: sys.stdout = save_stdout def blabla(): print("Foo blabla") # Redirect stdout to tqdm.write() (don't forget the `as save_stdout`) with stdout_redirect_to_tqdm() as save_stdout: # tqdm call need to specify sys.stdout, not sys.stderr (default) # and dynamic_ncols=True to autodetect console width for _ in tqdm(range(3), file=save_stdout, dynamic_ncols=True): blabla() sleep(.5) # After the `with`, printing is restored print('Done!') Monitoring thread, intervals and miniters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``tqdm`` implements a few tricks to to increase efficiency and reduce overhead. 1. Avoid unnecessary frequent bar refreshing: ``mininterval`` defines how long to wait between each refresh. ``tqdm`` always gets updated in the background, but it will diplay only every ``mininterval``. 2. Reduce number of calls to check system clock/time. 3. ``mininterval`` is more intuitive to configure than ``miniters``. A clever adjustment system ``dynamic_miniters`` will automatically adjust ``miniters`` to the amount of iterations that fit into time ``mininterval``. Essentially, ``tqdm`` will check if it's time to print without actually checking time. This behavior can be still be bypassed by manually setting ``miniters``. However, consider a case with a combination of fast and slow iterations. After a few fast iterations, ``dynamic_miniters`` will set ``miniters`` to a large number. When interation rate subsequently slows, ``miniters`` will remain large and thus reduce display update frequency. To address this: 4. ``maxinterval`` defines the maximum time between display refreshes. A concurrent monitoring thread checks for overdue updates and forces one where necessary. The monitoring thread should not have a noticeable overhead, and guarantees updates at least every 10 seconds by default. This value can be directly changed by setting the ``monitor_interval`` of any ``tqdm`` instance (i.e. ``t = tqdm.tqdm(...); t.monitor_interval = 2``). The monitor thread may be disabled application-wide by setting ``tqdm.tqdm.monitor_interval = 0`` before instantiatiation of any ``tqdm`` bar. Contributions ------------- All source code is hosted on `GitHub `__. Contributions are welcome. See the `CONTRIBUTING `__ file for more information. LICENCE ------- Open Source (OSI approved): |LICENCE| Citation information: |DOI-URI| Authors ------- Ranked by contributions. - Casper da Costa-Luis (casperdcl) - Stephen Larroque (lrq3000) - Hadrien Mary (hadim) - Noam Yorav-Raphael (noamraph)* - Ivan Ivanov (obiwanus) - Mikhail Korobov (kmike) `*` Original author |README-Hits| (Since 19 May 2016) .. |Logo| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif .. |Screenshot| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm.gif .. |Build-Status| image:: https://travis-ci.org/tqdm/tqdm.svg?branch=master :target: https://travis-ci.org/tqdm/tqdm .. |Coverage-Status| image:: https://coveralls.io/repos/tqdm/tqdm/badge.svg :target: https://coveralls.io/r/tqdm/tqdm .. |Branch-Coverage-Status| image:: https://codecov.io/github/tqdm/tqdm/coverage.svg?branch=master :target: https://codecov.io/github/tqdm/tqdm?branch=master .. |Codacy-Grade| image:: https://api.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177 :target: https://www.codacy.com/app/tqdm/tqdm?utm_source=github.com&utm_medium=referral&utm_content=tqdm/tqdm&utm_campaign=Badge_Grade .. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=2592000 :target: https://github.com/tqdm/tqdm/releases .. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/network .. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/stargazers .. |PyPI-Status| image:: https://img.shields.io/pypi/v/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |Conda-Forge-Status| image:: https://anaconda.org/conda-forge/tqdm/badges/version.svg :target: https://anaconda.org/conda-forge/tqdm .. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg :target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE .. |DOI-URI| image:: https://zenodo.org/badge/21637/tqdm/tqdm.svg :target: https://zenodo.org/badge/latestdoi/21637/tqdm/tqdm .. |Screenshot-Jupyter1| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-1.gif .. |Screenshot-Jupyter2| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-2.gif .. |Screenshot-Jupyter3| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-3.gif .. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif :target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif&style=social Keywords: progressbar progressmeter progress bar meter rate eta console terminal time Platform: any Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0) Classifier: License :: OSI Approved :: MIT License Classifier: Environment :: Console Classifier: Framework :: IPython Classifier: Operating System :: Microsoft :: Windows Classifier: Operating System :: MacOS :: MacOS X Classifier: Operating System :: POSIX Classifier: Operating System :: POSIX :: Linux Classifier: Operating System :: POSIX :: BSD Classifier: Operating System :: POSIX :: BSD :: FreeBSD Classifier: Operating System :: POSIX :: SunOS/Solaris Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.2 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Programming Language :: Python :: Implementation :: IronPython Classifier: Topic :: Software Development :: Libraries Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Software Development :: User Interfaces Classifier: Topic :: System :: Monitoring Classifier: Topic :: Terminals Classifier: Topic :: Utilities Classifier: Intended Audience :: Developers tqdm-4.11.2/setup.py0000775000175000017500000001771213041513621015041 0ustar cr45hincr45hin00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import os try: from setuptools import setup except ImportError: from distutils.core import setup import sys from subprocess import check_call from io import open as io_open # For Makefile parsing import shlex try: # pragma: no cover import ConfigParser import StringIO except ImportError: # pragma: no cover import configparser as ConfigParser import io as StringIO import re # Makefile auxiliary functions # RE_MAKE_CMD = re.compile('^\t(@\+?)(make)?', flags=re.M) def parse_makefile_aliases(filepath): ''' Parse a makefile to find commands and substitute variables. Expects a makefile with only aliases and a line return between each command. Returns a dict, with a list of commands for each alias. ''' # -- Parsing the Makefile using ConfigParser # Adding a fake section to make the Makefile a valid Ini file ini_str = '[root]\n' with io_open(filepath, mode='r') as fd: ini_str = ini_str + RE_MAKE_CMD.sub('\t', fd.read()) ini_fp = StringIO.StringIO(ini_str) # Parse using ConfigParser config = ConfigParser.RawConfigParser() config.readfp(ini_fp) # Fetch the list of aliases aliases = config.options('root') # -- Extracting commands for each alias commands = {} for alias in aliases: if alias.lower() in ['.phony']: continue # strip the first line return, and then split by any line return commands[alias] = config.get('root', alias).lstrip('\n').split('\n') # -- Commands substitution # Loop until all aliases are substituted by their commands: # Check each command of each alias, and if there is one command that is to # be substituted by an alias, try to do it right away. If this is not # possible because this alias itself points to other aliases , then stop # and put the current alias back in the queue to be processed again later. # Create the queue of aliases to process aliases_todo = list(commands.keys()) # Create the dict that will hold the full commands commands_new = {} # Loop until we have processed all aliases while aliases_todo: # Pick the first alias in the queue alias = aliases_todo.pop(0) # Create a new entry in the resulting dict commands_new[alias] = [] # For each command of this alias for cmd in commands[alias]: # Ignore self-referencing (alias points to itself) if cmd == alias: pass # Substitute full command elif cmd in aliases and cmd in commands_new: # Append all the commands referenced by the alias commands_new[alias].extend(commands_new[cmd]) # Delay substituting another alias, waiting for the other alias to # be substituted first elif cmd in aliases and cmd not in commands_new: # Delete the current entry to avoid other aliases # to reference this one wrongly (as it is empty) del commands_new[alias] aliases_todo.append(alias) break # Full command (no aliases) else: commands_new[alias].append(cmd) commands = commands_new del commands_new # -- Prepending prefix to avoid conflicts with standard setup.py commands # for alias in commands.keys(): # commands['make_'+alias] = commands[alias] # del commands[alias] return commands def execute_makefile_commands(commands, alias, verbose=False): cmds = commands[alias] for cmd in cmds: # Parse string in a shell-like fashion # (incl quoted strings and comments) parsed_cmd = shlex.split(cmd, comments=True) # Execute command if not empty (ie, not just a comment) if parsed_cmd: if verbose: print("Running command: " + cmd) # Launch the command and wait to finish (synchronized call) check_call(parsed_cmd, cwd=os.path.dirname(os.path.abspath(__file__))) # Main setup.py config # # Get version from tqdm/_version.py __version__ = None version_file = os.path.join(os.path.dirname(__file__), 'tqdm', '_version.py') with io_open(version_file, mode='r') as fd: exec(fd.read()) # Executing makefile commands if specified if sys.argv[1].lower().strip() == 'make': # Filename of the makefile fpath = os.path.join(os.path.dirname(__file__), 'Makefile') # Parse the makefile, substitute the aliases and extract the commands commands = parse_makefile_aliases(fpath) # If no alias (only `python setup.py make`), print the list of aliases if len(sys.argv) < 3 or sys.argv[-1] == '--help': print("Shortcut to use commands via aliases. List of aliases:") print('\n'.join(alias for alias in sorted(commands.keys()))) # Else process the commands for this alias else: arg = sys.argv[-1] # if unit testing, we do nothing (we just checked the makefile parsing) if arg == 'none': sys.exit(0) # else if the alias exists, we execute its commands elif arg in commands.keys(): execute_makefile_commands(commands, arg, verbose=True) # else the alias cannot be found else: raise Exception("Provided alias cannot be found: make " + arg) # Stop the processing of setup.py here: # It's important to avoid setup.py raising an error because of the command # not being standard sys.exit(0) # Python package config # README_rst = '' fndoc = os.path.join(os.path.dirname(__file__), 'README.rst') with io_open(fndoc, mode='r', encoding='utf-8') as fd: README_rst = fd.read() setup( name='tqdm', version=__version__, description='Fast, Extensible Progress Meter', license='MPLv2.0, MIT Licences', author='Noam Yorav-Raphael', author_email='noamraph@gmail.com', url='https://github.com/tqdm/tqdm', maintainer='tqdm developers', maintainer_email='python.tqdm@gmail.com', platforms=['any'], packages=['tqdm'], entry_points={'console_scripts': ['tqdm=tqdm._main:main'], }, long_description=README_rst, classifiers=[ # Trove classifiers # (https://pypi.python.org/pypi?%3Aaction=list_classifiers) 'Development Status :: 5 - Production/Stable', 'License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)', 'License :: OSI Approved :: MIT License', 'Environment :: Console', 'Framework :: IPython', 'Operating System :: Microsoft :: Windows', 'Operating System :: MacOS :: MacOS X', 'Operating System :: POSIX', 'Operating System :: POSIX :: Linux', 'Operating System :: POSIX :: BSD', 'Operating System :: POSIX :: BSD :: FreeBSD', 'Operating System :: POSIX :: SunOS/Solaris', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.2', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: Implementation :: PyPy', 'Programming Language :: Python :: Implementation :: IronPython', 'Topic :: Software Development :: Libraries', 'Topic :: Software Development :: Libraries :: Python Modules', 'Topic :: Software Development :: User Interfaces', 'Topic :: System :: Monitoring', 'Topic :: Terminals', 'Topic :: Utilities', 'Intended Audience :: Developers', ], keywords='progressbar progressmeter progress bar meter' ' rate eta console terminal time', test_suite='nose.collector', tests_require=['nose', 'flake8', 'coverage'], ) tqdm-4.11.2/tox.ini0000664000175000017500000000351513041134651014635 0ustar cr45hincr45hin00000000000000# Tox (http://tox.testrun.org/) is a tool for running tests # in multiple virtualenvs. This configuration file will run the # test suite on all supported python versions. To use it, "pip install tox" # and then run "tox" from this directory. [tox] # deprecation warning: py26, py33 envlist = py32, py33, py34, py26, py27, py35, py36, pypy, pypy3, pypy3.3-5.2-alpha1, flake8, setup.py, perf [testenv] passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH deps = nose nose-timer coverage<4 coveralls cython numpy pandas commands = nosetests --with-coverage --with-timer --cover-package=tqdm --ignore-files="tests_perf\.py" -d -v tqdm/ - coveralls [testenv:pypy] passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH deps = nose nose-timer coverage<4 coveralls commands = nosetests --with-coverage --with-timer --cover-package=tqdm --ignore-files="tests_p(erf|andas)\.py" -d -v tqdm/ - coveralls [testenv:pypy3] passenv = {[testenv:pypy]passenv} deps = {[testenv:pypy]deps} commands = {[testenv:pypy]commands} [testenv:pypy3.3-5.2-alpha1] passenv = {[testenv:pypy]passenv} deps = {[testenv:pypy]deps} virtualenv>=15.0.2 commands = {[testenv:pypy]commands} [testenv:py26] passenv = {[testenv:pypy]passenv} deps = {[testenv:pypy]deps} commands = {[testenv:pypy]commands} [testenv:py33] passenv = {[testenv:pypy]passenv} deps = {[testenv:pypy]deps} commands = {[testenv:pypy]commands} [testenv:flake8] deps = flake8 commands = flake8 --max-line-length=80 --count --statistics --exit-zero -j 8 --exclude .asv . [testenv:setup.py] deps = docutils pygments commands = python setup.py check --restructuredtext --metadata --strict python setup.py make none [testenv:perf] deps = nose nose-timer commands = nosetests --with-timer tqdm/tests/tests_perf.py -d -v tqdm-4.11.2/logo.png0000664000175000017500000002374713040270166015002 0ustar cr45hincr45hin00000000000000PNG  IHDRddpT IDATxwTϹN۾,mH@;64X/QcbF#j 4,Q$v DLl@TD. is~lݙYD{xks=?y#S7Gt$(co+7:m#}Xk/hn_#EVS;-Q ;.(b}d(PlKxE]b^Ls}>hKΫlht_@Z?^qaϘ2yN L&p4MD!5KhA0R".h=Wl:{k,< <}L}<T"qS|ɱl Y0z>" %ex<ʟ{jΉs,eKl~3㨧wgHY#"6ރ!Y|RAl-^as8]鼷l7|/qC_$/ta&A@,5Arҏ=+Vr_,kVѯ"#ʡSv>`bl4D, ol'Ŏ[|u3Q"hfntC`,ɦ06@lj5?!b3]#Dk/7-H7"` h_+`D4"M'ǧܣ#\. 4r 8"M?m dwh٭pQgNp:] w5'lp"vC,C cpuI bQ`\ H_Pxqsw\4W츀 !𳗮t:fRw nD1B'\(0~mqc&iG'>z./ z 5 e&cn(imܨ[vFLn SH&=)Ԝhv@s?8矯#/wH젡#T.$Qb-Ejz_ !7B-TfVK6MD$OUЀs7+IH[$hzO8kwstxgc,O-:jŲ1> EIF"\~-c.޾-nЈήC<u 4ỊDSJn `h,UO!-y1f bf;'h( I-\~0E9_MljR׺fY8LQ  >MϐS͠l/cQ%v0$;|GqQTТ+&2~m +_CAEPNEg`d W-W ȯW;ƍ^ 4g ^ɤH( V 1WQI=+ OdPGGIPrӴ kb =?H&zqYCU"@GqQU7e6mS5g|7XaڷH_M^RzaC{plF3jbD5T# ">FGI3(6nG#Y5 c&#`ظ")`iuq<1GY깼Zxb|` Ndi=}.<;0di X4JN8zvC2D!P%I5 댚yXL*uvB%$Bc;4C):(6G[7!{ ? k̬]D@eꔎc c}q+` ?e ߂OrJ/M>{>9d$&XJ\tf1k v~4 ;l8g?uH pCx&N45tŜmYG"2]ƾM~F?Ta"0KE]zEu` 8O}9M65\^g'Nh{ߵFּ& kyIp=2użG4Ah݁o;nU+y~"z^d¿)(#mQAY~]r0}钷5udK_MO~ùS)TZ# Ah9@vZFw':o-ěmng8Wm`VsuDD44p޳b/o1؆u7z?^Hf>}DLTN *F$ xKџFp޹ lZՈ]םIPO3>%EQG Y>p2h[NIkIgB鞬,#OP~F ^v&v1:BC^r?>,y.8̀QZxܖ2Qw~O`B Vw:K#.> c"7[>ϸG (;9FQtl1 `gBXV|7OPp%gS7s˘]琣qo1zK('%4 vH3`tl.yW~!Y60(72(kۜ9"JH; Ci40d=?ZGM;<7:X$`ebSI-cE3'KoDa@&\q,M 3-oLw-XiAeuJ)WEez@T"rb|dD KK>`TGv)) P g] +ЭXc(i0YT\i2|$!tYC_ $@}=y{`%GcauD;mʍjY |mĹ>.IS\ơd܆ ~rsn VԲc,V"O~Ic ">}*(pFӇo38ô/ ajxqDsOBǭԕZU8!(aL;QO*}~LACpHhm~zqo7P+v\AņncxXA/:,l.Îx@&!B!Ly!{]9jϚ<^yDT~k6;,crZ$wBZv_-I\[y~f+bc,;w|VeJm4QΣl(R'.|'oj[G!R2s7PJa0F1A) E̝zwz;}1 1VW Sm,z7T`fmAw5aI?) V+:=+j)庤W\\H~&mz掻c9>nTyRT${Hy;k:P> M}0@QNSK񨱩7R9dQ16 6u*3V|NMS9,wz%bQllpA QAP# ]+`X]={"8&퇷J _!e6Q ` ^Q.eem)g-͛S8|9x4fGH${fE<#fh hĤ ,[ w`nG0nt>z?Zg\%>dђEB}z4Cл V^AX(;Ai@*BXvWvIxO"j~'_]BA /F^p ES!є@[?!m"l;,=w z׶h_ÈhTVd)Lh=dX$19x)$I󮯭ŕlܨ"U(+vuA*js-ͤO@f dr!BέljE4>SJUE y 6&6cs9(l.;Hsǝ9ZX.!O{Mz2BD[ӜMLT@oDgd :ȀX~}I"Xb\ 1־U_Ym*v3<chZHH' &3̒Xw;O`;ZثMtDb",VLe[PN-{ dsEbQh=[\A(vHIi:C fVuWJLo֘^̶-BF6FևICVFO;8^mEQjRxE&:"VMs!]d%RpEq^\fpcj#KimC(xcD=E:'la -Oܗg;#C6I{[,~Ua<]$L0SCk=ӻ -C :2/$򻉋2yO 'BN-m s]`b``q!jӃ<4*Ug?O_qTe)  ߶dK4 S"(Q3U* KGݘ&eRƼ91 dR/kﶬ!|ҧ +D!-j\|ꊇC20;3{<Ho& h :Qp EVsu}lYf];@Y/a~8Ӑ~إ=288rGX+ݍdD.b6n"ļLВM"&= ̫YC'&+ڼ1"1C XC%\6j>$$M' xo[bo{ GBi(g<~E?u܅EO p.Uoܸ/dBYAw/#oҬy}KB[*_5Xk:03 74mi3nc?}L֨m(&Yjs^O]iB0㘍]-A_gOD<3D)mtbVֶyKvڻ(Op%z c7GԚ7jX:&%7}d}{ty-/_3;rڀXmb"B5g ^S!q>04%U;mK 7Q+i4Ը Q:D q7침4d5 )0odՃbctdZV5QP7WH!ȒM!A! a!,!ܨd# P$踹 ɍ3%H|.rX>aD!V.kGMg/)<wWۙ:?K|$HFf۟)w6CfIo)/<ۼL;0GN.5<%3оB{3e(gѺX ?7\hKg )PyG3'2+>t"dz{`:.]^bWXo"nݷ݈1+B 85[ Vķ{eUksyqF+UqM4?5yufؐ(Ӑ"T@- w36=}"2u Ky@!I\ǢEq铸0 3ҹr'˵[wl'T8LͱMko.Qxhwʓ+)M' vi!?u[n/geSXl q,6#˕؜li ԴXSqx>67]NE)NgM{&-cgItgfKT%T\h*U#٬yn+i,6e?y ;-XlR"_'VlJ*|EunMN_Ok S6ٱ[͸l$u'O0*4 G$is=őAWh1 nVMi8g~Q[oUWlpuƍɎTq6iY9" i7u'0GV|e)}r=1tcL v"X6Vi +0o#q[v6gK(*f}#Njܺܘŷs!9hFE:fн3:#?eYq\u'o<׮r~Uf=16LSJ{=!cqVBhb M1瓩3K{`Om`qDȽ㧫,~fP0-=BNwg{IDAT> }FSxܧĂ!r3z;H9ˌ[[j it%`& ׼ƍny ,}C :WBpޫ]?wHG˄G\w,\vM=Wwܘ3N{zEv#SDR=i!Y:o:dGw3 {.4lұSӛ6?eRnKxm!17wCͧwcp}SXڪe Zgyb[AceNjMK(FpYCWӡ!&@S>,k>af^xa6u ѳSkĠ>^y?ޜd xO(e .ۡ2x} @PS64FRs}ٞ^m_kӐɷo8oLCJ#0/+-sMTFAuV^W S(;e9G2 kʆpu;Omn,w&+f/\l"@'{Nj.y7EDDL&Zw{F#SԮ0>9Wva96~%!b ijh,nU@4r oeknZ\0I؋V}c񧩵^b"Dh/Q;.§;sʿm>>>oQ{CƝ8?K%SV:#X4U{,?ec8kY1T_'✑V 6 ,ӏ|53း*6\H%&+7{mϖn {Z[0 c=9SVoؓxc1dSMdK57.ywܞ^C}۸ld#-[m1IjYUu gr~ole=51Z]C٧c4o_F{.Elm;wWcY~Ov'Y dpEFSzuKgx7)(rk[A`4MEL91 5ɋ7bY+EoX07ߓPP>['>Wl>tx8qL?؁F$@%+scPH fqrk)s.͸Q7&]d%y"Elzpf5@<F O #s.=lwx!E'T3=k-l>}&#=pC$bő<}9 ;l uO'S}6}pSol9 p;y:EMn7n4>>2gLHF ! n,%병{c⹞,ÖM4?=O^enחob#kS,xs_PS currval1: etasamp = self._eta(pbar.maxval - currval1, pbar.currval - currval1, elapsed - elapsed1) weight = (pbar.currval / float(pbar.maxval)) ** 0.5 eta = (1 - weight) * eta + weight * etasamp return 'ETA: %s' % self.format_time(eta) class FileTransferSpeed(Widget): """Widget for showing the transfer speed (useful for file transfers).""" FORMAT = '%6.2f %s%s/s' PREFIXES = ' kMGTPEZY' __slots__ = ('unit',) def __init__(self, unit='B'): self.unit = unit def update(self, pbar): """Updates the widget with the current SI prefixed speed.""" if pbar.seconds_elapsed < 2e-6 or pbar.currval < 2e-6: # =~ 0 scaled = power = 0 else: speed = pbar.currval / pbar.seconds_elapsed power = int(math.log(speed, 1000)) scaled = speed / 1000.**power return self.FORMAT % (scaled, self.PREFIXES[power], self.unit) class AnimatedMarker(Widget): """An animated marker for the progress bar which defaults to appear as if it were rotating. """ __slots__ = ('markers', 'curmark') def __init__(self, markers='|/-\\'): self.markers = markers self.curmark = -1 def update(self, pbar): """Updates the widget to show the next marker or the first marker when finished""" if pbar.finished: return self.markers[0] self.curmark = (self.curmark + 1) % len(self.markers) return self.markers[self.curmark] # Alias for backwards compatibility RotatingMarker = AnimatedMarker class Counter(Widget): """Displays the current count.""" __slots__ = ('format_string',) def __init__(self, format='%d'): self.format_string = format def update(self, pbar): return self.format_string % pbar.currval class Percentage(Widget): """Displays the current percentage as a number with a percent sign.""" def update(self, pbar): return '%3d%%' % pbar.percentage() class FormatLabel(Timer): """Displays a formatted label.""" mapping = { 'elapsed': ('seconds_elapsed', Timer.format_time), 'finished': ('finished', None), 'last_update': ('last_update_time', None), 'max': ('maxval', None), 'seconds': ('seconds_elapsed', None), 'start': ('start_time', None), 'value': ('currval', None) } __slots__ = ('format_string',) def __init__(self, format): self.format_string = format def update(self, pbar): context = {} for name, (key, transform) in self.mapping.items(): try: value = getattr(pbar, key) if transform is None: context[name] = value else: context[name] = transform(value) except: pass return self.format_string % context class SimpleProgress(Widget): """Returns progress as a count of the total (e.g.: "5 of 47").""" __slots__ = ('sep',) def __init__(self, sep=' of '): self.sep = sep def update(self, pbar): return '%d%s%d' % (pbar.currval, self.sep, pbar.maxval) class Bar(WidgetHFill): """A progress bar which stretches to fill the line.""" __slots__ = ('marker', 'left', 'right', 'fill', 'fill_left') def __init__(self, marker='#', left='|', right='|', fill=' ', fill_left=True): """Creates a customizable progress bar. marker - string or updatable object to use as a marker left - string or updatable object to use as a left border right - string or updatable object to use as a right border fill - character to use for the empty part of the progress bar fill_left - whether to fill from the left or the right """ self.marker = marker self.left = left self.right = right self.fill = fill self.fill_left = fill_left def update(self, pbar, width): """Updates the progress bar and its subcomponents.""" left, marked, right = (format_updatable(i, pbar) for i in (self.left, self.marker, self.right)) width -= len(left) + len(right) # Marked must *always* have length of 1 if pbar.maxval: marked *= int(pbar.currval / pbar.maxval * width) else: marked = '' if self.fill_left: return '%s%s%s' % (left, marked.ljust(width, self.fill), right) else: return '%s%s%s' % (left, marked.rjust(width, self.fill), right) class ReverseBar(Bar): """A bar which has a marker which bounces from side to side.""" def __init__(self, marker='#', left='|', right='|', fill=' ', fill_left=False): """Creates a customizable progress bar. marker - string or updatable object to use as a marker left - string or updatable object to use as a left border right - string or updatable object to use as a right border fill - character to use for the empty part of the progress bar fill_left - whether to fill from the left or the right """ self.marker = marker self.left = left self.right = right self.fill = fill self.fill_left = fill_left class BouncingBar(Bar): def update(self, pbar, width): """Updates the progress bar and its subcomponents.""" left, marker, right = (format_updatable(i, pbar) for i in (self.left, self.marker, self.right)) width -= len(left) + len(right) if pbar.finished: return '%s%s%s' % (left, width * marker, right) position = int(pbar.currval % (width * 2 - 1)) if position > width: position = width * 2 - position lpad = self.fill * (position - 1) rpad = self.fill * (width - len(marker) - len(lpad)) # Swap if we want to bounce the other way if not self.fill_left: rpad, lpad = lpad, rpad return '%s%s%s%s%s' % (left, lpad, marker, rpad, right) tqdm-4.11.2/examples/progressbar/compat.pyc0000664000175000017500000000137712716102162021475 0ustar cr45hincr45hin00000000000000 Vc@sTdZyeWnek r*dZnXyeWnek rOdZnXdS(s=Compatibility methods and classes for the progressbar module.cCs-y|jSWntk r(|jSXdS(N(t__next__tAttributeErrortnext(titer((s6/home/cr45hin/gits/tqdm/examples/progressbar/compat.pyRs cCsx|D]}|rtSqWtS(N(tTruetFalse(titeratortitem((s6/home/cr45hin/gits/tqdm/examples/progressbar/compat.pytany*s N(t__doc__Rt NameErrorR(((s6/home/cr45hin/gits/tqdm/examples/progressbar/compat.pyts   tqdm-4.11.2/examples/progressbar/__init__.pyc0000664000175000017500000000240212716102162021737 0ustar cr45hincr45hin00000000000000 Vc@s@dZdZdZdZdZddlTddlTddlTdS(sMText progress bar library for Python. A text progress bar is typically used to display the progress of a long running operation, providing a visual cue that processing is underway. The ProgressBar class manages the current progress, and the format of the line is given by a number of widgets. A widget is an object that may display differently depending on the state of the progress bar. There are three types of widgets: - a string, which always shows itself - a ProgressBarWidget, which may return a different value every time its update method is called - a ProgressBarWidgetHFill, which is like ProgressBarWidget, except it expands to fill the remaining width of the line. The progressbar module is very easy to use, yet very powerful. It will also automatically enable features like auto-resizing when the system supports it. sNilton Volpatos$first-name dot last-name @ gmail.coms 2011-05-14s2.3i(t*N(t__doc__t __author__t__author_email__t__date__t __version__tcompattwidgetst progressbar(((s8/home/cr45hin/gits/tqdm/examples/progressbar/__init__.pyt(s  tqdm-4.11.2/examples/progressbar/progressbar.py0000664000175000017500000002233012640332257022376 0ustar cr45hincr45hin00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- # # progressbar - Text progress bar library for Python. # Copyright (c) 2005 Nilton Volpato # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public # License as published by the Free Software Foundation; either # version 2.1 of the License, or (at your option) any later version. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA """Main ProgressBar class.""" from __future__ import division import math import os import signal import sys import time try: from fcntl import ioctl from array import array import termios except ImportError: pass from compat import * # for: any, next import widgets class UnknownLength: pass class ProgressBar(object): """The ProgressBar class which updates and prints the bar. A common way of using it is like: >>> pbar = ProgressBar().start() >>> for i in range(100): ... # do something ... pbar.update(i+1) ... >>> pbar.finish() You can also use a ProgressBar as an iterator: >>> progress = ProgressBar() >>> for i in progress(some_iterable): ... # do something ... Since the progress bar is incredibly customizable you can specify different widgets of any type in any order. You can even write your own widgets! However, since there are already a good number of widgets you should probably play around with them before moving on to create your own widgets. The term_width parameter represents the current terminal width. If the parameter is set to an integer then the progress bar will use that, otherwise it will attempt to determine the terminal width falling back to 80 columns if the width cannot be determined. When implementing a widget's update method you are passed a reference to the current progress bar. As a result, you have access to the ProgressBar's methods and attributes. Although there is nothing preventing you from changing the ProgressBar you should treat it as read only. Useful methods and attributes include (Public API): - currval: current progress (0 <= currval <= maxval) - maxval: maximum (and final) value - finished: True if the bar has finished (reached 100%) - start_time: the time when start() method of ProgressBar was called - seconds_elapsed: seconds elapsed since start_time and last call to update - percentage(): progress in percent [0..100] """ __slots__ = ('currval', 'fd', 'finished', 'last_update_time', 'left_justify', 'maxval', 'next_update', 'num_intervals', 'poll', 'seconds_elapsed', 'signal_set', 'start_time', 'term_width', 'update_interval', 'widgets', '_time_sensitive', '__iterable') _DEFAULT_MAXVAL = 100 _DEFAULT_TERMSIZE = 80 _DEFAULT_WIDGETS = [widgets.Percentage(), ' ', widgets.Bar()] def __init__(self, maxval=None, widgets=None, term_width=None, poll=1, left_justify=True, fd=sys.stderr): """Initializes a progress bar with sane defaults.""" # Don't share a reference with any other progress bars if widgets is None: widgets = list(self._DEFAULT_WIDGETS) self.maxval = maxval self.widgets = widgets self.fd = fd self.left_justify = left_justify self.signal_set = False if term_width is not None: self.term_width = term_width else: try: self._handle_resize() signal.signal(signal.SIGWINCH, self._handle_resize) self.signal_set = True except (SystemExit, KeyboardInterrupt): raise except: self.term_width = self._env_size() self.__iterable = None self._update_widgets() self.currval = 0 self.finished = False self.last_update_time = None self.poll = poll self.seconds_elapsed = 0 self.start_time = None self.update_interval = 1 self.next_update = 0 def __call__(self, iterable): """Use a ProgressBar to iterate through an iterable.""" try: self.maxval = len(iterable) except: if self.maxval is None: self.maxval = UnknownLength self.__iterable = iter(iterable) return self def __iter__(self): return self def __next__(self): try: value = next(self.__iterable) if self.start_time is None: self.start() else: self.update(self.currval + 1) return value except StopIteration: if self.start_time is None: self.start() self.finish() raise # Create an alias so that Python 2.x won't complain about not being # an iterator. next = __next__ def _env_size(self): """Tries to find the term_width from the environment.""" return int(os.environ.get('COLUMNS', self._DEFAULT_TERMSIZE)) - 1 def _handle_resize(self, signum=None, frame=None): """Tries to catch resize signals sent from the terminal.""" h, w = array('h', ioctl(self.fd, termios.TIOCGWINSZ, '\0' * 8))[:2] self.term_width = w def percentage(self): """Returns the progress as a percentage.""" if self.currval >= self.maxval: return 100.0 return self.currval * 100.0 / self.maxval percent = property(percentage) def _format_widgets(self): result = [] expanding = [] width = self.term_width for index, widget in enumerate(self.widgets): if isinstance(widget, widgets.WidgetHFill): result.append(widget) expanding.insert(0, index) else: widget = widgets.format_updatable(widget, self) result.append(widget) width -= len(widget) count = len(expanding) while count: portion = max(int(math.ceil(width * 1. / count)), 0) index = expanding.pop() count -= 1 widget = result[index].update(self, portion) width -= len(widget) result[index] = widget return result def _format_line(self): """Joins the widgets and justifies the line.""" widgets = ''.join(self._format_widgets()) if self.left_justify: return widgets.ljust(self.term_width) else: return widgets.rjust(self.term_width) def _need_update(self): """Returns whether the ProgressBar should redraw the line.""" if self.currval >= self.next_update or self.finished: return True delta = time.time() - self.last_update_time return self._time_sensitive and delta > self.poll def _update_widgets(self): """Checks all widgets for the time sensitive bit.""" self._time_sensitive = any(getattr(w, 'TIME_SENSITIVE', False) for w in self.widgets) def update(self, value=None): """Updates the ProgressBar to a new value.""" if value is not None and value is not UnknownLength: if (self.maxval is not UnknownLength and not 0 <= value <= self.maxval): raise ValueError('Value out of range') self.currval = value if not self._need_update(): return if self.start_time is None: raise RuntimeError('You must call "start" before calling "update"') now = time.time() self.seconds_elapsed = now - self.start_time self.next_update = self.currval + self.update_interval self.fd.write(self._format_line() + '\r') self.last_update_time = now def start(self): """Starts measuring time, and prints the bar at 0%. It returns self so you can use it like this: >>> pbar = ProgressBar().start() >>> for i in range(100): ... # do something ... pbar.update(i+1) ... >>> pbar.finish() """ if self.maxval is None: self.maxval = self._DEFAULT_MAXVAL self.num_intervals = max(100, self.term_width) self.next_update = 0 if self.maxval is not UnknownLength: if self.maxval < 0: raise ValueError('Value out of range') self.update_interval = self.maxval / self.num_intervals self.start_time = self.last_update_time = time.time() self.update(0) return self def finish(self): """Puts the ProgressBar bar in the finished state.""" if self.finished: return self.finished = True self.update(self.maxval) self.fd.write('\n') if self.signal_set: signal.signal(signal.SIGWINCH, signal.SIG_DFL) tqdm-4.11.2/examples/progressbar/widgets.pyc0000664000175000017500000003464012716102162021657 0ustar cr45hincr45hin00000000000000 Vc@ sdZddlmZddlZddlZyddlmZmZWn ek rje Z dZnXede fiZ dZ de fd YZ d e fd YZ d e fd YZdefdYZdefdYZde fdYZde fdYZeZde fdYZde fdYZdefdYZde fdYZde fdYZd efd!YZd"efd#YZdS($sDefault ProgressBar widgets.i(tdivisionN(tABCMetatabstractmethodcC s|S(N((tfn((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyt stAbstractWidgetcC s$t|dr|j|S|SdS(Ntupdate(thasattrR(t updatabletpbar((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pytformat_updatable%s tWidgetcB s)eZdZeZdZedZRS(sThe base class for all widgets. The ProgressBar will call the widget's update value when the widget should be updated. The widget's size may change between calls, but the widget may display incorrectly if the size changes drastically and repeatedly. The boolean TIME_SENSITIVE informs the ProgressBar that it should be updated more often because it is time sensitive. cC sdS(sSUpdates the widget. pbar - a reference to the calling ProgressBar N((tselfR ((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR8s((t__name__t __module__t__doc__tFalsetTIME_SENSITIVEt __slots__RR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR *s t WidgetHFillcB seZdZedZRS(s The base class for all variable width widgets. This widget is much like the \hfill command in TeX, it will expand to fill the line. You can use more than one in the same line, and they will all have the same width, and together will fill the line. cC sdS(sUpdates the widget providing the total width the widget must fill. pbar - a reference to the calling ProgressBar width - The total width the widget must fill N((R R twidth((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRHs(R RRRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR@stTimercB s>eZdZdZeZddZedZdZ RS(s*Widget which displays the elapsed seconds.t format_stringsElapsed Time: %scC s ||_dS(N(R(R tformat((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyt__init__WscC sttjdt|S(s&Formats time as the string "HH:MM:SS".tseconds(tstrtdatetimet timedeltatint(R((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyt format_timeZscC s|j|j|jS(s,Updates the widget to show the elapsed time.(RRtseconds_elapsed(R R ((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRas(s format_string( R RRRtTrueRRt staticmethodRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRQs  tETAcB seZdZeZdZRS(s6Widget which attempts to estimate the time of arrival.cC sf|jdkrdS|jr0d|j|jS|j}||j|j|}d|j|SdS(s?Updates the widget to show the ETA or total time when finished.isETA: --:--:--sTime: %ssETA: %sN(tcurrvaltfinishedRRtmaxval(R R telapsedteta((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRls  (R RRR RR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR"gst AdaptiveETAcB s5eZdZeZdZdZdZdZRS(sWidget which attempts to estimate the time of arrival. Uses a weighted average of two estimates: 1) ETA based on the total progress and time elapsed so far 2) ETA based on the progress as per tha last 10 update reports The weight depends on the current progress so that to begin with the total progress is used and at the end only the most recent progress is used. i cC sU||f}t|ds5|g|jd|_n|jj||jjdS(Ntsamplesii(Rt NUM_SAMPLESR)tappendtpop(R R#R&tsample((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyt_update_sampless  cC s||t||S(N(tfloat(R R%R#R&((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyt_etascC s|jdkrdS|jr0d|j|jS|j}|j|j|\}}|j|j|j|}|j|kr|j|j||j|||}|jt|jd}d||||}nd|j|SdS(s?Updates the widget to show the ETA or total time when finished.isETA: --:--:--sTime: %sg?isETA: %sN(R#R$RRR.R0R%R/(R R R&tcurrval1telapsed1R'tetasamptweight((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs    ( R RRR RR*R.R0R(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR(ys    tFileTransferSpeedcB s5eZdZdZdZdZddZdZRS(sBWidget for showing the transfer speed (useful for file transfers).s %6.2f %s%s/ss kMGTPEZYtunittBcC s ||_dS(N(R6(R R6((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRscC s|jdks|jdkr+d}}n6|j|j}ttj|d}|d|}|j||j||jfS(s6Updates the widget with the current SI prefixed speed.g>iig@@(RR#RtmathtlogtFORMATtPREFIXESR6(R R tscaledtpowertspeed((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs  (sunit(R RRR:R;RRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR5s  tAnimatedMarkercB s)eZdZdZddZdZRS(saAn animated marker for the progress bar which defaults to appear as if it were rotating. tmarkerstcurmarks|/-\cC s||_d|_dS(Ni(R@RA(R R@((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs cC s?|jr|jdS|jdt|j|_|j|jS(sTUpdates the widget to show the next marker or the first marker when finishedii(R$R@RAtlen(R R ((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs (smarkersscurmark(R RRRRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR?s tCountercB s)eZdZdZddZdZRS(sDisplays the current count.Rs%dcC s ||_dS(N(R(R R((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRscC s|j|jS(N(RR#(R R ((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs(s format_string(R RRRRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRCs t PercentagecB seZdZdZRS(s@Displays the current percentage as a number with a percent sign.cC sd|jS(Ns%3d%%(t percentage(R R ((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRs(R RRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRDst FormatLabelcB sfeZdZidejfd6dd6dd6dd6dd6dd 6dd 6ZdZdZdZ RS(sDisplays a formatted label.RR&R$tlast_update_timet last_updateR%tmaxRt start_timetstartR#tvalueRcC s ||_dS(N(R(R R((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRscC s}i}xi|jjD]X\}\}}y<t||}|dkrS|||-sts%s%s%sN( RYRXRZRBR%RR#R\tljustR[trjust(R R RRYtmarkedRZ((R s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR*s % !  (smarkersleftsrightsfills fill_left(R RRRR RR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRWs   t ReverseBarcB s&eZdZddddedZRS(s9A bar which has a marker which bounces from side to side.R]R^R_cC s1||_||_||_||_||_dS(syCreates a customizable progress bar. marker - string or updatable object to use as a marker left - string or updatable object to use as a left border right - string or updatable object to use as a right border fill - character to use for the empty part of the progress bar fill_left - whether to fill from the left or the right N(RXRYRZR[R\(R RXRYRZR[R\((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyR@s    (R RRRR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRf=s t BouncingBarcB seZdZRS(c  sfd|j|j|jfD\}}}|t|t|8}jrid||||fStj|dd}||kr|d|}n|j|d}|j|t|t|}|js||}}nd|||||fS(s/Updates the progress bar and its subcomponents.c3 s|]}t|VqdS(N(R (R`Ra(R (s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pys Uss%s%s%siis %s%s%s%s%s( RYRXRZRBR$RR#R[R\( R R RRYRXRZtpositiontlpadtrpad((R s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRRs %  ! (R RR(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyRgQs(Rt __future__RRR8tabcRRt ImportErrortobjectRR R RRR"R(R5R?tRotatingMarkerRCRDRFRURWRfRg(((s7/home/cr45hin/gits/tqdm/examples/progressbar/widgets.pyts2     -   )tqdm-4.11.2/examples/progressbar/__init__.py0000664000175000017500000000357512640332257021616 0ustar cr45hincr45hin00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- # # progressbar - Text progress bar library for Python. # Copyright (c) 2005 Nilton Volpato # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public # License as published by the Free Software Foundation; either # version 2.1 of the License, or (at your option) any later version. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA """Text progress bar library for Python. A text progress bar is typically used to display the progress of a long running operation, providing a visual cue that processing is underway. The ProgressBar class manages the current progress, and the format of the line is given by a number of widgets. A widget is an object that may display differently depending on the state of the progress bar. There are three types of widgets: - a string, which always shows itself - a ProgressBarWidget, which may return a different value every time its update method is called - a ProgressBarWidgetHFill, which is like ProgressBarWidget, except it expands to fill the remaining width of the line. The progressbar module is very easy to use, yet very powerful. It will also automatically enable features like auto-resizing when the system supports it. """ __author__ = 'Nilton Volpato' __author_email__ = 'first-name dot last-name @ gmail.com' __date__ = '2011-05-14' __version__ = '2.3' from compat import * from widgets import * from progressbar import * tqdm-4.11.2/examples/progressbar/progressbar.pyc0000664000175000017500000002321612716102162022537 0ustar cr45hincr45hin00000000000000 Vc@ sdZddlmZddlZddlZddlZddlZddlZy0ddlm Z ddl m Z ddl Z Wne k rnXddl TddlZdd dYZd efd YZdS( sMain ProgressBar class.i(tdivisionN(tioctl(tarray(t*t UnknownLengthcB seZRS((t__name__t __module__(((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR*st ProgressBarcB seZdZd$ZdZdZejdejgZ d%d%d%de e j dZdZdZdZeZdZd%d%dZdZeeZdZdZdZd Zd%d!Zd"Zd#ZRS(&sThe ProgressBar class which updates and prints the bar. A common way of using it is like: >>> pbar = ProgressBar().start() >>> for i in range(100): ... # do something ... pbar.update(i+1) ... >>> pbar.finish() You can also use a ProgressBar as an iterator: >>> progress = ProgressBar() >>> for i in progress(some_iterable): ... # do something ... Since the progress bar is incredibly customizable you can specify different widgets of any type in any order. You can even write your own widgets! However, since there are already a good number of widgets you should probably play around with them before moving on to create your own widgets. The term_width parameter represents the current terminal width. If the parameter is set to an integer then the progress bar will use that, otherwise it will attempt to determine the terminal width falling back to 80 columns if the width cannot be determined. When implementing a widget's update method you are passed a reference to the current progress bar. As a result, you have access to the ProgressBar's methods and attributes. Although there is nothing preventing you from changing the ProgressBar you should treat it as read only. Useful methods and attributes include (Public API): - currval: current progress (0 <= currval <= maxval) - maxval: maximum (and final) value - finished: True if the bar has finished (reached 100%) - start_time: the time when start() method of ProgressBar was called - seconds_elapsed: seconds elapsed since start_time and last call to update - percentage(): progress in percent [0..100] tcurrvaltfdtfinishedtlast_update_timet left_justifytmaxvalt next_updatet num_intervalstpolltseconds_elapsedt signal_sett start_timet term_widthtupdate_intervaltwidgetst_time_sensitivet __iterableidiPt icC s!|dkrt|j}n||_||_||_||_t|_|dk rc||_ n_y-|j t j t j |j t |_Wn/ttfk rn|j|_ nXd|_|jd|_t|_d|_||_d|_d|_d|_d|_dS(s.Initializes a progress bar with sane defaults.iiN(tNonetlistt_DEFAULT_WIDGETSR RR R tFalseRRt_handle_resizetsignaltSIGWINCHtTruet SystemExittKeyboardInterruptt _env_sizet_ProgressBar__iterablet_update_widgetsRR R RRRRR(tselfR RRRR R ((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyt__init__bs6                   cC sKyt||_Wn"|jdkr8t|_q8nXt||_|S(s1Use a ProgressBar to iterate through an iterable.N(tlenR RRtiterR%(R'titerable((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyt__call__scC s|S(N((R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyt__iter__scC syGt|j}|jdkr.|jn|j|jd|SWn:tk r|jdkrs|jn|jnXdS(Ni( tnextR%RRtstarttupdateRt StopIterationtfinish(R'tvalue((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyt__next__s    cC s ttjjd|jdS(s2Tries to find the term_width from the environment.tCOLUMNSi(tinttostenvirontgett_DEFAULT_TERMSIZE(R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR$scC s8tdt|jtjdd \}}||_dS(s5Tries to catch resize signals sent from the terminal.thtiiNt(RRR ttermiost TIOCGWINSZR(R'tsignumtframeR;tw((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyRs+cC s(|j|jkrdS|jd|jS(s%Returns the progress as a percentage.gY@(RR (R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyt percentagescC sg}g}|j}x~t|jD]m\}}t|tjrc|j||jd|q%tj||}|j||t|8}q%Wt|}xv|rt t t j |d|d}|j }|d8}||j||}|t|8}|||sN(tanyRR(R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR&s cC s|dk rb|tk rb|jtk rVd|koA|jkn rVtdn||_n|jsrdS|jdkrtdntj}||j|_ |j|j |_ |j j |jd||_dS(s'Updates the ProgressBar to a new value.isValue out of rangeNs-You must call "start" before calling "update"s (RRR t ValueErrorRR]Rt RuntimeErrorR[RRRR twriteRZR (R'R3tnow((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR0s    cC s|jdkr|j|_ntd|j|_d|_|jtk r|jdkritdn|j|j|_ nt j |_ |_ |j d|S(sStarts measuring time, and prints the bar at 0%. It returns self so you can use it like this: >>> pbar = ProgressBar().start() >>> for i in range(100): ... # do something ... pbar.update(i+1) ... >>> pbar.finish() idisValue out of rangeN(R Rt_DEFAULT_MAXVALRJRRRRRbRR[RR R0(R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR/ s   cC s\|jr dSt|_|j|j|jjd|jrXtjtjtj ndS(s/Puts the ProgressBar bar in the finished state.Ns ( R R!R0R R RdRRR tSIG_DFL(R'((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR2)s   (scurrvalsfdsfinishedslast_update_times left_justifysmaxvals next_updates num_intervalsspollsseconds_elapseds signal_sets start_times term_widthsupdate_intervalswidgetss_time_sensitives __iterableN(RRt__doc__t __slots__RfR:Rt PercentagetBarRRR!tsyststderrR(R,R-R4R.R$RRCtpropertytpercentRURZR]R&R0R/R2(((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyR-s4) $          ((Rht __future__RRKR7RRlR[tfcntlRRR>t ImportErrortcompatRRtobjectR(((s;/home/cr45hin/gits/tqdm/examples/progressbar/progressbar.pyts         tqdm-4.11.2/examples/progressbar/compat.py0000664000175000017500000000266412640332257021340 0ustar cr45hincr45hin00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- # # progressbar - Text progress bar library for Python. # Copyright (c) 2005 Nilton Volpato # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public # License as published by the Free Software Foundation; either # version 2.1 of the License, or (at your option) any later version. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA """Compatibility methods and classes for the progressbar module.""" # Python 3.x (and backports) use a modified iterator syntax # This will allow 2.x to behave with 3.x iterators try: next except NameError: def next(iter): try: # Try new style iterators return iter.__next__() except AttributeError: # Fallback in case of a "native" iterator return iter.next() # Python < 2.5 does not have "any" try: any except NameError: def any(iterator): for item in iterator: if item: return True return False tqdm-4.11.2/examples/.git0000664000175000017500000000004112755154307015726 0ustar cr45hincr45hin00000000000000gitdir: ../.git/modules/examples tqdm-4.11.2/examples/tqdm_wget.py0000664000175000017500000000362613036011052017501 0ustar cr45hincr45hin00000000000000"""An example of wrapping manual tqdm updates for urllib reporthook. # urllib.urlretrieve documentation > If present, the hook function will be called once > on establishment of the network connection and once after each block read > thereafter. The hook will be passed three arguments; a count of blocks > transferred so far, a block size in bytes, and the total size of the file. Usage: tqdm_wget.py [options] Options: -h, --help Print this help message and exit -u URL, --url URL : string, optional The url to fetch. [default: http://www.doc.ic.ac.uk/~cod11/matryoshka.zip] -o FILE, --output FILE : string, optional The local file path in which to save the url [default: /dev/null]. """ import urllib from tqdm import tqdm from docopt import docopt def my_hook(t): """ Wraps tqdm instance. Don't forget to close() or __exit__() the tqdm instance once you're done with it (easiest using `with` syntax). Example ------- >>> with tqdm(...) as t: ... reporthook = my_hook(t) ... urllib.urlretrieve(..., reporthook=reporthook) """ last_b = [0] def inner(b=1, bsize=1, tsize=None): """ b : int, optional Number of blocks just transferred [default: 1]. bsize : int, optional Size of each block (in tqdm units) [default: 1]. tsize : int, optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: t.total = tsize t.update((b - last_b[0]) * bsize) last_b[0] = b return inner opts = docopt(__doc__) eg_link = opts['--url'] eg_file = eg_link.replace('/', ' ').split()[-1] with tqdm(unit='B', unit_scale=True, leave=True, miniters=1, desc=eg_file) as t: # all optional kwargs urllib.urlretrieve(eg_link, filename=opts['--output'], reporthook=my_hook(t), data=None) tqdm-4.11.2/examples/simple_examples.py0000664000175000017500000000264513040270166020705 0ustar cr45hincr45hin00000000000000""" # Simple tqdm examples and profiling # Benchmark for i in _range(int(1e8)): pass # Basic demo import tqdm for i in tqdm.trange(int(1e8)): pass # Some decorations import tqdm for i in tqdm.trange(int(1e8), miniters=int(1e6), ascii=True, desc="cool", dynamic_ncols=True): pass # Nested bars from tqdm import trange for i in trange(10): for j in trange(int(1e7), leave=False, unit_scale=True): pass # Experimental GUI demo import tqdm for i in tqdm.tgrange(int(1e8)): pass # Comparison to https://code.google.com/p/python-progressbar/ try: from progressbar.progressbar import ProgressBar except ImportError: pass else: for i in ProgressBar()(_range(int(1e8))): pass # Dynamic miniters benchmark from tqdm import trange for i in trange(int(1e8), miniters=None, mininterval=0.1, smoothing=0): pass # Fixed miniters benchmark from tqdm import trange for i in trange(int(1e8), miniters=4500000, mininterval=0.1, smoothing=0): pass """ from time import sleep from timeit import timeit import re # Simple demo from tqdm import trange for i in trange(16, leave=True): sleep(0.1) # Profiling/overhead tests stmts = filter(None, re.split(r'\n\s*#.*?\n', __doc__)) for s in stmts: print(s.replace('import tqdm\n', '')) print(timeit(stmt='try:\n\t_range = xrange' '\nexcept:\n\t_range = range\n' + s, number=1), 'seconds') tqdm-4.11.2/examples/7zx.py0000664000175000017500000001042013041513621016231 0ustar cr45hincr45hin00000000000000# -*- coding: utf-8 -*- """Usage: 7zx.py [--help | options] ... Options: -h, --help Print this help and exit -v, --version Print version and exit -c, --compressed Use compressed (instead of uncompressed) file sizes -s, --silent Do not print one row per zip file -y, --yes Assume yes to all queries (for extraction) -D=, --debug= Print various types of debugging information. Choices: CRITICAL|FATAL ERROR WARN(ING) [default: INFO] DEBUG NOTSET -d, --debug-trace Print lots of debugging information (-D NOTSET) """ from __future__ import print_function from docopt import docopt import logging as log import subprocess import re from tqdm import tqdm import pty import os import io __author__ = "Casper da Costa-Luis " __licence__ = "MPLv2.0" __version__ = "0.2.0" __license__ = __licence__ RE_SCN = re.compile("([0-9]+)\s+([0-9]+)\s+(.*)$", flags=re.M) def main(): args = docopt(__doc__, version=__version__) if args.pop('--debug-trace', False): args['--debug'] = "NOTSET" log.basicConfig(level=getattr(log, args['--debug'], log.INFO), format='%(levelname)s: %(message)s') log.debug(args) # Get compressed sizes zips = {} for fn in args['']: info = subprocess.check_output(["7z", "l", fn]).strip() finfo = RE_SCN.findall(info) # builtin test: last line should be total sizes log.debug(finfo) totals = map(int, finfo[-1][:2]) # log.debug(totals) for s in range(2): assert(sum(map(int, (inf[s] for inf in finfo[:-1]))) == totals[s]) fcomp = dict((n, int(c if args['--compressed'] else u)) for (u, c, n) in finfo[:-1]) # log.debug(fcomp) # zips : {'zipname' : {'filename' : int(size)}} zips[fn] = fcomp # Extract cmd7zx = ["7z", "x", "-bd"] if args['--yes']: cmd7zx += ["-y"] log.info("Extracting from {:d} file(s)".format(len(zips))) with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()), unit="B", unit_scale=True) as tall: for fn, fcomp in zips.items(): md, sd = pty.openpty() ex = subprocess.Popen(cmd7zx + [fn], bufsize=1, stdout=md, # subprocess.PIPE, stderr=subprocess.STDOUT) os.close(sd) with io.open(md, mode="rU", buffering=1) as m: with tqdm(total=sum(fcomp.values()), disable=len(zips) < 2, leave=False, unit="B", unit_scale=True) as t: while True: try: l_raw = m.readline() except IOError: break l = l_raw.strip() if l.startswith("Extracting"): exname = l.lstrip("Extracting").lstrip() s = fcomp.get(exname, 0) # 0 is likely folders t.update(s) tall.update(s) elif l: if not any(l.startswith(i) for i in ("7-Zip ", "p7zip Version ", "Everything is Ok", "Folders: ", "Files: ", "Size: ", "Compressed: ")): if l.startswith("Processing archive: "): if not args['--silent']: t.write(t.format_interval( t.start_t - tall.start_t) + ' ' + l.lstrip("Processing archive: ")) else: t.write(l) ex.wait() main.__doc__ = __doc__ if __name__ == "__main__": main() tqdm-4.11.2/examples/include_no_requirements.py0000664000175000017500000000034413040270166022432 0ustar cr45hincr45hin00000000000000# How to import tqdm without enforcing it as a dependency try: from tqdm import tqdm except ImportError: def tqdm(*args, **kwargs): if args: return args[0] return kwargs.get('iterable', None) tqdm-4.11.2/tqdm/0000775000175000017500000000000013041514547014271 5ustar cr45hincr45hin00000000000000tqdm-4.11.2/tqdm/_tqdm.py0000664000175000017500000012435413041513621015750 0ustar cr45hincr45hin00000000000000""" Customisable progressbar decorator for iterators. Includes a default (x)range iterator printing to stderr. Usage: >>> from tqdm import trange[, tqdm] >>> for i in trange(10): #same as: for i in tqdm(xrange(10)) ... ... """ from __future__ import absolute_import # integer division / : float, // : int from __future__ import division # compatibility functions and utilities from ._utils import _supports_unicode, _environ_cols_wrapper, _range, _unich, \ _term_move_up, _unicode, WeakSet, _basestring, _OrderedDict # native libraries import sys from numbers import Number from threading import Thread from time import time from time import sleep __author__ = {"github.com/": ["noamraph", "obiwanus", "kmike", "hadim", "casperdcl", "lrq3000"]} __all__ = ['tqdm', 'trange', 'TqdmTypeError', 'TqdmKeyError', 'TqdmDeprecationWarning'] class TqdmTypeError(TypeError): pass class TqdmKeyError(KeyError): pass class TqdmDeprecationWarning(Exception): # not suppressed if raised def __init__(self, msg, fp_write=None, *a, **k): if fp_write is not None: fp_write("\nTqdmDeprecationWarning: " + str(msg).rstrip() + '\n') else: super(TqdmDeprecationWarning, self).__init__(msg, *a, **k) class TMonitor(Thread): """ Monitoring thread for tqdm bars. Monitors if tqdm bars are taking too much time to display and readjusts miniters automatically if necessary. Parameters ---------- tqdm_cls : class tqdm class to use (can be core tqdm or a submodule). sleep_interval : fload Time to sleep between monitoring checks. """ # internal vars for unit testing _time = None _sleep = None def __init__(self, tqdm_cls, sleep_interval): # setcheckinterval is deprecated getattr(sys, 'setswitchinterval', getattr(sys, 'setcheckinterval'))(100) Thread.__init__(self) self.daemon = True # kill thread when main killed (KeyboardInterrupt) self.was_killed = False self.woken = 0 # last time woken up, to sync with monitor self.tqdm_cls = tqdm_cls self.sleep_interval = sleep_interval if TMonitor._time is not None: self._time = TMonitor._time else: self._time = time if TMonitor._sleep is not None: self._sleep = TMonitor._sleep else: self._sleep = sleep self.start() def exit(self): self.was_killed = True # self.join() # DO NOT, blocking event, slows down tqdm at closing return self.report() def run(self): cur_t = self._time() while True: # After processing and before sleeping, notify that we woke # Need to be done just before sleeping self.woken = cur_t # Sleep some time... self._sleep(self.sleep_interval) # Quit if killed # if self.exit_event.is_set(): # TODO: should work but does not... if self.was_killed: return # Then monitor! cur_t = self._time() # Check for each tqdm instance if one is waiting too long to print # NB: copy avoids size change during iteration RuntimeError for instance in self.tqdm_cls._instances.copy(): # Only if mininterval > 1 (else iterations are just slow) # and last refresh was longer than maxinterval in this instance if instance.miniters > 1 and \ (cur_t - instance.last_print_t) >= instance.maxinterval: # We force bypassing miniters on next iteration # dynamic_miniters should adjust mininterval automatically instance.miniters = 1 # Refresh now! (works only for manual tqdm) instance.refresh() def report(self): # return self.is_alive() # TODO: does not work... return not self.was_killed class tqdm(object): """ Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested. """ monitor_interval = 10 # set to 0 to disable the thread monitor = None @staticmethod def format_sizeof(num, suffix=''): """ Formats a number (greater than unity) with SI Order of Magnitude prefixes. Parameters ---------- num : float Number ( >= 1) to format. suffix : str, optional Post-postfix [default: '']. Returns ------- out : str Number with Order of Magnitude SI unit postfix. """ for unit in ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z']: if abs(num) < 999.95: if abs(num) < 99.95: if abs(num) < 9.995: return '{0:1.2f}'.format(num) + unit + suffix return '{0:2.1f}'.format(num) + unit + suffix return '{0:3.0f}'.format(num) + unit + suffix num /= 1000.0 return '{0:3.1f}Y'.format(num) + suffix @staticmethod def format_interval(t): """ Formats a number of seconds as a clock time, [H:]MM:SS Parameters ---------- t : int Number of seconds. Returns ------- out : str [H:]MM:SS """ mins, s = divmod(int(t), 60) h, m = divmod(mins, 60) if h: return '{0:d}:{1:02d}:{2:02d}'.format(h, m, s) else: return '{0:02d}:{1:02d}'.format(m, s) @staticmethod def status_printer(file): """ Manage the printing and in-place updating of a line of characters. Note that if the string is longer than a line, then in-place updating may not work (it will print a new line at each refresh). """ fp = file fp_flush = getattr(fp, 'flush', lambda: None) # pragma: no cover def fp_write(s): fp.write(_unicode(s)) fp_flush() last_len = [0] def print_status(s): len_s = len(s) fp_write('\r' + s + (' ' * max(last_len[0] - len_s, 0))) last_len[0] = len_s return print_status @staticmethod def format_meter(n, total, elapsed, ncols=None, prefix='', ascii=False, unit='it', unit_scale=False, rate=None, bar_format=None, postfix=None): """ Return a string-based progress bar given some parameters Parameters ---------- n : int Number of finished iterations. total : int The expected total number of iterations. If meaningless (), only basic progress statistics are displayed (no ETA). elapsed : float Number of seconds passed since start. ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progress meter to stay within this bound [default: None]. The fallback meter width is 10 for the progress bar + no limit for the iterations counter and statistics. If 0, will not print any meter (only stats). prefix : str, optional Prefix message (included in total width) [default: '']. ascii : bool, optional If not set, use unicode (smooth blocks) to fill the meter [default: False]. The fallback is to use ASCII characters (1-9 #). unit : str, optional The iteration unit [default: 'it']. unit_scale : bool, optional If set, the number of iterations will printed with an appropriate SI metric prefix (K = 10^3, M = 10^6, etc.) [default: False]. rate : float, optional Manual override for iteration rate. If [default: None], uses n/elapsed. bar_format : str, optional Specify a custom bar string formatting. May impact performance. [default: '{l_bar}{bar}{r_bar}'], where l_bar is '{desc}{percentage:3.0f}%|' and r_bar is '| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. postfix : str, optional Same as prefix but will be placed at the end as additional stats. Returns ------- out : Formatted meter and stats, ready to display. """ # sanity check: total if total and n > total: total = None format_interval = tqdm.format_interval elapsed_str = format_interval(elapsed) # if unspecified, attempt to use rate = average speed # (we allow manual override since predicting time is an arcane art) if rate is None and elapsed: rate = n / elapsed inv_rate = 1 / rate if (rate and (rate < 1)) else None format_sizeof = tqdm.format_sizeof rate_fmt = ((format_sizeof(inv_rate if inv_rate else rate) if unit_scale else '{0:5.2f}'.format(inv_rate if inv_rate else rate)) if rate else '?') \ + ('s' if inv_rate else unit) + '/' + (unit if inv_rate else 's') if unit_scale: n_fmt = format_sizeof(n) total_fmt = format_sizeof(total) if total else None else: n_fmt = str(n) total_fmt = str(total) # total is known: we can predict some stats if total: # fractional and percentage progress frac = n / total percentage = frac * 100 remaining_str = format_interval((total - n) / rate) \ if rate else '?' # format the stats displayed to the left and right sides of the bar l_bar = (prefix if prefix else '') + \ '{0:3.0f}%|'.format(percentage) r_bar = '| {0}/{1} [{2}<{3}, {4}{5}]'.format( n_fmt, total_fmt, elapsed_str, remaining_str, rate_fmt, ', '+postfix if postfix else '') if ncols == 0: return l_bar[:-1] + r_bar[1:] if bar_format: # Custom bar formatting # Populate a dict with all available progress indicators bar_args = {'n': n, 'n_fmt': n_fmt, 'total': total, 'total_fmt': total_fmt, 'percentage': percentage, 'rate': rate if inv_rate is None else inv_rate, 'rate_noinv': rate, 'rate_noinv_fmt': ((format_sizeof(rate) if unit_scale else '{0:5.2f}'.format(rate)) if rate else '?') + unit + '/s', 'rate_fmt': rate_fmt, 'elapsed': elapsed_str, 'remaining': remaining_str, 'l_bar': l_bar, 'r_bar': r_bar, 'desc': prefix if prefix else '', 'postfix': ', '+postfix if postfix else '', # 'bar': full_bar # replaced by procedure below } # Interpolate supplied bar format with the dict if '{bar}' in bar_format: # Format left/right sides of the bar, and format the bar # later in the remaining space (avoid breaking display) l_bar_user, r_bar_user = bar_format.split('{bar}') l_bar = l_bar_user.format(**bar_args) r_bar = r_bar_user.format(**bar_args) else: # Else no progress bar, we can just format and return return bar_format.format(**bar_args) # Formatting progress bar # space available for bar's display N_BARS = max(1, ncols - len(l_bar) - len(r_bar)) if ncols \ else 10 # format bar depending on availability of unicode/ascii chars if ascii: bar_length, frac_bar_length = divmod( int(frac * N_BARS * 10), 10) bar = '#' * bar_length frac_bar = chr(48 + frac_bar_length) if frac_bar_length \ else ' ' else: bar_length, frac_bar_length = divmod(int(frac * N_BARS * 8), 8) bar = _unich(0x2588) * bar_length frac_bar = _unich(0x2590 - frac_bar_length) \ if frac_bar_length else ' ' # whitespace padding if bar_length < N_BARS: full_bar = bar + frac_bar + \ ' ' * max(N_BARS - bar_length - 1, 0) else: full_bar = bar + \ ' ' * max(N_BARS - bar_length, 0) # Piece together the bar parts return l_bar + full_bar + r_bar # no total: no progressbar, ETA, just progress stats else: return (prefix if prefix else '') + '{0}{1} [{2}, {3}{4}]'.format( n_fmt, unit, elapsed_str, rate_fmt, ', '+postfix if postfix else '') def __new__(cls, *args, **kwargs): # Create a new instance instance = object.__new__(cls) # Add to the list of instances if "_instances" not in cls.__dict__: cls._instances = WeakSet() cls._instances.add(instance) # Create the monitoring thread if cls.monitor_interval and (cls.monitor is None or not cls.monitor.report()): cls.monitor = TMonitor(cls, cls.monitor_interval) # Return the instance return instance @classmethod def _get_free_pos(cls, instance=None): """ Skips specified instance """ try: return max(inst.pos for inst in cls._instances if inst is not instance) + 1 except ValueError as e: if "arg is an empty sequence" in str(e): return 0 raise # pragma: no cover @classmethod def _decr_instances(cls, instance): """ Remove from list and reposition other bars so that newer bars won't overlap previous bars """ try: # in case instance was explicitly positioned, it won't be in set cls._instances.remove(instance) for inst in cls._instances: if inst.pos > instance.pos: inst.pos -= 1 # Kill monitor if no instances are left if not cls._instances and cls.monitor: cls.monitor.exit() del cls.monitor cls.monitor = None except KeyError: pass @classmethod def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm (without overlap with bars) """ fp = file # Clear all bars inst_cleared = [] for inst in getattr(cls, '_instances', []): # Clear instance if in the target output file # or if write output + tqdm output are both either # sys.stdout or sys.stderr (because both are mixed in terminal) if inst.fp == fp or all(f in (sys.stdout, sys.stderr) for f in (fp, inst.fp)): inst.clear() inst_cleared.append(inst) # Write the message fp.write(s) fp.write(end) # Force refresh display of bars we cleared for inst in inst_cleared: # Avoid race conditions by checking that the instance started if hasattr(inst, 'start_t'): # pragma: nocover inst.refresh() # TODO: make list of all instances incl. absolutely positioned ones? @classmethod def pandas(tclass, *targs, **tkwargs): """ Registers the given `tqdm` class with pandas.core. ( frame.DataFrame | series.Series | groupby.DataFrameGroupBy | groupby.SeriesGroupBy ).progress_apply A new instance will be create every time `progress_apply` is called, and each instance will automatically close() upon completion. Parameters ---------- targs, tkwargs : arguments for the tqdm instance Examples -------- >>> import pandas as pd >>> import numpy as np >>> from tqdm import tqdm, tqdm_gui >>> >>> df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) >>> tqdm.pandas(ncols=50) # can use tqdm_gui, optional kwargs, etc >>> # Now you can use `progress_apply` instead of `apply` >>> df.groupby(0).progress_apply(lambda x: x**2) References ---------- https://stackoverflow.com/questions/18603270/ progress-indicator-during-pandas-operations-python """ from pandas.core.frame import DataFrame from pandas.core.series import Series from pandas.core.groupby import DataFrameGroupBy from pandas.core.groupby import SeriesGroupBy from pandas.core.groupby import GroupBy from pandas.core.groupby import PanelGroupBy from pandas import Panel deprecated_t = [tkwargs.pop('deprecated_t', None)] def inner_generator(df_function='apply'): def inner(df, func, *args, **kwargs): """ Parameters ---------- df : (DataFrame|Series)[GroupBy] Data (may be grouped). func : function To be applied on the (grouped) data. *args, *kwargs : optional Transmitted to `df.apply()`. """ # Precompute total iterations total = getattr(df, 'ngroups', None) if total is None: # not grouped total = len(df) if isinstance(df, Series) \ else df.size // len(df) else: total += 1 # pandas calls update once too many # Init bar if deprecated_t[0] is not None: t = deprecated_t[0] deprecated_t[0] = None else: t = tclass(*targs, total=total, **tkwargs) # Define bar updating wrapper def wrapper(*args, **kwargs): t.update() return func(*args, **kwargs) # Apply the provided function (in *args and **kwargs) # on the df using our wrapper (which provides bar updating) result = getattr(df, df_function)(wrapper, *args, **kwargs) # Close bar and return pandas calculation result t.close() return result return inner # Monkeypatch pandas to provide easy methods # Enable custom tqdm progress in pandas! Series.progress_apply = inner_generator() SeriesGroupBy.progress_apply = inner_generator() Series.progress_map = inner_generator('map') SeriesGroupBy.progress_map = inner_generator('map') DataFrame.progress_apply = inner_generator() DataFrameGroupBy.progress_apply = inner_generator() DataFrame.progress_applymap = inner_generator('applymap') Panel.progress_apply = inner_generator() PanelGroupBy.progress_apply = inner_generator() GroupBy.progress_apply = inner_generator() GroupBy.progress_aggregate = inner_generator('aggregate') GroupBy.progress_transform = inner_generator('transform') def __init__(self, iterable=None, desc=None, total=None, leave=True, file=sys.stderr, ncols=None, mininterval=0.1, maxinterval=10.0, miniters=None, ascii=None, disable=False, unit='it', unit_scale=False, dynamic_ncols=False, smoothing=0.3, bar_format=None, initial=0, position=None, postfix=None, gui=False, **kwargs): """ Parameters ---------- iterable : iterable, optional Iterable to decorate with a progressbar. Leave blank to manually manage the updates. desc : str, optional Prefix for the progressbar. total : int, optional The number of expected iterations. If unspecified, len(iterable) is used if possible. As a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If `gui` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int(9e9). leave : bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. file : `io.TextIOWrapper` or `io.StringIO`, optional Specifies where to output the progress messages [default: sys.stderr]. Uses `file.write(str)` and `file.flush()` methods. ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). mininterval : float, optional Minimum progress display update interval, in seconds [default: 0.1]. maxinterval : float, optional Maximum progress display update interval, in seconds [default: 10]. Automatically adjusts `miniters` to correspond to `mininterval` after long display update lag. Only works if `dynamic_miniters` or monitor thread is enabled. miniters : int, optional Minimum progress display update interval, in iterations. If 0 and `dynamic_miniters`, will automatically adjust to equal `mininterval` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and `mininterval` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. ascii : bool, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters `1-9 #`. disable : bool or None, optional Whether to disable the entire progressbar wrapper [default: False]. If set to None, disable on non-TTY. unit : str, optional String that will be used to define the unit of each iteration [default: it]. unit_scale : bool, optional If set, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. dynamic_ncols : bool, optional If set, constantly alters `ncols` to the environment (allowing for window resizes) [default: False]. smoothing : float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. bar_format : str, optional Specify a custom bar string formatting. May impact performance. If unspecified, will use '{l_bar}{bar}{r_bar}', where l_bar is '{desc}{percentage:3.0f}%|' and r_bar is '| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. initial : int, optional The initial counter value. Useful when restarting a progress bar [default: 0]. position : int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). postfix : dict, optional Specify additional stats to display at the end of the bar. gui : bool, optional WARNING: internal parameter - do not use. Use tqdm_gui(...) instead. If set, will attempt to use matplotlib animations for a graphical output [default: False]. Returns ------- out : decorated iterator. """ if disable is None and hasattr(file, "isatty") and not file.isatty(): disable = True if disable: self.iterable = iterable self.disable = disable self.pos = self._get_free_pos(self) self._instances.remove(self) return if kwargs: self.disable = True self.pos = self._get_free_pos(self) self._instances.remove(self) raise (TqdmDeprecationWarning("""\ `nested` is deprecated and automated. Use position instead for manual control. """, fp_write=getattr(file, 'write', sys.stderr.write)) if "nested" in kwargs else TqdmKeyError("Unknown argument(s): " + str(kwargs))) # Preprocess the arguments if total is None and iterable is not None: try: total = len(iterable) except (TypeError, AttributeError): total = None if ((ncols is None) and (file in (sys.stderr, sys.stdout))) or \ dynamic_ncols: # pragma: no cover if dynamic_ncols: dynamic_ncols = _environ_cols_wrapper() ncols = dynamic_ncols(file) else: ncols = _environ_cols_wrapper()(file) if miniters is None: miniters = 0 dynamic_miniters = True else: dynamic_miniters = False if mininterval is None: mininterval = 0 if maxinterval is None: maxinterval = 0 if ascii is None: ascii = not _supports_unicode(file) if bar_format and not ascii: # Convert bar format into unicode since terminal uses unicode bar_format = _unicode(bar_format) if smoothing is None: smoothing = 0 # Store the arguments self.iterable = iterable self.desc = desc + ': ' if desc else '' self.total = total self.leave = leave self.fp = file self.ncols = ncols self.mininterval = mininterval self.maxinterval = maxinterval self.miniters = miniters self.dynamic_miniters = dynamic_miniters self.ascii = ascii self.disable = disable self.unit = unit self.unit_scale = unit_scale self.gui = gui self.dynamic_ncols = dynamic_ncols self.smoothing = smoothing self.avg_time = None self._time = time self.bar_format = bar_format self.postfix = None if postfix: self.set_postfix(**postfix) # Init the iterations counters self.last_print_n = initial self.n = initial # if nested, at initial sp() call we replace '\r' by '\n' to # not overwrite the outer progress bar self.pos = self._get_free_pos(self) if position is None else position if not gui: # Initialize the screen printer self.sp = self.status_printer(self.fp) if self.pos: self.moveto(self.pos) self.sp(self.format_meter(self.n, total, 0, (dynamic_ncols(file) if dynamic_ncols else ncols), self.desc, ascii, unit, unit_scale, None, bar_format, self.postfix)) if self.pos: self.moveto(-self.pos) # Init the time counter self.last_print_t = self._time() # NB: Avoid race conditions by setting start_t at the very end of init self.start_t = self.last_print_t def __len__(self): return (self.iterable.shape[0] if hasattr(self.iterable, 'shape') else len(self.iterable)) if self.iterable is not None \ else self.total def __enter__(self): return self def __exit__(self, *exc): self.close() return False def __del__(self): self.close() def __repr__(self): return self.format_meter(self.n, self.total, self._time() - self.start_t, self.ncols, self.desc, self.ascii, self.unit, self.unit_scale, 1 / self.avg_time if self.avg_time else None, self.bar_format, self.postfix) def __lt__(self, other): return self.pos < other.pos def __le__(self, other): return (self < other) or (self == other) def __eq__(self, other): return self.pos == other.pos def __ne__(self, other): return not (self == other) def __gt__(self, other): return not (self <= other) def __ge__(self, other): return not (self < other) def __hash__(self): return id(self) def __iter__(self): ''' Backward-compatibility to use: for x in tqdm(iterable) ''' # Inlining instance variables as locals (speed optimisation) iterable = self.iterable # If the bar is disabled, then just walk the iterable # (note: keep this check outside the loop for performance) if self.disable: for obj in iterable: yield obj else: ncols = self.ncols mininterval = self.mininterval maxinterval = self.maxinterval miniters = self.miniters dynamic_miniters = self.dynamic_miniters unit = self.unit unit_scale = self.unit_scale ascii = self.ascii start_t = self.start_t last_print_t = self.last_print_t last_print_n = self.last_print_n n = self.n dynamic_ncols = self.dynamic_ncols smoothing = self.smoothing avg_time = self.avg_time bar_format = self.bar_format _time = self._time format_meter = self.format_meter try: sp = self.sp except AttributeError: raise TqdmDeprecationWarning("""\ Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)` """, fp_write=getattr(self.fp, 'write', sys.stderr.write)) for obj in iterable: yield obj # Update and print the progressbar. # Note: does not call self.update(1) for speed optimisation. n += 1 # check the counter first (avoid calls to time()) if n - last_print_n >= self.miniters: miniters = self.miniters # watch monitoring thread changes delta_t = _time() - last_print_t if delta_t >= mininterval: cur_t = _time() delta_it = n - last_print_n elapsed = cur_t - start_t # optimised if in inner loop # EMA (not just overall average) if smoothing and delta_t and delta_it: avg_time = delta_t / delta_it \ if avg_time is None \ else smoothing * delta_t / delta_it + \ (1 - smoothing) * avg_time if self.pos: self.moveto(self.pos) # Printing the bar's update sp(format_meter( n, self.total, elapsed, (dynamic_ncols(self.fp) if dynamic_ncols else ncols), self.desc, ascii, unit, unit_scale, 1 / avg_time if avg_time else None, bar_format, self.postfix)) if self.pos: self.moveto(-self.pos) # If no `miniters` was specified, adjust automatically # to the max iteration rate seen so far between 2 prints if dynamic_miniters: if maxinterval and delta_t >= maxinterval: # Adjust miniters to time interval by rule of 3 if mininterval: # Set miniters to correspond to mininterval miniters = delta_it * mininterval / delta_t else: # Set miniters to correspond to maxinterval miniters = delta_it * maxinterval / delta_t elif smoothing: # EMA-weight miniters to converge # towards the timeframe of mininterval miniters = smoothing * delta_it * \ (mininterval / delta_t if mininterval and delta_t else 1) + \ (1 - smoothing) * miniters else: # Maximum nb of iterations between 2 prints miniters = max(miniters, delta_it) # Store old values for next call self.n = self.last_print_n = last_print_n = n self.last_print_t = last_print_t = cur_t self.miniters = miniters # Closing the progress bar. # Update some internal variables for close(). self.last_print_n = last_print_n self.n = n self.miniters = miniters self.close() def update(self, n=1): """ Manually update the progress bar, useful for streams such as reading files. E.g.: >>> t = tqdm(total=filesize) # Initialise >>> for current_buffer in stream: ... ... ... t.update(len(current_buffer)) >>> t.close() The last line is highly recommended, but possibly not necessary if `t.update()` will be called in such a way that `filesize` will be exactly reached and printed. Parameters ---------- n : int Increment to add to the internal counter of iterations [default: 1]. """ if self.disable: return if n < 0: raise ValueError("n ({0}) cannot be negative".format(n)) self.n += n if self.n - self.last_print_n >= self.miniters: # We check the counter first, to reduce the overhead of time() delta_t = self._time() - self.last_print_t if delta_t >= self.mininterval: cur_t = self._time() delta_it = self.n - self.last_print_n # should be n? elapsed = cur_t - self.start_t # EMA (not just overall average) if self.smoothing and delta_t and delta_it: self.avg_time = delta_t / delta_it \ if self.avg_time is None \ else self.smoothing * delta_t / delta_it + \ (1 - self.smoothing) * self.avg_time if not hasattr(self, "sp"): raise TqdmDeprecationWarning("""\ Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)` """, fp_write=getattr(self.fp, 'write', sys.stderr.write)) if self.pos: self.moveto(self.pos) # Print bar's update self.sp(self.format_meter( self.n, self.total, elapsed, (self.dynamic_ncols(self.fp) if self.dynamic_ncols else self.ncols), self.desc, self.ascii, self.unit, self.unit_scale, 1 / self.avg_time if self.avg_time else None, self.bar_format, self.postfix)) if self.pos: self.moveto(-self.pos) # If no `miniters` was specified, adjust automatically to the # maximum iteration rate seen so far between two prints. # e.g.: After running `tqdm.update(5)`, subsequent # calls to `tqdm.update()` will only cause an update after # at least 5 more iterations. if self.dynamic_miniters: if self.maxinterval and delta_t >= self.maxinterval: if self.mininterval: self.miniters = delta_it * self.mininterval \ / delta_t else: self.miniters = delta_it * self.maxinterval \ / delta_t elif self.smoothing: self.miniters = self.smoothing * delta_it * \ (self.mininterval / delta_t if self.mininterval and delta_t else 1) + \ (1 - self.smoothing) * self.miniters else: self.miniters = max(self.miniters, delta_it) # Store old values for next call self.last_print_n = self.n self.last_print_t = cur_t def close(self): """ Cleanup and (if leave=False) close the progressbar. """ if self.disable: return # Prevent multiple closures self.disable = True # decrement instance pos and remove from internal set pos = self.pos self._decr_instances(self) # GUI mode if not hasattr(self, "sp"): return # annoyingly, _supports_unicode isn't good enough def fp_write(s): self.fp.write(_unicode(s)) try: fp_write('') except ValueError as e: if 'closed' in str(e): return raise # pragma: no cover if pos: self.moveto(pos) if self.leave: if self.last_print_n < self.n: cur_t = self._time() # stats for overall rate (no weighted average) self.sp(self.format_meter( self.n, self.total, cur_t - self.start_t, (self.dynamic_ncols(self.fp) if self.dynamic_ncols else self.ncols), self.desc, self.ascii, self.unit, self.unit_scale, None, self.bar_format, self.postfix)) if pos: self.moveto(-pos) else: fp_write('\n') else: self.sp('') # clear up last bar if pos: self.moveto(-pos) else: fp_write('\r') def unpause(self): """ Restart tqdm timer from last print time. """ cur_t = self._time() self.start_t += cur_t - self.last_print_t self.last_print_t = cur_t def set_description(self, desc=None): """ Set/modify description of the progress bar. """ self.desc = desc + ': ' if desc else '' def set_postfix(self, ordered_dict=None, **kwargs): """ Set/modify postfix (additional stats) with automatic formatting based on datatype. """ # Sort in alphabetical order to be more deterministic postfix = _OrderedDict([] if ordered_dict is None else ordered_dict) for key in sorted(kwargs.keys()): postfix[key] = kwargs[key] # Preprocess stats according to datatype for key in postfix.keys(): # Number: limit the length of the string if isinstance(postfix[key], Number): postfix[key] = '{0:2.3g}'.format(postfix[key]) # Else for any other type, try to get the string conversion elif not isinstance(postfix[key], _basestring): postfix[key] = str(postfix[key]) # Else if it's a string, don't need to preprocess anything # Stitch together to get the final postfix self.postfix = ', '.join(key + '=' + postfix[key].strip() for key in postfix.keys()) def moveto(self, n): self.fp.write(_unicode('\n' * n + _term_move_up() * -n)) def clear(self, nomove=False): """ Clear current bar display """ if self.disable: return if not nomove: self.moveto(self.pos) # clear up the bar (can't rely on sp('')) self.fp.write('\r') self.fp.write(' ' * (self.ncols if self.ncols else 10)) self.fp.write('\r') # place cursor back at the beginning of line if not nomove: self.moveto(-self.pos) def refresh(self): """ Force refresh the display of this bar """ if self.disable: return self.moveto(self.pos) # clear up this line's content (whatever there was) self.clear(nomove=True) # Print current/last bar state self.fp.write(self.__repr__()) self.moveto(-self.pos) def trange(*args, **kwargs): """ A shortcut for tqdm(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ return tqdm(_range(*args), **kwargs) tqdm-4.11.2/tqdm/_main.py0000664000175000017500000001150513040270166015723 0ustar cr45hincr45hin00000000000000from ._tqdm import tqdm, TqdmTypeError, TqdmKeyError from ._version import __version__ # NOQA import sys import re __all__ = ["main"] def cast(val, typ): # sys.stderr.write('\ndebug | `val:type`: `' + val + ':' + typ + '`.\n') if typ == 'bool': if (val == 'True') or (val == ''): return True elif val == 'False': return False else: raise TqdmTypeError(val + ' : ' + typ) try: return eval(typ + '("' + val + '")') except: if (typ == 'chr'): return chr(ord(eval('"' + val + '"'))) else: raise TqdmTypeError(val + ' : ' + typ) def posix_pipe(fin, fout, delim='\n', buf_size=256, callback=lambda int: None # pragma: no cover ): """ Params ------ fin : file with `read(buf_size : int)` method fout : file with `write` (and optionally `flush`) methods. callback : function(int), e.g.: `tqdm.update` """ fp_write = fout.write tmp = '' if not delim: while True: tmp = fin.read(buf_size) # flush at EOF if not tmp: getattr(fout, 'flush', lambda: None)() # pragma: no cover return fp_write(tmp) callback(len(tmp)) return buf = '' # n = 0 while True: tmp = fin.read(buf_size) # flush at EOF if not tmp: if buf: fp_write(buf) callback(1 + buf.count(delim)) # n += 1 + buf.count(delim) getattr(fout, 'flush', lambda: None)() # pragma: no cover return # n while True: try: i = tmp.index(delim) except ValueError: buf += tmp break else: fp_write(buf + tmp[:i + len(delim)]) callback(1) # n += 1 buf = '' tmp = tmp[i + len(delim):] # ((opt, type), ... ) RE_OPTS = re.compile(r'\n {8}(\S+)\s{2,}:\s*([^\s,]+)') # better split method assuming no positional args RE_SHLEX = re.compile(r'\s*--?([^\s=]+)(?:\s*|=|$)') # TODO: add custom support for some of the following? UNSUPPORTED_OPTS = ('iterable', 'gui', 'out', 'file') # The 8 leading spaces are required for consistency CLI_EXTRA_DOC = r""" Extra CLI Options ----------------- delim : chr, optional Delimiting character [default: '\n']. Use '\0' for null. N.B.: on Windows systems, Python converts '\n' to '\r\n'. buf_size : int, optional String buffer size in bytes [default: 256] used when `delim` is specified. bytes : bool, optional If true, will count bytes and ignore `delim`. """ def main(fp=sys.stderr): """ Paramters (internal use only) --------- fp : file-like object for tqdm """ d = tqdm.__init__.__doc__ + CLI_EXTRA_DOC opt_types = dict(RE_OPTS.findall(d)) for o in UNSUPPORTED_OPTS: opt_types.pop(o) # d = RE_OPTS.sub(r' --\1=<\1> : \2', d) split = RE_OPTS.split(d) opt_types_desc = zip(split[1::3], split[2::3], split[3::3]) d = ''.join('\n --{0}=<{0}> : {1}{2}'.format(*otd) for otd in opt_types_desc if otd[0] not in UNSUPPORTED_OPTS) d = """Usage: tqdm [--help | options] Options: -h, --help Print this help and exit -v, --version Print version and exit """ + d.strip('\n') + '\n' # opts = docopt(d, version=__version__) if any(v in sys.argv for v in ('-v', '--version')): sys.stdout.write(__version__ + '\n') sys.exit(0) elif any(v in sys.argv for v in ('-h', '--help')): sys.stdout.write(d + '\n') sys.exit(0) argv = RE_SHLEX.split(' '.join(sys.argv)) opts = dict(zip(argv[1::2], argv[2::2])) tqdm_args = {'file': fp} try: for (o, v) in opts.items(): try: tqdm_args[o] = cast(v, opt_types[o]) except KeyError as e: raise TqdmKeyError(str(e)) # fp.write('\ndebug | args: ' + str(tqdm_args) + '\n') except: fp.write('\nError:\nUsage:\n tqdm [--help | options]\n') for i in sys.stdin: sys.stdout.write(i) raise else: buf_size = tqdm_args.pop('buf_size', 256) delim = tqdm_args.pop('delim', '\n') delim_per_char = tqdm_args.pop('bytes', False) if delim_per_char: with tqdm(**tqdm_args) as t: posix_pipe(sys.stdin, sys.stdout, '', buf_size, t.update) elif delim == '\n': for i in tqdm(sys.stdin, **tqdm_args): sys.stdout.write(i) else: with tqdm(**tqdm_args) as t: posix_pipe(sys.stdin, sys.stdout, delim, buf_size, t.update) tqdm-4.11.2/tqdm/_tqdm_notebook.py0000664000175000017500000002013213040270166017640 0ustar cr45hincr45hin00000000000000""" IPython/Jupyter Notebook progressbar decorator for iterators. Includes a default (x)range iterator printing to stderr. Usage: >>> from tqdm_notebook import tnrange[, tqdm_notebook] >>> for i in tnrange(10): #same as: for i in tqdm_notebook(xrange(10)) ... ... """ # future division is important to divide integers and get as # a result precise floating numbers (instead of truncated int) from __future__ import division, absolute_import # import compatibility functions and utilities import sys from ._utils import _range # to inherit from the tqdm class from ._tqdm import tqdm if True: # pragma: no cover # import IPython/Jupyter base widget and display utilities try: # IPython 4.x import ipywidgets IPY = 4 except ImportError: # IPython 3.x / 2.x IPY = 32 import warnings with warnings.catch_warnings(): ipy_deprecation_msg = "The `IPython.html` package" \ " has been deprecated" warnings.filterwarnings('error', message=".*" + ipy_deprecation_msg + ".*") try: import IPython.html.widgets as ipywidgets except Warning as e: if ipy_deprecation_msg not in str(e): raise warnings.simplefilter('ignore') try: import IPython.html.widgets as ipywidgets # NOQA except ImportError: pass except ImportError: pass try: # IPython 4.x / 3.x if IPY == 32: from IPython.html.widgets import IntProgress, HBox, HTML IPY = 3 else: from ipywidgets import IntProgress, HBox, HTML except ImportError: try: # IPython 2.x from IPython.html.widgets import IntProgressWidget as IntProgress from IPython.html.widgets import ContainerWidget as HBox from IPython.html.widgets import HTML IPY = 2 except ImportError: IPY = 0 try: from IPython.display import display # , clear_output except ImportError: pass # HTML encoding try: # Py3 from html import escape except ImportError: # Py2 from cgi import escape __author__ = {"github.com/": ["lrq3000", "casperdcl", "alexanderkuk"]} __all__ = ['tqdm_notebook', 'tnrange'] class tqdm_notebook(tqdm): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ @staticmethod def status_printer(_, total=None, desc=None): """ Manage the printing of an IPython/Jupyter Notebook progress bar widget. """ # Fallback to text bar if there's no total # DEPRECATED: replaced with an 'info' style bar # if not total: # return super(tqdm_notebook, tqdm_notebook).status_printer(file) # fp = file # Prepare IPython progress bar if total: pbar = IntProgress(min=0, max=total) else: # No total? Show info style bar with no progress tqdm status pbar = IntProgress(min=0, max=1) pbar.value = 1 pbar.bar_style = 'info' if desc: pbar.description = desc # Prepare status text ptext = HTML() # Only way to place text to the right of the bar is to use a container container = HBox(children=[pbar, ptext]) display(container) def print_status(s='', close=False, bar_style=None): # Note: contrary to native tqdm, s='' does NOT clear bar # goal is to keep all infos if error happens so user knows # at which iteration the loop failed. # Clear previous output (really necessary?) # clear_output(wait=1) # Get current iteration value from format_meter string if total: n = None if s: npos = s.find(r'/|/') # cause we use bar_format=r'{n}|...' # Check that n can be found in s (else n > total) if npos >= 0: n = int(s[:npos]) # get n from string s = s[npos + 3:] # remove from string # Update bar with current n value if n is not None: pbar.value = n # Print stats if s: # never clear the bar (signal: s='') s = s.replace('||', '') # remove inesthetical pipes s = escape(s) # html escape special characters (like '?') ptext.value = s # Change bar style if bar_style: # Hack-ish way to avoid the danger bar_style being overriden by # success because the bar gets closed after the error... if not (pbar.bar_style == 'danger' and bar_style == 'success'): pbar.bar_style = bar_style # Special signal to close the bar if close and pbar.bar_style != 'danger': # hide only if no error container.visible = False return print_status @classmethod def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm_notebook (just an alias for print) """ # Just an alias for print because overlap is impossible with ipywidgets file.write(s) file.write(end) def __init__(self, *args, **kwargs): # Setup default output if kwargs.get('file', sys.stderr) is sys.stderr: kwargs['file'] = sys.stdout # avoid the red block in IPython # Remove the bar from the printed string, only print stats if not kwargs.get('bar_format', None): kwargs['bar_format'] = r'{n}/|/{l_bar}{r_bar}' # Initialize parent class + avoid printing by using gui=True kwargs['gui'] = True super(tqdm_notebook, self).__init__(*args, **kwargs) if self.disable or not kwargs['gui']: return # Delete first pbar generated from super() (wrong total and text) # DEPRECATED by using gui=True # self.sp('', close=True) # Replace with IPython progress bar display (with correct total) self.sp = self.status_printer(self.fp, self.total, self.desc) self.desc = None # trick to place description before the bar # Print initial bar state if not self.disable: self.sp(self.__repr__()) # same as self.refresh without clearing def __iter__(self, *args, **kwargs): try: for obj in super(tqdm_notebook, self).__iter__(*args, **kwargs): # return super(tqdm...) will not catch exception yield obj # NB: except ... [ as ...] breaks IPython async KeyboardInterrupt except: self.sp(bar_style='danger') raise def update(self, *args, **kwargs): try: super(tqdm_notebook, self).update(*args, **kwargs) except Exception as exc: # cannot catch KeyboardInterrupt when using manual tqdm # as the interrupt will most likely happen on another statement self.sp(bar_style='danger') raise exc def close(self, *args, **kwargs): super(tqdm_notebook, self).close(*args, **kwargs) # If it was not run in a notebook, sp is not assigned, check for it if hasattr(self, 'sp'): # Try to detect if there was an error or KeyboardInterrupt # in manual mode: if n < total, things probably got wrong if self.total and self.n < self.total: self.sp(bar_style='danger') else: if self.leave: self.sp(bar_style='success') else: self.sp(close=True) def moveto(self, *args, **kwargs): # void -> avoid extraneous `\n` in IPython output cell return def tnrange(*args, **kwargs): """ A shortcut for tqdm_notebook(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ return tqdm_notebook(_range(*args), **kwargs) tqdm-4.11.2/tqdm/_version.py0000664000175000017500000000421713041514474016472 0ustar cr45hincr45hin00000000000000# Definition of the version number import os from io import open as io_open __all__ = ["__version__"] # major, minor, patch, -extra version_info = 4, 11, 2 # Nice string for the version __version__ = '.'.join(map(str, version_info)) # auto -extra based on commit hash (if not tagged as release) scriptdir = os.path.dirname(__file__) gitdir = os.path.abspath(os.path.join(scriptdir, "..", ".git")) if os.path.isdir(gitdir): # pragma: nocover extra = None # Open config file to check if we are in tqdm project with io_open(os.path.join(gitdir, "config"), 'r') as fh_config: if 'tqdm' in fh_config.read(): # Open the HEAD file with io_open(os.path.join(gitdir, "HEAD"), 'r') as fh_head: extra = fh_head.readline().strip() # in a branch => HEAD points to file containing last commit if 'ref:' in extra: # reference file path ref_file = extra[5:] branch_name = ref_file.rsplit('/', 1)[-1] ref_file_path = os.path.abspath(os.path.join(gitdir, ref_file)) # check that we are in git folder # (by stripping the git folder from the ref file path) if os.path.relpath( ref_file_path, gitdir).replace('\\', '/') != ref_file: # out of git folder extra = None else: # open the ref file with io_open(ref_file_path, 'r') as fh_branch: commit_hash = fh_branch.readline().strip() extra = commit_hash[:8] if branch_name != "master": extra += '.' + branch_name # detached HEAD mode, already have commit hash else: extra = extra[:8] # Append commit hash (and branch) to version string if not tagged if extra is not None: with io_open(os.path.join(gitdir, "refs", "tags", 'v' + __version__)) as fdv: if fdv.readline().strip()[:8] != extra[:8]: __version__ += '-' + extra tqdm-4.11.2/tqdm/_tqdm_gui.py0000664000175000017500000003222513040270166016612 0ustar cr45hincr45hin00000000000000""" GUI progressbar decorator for iterators. Includes a default (x)range iterator printing to stderr. Usage: >>> from tqdm_gui import tgrange[, tqdm_gui] >>> for i in tgrange(10): #same as: for i in tqdm_gui(xrange(10)) ... ... """ # future division is important to divide integers and get as # a result precise floating numbers (instead of truncated int) from __future__ import division, absolute_import # import compatibility functions and utilities import sys from time import time from ._utils import _range # to inherit from the tqdm class from ._tqdm import tqdm __author__ = {"github.com/": ["casperdcl", "lrq3000"]} __all__ = ['tqdm_gui', 'tgrange'] class tqdm_gui(tqdm): # pragma: no cover """ Experimental GUI version of tqdm! """ @classmethod def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm_gui (just an alias for print) """ # TODO: print text on GUI? file.write(s) file.write(end) def __init__(self, *args, **kwargs): import matplotlib as mpl import matplotlib.pyplot as plt from collections import deque kwargs['gui'] = True super(tqdm_gui, self).__init__(*args, **kwargs) # Initialize the GUI display if self.disable or not kwargs['gui']: return self.fp.write('Warning: GUI is experimental/alpha\n') self.mpl = mpl self.plt = plt self.sp = None # Remember if external environment uses toolbars self.toolbar = self.mpl.rcParams['toolbar'] self.mpl.rcParams['toolbar'] = 'None' self.mininterval = max(self.mininterval, 0.5) self.fig, ax = plt.subplots(figsize=(9, 2.2)) # self.fig.subplots_adjust(bottom=0.2) if self.total: self.xdata = [] self.ydata = [] self.zdata = [] else: self.xdata = deque([]) self.ydata = deque([]) self.zdata = deque([]) self.line1, = ax.plot(self.xdata, self.ydata, color='b') self.line2, = ax.plot(self.xdata, self.zdata, color='k') ax.set_ylim(0, 0.001) if self.total: ax.set_xlim(0, 100) ax.set_xlabel('percent') self.fig.legend((self.line1, self.line2), ('cur', 'est'), loc='center right') # progressbar self.hspan = plt.axhspan(0, 0.001, xmin=0, xmax=0, color='g') else: # ax.set_xlim(-60, 0) ax.set_xlim(0, 60) ax.invert_xaxis() ax.set_xlabel('seconds') ax.legend(('cur', 'est'), loc='lower left') ax.grid() # ax.set_xlabel('seconds') ax.set_ylabel((self.unit if self.unit else 'it') + '/s') if self.unit_scale: plt.ticklabel_format(style='sci', axis='y', scilimits=(0, 0)) ax.yaxis.get_offset_text().set_x(-0.15) # Remember if external environment is interactive self.wasion = plt.isinteractive() plt.ion() self.ax = ax def __iter__(self): # TODO: somehow allow the following: # if not self.gui: # return super(tqdm_gui, self).__iter__() iterable = self.iterable if self.disable: for obj in iterable: yield obj return # ncols = self.ncols mininterval = self.mininterval maxinterval = self.maxinterval miniters = self.miniters dynamic_miniters = self.dynamic_miniters unit = self.unit unit_scale = self.unit_scale ascii = self.ascii start_t = self.start_t last_print_t = self.last_print_t last_print_n = self.last_print_n n = self.n # dynamic_ncols = self.dynamic_ncols smoothing = self.smoothing avg_time = self.avg_time bar_format = self.bar_format plt = self.plt ax = self.ax xdata = self.xdata ydata = self.ydata zdata = self.zdata line1 = self.line1 line2 = self.line2 for obj in iterable: yield obj # Update and print the progressbar. # Note: does not call self.update(1) for speed optimisation. n += 1 delta_it = n - last_print_n # check the counter first (avoid calls to time()) if delta_it >= miniters: cur_t = time() delta_t = cur_t - last_print_t if delta_t >= mininterval: elapsed = cur_t - start_t # EMA (not just overall average) if smoothing and delta_t: avg_time = delta_t / delta_it \ if avg_time is None \ else smoothing * delta_t / delta_it + \ (1 - smoothing) * avg_time # Inline due to multiple calls total = self.total # instantaneous rate y = delta_it / delta_t # overall rate z = n / elapsed # update line data xdata.append(n * 100.0 / total if total else cur_t) ydata.append(y) zdata.append(z) # Discard old values # xmin, xmax = ax.get_xlim() # if (not total) and elapsed > xmin * 1.1: if (not total) and elapsed > 66: xdata.popleft() ydata.popleft() zdata.popleft() ymin, ymax = ax.get_ylim() if y > ymax or z > ymax: ymax = 1.1 * y ax.set_ylim(ymin, ymax) ax.figure.canvas.draw() if total: line1.set_data(xdata, ydata) line2.set_data(xdata, zdata) try: poly_lims = self.hspan.get_xy() except AttributeError: self.hspan = plt.axhspan(0, 0.001, xmin=0, xmax=0, color='g') poly_lims = self.hspan.get_xy() poly_lims[0, 1] = ymin poly_lims[1, 1] = ymax poly_lims[2] = [n / total, ymax] poly_lims[3] = [poly_lims[2, 0], ymin] if len(poly_lims) > 4: poly_lims[4, 1] = ymin self.hspan.set_xy(poly_lims) else: t_ago = [cur_t - i for i in xdata] line1.set_data(t_ago, ydata) line2.set_data(t_ago, zdata) ax.set_title(self.format_meter( n, total, elapsed, 0, self.desc, ascii, unit, unit_scale, 1 / avg_time if avg_time else None, bar_format), fontname="DejaVu Sans Mono", fontsize=11) plt.pause(1e-9) # If no `miniters` was specified, adjust automatically # to the maximum iteration rate seen so far. if dynamic_miniters: if maxinterval and delta_t > maxinterval: # Set miniters to correspond to maxinterval miniters = delta_it * maxinterval / delta_t elif mininterval and delta_t: # EMA-weight miniters to converge # towards the timeframe of mininterval miniters = smoothing * delta_it * mininterval \ / delta_t + (1 - smoothing) * miniters else: miniters = smoothing * delta_it + \ (1 - smoothing) * miniters # Store old values for next call last_print_n = n last_print_t = cur_t # Closing the progress bar. # Update some internal variables for close(). self.last_print_n = last_print_n self.n = n self.close() def update(self, n=1): # if not self.gui: # return super(tqdm_gui, self).close() if self.disable: return if n < 0: n = 1 self.n += n delta_it = self.n - self.last_print_n # should be n? if delta_it >= self.miniters: # We check the counter first, to reduce the overhead of time() cur_t = time() delta_t = cur_t - self.last_print_t if delta_t >= self.mininterval: elapsed = cur_t - self.start_t # EMA (not just overall average) if self.smoothing and delta_t: self.avg_time = delta_t / delta_it \ if self.avg_time is None \ else self.smoothing * delta_t / delta_it + \ (1 - self.smoothing) * self.avg_time # Inline due to multiple calls total = self.total ax = self.ax # instantaneous rate y = delta_it / delta_t # smoothed rate z = self.n / elapsed # update line data self.xdata.append(self.n * 100.0 / total if total else cur_t) self.ydata.append(y) self.zdata.append(z) # Discard old values if (not total) and elapsed > 66: self.xdata.popleft() self.ydata.popleft() self.zdata.popleft() ymin, ymax = ax.get_ylim() if y > ymax or z > ymax: ymax = 1.1 * y ax.set_ylim(ymin, ymax) ax.figure.canvas.draw() if total: self.line1.set_data(self.xdata, self.ydata) self.line2.set_data(self.xdata, self.zdata) try: poly_lims = self.hspan.get_xy() except AttributeError: self.hspan = self.plt.axhspan(0, 0.001, xmin=0, xmax=0, color='g') poly_lims = self.hspan.get_xy() poly_lims[0, 1] = ymin poly_lims[1, 1] = ymax poly_lims[2] = [self.n / total, ymax] poly_lims[3] = [poly_lims[2, 0], ymin] if len(poly_lims) > 4: poly_lims[4, 1] = ymin self.hspan.set_xy(poly_lims) else: t_ago = [cur_t - i for i in self.xdata] self.line1.set_data(t_ago, self.ydata) self.line2.set_data(t_ago, self.zdata) ax.set_title(self.format_meter( self.n, total, elapsed, 0, self.desc, self.ascii, self.unit, self.unit_scale, 1 / self.avg_time if self.avg_time else None, self.bar_format), fontname="DejaVu Sans Mono", fontsize=11) self.plt.pause(1e-9) # If no `miniters` was specified, adjust automatically to the # maximum iteration rate seen so far. # e.g.: After running `tqdm.update(5)`, subsequent # calls to `tqdm.update()` will only cause an update after # at least 5 more iterations. if self.dynamic_miniters: if self.maxinterval and delta_t > self.maxinterval: self.miniters = self.miniters * self.maxinterval \ / delta_t elif self.mininterval and delta_t: self.miniters = self.smoothing * delta_it \ * self.mininterval / delta_t + \ (1 - self.smoothing) * self.miniters else: self.miniters = self.smoothing * delta_it + \ (1 - self.smoothing) * self.miniters # Store old values for next call self.last_print_n = self.n self.last_print_t = cur_t def close(self): # if not self.gui: # return super(tqdm_gui, self).close() if self.disable: return self.disable = True self._instances.remove(self) # Restore toolbars self.mpl.rcParams['toolbar'] = self.toolbar # Return to non-interactive mode if not self.wasion: self.plt.ioff() if not self.leave: self.plt.close(self.fig) def tgrange(*args, **kwargs): """ A shortcut for tqdm_gui(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ return tqdm_gui(_range(*args), **kwargs) tqdm-4.11.2/tqdm/__main__.py0000664000175000017500000000003713036011052016346 0ustar cr45hincr45hin00000000000000from ._main import main main() tqdm-4.11.2/tqdm/tests/0000775000175000017500000000000013041514547015433 5ustar cr45hincr45hin00000000000000tqdm-4.11.2/tqdm/tests/tests_pandas.py0000664000175000017500000001025713040270166020475 0ustar cr45hincr45hin00000000000000from nose.plugins.skip import SkipTest from tqdm import tqdm from tests_tqdm import with_setup, pretest, posttest, StringIO, closing @with_setup(pretest, posttest) def test_pandas_groupby_apply(): """ Test pandas.DataFrame.groupby(...).progress_apply """ try: from numpy.random import randint import pandas as pd except: raise SkipTest with closing(StringIO()) as our_file: tqdm.pandas(file=our_file, leave=False, ascii=True) df = pd.DataFrame(randint(0, 50, (500, 3))) df.groupby(0).progress_apply(lambda x: None) dfs = pd.DataFrame(randint(0, 50, (500, 3)), columns=list('abc')) dfs.groupby(['a']).progress_apply(lambda x: None) our_file.seek(0) # don't expect final output since no `leave` and # high dynamic `miniters` nexres = '100%|##########|' if nexres in our_file.read(): our_file.seek(0) raise AssertionError("\nDid not expect:\n{0}\nIn:{1}\n".format( nexres, our_file.read())) @with_setup(pretest, posttest) def test_pandas_apply(): """ Test pandas.DataFrame[.series].progress_apply """ try: from numpy.random import randint import pandas as pd except: raise SkipTest with closing(StringIO()) as our_file: tqdm.pandas(file=our_file, leave=True, ascii=True) df = pd.DataFrame(randint(0, 50, (500, 3))) df.progress_apply(lambda x: None) dfs = pd.DataFrame(randint(0, 50, (500, 3)), columns=list('abc')) dfs.a.progress_apply(lambda x: None) our_file.seek(0) if our_file.read().count('100%') < 2: our_file.seek(0) raise AssertionError("\nExpected:\n{0}\nIn:{1}\n".format( '100% at least twice', our_file.read())) @with_setup(pretest, posttest) def test_pandas_map(): """ Test pandas.Series.progress_map """ try: from numpy.random import randint import pandas as pd except: raise SkipTest with closing(StringIO()) as our_file: tqdm.pandas(file=our_file, leave=True, ascii=True) dfs = pd.DataFrame(randint(0, 50, (500, 3)), columns=list('abc')) dfs.a.progress_map(lambda x: None) if our_file.getvalue().count('100%') < 1: raise AssertionError("\nExpected:\n{0}\nIn:{1}\n".format( '100% at least twice', our_file.getvalue())) @with_setup(pretest, posttest) def test_pandas_leave(): """ Test pandas with `leave=True` """ try: from numpy.random import randint import pandas as pd except: raise SkipTest with closing(StringIO()) as our_file: df = pd.DataFrame(randint(0, 100, (1000, 6))) tqdm.pandas(file=our_file, leave=True, ascii=True) df.groupby(0).progress_apply(lambda x: None) our_file.seek(0) exres = '100%|##########| 101/101' if exres not in our_file.read(): our_file.seek(0) raise AssertionError("\nExpected:\n{0}\nIn:{1}\n".format( exres, our_file.read())) @with_setup(pretest, posttest) def test_pandas_deprecation(): """ Test bar object instance as argument deprecation """ try: from numpy.random import randint from tqdm import tqdm_pandas import pandas as pd except: raise SkipTest with closing(StringIO()) as our_file: tqdm_pandas(tqdm(file=our_file, leave=False, ascii=True, ncols=20)) df = pd.DataFrame(randint(0, 50, (500, 3))) df.groupby(0).progress_apply(lambda x: None) # Check deprecation message assert "TqdmDeprecationWarning" in our_file.getvalue() assert "instead of `tqdm_pandas(tqdm(...))`" in our_file.getvalue() with closing(StringIO()) as our_file: tqdm_pandas(tqdm, file=our_file, leave=False, ascii=True, ncols=20) df = pd.DataFrame(randint(0, 50, (500, 3))) df.groupby(0).progress_apply(lambda x: None) # Check deprecation message assert "TqdmDeprecationWarning" in our_file.getvalue() assert "instead of `tqdm_pandas(tqdm, ...)`" in our_file.getvalue() tqdm-4.11.2/tqdm/tests/tests_tqdm.py0000664000175000017500000015164113041513621020174 0ustar cr45hincr45hin00000000000000# Advice: use repr(our_file.read()) to print the full output of tqdm # (else '\r' will replace the previous lines and you'll see only the latest. from __future__ import unicode_literals import sys import csv import re import os from nose import with_setup from nose.plugins.skip import SkipTest from nose.tools import assert_raises from time import sleep from tqdm import tqdm from tqdm import trange from tqdm import TqdmDeprecationWarning from tqdm._tqdm import TMonitor try: from StringIO import StringIO except ImportError: from io import StringIO from io import IOBase # to support unicode strings class DeprecationError(Exception): pass # Ensure we can use `with closing(...) as ... :` syntax if getattr(StringIO, '__exit__', False) and \ getattr(StringIO, '__enter__', False): def closing(arg): return arg else: from contextlib import closing try: _range = xrange except NameError: _range = range try: _unicode = unicode except NameError: _unicode = str nt_and_no_colorama = False if os.name == 'nt': try: import colorama # NOQA except ImportError: nt_and_no_colorama = True # Regex definitions # List of control characters CTRLCHR = [r'\r', r'\n', r'\x1b\[A'] # Need to escape [ for regex # Regular expressions compilation RE_rate = re.compile(r'(\d+\.\d+)it/s') RE_ctrlchr = re.compile("(%s)" % '|'.join(CTRLCHR)) # Match control chars RE_ctrlchr_excl = re.compile('|'.join(CTRLCHR)) # Match and exclude ctrl chars RE_pos = re.compile(r'((\x1b\[A|\r|\n)+((pos\d+) bar:\s+\d+%|\s{3,6})?)') # NOQA class DiscreteTimer(object): '''Virtual discrete time manager, to precisely control time for tests''' def __init__(self): self.t = 0.0 def sleep(self, t): '''Sleep = increment the time counter (almost no CPU used)''' self.t += t def time(self): '''Get the current time''' return self.t class FakeSleep(object): '''Wait until the discrete timer reached the required time''' def __init__(self, dtimer): self.dtimer = dtimer def sleep(self, t): end = t + self.dtimer.t while(self.dtimer.t < end): sleep(0.0000001) # sleep a bit to interrupt (instead of pass) def cpu_timify(t, timer=None): '''Force tqdm to use the specified timer instead of system-wide time()''' if timer is None: timer = DiscreteTimer() t._time = timer.time t._sleep = timer.sleep t.start_t = t.last_print_t = t._time() return timer def pretest(): if getattr(tqdm, "_instances", False): n = len(tqdm._instances) if n: tqdm._instances.clear() raise EnvironmentError( "{0} `tqdm` instances still in existence PRE-test".format(n)) def posttest(): if getattr(tqdm, "_instances", False): n = len(tqdm._instances) if n: tqdm._instances.clear() raise EnvironmentError( "{0} `tqdm` instances still in existence POST-test".format(n)) class UnicodeIO(IOBase): ''' Unicode version of StringIO ''' def __init__(self, *args, **kwargs): super(UnicodeIO, self).__init__(*args, **kwargs) self.encoding = 'U8' # io.StringIO supports unicode, but no encoding self.text = '' self.cursor = 0 def __len__(self): return len(self.text) def seek(self, offset): self.cursor = offset def tell(self): return self.cursor def write(self, s): self.text = self.text[:self.cursor] + s + \ self.text[self.cursor + len(s):] self.cursor += len(s) def read(self, n=-1): _cur = self.cursor self.cursor = len(self) if n < 0 \ else min(_cur + n, len(self)) return self.text[_cur:self.cursor] def getvalue(self): return self.text def get_bar(all_bars, i): """ Get a specific update from a whole bar traceback """ # Split according to any used control characters bars_split = RE_ctrlchr_excl.split(all_bars) bars_split = list(filter(None, bars_split)) # filter out empty splits return bars_split[i] def progressbar_rate(bar_str): return float(RE_rate.search(bar_str).group(1)) def squash_ctrlchars(s): """ Apply control characters in a string just like a terminal display """ # List of supported control codes ctrlcodes = [r'\r', r'\n', r'\x1b\[A'] # Init variables curline = 0 # current line in our fake terminal lines = [''] # state of our fake terminal # Split input string by control codes RE_ctrl = re.compile("(%s)" % ("|".join(ctrlcodes)), flags=re.DOTALL) s_split = RE_ctrl.split(s) s_split = filter(None, s_split) # filter out empty splits # For each control character or message for nextctrl in s_split: # If it's a control character, apply it if nextctrl == '\r': # Carriage return # Go to the beginning of the line # simplified here: we just empty the string lines[curline] = '' elif nextctrl == '\n': # Newline # Go to the next line if curline < (len(lines) - 1): # If already exists, just move cursor curline += 1 else: # Else the new line is created lines.append('') curline += 1 elif nextctrl == '\x1b[A': # Move cursor up if curline > 0: curline -= 1 else: raise ValueError("Cannot go up, anymore!") # Else, it is a message, we print it on current line else: lines[curline] += nextctrl return lines def test_format_interval(): """ Test time interval format """ format_interval = tqdm.format_interval assert format_interval(60) == '01:00' assert format_interval(6160) == '1:42:40' assert format_interval(238113) == '66:08:33' def test_format_meter(): """ Test statistics and progress bar formatting """ try: unich = unichr except NameError: unich = chr format_meter = tqdm.format_meter assert format_meter(0, 1000, 13) == \ " 0%| | 0/1000 [00:13= (bigstep - 1) and \ ((i - (bigstep - 1)) % smallstep) == 0: timer.sleep(1e-2) if i >= 3 * bigstep: break our_file.seek(0) assert "15%" in our_file.read() # Test different behavior with and without mininterval timer = DiscreteTimer() total = 1000 mininterval = 0.1 maxinterval = 10 with closing(StringIO()) as our_file: with tqdm(total=total, file=our_file, miniters=None, smoothing=1, mininterval=mininterval, maxinterval=maxinterval) as tm1: with tqdm(total=total, file=our_file, miniters=None, smoothing=1, mininterval=0, maxinterval=maxinterval) as tm2: cpu_timify(tm1, timer) cpu_timify(tm2, timer) # Fast iterations, check if dynamic_miniters triggers timer.sleep(mininterval) # to force update for t1 tm1.update(total/2) tm2.update(total/2) assert int(tm1.miniters) == tm2.miniters == total/2 # Slow iterations, check different miniters if mininterval timer.sleep(maxinterval*2) tm1.update(total/2) tm2.update(total/2) res = [tm1.miniters, tm2.miniters] assert res == [ (total/2)*mininterval/(maxinterval*2), (total/2)*maxinterval/(maxinterval*2) ] # Same with iterable based tqdm timer1 = DiscreteTimer() # need 2 timers for each bar because zip not work timer2 = DiscreteTimer() total = 100 mininterval = 0.1 maxinterval = 10 with closing(StringIO()) as our_file: t1 = tqdm(_range(total), file=our_file, miniters=None, smoothing=1, mininterval=mininterval, maxinterval=maxinterval) t2 = tqdm(_range(total), file=our_file, miniters=None, smoothing=1, mininterval=0, maxinterval=maxinterval) cpu_timify(t1, timer1) cpu_timify(t2, timer2) for i in t1: if i == ((total/2)-2): timer1.sleep(mininterval) if i == (total-1): timer1.sleep(maxinterval*2) for i in t2: if i == ((total/2)-2): timer2.sleep(mininterval) if i == (total-1): timer2.sleep(maxinterval*2) assert t1.miniters == 0.255 assert t2.miniters == 0.5 t1.close() t2.close() @with_setup(pretest, posttest) def test_min_iters(): """ Test miniters """ with closing(StringIO()) as our_file: for _ in tqdm(_range(3), file=our_file, leave=True, miniters=4): our_file.write('blank\n') our_file.seek(0) assert '\nblank\nblank\n' in our_file.read() with closing(StringIO()) as our_file: for _ in tqdm(_range(3), file=our_file, leave=True, miniters=1): our_file.write('blank\n') our_file.seek(0) # assume automatic mininterval = 0 means intermediate output assert '| 3/3 ' in our_file.read() @with_setup(pretest, posttest) def test_dynamic_min_iters(): """ Test purely dynamic miniters (and manual updates and __del__) """ with closing(StringIO()) as our_file: total = 10 t = tqdm(total=total, file=our_file, miniters=None, mininterval=0, smoothing=1) t.update() # Increase 3 iterations t.update(3) # The next two iterations should be skipped because of dynamic_miniters t.update() t.update() # The third iteration should be displayed t.update() our_file.seek(0) out = our_file.read() assert t.dynamic_miniters t.__del__() # simulate immediate del gc assert ' 0%| | 0/10 [00:00<' in out assert '40%' in out assert '50%' not in out assert '60%' not in out assert '70%' in out # Check with smoothing=0, miniters should be set to max update seen so far with closing(StringIO()) as our_file: total = 10 t = tqdm(total=total, file=our_file, miniters=None, mininterval=0, smoothing=0) t.update() t.update(2) t.update(5) # this should be stored as miniters t.update(1) our_file.seek(0) out = our_file.read() assert t.dynamic_miniters and not t.smoothing assert t.miniters == 5 t.close() # Check iterable based tqdm with closing(StringIO()) as our_file: t = tqdm(_range(10), file=our_file, miniters=None, mininterval=None, smoothing=0.5) for _ in t: pass assert t.dynamic_miniters # No smoothing with closing(StringIO()) as our_file: t = tqdm(_range(10), file=our_file, miniters=None, mininterval=None, smoothing=0) for _ in t: pass assert t.dynamic_miniters # No dynamic_miniters (miniters is fixed manually) with closing(StringIO()) as our_file: t = tqdm(_range(10), file=our_file, miniters=1, mininterval=None) for _ in t: pass assert not t.dynamic_miniters @with_setup(pretest, posttest) def test_big_min_interval(): """ Test large mininterval """ with closing(StringIO()) as our_file: for _ in tqdm(_range(2), file=our_file, mininterval=1E10): pass our_file.seek(0) assert '50%' not in our_file.read() with closing(StringIO()) as our_file: with tqdm(_range(2), file=our_file, mininterval=1E10) as t: t.update() t.update() our_file.seek(0) assert '50%' not in our_file.read() @with_setup(pretest, posttest) def test_smoothed_dynamic_min_iters(): """ Test smoothed dynamic miniters """ timer = DiscreteTimer() with closing(StringIO()) as our_file: with tqdm(total=100, file=our_file, miniters=None, mininterval=0, smoothing=0.5, maxinterval=0) as t: cpu_timify(t, timer) # Increase 10 iterations at once t.update(10) # The next iterations should be partially skipped for _ in _range(2): t.update(4) for _ in _range(20): t.update() our_file.seek(0) out = our_file.read() assert t.dynamic_miniters assert ' 0%| | 0/100 [00:00<' in out assert '10%' in out assert '14%' not in out assert '18%' in out assert '20%' not in out assert '25%' in out assert '30%' not in out assert '32%' in out @with_setup(pretest, posttest) def test_smoothed_dynamic_min_iters_with_min_interval(): """ Test smoothed dynamic miniters with mininterval """ timer = DiscreteTimer() # In this test, `miniters` should gradually decline total = 100 with closing(StringIO()) as our_file: # Test manual updating tqdm with tqdm(total=total, file=our_file, miniters=None, mininterval=1e-3, smoothing=1, maxinterval=0) as t: cpu_timify(t, timer) t.update(10) timer.sleep(1e-2) for _ in _range(4): t.update() timer.sleep(1e-2) our_file.seek(0) out = our_file.read() assert t.dynamic_miniters with closing(StringIO()) as our_file: # Test iteration-based tqdm with tqdm(_range(total), file=our_file, miniters=None, mininterval=0.01, smoothing=1, maxinterval=0) as t2: cpu_timify(t2, timer) for i in t2: if i >= 10: timer.sleep(0.1) if i >= 14: break our_file.seek(0) out2 = our_file.read() assert t.dynamic_miniters assert ' 0%| | 0/100 [00:00<' in out assert '11%' in out and '11%' in out2 # assert '12%' not in out and '12%' in out2 assert '13%' in out and '13%' in out2 assert '14%' in out and '14%' in out2 @with_setup(pretest, posttest) def test_disable(): """ Test disable """ with closing(StringIO()) as our_file: for _ in tqdm(_range(3), file=our_file, disable=True): pass our_file.seek(0) assert our_file.read() == '' with closing(StringIO()) as our_file: progressbar = tqdm(total=3, file=our_file, miniters=1, disable=True) progressbar.update(3) progressbar.close() our_file.seek(0) assert our_file.read() == '' @with_setup(pretest, posttest) def test_unit(): """ Test SI unit prefix """ with closing(StringIO()) as our_file: for _ in tqdm(_range(3), file=our_file, miniters=1, unit="bytes"): pass our_file.seek(0) assert 'bytes/s' in our_file.read() @with_setup(pretest, posttest) def test_ascii(): """ Test ascii/unicode bar """ # Test ascii autodetection with closing(StringIO()) as our_file: with tqdm(total=10, file=our_file, ascii=None) as t: assert t.ascii # TODO: this may fail in the future # Test ascii bar with closing(StringIO()) as our_file: for _ in tqdm(_range(3), total=15, file=our_file, miniters=1, mininterval=0, ascii=True): pass our_file.seek(0) res = our_file.read().strip("\r").split("\r") assert '7%|6' in res[1] assert '13%|#3' in res[2] assert '20%|##' in res[3] # Test unicode bar with closing(UnicodeIO()) as our_file: with tqdm(total=15, file=our_file, ascii=False, mininterval=0) as t: for _ in _range(3): t.update() our_file.seek(0) res = our_file.read().strip("\r").split("\r") assert "7%|\u258b" in res[1] assert "13%|\u2588\u258e" in res[2] assert "20%|\u2588\u2588" in res[3] @with_setup(pretest, posttest) def test_update(): """ Test manual creation and updates """ with closing(StringIO()) as our_file: with tqdm(total=2, file=our_file, miniters=1, mininterval=0) \ as progressbar: assert len(progressbar) == 2 progressbar.update(2) our_file.seek(0) assert '| 2/2' in our_file.read() progressbar.desc = 'dynamically notify of 4 increments in total' progressbar.total = 4 try: progressbar.update(-10) except ValueError as e: if str(e) != "n (-10) cannot be negative": raise progressbar.update() # should default to +1 else: raise ValueError("Should not support negative updates") our_file.seek(0) res = our_file.read() assert '| 3/4 ' in res assert 'dynamically notify of 4 increments in total' in res @with_setup(pretest, posttest) def test_close(): """ Test manual creation and closure and n_instances """ # With `leave` option with closing(StringIO()) as our_file: progressbar = tqdm(total=3, file=our_file, miniters=10) progressbar.update(3) assert '| 3/3 ' not in our_file.getvalue() # Should be blank assert len(tqdm._instances) == 1 progressbar.close() assert len(tqdm._instances) == 0 assert '| 3/3 ' in our_file.getvalue() # Without `leave` option with closing(StringIO()) as our_file: progressbar = tqdm(total=3, file=our_file, miniters=10, leave=False) progressbar.update(3) progressbar.close() assert '| 3/3 ' not in our_file.getvalue() # Should be blank # With all updates with closing(StringIO()) as our_file: assert len(tqdm._instances) == 0 with tqdm(total=3, file=our_file, miniters=0, mininterval=0, leave=True) as progressbar: assert len(tqdm._instances) == 1 progressbar.update(3) res = our_file.getvalue() assert '| 3/3 ' in res # Should be blank # close() called assert len(tqdm._instances) == 0 our_file.seek(0) exres = res + '\n' if exres != our_file.read(): our_file.seek(0) raise AssertionError("\nExpected:\n{0}\nGot:{1}\n".format( exres, our_file.read())) # Closing after the output stream has closed with closing(StringIO()) as our_file: t = tqdm(total=2, file=our_file) t.update() t.update() t.close() @with_setup(pretest, posttest) def test_smoothing(): """ Test exponential weighted average smoothing """ timer = DiscreteTimer() # -- Test disabling smoothing with closing(StringIO()) as our_file: with tqdm(_range(3), file=our_file, smoothing=None, leave=True) as t: cpu_timify(t, timer) for _ in t: pass our_file.seek(0) assert '| 3/3 ' in our_file.read() # -- Test smoothing # Compile the regex to find the rate # 1st case: no smoothing (only use average) with closing(StringIO()) as our_file2: with closing(StringIO()) as our_file: t = tqdm(_range(3), file=our_file2, smoothing=None, leave=True, miniters=1, mininterval=0) cpu_timify(t, timer) with tqdm(_range(3), file=our_file, smoothing=None, leave=True, miniters=1, mininterval=0) as t2: cpu_timify(t2, timer) for i in t2: # Sleep more for first iteration and # see how quickly rate is updated if i == 0: timer.sleep(0.01) else: # Need to sleep in all iterations # to calculate smoothed rate # (else delta_t is 0!) timer.sleep(0.001) t.update() n_old = len(tqdm._instances) t.close() assert len(tqdm._instances) == n_old - 1 # Get result for iter-based bar a = progressbar_rate(get_bar(our_file.getvalue(), 3)) # Get result for manually updated bar a2 = progressbar_rate(get_bar(our_file2.getvalue(), 3)) # 2nd case: use max smoothing (= instant rate) with closing(StringIO()) as our_file2: with closing(StringIO()) as our_file: t = tqdm(_range(3), file=our_file2, smoothing=1, leave=True, miniters=1, mininterval=0) cpu_timify(t, timer) with tqdm(_range(3), file=our_file, smoothing=1, leave=True, miniters=1, mininterval=0) as t2: cpu_timify(t2, timer) for i in t2: if i == 0: timer.sleep(0.01) else: timer.sleep(0.001) t.update() t.close() # Get result for iter-based bar b = progressbar_rate(get_bar(our_file.getvalue(), 3)) # Get result for manually updated bar b2 = progressbar_rate(get_bar(our_file2.getvalue(), 3)) # 3rd case: use medium smoothing with closing(StringIO()) as our_file2: with closing(StringIO()) as our_file: t = tqdm(_range(3), file=our_file2, smoothing=0.5, leave=True, miniters=1, mininterval=0) cpu_timify(t, timer) t2 = tqdm(_range(3), file=our_file, smoothing=0.5, leave=True, miniters=1, mininterval=0) cpu_timify(t2, timer) for i in t2: if i == 0: timer.sleep(0.01) else: timer.sleep(0.001) t.update() t2.close() t.close() # Get result for iter-based bar c = progressbar_rate(get_bar(our_file.getvalue(), 3)) # Get result for manually updated bar c2 = progressbar_rate(get_bar(our_file2.getvalue(), 3)) # Check that medium smoothing's rate is between no and max smoothing rates assert a < c < b assert a2 < c2 < b2 @with_setup(pretest, posttest) def test_deprecated_nested(): """ Test nested progress bars """ if nt_and_no_colorama: raise SkipTest # TODO: test degradation on windows without colorama? # Artificially test nested loop printing # Without leave our_file = StringIO() try: tqdm(total=2, file=our_file, nested=True) except TqdmDeprecationWarning: if """`nested` is deprecated and automated.\ Use position instead for manual control.""" not in our_file.getvalue(): raise else: raise DeprecationError("Should not allow nested kwarg") @with_setup(pretest, posttest) def test_bar_format(): """ Test custom bar formatting """ with closing(StringIO()) as our_file: bar_format = r'{l_bar}{bar}|{n_fmt}/{total_fmt}-{n}/{total}{percentage}{rate}{rate_fmt}{elapsed}{remaining}' # NOQA for _ in trange(2, file=our_file, leave=True, bar_format=bar_format): pass out = our_file.getvalue() assert "\r 0%| |0/2-0/20.0None?it/s00:00?\r" in out # Test unicode string auto conversion with closing(StringIO()) as our_file: bar_format = r'hello world' with tqdm(ascii=False, bar_format=bar_format, file=our_file) as t: assert isinstance(t.bar_format, _unicode) @with_setup(pretest, posttest) def test_unpause(): """ Test unpause """ timer = DiscreteTimer() with closing(StringIO()) as our_file: t = trange(10, file=our_file, leave=True, mininterval=0) cpu_timify(t, timer) timer.sleep(0.01) t.update() timer.sleep(0.01) t.update() timer.sleep(0.1) # longer wait time t.unpause() timer.sleep(0.01) t.update() timer.sleep(0.01) t.update() t.close() r_before = progressbar_rate(get_bar(our_file.getvalue(), 2)) r_after = progressbar_rate(get_bar(our_file.getvalue(), 3)) assert r_before == r_after @with_setup(pretest, posttest) def test_position(): """ Test positioned progress bars """ if nt_and_no_colorama: raise SkipTest # Artificially test nested loop printing # Without leave our_file = StringIO() t = tqdm(total=2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos2 bar', leave=False, position=2) t.update() t.close() our_file.seek(0) out = our_file.read() res = [m[0] for m in RE_pos.findall(out)] exres = ['\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\n\r ', '\x1b[A\x1b[A'] if res != exres: raise AssertionError("\nExpected:\n{0}\nGot:\n{1}\nRaw:\n{2}\n".format( str(exres), str(res), str([out]))) # Test iteration-based tqdm positioning our_file = StringIO() for _ in trange(2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos0 bar', position=0): for _ in trange(2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos1 bar', position=1): for _ in trange(2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos2 bar', position=2): pass our_file.seek(0) out = our_file.read() res = [m[0] for m in RE_pos.findall(out)] exres = ['\rpos0 bar: 0%', '\n\rpos1 bar: 0%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\n\rpos2 bar: 100%', '\x1b[A\x1b[A\n\n\x1b[A\x1b[A\n\rpos1 bar: 50%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\n\rpos2 bar: 100%', '\x1b[A\x1b[A\n\n\x1b[A\x1b[A\n\rpos1 bar: 100%', '\x1b[A\n\x1b[A\rpos0 bar: 50%', '\n\rpos1 bar: 0%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\n\rpos2 bar: 100%', '\x1b[A\x1b[A\n\n\x1b[A\x1b[A\n\rpos1 bar: 50%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\n\rpos2 bar: 100%', '\x1b[A\x1b[A\n\n\x1b[A\x1b[A\n\rpos1 bar: 100%', '\x1b[A\n\x1b[A\rpos0 bar: 100%', '\n'] if res != exres: raise AssertionError("\nExpected:\n{0}\nGot:\n{1}\nRaw:\n{2}\n".format( str(exres), str(res), str([out]))) # Test manual tqdm positioning our_file = StringIO() t1 = tqdm(total=2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos0 bar', position=0) t2 = tqdm(total=2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos1 bar', position=1) t3 = tqdm(total=2, file=our_file, miniters=1, mininterval=0, maxinterval=0, desc='pos2 bar', position=2) for _ in _range(2): t1.update() t3.update() t2.update() our_file.seek(0) out = our_file.read() res = [m[0] for m in RE_pos.findall(out)] exres = ['\rpos0 bar: 0%', '\n\rpos1 bar: 0%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\rpos0 bar: 50%', '\n\n\rpos2 bar: 50%', '\x1b[A\x1b[A\n\rpos1 bar: 50%', '\x1b[A\rpos0 bar: 100%', '\n\n\rpos2 bar: 100%', '\x1b[A\x1b[A\n\rpos1 bar: 100%', '\x1b[A'] if res != exres: raise AssertionError("\nExpected:\n{0}\nGot:\n{1}\nRaw:\n{2}\n".format( str(exres), str(res), str([out]))) t1.close() t2.close() t3.close() # Test auto repositionning of bars when a bar is prematurely closed # tqdm._instances.clear() # reset number of instances with closing(StringIO()) as our_file: t1 = tqdm(total=10, file=our_file, desc='pos0 bar', mininterval=0) t2 = tqdm(total=10, file=our_file, desc='pos1 bar', mininterval=0) t3 = tqdm(total=10, file=our_file, desc='pos2 bar', mininterval=0) res = [m[0] for m in RE_pos.findall(our_file.getvalue())] exres = ['\rpos0 bar: 0%', '\n\rpos1 bar: 0%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A'] if res != exres: raise AssertionError( "\nExpected:\n{0}\nGot:\n{1}\n".format( str(exres), str(res))) t2.close() t4 = tqdm(total=10, file=our_file, desc='pos3 bar', mininterval=0) t1.update(1) t3.update(1) t4.update(1) res = [m[0] for m in RE_pos.findall(our_file.getvalue())] exres = ['\rpos0 bar: 0%', '\n\rpos1 bar: 0%', '\x1b[A\n\n\rpos2 bar: 0%', '\x1b[A\x1b[A\n\x1b[A\n\n\rpos3 bar: 0%', '\x1b[A\x1b[A\rpos0 bar: 10%', '\n\rpos2 bar: 10%', '\x1b[A\n\n\rpos3 bar: 10%', '\x1b[A\x1b[A'] if res != exres: raise AssertionError( "\nExpected:\n{0}\nGot:\n{1}\n".format( str(exres), str(res))) t4.close() t3.close() t1.close() @with_setup(pretest, posttest) def test_set_description(): """ Test set description """ with closing(StringIO()) as our_file: with tqdm(desc='Hello', file=our_file) as t: assert t.desc == 'Hello: ' t.set_description('World') assert t.desc == 'World: ' t.set_description() assert t.desc == '' @with_setup(pretest, posttest) def test_deprecated_gui(): """ Test internal GUI properties """ # Check: StatusPrinter iff gui is disabled with closing(StringIO()) as our_file: t = tqdm(total=2, gui=True, file=our_file, miniters=1, mininterval=0) assert not hasattr(t, "sp") try: t.update(1) except TqdmDeprecationWarning as e: if 'Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)`' \ not in our_file.getvalue(): raise else: raise DeprecationError('Should not allow manual gui=True without' ' overriding __iter__() and update()') finally: t._instances.clear() # t.close() # len(tqdm._instances) += 1 # undo the close() decrement t = tqdm(_range(3), gui=True, file=our_file, miniters=1, mininterval=0) try: for _ in t: pass except TqdmDeprecationWarning as e: if 'Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)`' \ not in our_file.getvalue(): raise e else: raise DeprecationError('Should not allow manual gui=True without' ' overriding __iter__() and update()') finally: t._instances.clear() # t.close() # len(tqdm._instances) += 1 # undo the close() decrement with tqdm(total=1, gui=False, file=our_file) as t: assert hasattr(t, "sp") @with_setup(pretest, posttest) def test_cmp(): """ Test comparison functions """ with closing(StringIO()) as our_file: t0 = tqdm(total=10, file=our_file) t1 = tqdm(total=10, file=our_file) t2 = tqdm(total=10, file=our_file) assert t0 < t1 assert t2 >= t0 assert t0 <= t2 t3 = tqdm(total=10, file=our_file) t4 = tqdm(total=10, file=our_file) t5 = tqdm(total=10, file=our_file) t5.close() t6 = tqdm(total=10, file=our_file) assert t3 != t4 assert t3 > t2 assert t5 == t6 t6.close() t4.close() t3.close() t2.close() t1.close() t0.close() @with_setup(pretest, posttest) def test_repr(): """ Test representation """ with closing(StringIO()) as our_file: with tqdm(total=10, ascii=True, file=our_file) as t: assert str(t) == ' 0%| | 0/10 [00:00= timeend and t.miniters == 1): timer.sleep(1) # Force monitor to wake up if it woken too soon sleep(0.000001) # sleep to allow interrupt (instead of pass) assert t.miniters == 1 # check that monitor corrected miniters # Note: at this point, there may be a race condition: monitor saved # current woken time but timer.sleep() happen just before monitor # sleep. To fix that, either sleep here or increase time in a loop # to ensure that monitor wakes up at some point. # Try again but already at miniters = 1 so nothing will be done timer.sleep(maxinterval*2) t.update(2) timeend = timer.time() while not (t.monitor.woken >= timeend): timer.sleep(1) # Force monitor to wake up if it woken too soon sleep(0.000001) # Wait for the monitor to get out of sleep's loop and update tqdm.. assert t.miniters == 1 # check that monitor corrected miniters # 3- Check that class var monitor is deleted if no instance left assert tqdm.monitor is None # 4- Test on multiple bars, one not needing miniters adjustment total = 1000 # Setup a discrete timer timer = DiscreteTimer() # And a fake sleeper sleeper = FakeSleep(timer) # Setup TMonitor to use the timer TMonitor._time = timer.time TMonitor._sleep = sleeper.sleep with closing(StringIO()) as our_file: with tqdm(total=total, file=our_file, miniters=500, mininterval=0.1, maxinterval=maxinterval) as t1: # Set high maxinterval for t2 so monitor does not need to adjust it with tqdm(total=total, file=our_file, miniters=500, mininterval=0.1, maxinterval=1E5) as t2: cpu_timify(t1, timer) cpu_timify(t2, timer) # Do a lot of iterations in a small timeframe timer.sleep(5) t1.update(500) t2.update(500) assert t1.miniters == 500 assert t2.miniters == 500 # Then do 1 it after monitor interval, so that monitor kicks in timer.sleep(maxinterval*2) t1.update(1) t2.update(1) # Wait for the monitor to get out of sleep and update tqdm timeend = timer.time() while not (t.monitor.woken >= timeend and t1.miniters == 1): timer.sleep(1) sleep(0.000001) assert t1.miniters == 1 # check that monitor corrected miniters assert t2.miniters == 500 # check that t2 was not adjusted @with_setup(pretest, posttest) def test_postfix(): """Test postfix""" postfix = {'float': 0.321034, 'gen': 543, 'str': 'h', 'lst': [2]} postfix_order = (('w', 'w'), ('a', 0)) # no need for OrderedDict expected = ['float=0.321', 'gen=543', 'lst=[2]', 'str=h'] expected_order = ['w=w', 'a=0', 'float=0.321', 'gen=543', 'lst=[2]', 'str=h'] # Test postfix set at init with closing(StringIO()) as our_file: with tqdm(total=10, file=our_file, desc='pos0 bar', bar_format='{r_bar}', postfix=postfix) as t1: t1.refresh() out = our_file.getvalue() # Test postfix set after init with closing(StringIO()) as our_file: with trange(10, file=our_file, desc='pos1 bar', bar_format='{r_bar}', postfix=None) as t2: t2.set_postfix(**postfix) t2.refresh() out2 = our_file.getvalue() # Order of items in dict may change, so need a loop to check per item for res in expected: assert res in out assert res in out2 # Test postfix set after init and with ordered dict with closing(StringIO()) as our_file: with trange(10, file=our_file, desc='pos2 bar', bar_format='{r_bar}', postfix=None) as t3: t3.set_postfix(postfix_order, **postfix) t3.refresh() out3 = our_file.getvalue() out3 = out3[1:-1].split(', ')[3:] assert out3 == expected_order tqdm-4.11.2/tqdm/tests/tests_main.py0000664000175000017500000000542213040270166020151 0ustar cr45hincr45hin00000000000000import sys import subprocess from tqdm import main, TqdmKeyError, TqdmTypeError from tests_tqdm import with_setup, pretest, posttest, _range, closing, UnicodeIO def _sh(*cmd, **kwargs): return subprocess.Popen(cmd, stdout=subprocess.PIPE, **kwargs).communicate()[0].decode('utf-8') # WARNING: this should be the last test as it messes with sys.stdin, argv @with_setup(pretest, posttest) def test_main(): """ Test command line pipes """ ls_out = _sh('ls').replace('\r\n', '\n') ls = subprocess.Popen(('ls'), stdout=subprocess.PIPE, stderr=subprocess.STDOUT) res = _sh(sys.executable, '-c', 'from tqdm import main; main()', stdin=ls.stdout, stderr=subprocess.STDOUT) ls.wait() # actual test: assert (ls_out in res.replace('\r\n', '\n')) # semi-fake test which gets coverage: _SYS = sys.stdin, sys.argv with closing(UnicodeIO()) as sys.stdin: sys.argv = ['', '--desc', 'Test CLI delims', '--ascii', 'True', '--delim', r'\0', '--buf_size', '64'] sys.stdin.write('\0'.join(map(str, _range(int(1e3))))) sys.stdin.seek(0) main() IN_DATA_LIST = map(str, _range(int(1e3))) sys.stdin = IN_DATA_LIST sys.argv = ['', '--desc', 'Test CLI pipes', '--ascii', 'True', '--unit_scale', 'True'] import tqdm.__main__ # NOQA IN_DATA = '\0'.join(IN_DATA_LIST) with closing(UnicodeIO()) as sys.stdin: sys.stdin.write(IN_DATA) sys.stdin.seek(0) sys.argv = ['', '--ascii', '--bytes'] with closing(UnicodeIO()) as fp: main(fp=fp) assert (str(len(IN_DATA)) in fp.getvalue()) sys.stdin = IN_DATA_LIST sys.argv = ['', '-ascii', '--unit_scale', 'False', '--desc', 'Test CLI errors'] main() sys.argv = ['', '-ascii', '-unit_scale', '--bad_arg_u_ment', 'foo'] try: main() except TqdmKeyError as e: if 'bad_arg_u_ment' not in str(e): raise else: raise TqdmKeyError('bad_arg_u_ment') sys.argv = ['', '-ascii', '-unit_scale', 'invalid_bool_value'] try: main() except TqdmTypeError as e: if 'invalid_bool_value' not in str(e): raise else: raise TqdmTypeError('invalid_bool_value') sys.argv = ['', '-ascii', '--total', 'invalid_int_value'] try: main() except TqdmTypeError as e: if 'invalid_int_value' not in str(e): raise else: raise TqdmTypeError('invalid_int_value') for i in ('-h', '--help', '-v', '--version'): sys.argv = ['', i] try: main() except SystemExit: pass # clean up sys.stdin, sys.argv = _SYS tqdm-4.11.2/tqdm/tests/tests_perf.py0000664000175000017500000002324413041513621020160 0ustar cr45hincr45hin00000000000000from __future__ import print_function, division from nose.plugins.skip import SkipTest from contextlib import contextmanager import sys from time import sleep, time from tqdm import trange from tqdm import tqdm from tests_tqdm import with_setup, pretest, posttest, StringIO, closing, _range # Use relative/cpu timer to have reliable timings when there is a sudden load try: from time import process_time except ImportError: from time import clock process_time = clock def get_relative_time(prevtime=0): return process_time() - prevtime def cpu_sleep(t): '''Sleep the given amount of cpu time''' start = process_time() while((process_time() - start) < t): pass def checkCpuTime(sleeptime=0.2): '''Check if cpu time works correctly''' if checkCpuTime.passed: return True # First test that sleeping does not consume cputime start1 = process_time() sleep(sleeptime) t1 = process_time() - start1 # secondly check by comparing to cpusleep (where we actually do something) start2 = process_time() cpu_sleep(sleeptime) t2 = process_time() - start2 if (abs(t1) < 0.0001 and (t1 < t2 / 10)): return True raise SkipTest checkCpuTime.passed = False @contextmanager def relative_timer(): start = process_time() def elapser(): return process_time() - start yield lambda: elapser() spent = process_time() - start def elapser(): # NOQA return spent def retry_on_except(n=3): def wrapper(fn): def test_inner(): for i in range(1, n + 1): try: checkCpuTime() fn() except SkipTest: if i >= n: raise else: return test_inner.__doc__ = fn.__doc__ return test_inner return wrapper class MockIO(StringIO): """ Wraps StringIO to mock a file with no I/O """ def write(self, data): return def simple_progress(iterable=None, total=None, file=sys.stdout, desc='', leave=False, miniters=1, mininterval=0.1, width=60): """ Simple progress bar reproducing tqdm's major features """ n = [0] # use a closure start_t = [time()] last_n = [0] last_t = [0] if iterable is not None: total = len(iterable) def format_interval(t): mins, s = divmod(int(t), 60) h, m = divmod(mins, 60) if h: return '{0:d}:{1:02d}:{2:02d}'.format(h, m, s) else: return '{0:02d}:{1:02d}'.format(m, s) def update_and_print(i=1): n[0] += i if (n[0] - last_n[0]) >= miniters: last_n[0] = n[0] if (time() - last_t[0]) >= mininterval: last_t[0] = time() # last_t[0] == current time spent = last_t[0] - start_t[0] spent_fmt = format_interval(spent) rate = n[0] / spent if spent > 0 else 0 if 0.0 < rate < 1.0: rate_fmt = "%.2fs/it" % (1.0 / rate) else: rate_fmt = "%.2fit/s" % rate frac = n[0] / total percentage = int(frac * 100) eta = (total - n[0]) / rate if rate > 0 else 0 eta_fmt = format_interval(eta) bar = "#" * int(frac * width) barfill = " " * int((1.0 - frac) * width) bar_length, frac_bar_length = divmod( int(frac * width * 10), 10) bar = '#' * bar_length frac_bar = chr(48 + frac_bar_length) if frac_bar_length \ else ' ' file.write("\r%s %i%%|%s%s%s| %i/%i [%s<%s, %s]" % (desc, percentage, bar, frac_bar, barfill, n[0], total, spent_fmt, eta_fmt, rate_fmt)) if n[0] == total and leave: file.write("\n") file.flush() def update_and_yield(): for elt in iterable: yield elt update_and_print() update_and_print(0) if iterable is not None: return update_and_yield() else: return update_and_print @with_setup(pretest, posttest) @retry_on_except() def test_iter_overhead(): """ Test overhead of iteration based tqdm """ total = int(1e6) with closing(MockIO()) as our_file: a = 0 with relative_timer() as time_tqdm: for i in trange(total, file=our_file): a += i assert(a == (total * total - total) / 2.0) a = 0 with relative_timer() as time_bench: for i in _range(total): a += i our_file.write(a) # Compute relative overhead of tqdm against native range() if time_tqdm() > 9 * time_bench(): raise AssertionError('trange(%g): %f, range(%g): %f' % (total, time_tqdm(), total, time_bench())) @with_setup(pretest, posttest) @retry_on_except() def test_manual_overhead(): """ Test overhead of manual tqdm """ total = int(1e6) with closing(MockIO()) as our_file: with tqdm(total=total * 10, file=our_file, leave=True) as t: a = 0 with relative_timer() as time_tqdm: for i in _range(total): a += i t.update(10) a = 0 with relative_timer() as time_bench: for i in _range(total): a += i our_file.write(a) # Compute relative overhead of tqdm against native range() if time_tqdm() > 10 * time_bench(): raise AssertionError('tqdm(%g): %f, range(%g): %f' % (total, time_tqdm(), total, time_bench())) @with_setup(pretest, posttest) @retry_on_except() def test_iter_overhead_hard(): """ Test overhead of iteration based tqdm (hard) """ total = int(1e5) with closing(MockIO()) as our_file: a = 0 with relative_timer() as time_tqdm: for i in trange(total, file=our_file, leave=True, miniters=1, mininterval=0, maxinterval=0): a += i assert(a == (total * total - total) / 2.0) a = 0 with relative_timer() as time_bench: for i in _range(total): a += i our_file.write(("%i" % a) * 40) # Compute relative overhead of tqdm against native range() try: assert(time_tqdm() < 60 * time_bench()) except AssertionError: raise AssertionError('trange(%g): %f, range(%g): %f' % (total, time_tqdm(), total, time_bench())) @with_setup(pretest, posttest) @retry_on_except() def test_manual_overhead_hard(): """ Test overhead of manual tqdm (hard) """ total = int(1e5) with closing(MockIO()) as our_file: t = tqdm(total=total * 10, file=our_file, leave=True, miniters=1, mininterval=0, maxinterval=0) a = 0 with relative_timer() as time_tqdm: for i in _range(total): a += i t.update(10) a = 0 with relative_timer() as time_bench: for i in _range(total): a += i our_file.write(("%i" % a) * 40) # Compute relative overhead of tqdm against native range() try: assert(time_tqdm() < 100 * time_bench()) except AssertionError: raise AssertionError('tqdm(%g): %f, range(%g): %f' % (total, time_tqdm(), total, time_bench())) @with_setup(pretest, posttest) @retry_on_except() def test_iter_overhead_simplebar_hard(): """ Test overhead of iteration based tqdm vs simple progress bar (hard) """ total = int(1e4) with closing(MockIO()) as our_file: a = 0 with relative_timer() as time_tqdm: for i in trange(total, file=our_file, leave=True, miniters=1, mininterval=0, maxinterval=0): a += i assert(a == (total * total - total) / 2.0) a = 0 with relative_timer() as time_bench: for i in simple_progress(_range(total), file=our_file, leave=True, miniters=1, mininterval=0): a += i # Compute relative overhead of tqdm against native range() try: assert(time_tqdm() < 2 * time_bench()) except AssertionError: raise AssertionError('trange(%g): %f, simple_progress(%g): %f' % (total, time_tqdm(), total, time_bench())) @with_setup(pretest, posttest) @retry_on_except() def test_manual_overhead_simplebar_hard(): """ Test overhead of manual tqdm vs simple progress bar (hard) """ total = int(1e4) with closing(MockIO()) as our_file: t = tqdm(total=total * 10, file=our_file, leave=True, miniters=1, mininterval=0, maxinterval=0) a = 0 with relative_timer() as time_tqdm: for i in _range(total): a += i t.update(10) simplebar_update = simple_progress(total=total, file=our_file, leave=True, miniters=1, mininterval=0) a = 0 with relative_timer() as time_bench: for i in _range(total): a += i simplebar_update(10) # Compute relative overhead of tqdm against native range() try: assert(time_tqdm() < 2 * time_bench()) except AssertionError: raise AssertionError('tqdm(%g): %f, simple_progress(%g): %f' % (total, time_tqdm(), total, time_bench())) tqdm-4.11.2/tqdm/tests/tests_version.py0000664000175000017500000000055213041513621020706 0ustar cr45hincr45hin00000000000000import re def test_version(): """ Test version string """ from tqdm import __version__ version_parts = re.split('[.-]', __version__) assert 3 <= len(version_parts) # must have at least Major.minor.patch try: map(int, version_parts[:3]) except ValueError: raise TypeError('Version Major.minor.patch must be 3 integers') tqdm-4.11.2/tqdm/_tqdm_pandas.py0000664000175000017500000000311013040270166017263 0ustar cr45hincr45hin00000000000000import sys __author__ = "github.com/casperdcl" __all__ = ['tqdm_pandas'] def tqdm_pandas(tclass, *targs, **tkwargs): """ Registers the given `tqdm` instance with `pandas.core.groupby.DataFrameGroupBy.progress_apply`. It will even close() the `tqdm` instance upon completion. Parameters ---------- tclass : tqdm class you want to use (eg, tqdm, tqdm_notebook, etc) targs and tkwargs : arguments for the tqdm instance Examples -------- >>> import pandas as pd >>> import numpy as np >>> from tqdm import tqdm, tqdm_pandas >>> >>> df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) >>> tqdm_pandas(tqdm, leave=True) # can use tqdm_gui, optional kwargs, etc >>> # Now you can use `progress_apply` instead of `apply` >>> df.groupby(0).progress_apply(lambda x: x**2) References ---------- https://stackoverflow.com/questions/18603270/ progress-indicator-during-pandas-operations-python """ from tqdm import TqdmDeprecationWarning if isinstance(tclass, type) or (getattr(tclass, '__name__', '').startswith( 'tqdm_')): # delayed adapter case TqdmDeprecationWarning("""\ Please use `tqdm.pandas(...)` instead of `tqdm_pandas(tqdm, ...)`. """, fp_write=getattr(tkwargs.get('file', None), 'write', sys.stderr.write)) tclass.pandas(*targs, **tkwargs) else: TqdmDeprecationWarning("""\ Please use `tqdm.pandas(...)` instead of `tqdm_pandas(tqdm(...))`. """, fp_write=getattr(tclass.fp, 'write', sys.stderr.write)) type(tclass).pandas(deprecated_t=tclass) tqdm-4.11.2/tqdm/__init__.py0000664000175000017500000000175013040270166016400 0ustar cr45hincr45hin00000000000000from ._tqdm import tqdm from ._tqdm import trange from ._tqdm_gui import tqdm_gui from ._tqdm_gui import tgrange from ._tqdm_pandas import tqdm_pandas from ._main import main from ._version import __version__ # NOQA from ._tqdm import TqdmTypeError, TqdmKeyError, TqdmDeprecationWarning __all__ = ['tqdm', 'tqdm_gui', 'trange', 'tgrange', 'tqdm_pandas', 'tqdm_notebook', 'tnrange', 'main', 'TqdmTypeError', 'TqdmKeyError', 'TqdmDeprecationWarning', '__version__'] def tqdm_notebook(*args, **kwargs): # pragma: no cover """See tqdm._tqdm_notebook.tqdm_notebook for full documentation""" from ._tqdm_notebook import tqdm_notebook as _tqdm_notebook return _tqdm_notebook(*args, **kwargs) def tnrange(*args, **kwargs): # pragma: no cover """ A shortcut for tqdm_notebook(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ from ._tqdm_notebook import tnrange as _tnrange return _tnrange(*args, **kwargs) tqdm-4.11.2/tqdm/_utils.py0000664000175000017500000001417113041513621016136 0ustar cr45hincr45hin00000000000000import os import subprocess from platform import system as _curos CUR_OS = _curos() IS_WIN = CUR_OS in ['Windows', 'cli'] IS_NIX = (not IS_WIN) and any( CUR_OS.startswith(i) for i in ['CYGWIN', 'MSYS', 'Linux', 'Darwin', 'SunOS', 'FreeBSD']) # Py2/3 compat. Empty conditional to avoid coverage if True: # pragma: no cover try: _range = xrange except NameError: _range = range try: _unich = unichr except NameError: _unich = chr try: _unicode = unicode except NameError: _unicode = str try: if IS_WIN: import colorama colorama.init() else: colorama = None except ImportError: colorama = None try: from weakref import WeakSet except ImportError: WeakSet = set try: _basestring = basestring except NameError: _basestring = str try: # py>=2.7,>=3.1 from collections import OrderedDict as _OrderedDict except ImportError: try: # older Python versions with backported ordereddict lib from ordereddict import OrderedDict as _OrderedDict except ImportError: # older Python versions without ordereddict lib # Py2.6,3.0 compat, from PEP 372 from collections import MutableMapping class _OrderedDict(dict, MutableMapping): # Methods with direct access to underlying attributes def __init__(self, *args, **kwds): if len(args) > 1: raise TypeError('expected at 1 argument, got %d', len(args)) if not hasattr(self, '_keys'): self._keys = [] self.update(*args, **kwds) def clear(self): del self._keys[:] dict.clear(self) def __setitem__(self, key, value): if key not in self: self._keys.append(key) dict.__setitem__(self, key, value) def __delitem__(self, key): dict.__delitem__(self, key) self._keys.remove(key) def __iter__(self): return iter(self._keys) def __reversed__(self): return reversed(self._keys) def popitem(self): if not self: raise KeyError key = self._keys.pop() value = dict.pop(self, key) return key, value def __reduce__(self): items = [[k, self[k]] for k in self] inst_dict = vars(self).copy() inst_dict.pop('_keys', None) return (self.__class__, (items,), inst_dict) # Methods with indirect access via the above methods setdefault = MutableMapping.setdefault update = MutableMapping.update pop = MutableMapping.pop keys = MutableMapping.keys values = MutableMapping.values items = MutableMapping.items def __repr__(self): pairs = ', '.join(map('%r: %r'.__mod__, self.items())) return '%s({%s})' % (self.__class__.__name__, pairs) def copy(self): return self.__class__(self) @classmethod def fromkeys(cls, iterable, value=None): d = cls() for key in iterable: d[key] = value return d def _is_utf(encoding): return encoding.lower().startswith('utf-') or ('U8' == encoding) def _supports_unicode(file): return _is_utf(file.encoding) if ( getattr(file, 'encoding', None) or # FakeStreams from things like bpython-curses can lie getattr(file, 'interface', None)) else False # pragma: no cover def _environ_cols_wrapper(): # pragma: no cover """ Return a function which gets width and height of console (linux,osx,windows,cygwin). """ _environ_cols = None if IS_WIN: _environ_cols = _environ_cols_windows if _environ_cols is None: _environ_cols = _environ_cols_tput if IS_NIX: _environ_cols = _environ_cols_linux return _environ_cols def _environ_cols_windows(fp): # pragma: no cover try: from ctypes import windll, create_string_buffer import struct from sys import stdin, stdout io_handle = None if fp == stdin: io_handle = -10 elif fp == stdout: io_handle = -11 else: # assume stderr io_handle = -12 h = windll.kernel32.GetStdHandle(io_handle) csbi = create_string_buffer(22) res = windll.kernel32.GetConsoleScreenBufferInfo(h, csbi) if res: (_bufx, _bufy, _curx, _cury, _wattr, left, _top, right, _bottom, _maxx, _maxy) = struct.unpack("hhhhHhhhhhh", csbi.raw) # nlines = bottom - top + 1 return right - left # +1 except: pass return None def _environ_cols_tput(*args): # pragma: no cover """ cygwin xterm (windows) """ try: import shlex cols = int(subprocess.check_call(shlex.split('tput cols'))) # rows = int(subprocess.check_call(shlex.split('tput lines'))) return cols except: pass return None def _environ_cols_linux(fp): # pragma: no cover try: from termios import TIOCGWINSZ from fcntl import ioctl from array import array except ImportError: return None else: try: return array('h', ioctl(fp, TIOCGWINSZ, '\0' * 8))[1] except: try: from os.environ import get except ImportError: return None else: return int(get('COLUMNS', 1)) - 1 def _term_move_up(): # pragma: no cover return '' if (os.name == 'nt') and (colorama is None) else '\x1b[A' tqdm-4.11.2/README.rst0000664000175000017500000006427613041134651015024 0ustar cr45hincr45hin00000000000000|Logo| tqdm ==== |PyPI-Status| |PyPI-Versions| |Conda-Forge-Status| |Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |DOI-URI| |LICENCE| ``tqdm`` means "progress" in Arabic (taqadum, تقدّم) and an abbreviation for "I love you so much" in Spanish (te quiero demasiado). Instantly make your loops show a smart progress meter - just wrap any iterable with ``tqdm(iterable)``, and you're done! .. code:: python from tqdm import tqdm for i in tqdm(range(10000)): ... ``76%|████████████████████████████         | 7568/10000 [00:33<00:10, 229.00it/s]`` ``trange(N)`` can be also used as a convenient shortcut for ``tqdm(xrange(N))``. |Screenshot| REPL: `ptpython `__ It can also be executed as a module with pipes: .. code:: sh $ seq 9999999 | tqdm --unit_scale | wc -l 10.0Mit [00:02, 3.58Mit/s] 9999999 $ 7z a -bd -r backup.7z docs/ | grep Compressing | \ tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] Overhead is low -- about 60ns per iteration (80ns with ``tqdm_gui``), and is unit tested against performance regression. By comparison, the well established `ProgressBar `__ has an 800ns/iter overhead. In addition to its low overhead, ``tqdm`` uses smart algorithms to predict the remaining time and to skip unnecessary iteration displays, which allows for a negligible overhead in most cases. ``tqdm`` works on any platform (Linux, Windows, Mac, FreeBSD, Solaris/SunOS), in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks. ``tqdm`` does not require any library (not even curses!) to run, just a vanilla Python interpreter will do and an environment supporting ``carriage return \r`` and ``line feed \n`` control characters. ------------------------------------------ .. contents:: Table of contents :backlinks: top :local: Installation ------------ Latest PyPI stable release ~~~~~~~~~~~~~~~~~~~~~~~~~~ |PyPI-Status| .. code:: sh pip install tqdm Latest development release on github ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |GitHub-Status| |GitHub-Stars| |GitHub-Forks| Pull and install in the current directory: .. code:: sh pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm Latest Conda release ~~~~~~~~~~~~~~~~~~~~ |Conda-Forge-Status| .. code:: sh conda install -c conda-forge tqdm Changelog --------- The list of all changes is available either on GitHub's Releases: |GitHub-Status| or on crawlers such as `allmychanges.com `_. Usage ----- ``tqdm`` is very versatile and can be used in a number of ways. The three main ones are given below. Iterable-based ~~~~~~~~~~~~~~ Wrap ``tqdm()`` around any iterable: .. code:: python text = "" for char in tqdm(["a", "b", "c", "d"]): text = text + char ``trange(i)`` is a special optimised instance of ``tqdm(range(i))``: .. code:: python for i in trange(100): pass Instantiation outside of the loop allows for manual control over ``tqdm()``: .. code:: python pbar = tqdm(["a", "b", "c", "d"]) for char in pbar: pbar.set_description("Processing %s" % char) Manual ~~~~~~ Manual control on ``tqdm()`` updates by using a ``with`` statement: .. code:: python with tqdm(total=100) as pbar: for i in range(10): pbar.update(10) If the optional variable ``total`` (or an iterable with ``len()``) is provided, predictive stats are displayed. ``with`` is also optional (you can just assign ``tqdm()`` to a variable, but in this case don't forget to ``del`` or ``close()`` at the end: .. code:: python pbar = tqdm(total=100) for i in range(10): pbar.update(10) pbar.close() Module ~~~~~~ Perhaps the most wonderful use of ``tqdm`` is in a script or on the command line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass through all ``stdin`` to ``stdout`` while printing progress to ``stderr``. The example below demonstrated counting the number of lines in all python files in the current directory, with timing information included. .. code:: sh $ time find . -name '*.py' -exec cat \{} \; | wc -l 857365 real 0m3.458s user 0m0.274s sys 0m3.325s $ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 857366it [00:03, 246471.31it/s] 857365 real 0m3.585s user 0m0.862s sys 0m3.358s Note that the usual arguments for ``tqdm`` can also be specified. .. code:: sh $ find . -name '*.py' -exec cat \{} \; | tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|███████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s] Backing up a large directory? .. code:: sh $ 7z a -bd -r backup.7z docs/ | grep Compressing | tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] FAQ and Known Issues -------------------- The most common issues relate to excessive output on multiple lines, instead of a neat one-line progress bar. - Consoles in general: require support for carriage return (``CR``, ``\r``). - Nested progress bars: * Consoles in general: require support for moving cursors up to the previous line. For example, `IDLE `__, `ConEmu `__ and `PyCharm `__ (also `here `__ and `here `__) lack full support. * Windows: additionally may require the python module ``colorama``. - Wrapping enumerated iterables: use ``enumerate(tqdm(...))`` instead of ``tqdm(enumerate(...))`. The same applies to ``numpy.ndenumerate``. This is because enumerate functions tend to hide the length of iterables. ``tqdm`` does not. - Wrapping zipped iterables has similar issues due to internal optimisations. ``tqdm(zip(a, b))`` should be replaced with ``zip(tqdm(a), b)`` or even ``zip(tqdm(a), tqdm(b))`` If you come across any other difficulties, browse/open issues `here `__. Documentation ------------- |PyPI-Versions| |README-Hits| (Since 19 May 2016) .. code:: python class tqdm(object): """ Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested. """ def __init__(self, iterable=None, desc=None, total=None, leave=True, file=sys.stderr, ncols=None, mininterval=0.1, maxinterval=10.0, miniters=None, ascii=None, disable=False, unit='it', unit_scale=False, dynamic_ncols=False, smoothing=0.3, bar_format=None, initial=0, position=None, postfix=None): Parameters ~~~~~~~~~~ * iterable : iterable, optional Iterable to decorate with a progressbar. Leave blank to manually manage the updates. * desc : str, optional Prefix for the progressbar. * total : int, optional The number of expected iterations. If (default: None), len(iterable) is used if possible. As a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If ``gui`` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int(9e9). * leave : bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. * file : ``io.TextIOWrapper`` or ``io.StringIO``, optional Specifies where to output the progress messages [default: sys.stderr]. Uses ``file.write(str)`` and ``file.flush()`` methods. * ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). * mininterval : float, optional Minimum progress display update interval, in seconds [default: 0.1]. * maxinterval : float, optional Maximum progress display update interval, in seconds [default: 10]. Automatically adjusts ``miniters`` to correspond to ``mininterval`` after long display update lag. Only works if ``dynamic_miniters`` or monitor thread is enabled. * miniters : int, optional Minimum progress display update interval, in iterations. If 0 and ``dynamic_miniters``, will automatically adjust to equal ``mininterval`` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and ``mininterval`` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. * ascii : bool, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters ``1-9 #``. * disable : bool, optional Whether to disable the entire progressbar wrapper [default: False]. * unit : str, optional String that will be used to define the unit of each iteration [default: it]. * unit_scale : bool, optional If set, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. * dynamic_ncols : bool, optional If set, constantly alters ``ncols`` to the environment (allowing for window resizes) [default: False]. * smoothing : float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. * bar_format : str, optional Specify a custom bar string formatting. May impact performance. If unspecified, will use '{l_bar}{bar}{r_bar}', where l_bar is '{desc}{percentage:3.0f}%|' and r_bar is '| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. * initial : int, optional The initial counter value. Useful when restarting a progress bar [default: 0]. * position : int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). * postfix : dict, optional Specify additional stats to display at the end of the bar. Extra CLI Options ~~~~~~~~~~~~~~~~~ * delim : chr, optional Delimiting character [default: '\n']. Use '\0' for null. N.B.: on Windows systems, Python converts '\n' to '\r\n'. * buf_size : int, optional String buffer size in bytes [default: 256] used when ``delim`` is specified. * bytes : bool, optional If true, will count bytes and ignore ``delim``. Returns ~~~~~~~ * out : decorated iterator. .. code:: python def update(self, n=1): """ Manually update the progress bar, useful for streams such as reading files. E.g.: >>> t = tqdm(total=filesize) # Initialise >>> for current_buffer in stream: ... ... ... t.update(len(current_buffer)) >>> t.close() The last line is highly recommended, but possibly not necessary if ``t.update()`` will be called in such a way that ``filesize`` will be exactly reached and printed. Parameters ---------- n : int Increment to add to the internal counter of iterations [default: 1]. """ def close(self): """ Cleanup and (if leave=False) close the progressbar. """ def clear(self): """ Clear current bar display """ def refresh(self): """ Force refresh the display of this bar """ def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm (without overlap with bars) """ def set_description(self, desc=None): """ Set/modify description of the progress bar. """ def set_postfix(self, **kwargs): """ Set/modify postfix (additional stats) with automatic formatting based on datatype. """ def trange(*args, **kwargs): """ A shortcut for tqdm(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ class tqdm_gui(tqdm): """ Experimental GUI version of tqdm! """ def tgrange(*args, **kwargs): """ Experimental GUI version of trange! """ class tqdm_notebook(tqdm): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ def tnrange(*args, **kwargs): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ Examples and Advanced Usage --------------------------- - See the `examples `__ folder; - import the module and run ``help()``, or - consult the `wiki `__. - this has an `excellent article `__ on how to make a **great** progressbar. Description and additional stats ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Custom information can be displayed and updated dynamically on ``tqdm`` bars with the ``desc`` and ``postfix`` arguments: .. code:: python from tqdm import trange from random import random, randint from time import sleep t = trange(100) for i in t: # Description will be displayed on the left t.set_description('GEN %i' % i) # Postfix will be displayed on the right, and will format automatically # based on argument's datatype t.set_postfix(loss=random(), gen=randint(1,999), str='h', lst=[1, 2]) sleep(0.1) Nested progress bars ~~~~~~~~~~~~~~~~~~~~ ``tqdm`` supports nested progress bars. Here's an example: .. code:: python from tqdm import trange from time import sleep for i in trange(10, desc='1st loop'): for j in trange(5, desc='2nd loop', leave=False): for k in trange(100, desc='3nd loop'): sleep(0.01) On Windows `colorama `__ will be used if available to produce a beautiful nested display. For manual control over positioning (e.g. for multi-threaded use), you may specify `position=n` where `n=0` for the outermost bar, `n=1` for the next, and so on. Hooks and callbacks ~~~~~~~~~~~~~~~~~~~ ``tqdm`` can easily support callbacks/hooks and manual updates. Here's an example with ``urllib``: **urllib.urlretrieve documentation** | [...] | If present, the hook function will be called once | on establishment of the network connection and once after each block read | thereafter. The hook will be passed three arguments; a count of blocks | transferred so far, a block size in bytes, and the total size of the file. | [...] .. code:: python import urllib from tqdm import tqdm def my_hook(t): """ Wraps tqdm instance. Don't forget to close() or __exit__() the tqdm instance once you're done with it (easiest using `with` syntax). Example ------- >>> with tqdm(...) as t: ... reporthook = my_hook(t) ... urllib.urlretrieve(..., reporthook=reporthook) """ last_b = [0] def inner(b=1, bsize=1, tsize=None): """ b : int, optional Number of blocks just transferred [default: 1]. bsize : int, optional Size of each block (in tqdm units) [default: 1]. tsize : int, optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: t.total = tsize t.update((b - last_b[0]) * bsize) last_b[0] = b return inner eg_link = 'http://www.doc.ic.ac.uk/~cod11/matryoshka.zip' with tqdm(unit='B', unit_scale=True, miniters=1, desc=eg_link.split('/')[-1]) as t: # all optional kwargs urllib.urlretrieve(eg_link, filename='/dev/null', reporthook=my_hook(t), data=None) It is recommend to use ``miniters=1`` whenever there is potentially large differences in iteration speed (e.g. downloading a file over a patchy connection). Pandas Integration ~~~~~~~~~~~~~~~~~~ Due to popular demand we've added support for ``pandas`` -- here's an example for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``: .. code:: python import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.) tqdm.pandas(desc="my bar!") # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2) In case you're interested in how this works (and how to modify it for your own callbacks), see the `examples `__ folder or import the module and run ``help()``. IPython/Jupyter Integration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ IPython/Jupyter is supported via the `tqdm_notebook` submodule: .. code:: python from tqdm import tnrange, tqdm_notebook from time import sleep for i in tnrange(10, desc='1st loop'): for j in tqdm_notebook(xrange(100), desc='2nd loop'): sleep(0.01) In addition to `tqdm` features, the submodule provides a native Jupyter widget (compatible with IPython v1-v4 and Jupyter), fully working nested bars and color hints (blue: normal, green: completed, red: error/interrupt, light blue: no ETA); as demonstrated below. |Screenshot-Jupyter1| |Screenshot-Jupyter2| |Screenshot-Jupyter3| Writing messages ~~~~~~~~~~~~~~~~ Since ``tqdm`` uses a simple printing mechanism to display progress bars, you should not write any message in the terminal using ``print()``. To write messages in the terminal without any collision with ``tqdm`` bar display, a ``.write()`` method is provided: .. code:: python from tqdm import tqdm, trange from time import sleep bar = trange(10) for i in bar: # Print using tqdm class method .write() sleep(0.1) if not (i % 3): tqdm.write("Done task %i" % i) # Can also use bar.write() By default, this will print to standard output ``sys.stdout``. but you can specify any file-like object using the ``file`` argument. For example, this can be used to redirect the messages writing to a log file or class. Redirecting writing ~~~~~~~~~~~~~~~~~~~ If using a library that can print messages to the console, editing the library by replacing ``print()`` with ``tqdm.write()`` may not be desirable. In that case, redirecting ``sys.stdout`` to ``tqdm.write()`` is an option. To redirect ``sys.stdout``, create a file-like class that will write any input string to ``tqdm.write()``, and supply the arguments ``file=sys.stdout, dynamic_ncols=True``. A reusable canonical example is given below: .. code:: python from time import sleep import contextlib import sys from tqdm import tqdm class DummyTqdmFile(object): """Dummy file-like that will write to tqdm""" file = None def __init__(self, file): self.file = file def write(self, x): # Avoid print() second call (useless \n) if len(x.rstrip()) > 0: tqdm.write(x, file=self.file) @contextlib.contextmanager def stdout_redirect_to_tqdm(): save_stdout = sys.stdout try: sys.stdout = DummyTqdmFile(sys.stdout) yield save_stdout # Relay exceptions except Exception as exc: raise exc # Always restore sys.stdout if necessary finally: sys.stdout = save_stdout def blabla(): print("Foo blabla") # Redirect stdout to tqdm.write() (don't forget the `as save_stdout`) with stdout_redirect_to_tqdm() as save_stdout: # tqdm call need to specify sys.stdout, not sys.stderr (default) # and dynamic_ncols=True to autodetect console width for _ in tqdm(range(3), file=save_stdout, dynamic_ncols=True): blabla() sleep(.5) # After the `with`, printing is restored print('Done!') Monitoring thread, intervals and miniters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``tqdm`` implements a few tricks to to increase efficiency and reduce overhead. 1. Avoid unnecessary frequent bar refreshing: ``mininterval`` defines how long to wait between each refresh. ``tqdm`` always gets updated in the background, but it will diplay only every ``mininterval``. 2. Reduce number of calls to check system clock/time. 3. ``mininterval`` is more intuitive to configure than ``miniters``. A clever adjustment system ``dynamic_miniters`` will automatically adjust ``miniters`` to the amount of iterations that fit into time ``mininterval``. Essentially, ``tqdm`` will check if it's time to print without actually checking time. This behavior can be still be bypassed by manually setting ``miniters``. However, consider a case with a combination of fast and slow iterations. After a few fast iterations, ``dynamic_miniters`` will set ``miniters`` to a large number. When interation rate subsequently slows, ``miniters`` will remain large and thus reduce display update frequency. To address this: 4. ``maxinterval`` defines the maximum time between display refreshes. A concurrent monitoring thread checks for overdue updates and forces one where necessary. The monitoring thread should not have a noticeable overhead, and guarantees updates at least every 10 seconds by default. This value can be directly changed by setting the ``monitor_interval`` of any ``tqdm`` instance (i.e. ``t = tqdm.tqdm(...); t.monitor_interval = 2``). The monitor thread may be disabled application-wide by setting ``tqdm.tqdm.monitor_interval = 0`` before instantiatiation of any ``tqdm`` bar. Contributions ------------- All source code is hosted on `GitHub `__. Contributions are welcome. See the `CONTRIBUTING `__ file for more information. LICENCE ------- Open Source (OSI approved): |LICENCE| Citation information: |DOI-URI| Authors ------- Ranked by contributions. - Casper da Costa-Luis (casperdcl) - Stephen Larroque (lrq3000) - Hadrien Mary (hadim) - Noam Yorav-Raphael (noamraph)* - Ivan Ivanov (obiwanus) - Mikhail Korobov (kmike) `*` Original author |README-Hits| (Since 19 May 2016) .. |Logo| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif .. |Screenshot| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm.gif .. |Build-Status| image:: https://travis-ci.org/tqdm/tqdm.svg?branch=master :target: https://travis-ci.org/tqdm/tqdm .. |Coverage-Status| image:: https://coveralls.io/repos/tqdm/tqdm/badge.svg :target: https://coveralls.io/r/tqdm/tqdm .. |Branch-Coverage-Status| image:: https://codecov.io/github/tqdm/tqdm/coverage.svg?branch=master :target: https://codecov.io/github/tqdm/tqdm?branch=master .. |Codacy-Grade| image:: https://api.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177 :target: https://www.codacy.com/app/tqdm/tqdm?utm_source=github.com&utm_medium=referral&utm_content=tqdm/tqdm&utm_campaign=Badge_Grade .. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=2592000 :target: https://github.com/tqdm/tqdm/releases .. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/network .. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/stargazers .. |PyPI-Status| image:: https://img.shields.io/pypi/v/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |Conda-Forge-Status| image:: https://anaconda.org/conda-forge/tqdm/badges/version.svg :target: https://anaconda.org/conda-forge/tqdm .. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg :target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE .. |DOI-URI| image:: https://zenodo.org/badge/21637/tqdm/tqdm.svg :target: https://zenodo.org/badge/latestdoi/21637/tqdm/tqdm .. |Screenshot-Jupyter1| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-1.gif .. |Screenshot-Jupyter2| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-2.gif .. |Screenshot-Jupyter3| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-3.gif .. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif :target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif&style=social tqdm-4.11.2/LICENCE0000664000175000017500000000351213036011052014275 0ustar cr45hincr45hin00000000000000`tqdm` is a product of collaborative work. Unless otherwise stated, all authors (see commit logs) retain copyright for their respective work, and release the work under the MIT licence (text below). Exceptions or notable authors are listed below in reverse chronological order: * MPLv2.0 (text below) 2015-2016 (c) Casper da Costa-Luis [casperdcl](https://github.com/casperdcl). * MIT 2016 (c) [PR #96] on behalf of Google Inc. [PR #96]: https://github.com/tqdm/tqdm/pull/96 Mozilla Public Licence (MPL) v. 2.0 - Exhibit A ----------------------------------------------- This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. MIT License (MIT) ----------------- Copyright (c) 2013 noamraph Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. tqdm-4.11.2/CONTRIBUTING.md0000664000175000017500000001701213041134651015550 0ustar cr45hincr45hin00000000000000HOW TO CONTRIBUTE TO TQDM ========================= This file describes how to - contribute changes to the project, and - upload released to the pypi repository. Most of the management commands have been directly placed inside the Makefile: ``` make [] # on UNIX-like environments python setup.py make [] # if make is unavailable ``` Use the alias `help` (or leave blank) to list all available aliases. HOW TO COMMIT CONTRIBUTIONS --------------------------- Contributions to the project are made using the "Fork & Pull" model. The typical steps would be: 1. create an account on [github](https://github.com) 2. fork [tqdm](https://github.com/tqdm/tqdm) 3. make a local clone: `git clone https://github.com/your_account/tqdm.git` 4. make changes on the local copy 5. test (see below) and commit changes `git commit -a -m "my message"` 6. `push` to your github account: `git push origin` 7. create a Pull Request (PR) from your github fork (go to your fork's webpage and click on "Pull Request." You can then add a message to describe your proposal.) TESTING ------- To test functionality (such as before submitting a Pull Request), there are a number of unit tests. Standard unit tests ~~~~~~~~~~~~~~~~~~~ The standard way to run the tests: - install `tox` - `cd` to the root of the `tqdm` directory (in the same folder as this file) - run the following command: ``` [python setup.py] make test # or: tox --skip-missing-interpreters ``` This will build the module and run the tests in a virtual environment. Errors and coverage rates will be output to the console/log. (Ignore missing interpreters errors - these are due to the local machine missing certain versions of Python.) Note: to install all versions of the Python interpreter that are specified in [tox.ini](https://raw.githubusercontent.com/tqdm/tqdm/master/tox.ini), you can use `MiniConda` to install a minimal setup. You must also make sure that each distribution has an alias to call the Python interpreter: `python27` for Python 2.7's interpreter, `python32` for Python 3.2's, etc. Alternative unit tests with Nose ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Alternatively, use `nose` to run the tests just for the current Python version: - install `nose` and `flake8` - run the following command: ``` [python setup.py] make alltests ``` MANAGE A NEW RELEASE ===================== This section is intended for the project's maintainers and describes how to build and upload a new release. Once again, `[python setup.py] make []` will help. SEMANTIC VERSIONING ------------------- The tqdm repository managers should: - regularly bump the version number in the file [_version.py](https://raw.githubusercontent.com/tqdm/tqdm/master/tqdm/_version.py) - follow the [Semantic Versioning](http://semver.org/) convention - take care of this (instead of users) to avoid PR conflicts solely due to the version file bumping Note: tools can be used to automate this process, such as [bumpversion](https://github.com/peritus/bumpversion) or [python-semanticversion](https://github.com/rbarrois/python-semanticversion/). CHECKING SETUP.PY ----------------- To check that the `setup.py` file is compliant with PyPi requirements (e.g. version number; reStructuredText in README.rst) use: ``` [python setup.py] make testsetup ``` To upload just metadata (including overwriting mistakenly uploaded metadata) to PyPi, use: ``` [python setup.py] make pypimeta ``` MERGING PULL REQUESTS --------------------- This section describes how to cleanly merge PRs. 1 Rebase ~~~~~~~~ From your project repository, merge and test (replace `pr-branch-name` as appropriate): ``` git fetch origin git checkout -b pr-branch-name origin/pr-branch-name git rebase master ``` If there are conflicts: ``` git mergetool git rebase --continue ``` 2 Push ~~~~~~ Update branch with the rebased history: ``` git push origin pr-branch-name --force ``` Non maintainers can stop here. Note: NEVER just `git push --force` (this will push all local branches, overwriting remotes). 3 Merge ~~~~~~~ ``` git checkout master git merge --no-ff pr-branch-name ``` 4 Test ~~~~~~ ``` [python setup.py] make alltests ``` 5 Version ~~~~~~~~~ Modify tqdm/_version.py and ammend the last (merge) commit: ``` git add tqdm/_version.py git commit --amend # Add "+ bump version" in the commit message ``` 6 Push to master ~~~~~~~~~~~~~~~~ ``` git push origin master ``` BUILDING A RELEASE AND UPLOADING TO PYPI ---------------------------------------- Formally publishing requires additional steps: testing and tagging. Test ~~~~ - ensure that all online CI tests have passed - check `setup.py` and `MANIFEST.in` - which define the packaging process and info that will be uploaded to [pypi](pypi.python.org) - using `[python setup.py] make installdev` Tag ~~~ - ensure the version has been bumped, committed **and** tagged. The tag format is `v{major}.{minor}.{patch}`, for example: `v4.4.1`. The current commit's tag is used in the version checking process. If the current commit is not tagged appropriately, the version will display as `v{major}.{minor}.{patch}-{commit_hash}`. Upload ~~~~~~ Build tqdm into a distributable python package: ``` [python setup.py] make build ``` This will generate several builds in the `dist/` folder. On non-windows machines the windows `exe` installer may fail to build. This is normal. Finally, upload everything to pypi. This can be done easily using the [twine](https://github.com/pypa/twine) module: ``` [python setup.py] make pypi ``` Also, the new release can (should) be added to `github` by creating a new release from the web interface; uploading packages from the `dist/` folder created by `[python setup.py] make build`. Notes ~~~~~ - you can also test on the pypi test servers `testpypi.python.org/pypi` before the real deployment - in case of a mistake, you can delete an uploaded release on pypi, but you cannot re-upload another with the same version number - in case of a mistake in the metadata on pypi (e.g. bad README), updating just the metadata is possible: `[python setup.py] make pypimeta` UPDATING GH-PAGES ----------------- The most important file is README.rst, which sould always be kept up-to-date and in sync with the in-line source documentation. This will affect all of the following: - The [main repository site](https://github.com/tqdm/tqdm) which automatically serves the latest README.rst as well as links to all of github's features. This is the preferred online referral link for tqdm. - The [PyPi mirror](https://pypi.python.org/pypi/tqdm) which automatically serves the latest release built from README.rst as well as links to past releases. - Many external web crawlers. Additionally (less maintained), there exists: - A [wiki](https://github.com/tqdm/tqdm/wiki) which is publicly editable. - The [gh-pages project](https://tqdm.github.io/tqdm/) which is built from the [gh-pages branch](https://github.com/tqdm/tqdm/tree/gh-pages), which is built using [asv](https://github.com/spacetelescope/asv/). - The [gh-pages root](https://tqdm.github.io/) which is built from a separate outdated [github.io repo](https://github.com/tqdm/tqdm.github.io). QUICK DEV SUMMARY ----------------- For expereinced devs, once happy with local master: 1. bump version in `tqdm/_version.py` 2. test (`[python setup.py] make alltests`) 3. `git commit [--amend] # -m "bump version"` 4. `git push` 5. wait for tests to pass a) in case of failure, fix and go back to (2) 6. `git tag vM.m.p && git push --tags` 7. `[python setup.py] make distclean` 8. `[python setup.py] make build` 9. `[python setup.py] make pypi` tqdm-4.11.2/Makefile0000664000175000017500000000475613040270166014773 0ustar cr45hincr45hin00000000000000# IMPORTANT: for compatibility with `python setup.py make [alias]`, ensure: # 1. Every alias is preceded by @[+]make (eg: @make alias) # 2. A maximum of one @make alias or command per line # # Sample makefile compatible with `python setup.py make`: #``` #all: # @make test # @make install #test: # nosetest #install: # python setup.py install #``` .PHONY: alltests all flake8 test testnose testsetup testcoverage testperf testtimer distclean coverclean prebuildclean clean installdev install build pypimeta pypi none help: @python setup.py make alltests: @+make testcoverage @+make testperf @+make flake8 @+make testsetup all: @+make alltests @+make build flake8: @+flake8 --max-line-length=80 --count --statistics --exit-zero -j 8 --exclude .asv . test: tox --skip-missing-interpreters testnose: nosetests tqdm -d -v testsetup: python setup.py check --restructuredtext --strict python setup.py make none testcoverage: @make coverclean nosetests tqdm --with-coverage --cover-package=tqdm --cover-erase --cover-min-percentage=80 --ignore-files="tests_perf\.py" -d -v testperf: # do not use coverage (which is extremely slow) nosetests tqdm/tests/tests_perf.py -d -v testtimer: nosetests tqdm --with-timer -d -v testasv: asv run -j 8 HEAD~3..HEAD @make viewasv testasvfull: asv run -j 8 v1.0.0..master @make testasv viewasv: asv publish asv preview distclean: @+make coverclean @+make prebuildclean @+make clean prebuildclean: @+python -c "import shutil; shutil.rmtree('build', True)" @+python -c "import shutil; shutil.rmtree('dist', True)" @+python -c "import shutil; shutil.rmtree('tqdm.egg-info', True)" coverclean: @+python -c "import os; os.remove('.coverage') if os.path.exists('.coverage') else None" clean: @+python -c "import os; import glob; [os.remove(i) for i in glob.glob('*.py[co]')]" @+python -c "import os; import glob; [os.remove(i) for i in glob.glob('tqdm/*.py[co]')]" @+python -c "import os; import glob; [os.remove(i) for i in glob.glob('tqdm/tests/*.py[co]')]" @+python -c "import os; import glob; [os.remove(i) for i in glob.glob('tqdm/examples/*.py[co]')]" installdev: python setup.py develop --uninstall python setup.py develop install: python setup.py install build: @make prebuildclean python setup.py sdist --formats=gztar,zip bdist_wheel python setup.py bdist_wininst pypimeta: python setup.py register pypi: twine upload dist/* buildupload: @make testsetup @make build @make pypimeta @make pypi none: # used for unit testing tqdm-4.11.2/PKG-INFO0000664000175000017500000010432113041514547014422 0ustar cr45hincr45hin00000000000000Metadata-Version: 1.1 Name: tqdm Version: 4.11.2 Summary: Fast, Extensible Progress Meter Home-page: https://github.com/tqdm/tqdm Author: tqdm developers Author-email: python.tqdm@gmail.com License: MPLv2.0, MIT Licences Description: |Logo| tqdm ==== |PyPI-Status| |PyPI-Versions| |Conda-Forge-Status| |Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |DOI-URI| |LICENCE| ``tqdm`` means "progress" in Arabic (taqadum, تقدّم) and an abbreviation for "I love you so much" in Spanish (te quiero demasiado). Instantly make your loops show a smart progress meter - just wrap any iterable with ``tqdm(iterable)``, and you're done! .. code:: python from tqdm import tqdm for i in tqdm(range(10000)): ... ``76%|████████████████████████████         | 7568/10000 [00:33<00:10, 229.00it/s]`` ``trange(N)`` can be also used as a convenient shortcut for ``tqdm(xrange(N))``. |Screenshot| REPL: `ptpython `__ It can also be executed as a module with pipes: .. code:: sh $ seq 9999999 | tqdm --unit_scale | wc -l 10.0Mit [00:02, 3.58Mit/s] 9999999 $ 7z a -bd -r backup.7z docs/ | grep Compressing | \ tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] Overhead is low -- about 60ns per iteration (80ns with ``tqdm_gui``), and is unit tested against performance regression. By comparison, the well established `ProgressBar `__ has an 800ns/iter overhead. In addition to its low overhead, ``tqdm`` uses smart algorithms to predict the remaining time and to skip unnecessary iteration displays, which allows for a negligible overhead in most cases. ``tqdm`` works on any platform (Linux, Windows, Mac, FreeBSD, Solaris/SunOS), in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks. ``tqdm`` does not require any library (not even curses!) to run, just a vanilla Python interpreter will do and an environment supporting ``carriage return \r`` and ``line feed \n`` control characters. ------------------------------------------ .. contents:: Table of contents :backlinks: top :local: Installation ------------ Latest PyPI stable release ~~~~~~~~~~~~~~~~~~~~~~~~~~ |PyPI-Status| .. code:: sh pip install tqdm Latest development release on github ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |GitHub-Status| |GitHub-Stars| |GitHub-Forks| Pull and install in the current directory: .. code:: sh pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm Latest Conda release ~~~~~~~~~~~~~~~~~~~~ |Conda-Forge-Status| .. code:: sh conda install -c conda-forge tqdm Changelog --------- The list of all changes is available either on GitHub's Releases: |GitHub-Status| or on crawlers such as `allmychanges.com `_. Usage ----- ``tqdm`` is very versatile and can be used in a number of ways. The three main ones are given below. Iterable-based ~~~~~~~~~~~~~~ Wrap ``tqdm()`` around any iterable: .. code:: python text = "" for char in tqdm(["a", "b", "c", "d"]): text = text + char ``trange(i)`` is a special optimised instance of ``tqdm(range(i))``: .. code:: python for i in trange(100): pass Instantiation outside of the loop allows for manual control over ``tqdm()``: .. code:: python pbar = tqdm(["a", "b", "c", "d"]) for char in pbar: pbar.set_description("Processing %s" % char) Manual ~~~~~~ Manual control on ``tqdm()`` updates by using a ``with`` statement: .. code:: python with tqdm(total=100) as pbar: for i in range(10): pbar.update(10) If the optional variable ``total`` (or an iterable with ``len()``) is provided, predictive stats are displayed. ``with`` is also optional (you can just assign ``tqdm()`` to a variable, but in this case don't forget to ``del`` or ``close()`` at the end: .. code:: python pbar = tqdm(total=100) for i in range(10): pbar.update(10) pbar.close() Module ~~~~~~ Perhaps the most wonderful use of ``tqdm`` is in a script or on the command line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass through all ``stdin`` to ``stdout`` while printing progress to ``stderr``. The example below demonstrated counting the number of lines in all python files in the current directory, with timing information included. .. code:: sh $ time find . -name '*.py' -exec cat \{} \; | wc -l 857365 real 0m3.458s user 0m0.274s sys 0m3.325s $ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 857366it [00:03, 246471.31it/s] 857365 real 0m3.585s user 0m0.862s sys 0m3.358s Note that the usual arguments for ``tqdm`` can also be specified. .. code:: sh $ find . -name '*.py' -exec cat \{} \; | tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|███████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s] Backing up a large directory? .. code:: sh $ 7z a -bd -r backup.7z docs/ | grep Compressing | tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] FAQ and Known Issues -------------------- The most common issues relate to excessive output on multiple lines, instead of a neat one-line progress bar. - Consoles in general: require support for carriage return (``CR``, ``\r``). - Nested progress bars: * Consoles in general: require support for moving cursors up to the previous line. For example, `IDLE `__, `ConEmu `__ and `PyCharm `__ (also `here `__ and `here `__) lack full support. * Windows: additionally may require the python module ``colorama``. - Wrapping enumerated iterables: use ``enumerate(tqdm(...))`` instead of ``tqdm(enumerate(...))`. The same applies to ``numpy.ndenumerate``. This is because enumerate functions tend to hide the length of iterables. ``tqdm`` does not. - Wrapping zipped iterables has similar issues due to internal optimisations. ``tqdm(zip(a, b))`` should be replaced with ``zip(tqdm(a), b)`` or even ``zip(tqdm(a), tqdm(b))`` If you come across any other difficulties, browse/open issues `here `__. Documentation ------------- |PyPI-Versions| |README-Hits| (Since 19 May 2016) .. code:: python class tqdm(object): """ Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested. """ def __init__(self, iterable=None, desc=None, total=None, leave=True, file=sys.stderr, ncols=None, mininterval=0.1, maxinterval=10.0, miniters=None, ascii=None, disable=False, unit='it', unit_scale=False, dynamic_ncols=False, smoothing=0.3, bar_format=None, initial=0, position=None, postfix=None): Parameters ~~~~~~~~~~ * iterable : iterable, optional Iterable to decorate with a progressbar. Leave blank to manually manage the updates. * desc : str, optional Prefix for the progressbar. * total : int, optional The number of expected iterations. If (default: None), len(iterable) is used if possible. As a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If ``gui`` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int(9e9). * leave : bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. * file : ``io.TextIOWrapper`` or ``io.StringIO``, optional Specifies where to output the progress messages [default: sys.stderr]. Uses ``file.write(str)`` and ``file.flush()`` methods. * ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). * mininterval : float, optional Minimum progress display update interval, in seconds [default: 0.1]. * maxinterval : float, optional Maximum progress display update interval, in seconds [default: 10]. Automatically adjusts ``miniters`` to correspond to ``mininterval`` after long display update lag. Only works if ``dynamic_miniters`` or monitor thread is enabled. * miniters : int, optional Minimum progress display update interval, in iterations. If 0 and ``dynamic_miniters``, will automatically adjust to equal ``mininterval`` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and ``mininterval`` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. * ascii : bool, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters ``1-9 #``. * disable : bool, optional Whether to disable the entire progressbar wrapper [default: False]. * unit : str, optional String that will be used to define the unit of each iteration [default: it]. * unit_scale : bool, optional If set, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. * dynamic_ncols : bool, optional If set, constantly alters ``ncols`` to the environment (allowing for window resizes) [default: False]. * smoothing : float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. * bar_format : str, optional Specify a custom bar string formatting. May impact performance. If unspecified, will use '{l_bar}{bar}{r_bar}', where l_bar is '{desc}{percentage:3.0f}%|' and r_bar is '| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. * initial : int, optional The initial counter value. Useful when restarting a progress bar [default: 0]. * position : int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). * postfix : dict, optional Specify additional stats to display at the end of the bar. Extra CLI Options ~~~~~~~~~~~~~~~~~ * delim : chr, optional Delimiting character [default: '\n']. Use '\0' for null. N.B.: on Windows systems, Python converts '\n' to '\r\n'. * buf_size : int, optional String buffer size in bytes [default: 256] used when ``delim`` is specified. * bytes : bool, optional If true, will count bytes and ignore ``delim``. Returns ~~~~~~~ * out : decorated iterator. .. code:: python def update(self, n=1): """ Manually update the progress bar, useful for streams such as reading files. E.g.: >>> t = tqdm(total=filesize) # Initialise >>> for current_buffer in stream: ... ... ... t.update(len(current_buffer)) >>> t.close() The last line is highly recommended, but possibly not necessary if ``t.update()`` will be called in such a way that ``filesize`` will be exactly reached and printed. Parameters ---------- n : int Increment to add to the internal counter of iterations [default: 1]. """ def close(self): """ Cleanup and (if leave=False) close the progressbar. """ def clear(self): """ Clear current bar display """ def refresh(self): """ Force refresh the display of this bar """ def write(cls, s, file=sys.stdout, end="\n"): """ Print a message via tqdm (without overlap with bars) """ def set_description(self, desc=None): """ Set/modify description of the progress bar. """ def set_postfix(self, **kwargs): """ Set/modify postfix (additional stats) with automatic formatting based on datatype. """ def trange(*args, **kwargs): """ A shortcut for tqdm(xrange(*args), **kwargs). On Python3+ range is used instead of xrange. """ class tqdm_gui(tqdm): """ Experimental GUI version of tqdm! """ def tgrange(*args, **kwargs): """ Experimental GUI version of trange! """ class tqdm_notebook(tqdm): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ def tnrange(*args, **kwargs): """ Experimental IPython/Jupyter Notebook widget using tqdm! """ Examples and Advanced Usage --------------------------- - See the `examples `__ folder; - import the module and run ``help()``, or - consult the `wiki `__. - this has an `excellent article `__ on how to make a **great** progressbar. Description and additional stats ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Custom information can be displayed and updated dynamically on ``tqdm`` bars with the ``desc`` and ``postfix`` arguments: .. code:: python from tqdm import trange from random import random, randint from time import sleep t = trange(100) for i in t: # Description will be displayed on the left t.set_description('GEN %i' % i) # Postfix will be displayed on the right, and will format automatically # based on argument's datatype t.set_postfix(loss=random(), gen=randint(1,999), str='h', lst=[1, 2]) sleep(0.1) Nested progress bars ~~~~~~~~~~~~~~~~~~~~ ``tqdm`` supports nested progress bars. Here's an example: .. code:: python from tqdm import trange from time import sleep for i in trange(10, desc='1st loop'): for j in trange(5, desc='2nd loop', leave=False): for k in trange(100, desc='3nd loop'): sleep(0.01) On Windows `colorama `__ will be used if available to produce a beautiful nested display. For manual control over positioning (e.g. for multi-threaded use), you may specify `position=n` where `n=0` for the outermost bar, `n=1` for the next, and so on. Hooks and callbacks ~~~~~~~~~~~~~~~~~~~ ``tqdm`` can easily support callbacks/hooks and manual updates. Here's an example with ``urllib``: **urllib.urlretrieve documentation** | [...] | If present, the hook function will be called once | on establishment of the network connection and once after each block read | thereafter. The hook will be passed three arguments; a count of blocks | transferred so far, a block size in bytes, and the total size of the file. | [...] .. code:: python import urllib from tqdm import tqdm def my_hook(t): """ Wraps tqdm instance. Don't forget to close() or __exit__() the tqdm instance once you're done with it (easiest using `with` syntax). Example ------- >>> with tqdm(...) as t: ... reporthook = my_hook(t) ... urllib.urlretrieve(..., reporthook=reporthook) """ last_b = [0] def inner(b=1, bsize=1, tsize=None): """ b : int, optional Number of blocks just transferred [default: 1]. bsize : int, optional Size of each block (in tqdm units) [default: 1]. tsize : int, optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: t.total = tsize t.update((b - last_b[0]) * bsize) last_b[0] = b return inner eg_link = 'http://www.doc.ic.ac.uk/~cod11/matryoshka.zip' with tqdm(unit='B', unit_scale=True, miniters=1, desc=eg_link.split('/')[-1]) as t: # all optional kwargs urllib.urlretrieve(eg_link, filename='/dev/null', reporthook=my_hook(t), data=None) It is recommend to use ``miniters=1`` whenever there is potentially large differences in iteration speed (e.g. downloading a file over a patchy connection). Pandas Integration ~~~~~~~~~~~~~~~~~~ Due to popular demand we've added support for ``pandas`` -- here's an example for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``: .. code:: python import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.) tqdm.pandas(desc="my bar!") # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2) In case you're interested in how this works (and how to modify it for your own callbacks), see the `examples `__ folder or import the module and run ``help()``. IPython/Jupyter Integration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ IPython/Jupyter is supported via the `tqdm_notebook` submodule: .. code:: python from tqdm import tnrange, tqdm_notebook from time import sleep for i in tnrange(10, desc='1st loop'): for j in tqdm_notebook(xrange(100), desc='2nd loop'): sleep(0.01) In addition to `tqdm` features, the submodule provides a native Jupyter widget (compatible with IPython v1-v4 and Jupyter), fully working nested bars and color hints (blue: normal, green: completed, red: error/interrupt, light blue: no ETA); as demonstrated below. |Screenshot-Jupyter1| |Screenshot-Jupyter2| |Screenshot-Jupyter3| Writing messages ~~~~~~~~~~~~~~~~ Since ``tqdm`` uses a simple printing mechanism to display progress bars, you should not write any message in the terminal using ``print()``. To write messages in the terminal without any collision with ``tqdm`` bar display, a ``.write()`` method is provided: .. code:: python from tqdm import tqdm, trange from time import sleep bar = trange(10) for i in bar: # Print using tqdm class method .write() sleep(0.1) if not (i % 3): tqdm.write("Done task %i" % i) # Can also use bar.write() By default, this will print to standard output ``sys.stdout``. but you can specify any file-like object using the ``file`` argument. For example, this can be used to redirect the messages writing to a log file or class. Redirecting writing ~~~~~~~~~~~~~~~~~~~ If using a library that can print messages to the console, editing the library by replacing ``print()`` with ``tqdm.write()`` may not be desirable. In that case, redirecting ``sys.stdout`` to ``tqdm.write()`` is an option. To redirect ``sys.stdout``, create a file-like class that will write any input string to ``tqdm.write()``, and supply the arguments ``file=sys.stdout, dynamic_ncols=True``. A reusable canonical example is given below: .. code:: python from time import sleep import contextlib import sys from tqdm import tqdm class DummyTqdmFile(object): """Dummy file-like that will write to tqdm""" file = None def __init__(self, file): self.file = file def write(self, x): # Avoid print() second call (useless \n) if len(x.rstrip()) > 0: tqdm.write(x, file=self.file) @contextlib.contextmanager def stdout_redirect_to_tqdm(): save_stdout = sys.stdout try: sys.stdout = DummyTqdmFile(sys.stdout) yield save_stdout # Relay exceptions except Exception as exc: raise exc # Always restore sys.stdout if necessary finally: sys.stdout = save_stdout def blabla(): print("Foo blabla") # Redirect stdout to tqdm.write() (don't forget the `as save_stdout`) with stdout_redirect_to_tqdm() as save_stdout: # tqdm call need to specify sys.stdout, not sys.stderr (default) # and dynamic_ncols=True to autodetect console width for _ in tqdm(range(3), file=save_stdout, dynamic_ncols=True): blabla() sleep(.5) # After the `with`, printing is restored print('Done!') Monitoring thread, intervals and miniters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``tqdm`` implements a few tricks to to increase efficiency and reduce overhead. 1. Avoid unnecessary frequent bar refreshing: ``mininterval`` defines how long to wait between each refresh. ``tqdm`` always gets updated in the background, but it will diplay only every ``mininterval``. 2. Reduce number of calls to check system clock/time. 3. ``mininterval`` is more intuitive to configure than ``miniters``. A clever adjustment system ``dynamic_miniters`` will automatically adjust ``miniters`` to the amount of iterations that fit into time ``mininterval``. Essentially, ``tqdm`` will check if it's time to print without actually checking time. This behavior can be still be bypassed by manually setting ``miniters``. However, consider a case with a combination of fast and slow iterations. After a few fast iterations, ``dynamic_miniters`` will set ``miniters`` to a large number. When interation rate subsequently slows, ``miniters`` will remain large and thus reduce display update frequency. To address this: 4. ``maxinterval`` defines the maximum time between display refreshes. A concurrent monitoring thread checks for overdue updates and forces one where necessary. The monitoring thread should not have a noticeable overhead, and guarantees updates at least every 10 seconds by default. This value can be directly changed by setting the ``monitor_interval`` of any ``tqdm`` instance (i.e. ``t = tqdm.tqdm(...); t.monitor_interval = 2``). The monitor thread may be disabled application-wide by setting ``tqdm.tqdm.monitor_interval = 0`` before instantiatiation of any ``tqdm`` bar. Contributions ------------- All source code is hosted on `GitHub `__. Contributions are welcome. See the `CONTRIBUTING `__ file for more information. LICENCE ------- Open Source (OSI approved): |LICENCE| Citation information: |DOI-URI| Authors ------- Ranked by contributions. - Casper da Costa-Luis (casperdcl) - Stephen Larroque (lrq3000) - Hadrien Mary (hadim) - Noam Yorav-Raphael (noamraph)* - Ivan Ivanov (obiwanus) - Mikhail Korobov (kmike) `*` Original author |README-Hits| (Since 19 May 2016) .. |Logo| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif .. |Screenshot| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm.gif .. |Build-Status| image:: https://travis-ci.org/tqdm/tqdm.svg?branch=master :target: https://travis-ci.org/tqdm/tqdm .. |Coverage-Status| image:: https://coveralls.io/repos/tqdm/tqdm/badge.svg :target: https://coveralls.io/r/tqdm/tqdm .. |Branch-Coverage-Status| image:: https://codecov.io/github/tqdm/tqdm/coverage.svg?branch=master :target: https://codecov.io/github/tqdm/tqdm?branch=master .. |Codacy-Grade| image:: https://api.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177 :target: https://www.codacy.com/app/tqdm/tqdm?utm_source=github.com&utm_medium=referral&utm_content=tqdm/tqdm&utm_campaign=Badge_Grade .. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=2592000 :target: https://github.com/tqdm/tqdm/releases .. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/network .. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg :target: https://github.com/tqdm/tqdm/stargazers .. |PyPI-Status| image:: https://img.shields.io/pypi/v/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |PyPI-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg :target: https://pypi.python.org/pypi/tqdm .. |Conda-Forge-Status| image:: https://anaconda.org/conda-forge/tqdm/badges/version.svg :target: https://anaconda.org/conda-forge/tqdm .. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg :target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE .. |DOI-URI| image:: https://zenodo.org/badge/21637/tqdm/tqdm.svg :target: https://zenodo.org/badge/latestdoi/21637/tqdm/tqdm .. |Screenshot-Jupyter1| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-1.gif .. |Screenshot-Jupyter2| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-2.gif .. |Screenshot-Jupyter3| image:: https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-3.gif .. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif :target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif&style=social Keywords: progressbar progressmeter progress bar meter rate eta console terminal time Platform: any Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0) Classifier: License :: OSI Approved :: MIT License Classifier: Environment :: Console Classifier: Framework :: IPython Classifier: Operating System :: Microsoft :: Windows Classifier: Operating System :: MacOS :: MacOS X Classifier: Operating System :: POSIX Classifier: Operating System :: POSIX :: Linux Classifier: Operating System :: POSIX :: BSD Classifier: Operating System :: POSIX :: BSD :: FreeBSD Classifier: Operating System :: POSIX :: SunOS/Solaris Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.2 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Programming Language :: Python :: Implementation :: IronPython Classifier: Topic :: Software Development :: Libraries Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Software Development :: User Interfaces Classifier: Topic :: System :: Monitoring Classifier: Topic :: Terminals Classifier: Topic :: Utilities Classifier: Intended Audience :: Developers