xdoctest.core module

Core methods used by xdoctest runner and plugin code to statically extract doctests from a module or package.

The following is a glossary of terms and jargon used in this repo.

  • callname - the name of a callable function, method, class etc… e.g. myfunc, MyClass, or MyClass.some_method.

  • got / want - a test that produces stdout or a value to check. Whatever is produced is what you “got” and whatever is expected is what you “want”. See xdoctest.checker for more details.

  • directives - special in-doctest comments that change the behavior of the doctests at runtime. See xdoctest.directive for more details.

  • chevrons - the three cheverons (``>>> ``) or right angle brakets are the

    standard prefix for a doctest, also referred to as a PS1 line in the parser.

  • zero-args - a function that can be called without any arguments.

  • freeform style - This is the term used to refer to a doctest that could be

    anywhere in the docstring. The alternative are structured doctests where they are only expected in known positions like in “Example blocks” for google and numpy style docstrings.

  • TODO - complete this list (Make an issue or PR if there is any term you don’t

    immediately understand!).

xdoctest.core.parse_freeform_docstr_examples(docstr, callname=None, modpath=None, lineno=1, fpath=None, asone=True)[source]

Finds free-form doctests in a docstring. This is similar to the original doctests because these tests do not requires a google/numpy style header.

Some care is taken to avoid enabling tests that look like disabled google doctests or scripts.

Parameters:

  • docstr (str) – an extracted docstring

  • callname (str | None) – the name of the callable (e.g. function, class, or method) that this docstring belongs to.

  • modpath (str | PathLike | None) – original module the docstring is from

  • lineno (int) – the line number (starting from 1) of the docstring. i.e. if you were to go to this line number in the source file the starting quotes of the docstr would be on this line. Defaults to 1.

  • fpath (str | PathLike | None) – the file that the docstring is from (if the file was not a module, needed for backwards compatibility)

  • asone (bool) – if False doctests are broken into multiple examples based on spacing, otherwise they are executed as a single unit. Defaults to True.

Yields:

xdoctest.doctest_example.DocTest – doctest object

Raises:

xdoctest.exceptions.DoctestParseError – if an error occurs in parsing

CommandLine

python -m xdoctest.core parse_freeform_docstr_examples

Example

>>> # TODO: move this to unit tests and make the doctest simpler
>>> from xdoctest import core
>>> from xdoctest import utils
>>> docstr = utils.codeblock(
>>>     '''
>>>     freeform
>>>     >>> doctest
>>>     >>> hasmultilines
>>>     whoppie
>>>     >>> 'but this is the same doctest'
>>>
>>>     >>> secondone
>>>
>>>     Script:
>>>         >>> 'special case, dont parse me'
>>>
>>>     DisableDoctest:
>>>         >>> 'special case, dont parse me'
>>>         want
>>>
>>>     AnythingElse:
>>>         >>> 'general case, parse me'
>>>        want
>>>     ''')
>>> examples = list(parse_freeform_docstr_examples(docstr, asone=True))
>>> assert len(examples) == 1
>>> examples = list(parse_freeform_docstr_examples(docstr, asone=False))
>>> assert len(examples) == 3
xdoctest.core.parse_google_docstr_examples(docstr, callname=None, modpath=None, lineno=1, fpath=None, eager_parse=True)[source]

Parses Google-style doctests from a docstr and generates example objects

Parameters:

  • docstr (str) – an extracted docstring

  • callname (str | None) – the name of the callable (e.g. function, class, or method) that this docstring belongs to.

  • modpath (str | PathLike | None) – original module the docstring is from

  • lineno (int) – the line number (starting from 1) of the docstring. i.e. if you were to go to this line number in the source file the starting quotes of the docstr would be on this line. Defaults to 1.

  • fpath (str | PathLike | None) – the file that the docstring is from (if the file was not a module, needed for backwards compatibility)

  • eager_parse (bool) – if True eagerly evaluate the parser inside the google example blocks. Defaults to True.

Yields:

xdoctest.doctest_example.DocTest – doctest object

Raises:
xdoctest.core.parse_auto_docstr_examples(docstr, *args, **kwargs)[source]

First try to parse google style, but if no tests are found use freeform style.

xdoctest.core.parse_docstr_examples(docstr, callname=None, modpath=None, lineno=1, style='auto', fpath=None, parser_kw=None)[source]

Parses doctests from a docstr and generates example objects. The style influences which tests are found.

Parameters:

  • docstr (str) – a previously extracted docstring

  • callname (str | None) – the name of the callable (e.g. function, class, or method) that this docstring belongs to.

  • modpath (str | PathLike | None) – original module the docstring is from

  • lineno (int) – the line number (starting from 1) of the docstring. i.e. if you were to go to this line number in the source file the starting quotes of the docstr would be on this line. Defaults to 1.

  • style (str) – expected doctest style, which can be “google”, “freeform”, or “auto”. Defaults to ‘auto’.

  • fpath (str | PathLike | None) – the file that the docstring is from (if the file was not a module, needed for backwards compatibility)

  • parser_kw (dict | None) – passed to the parser as keyword args

Yields:

xdoctest.doctest_example.DocTest – parsed example

CommandLine

python -m xdoctest.core parse_docstr_examples

Example

>>> from xdoctest.core import *
>>> from xdoctest import utils
>>> docstr = utils.codeblock(
...    '''
...    >>> 1 + 1  # xdoctest: +SKIP
...    2
...    >>> 2 + 2
...    4
...    ''')
>>> examples = list(parse_docstr_examples(docstr, 'name', fpath='foo.txt', style='freeform'))
>>> print(len(examples))
1
>>> examples = list(parse_docstr_examples(docstr, fpath='foo.txt'))
xdoctest.core.package_calldefs(pkg_identifier, exclude=[], ignore_syntax_errors=True, analysis='auto')[source]

Statically generates all callable definitions in a module or package

Parameters:

  • pkg_identifier (str | ModuleType) – path to or name of the module to be tested (or the live module itself, which is not recommended)

  • exclude (List[str]) – glob-patterns of file names to exclude

  • ignore_syntax_errors (bool) – if False raise an error when syntax errors occur in a doctest Defaults to True.

  • analysis (str) – if ‘static’, only static analysis is used to parse call definitions. If ‘auto’, uses dynamic analysis for compiled python extensions, but static analysis elsewhere, if ‘dynamic’, then dynamic analysis is used to parse all calldefs. Defaults to ‘auto’.

Yields:
Tuple[Dict[str, xdoctest.static_analysis.CallDefNode], str | ModuleType] -

  • item[0]: the mapping of callnames-to-calldefs

  • item[1]: the path to the file containing the doctest (usually a module) or the module itself

Example

>>> pkg_identifier = 'xdoctest.core'
>>> testables = list(package_calldefs(pkg_identifier))
>>> assert len(testables) == 1
>>> calldefs, modpath = testables[0]
>>> assert util_import.modpath_to_modname(modpath) == pkg_identifier
>>> assert 'package_calldefs' in calldefs
xdoctest.core.parse_calldefs(module_identifier, analysis='auto')[source]

Parse calldefs from a single module using either static or dynamic analysis.

Parameters:

  • module_identifier (str | ModuleType) – path to or name of the module to be tested (or the live module itself, which is not recommended)

  • analysis (str, default=’auto’) – if ‘static’, only static analysis is used to parse call definitions. If ‘auto’, uses dynamic analysis for compiled python extensions, but static analysis elsewhere, if ‘dynamic’, then dynamic analysis is used to parse all calldefs.

Returns:

the mapping of callnames-to-calldefs within the module.

Return type:

Dict[str, xdoctest.static_analysis.CallDefNode]

xdoctest.core.parse_doctestables(module_identifier, exclude=[], style='auto', ignore_syntax_errors=True, parser_kw={}, analysis='auto')[source]

Parses all doctests within top-level callables of a module and generates example objects. The style influences which tests are found.

Parameters:

  • module_identifier (str | PathLike | ModuleType) – path or name of a module or a module itself (we prefer a path)

  • exclude (List[str]) – glob-patterns of file names to exclude

  • style (str) – expected doctest style (e.g. google, freeform, auto)

  • ignore_syntax_errors (bool, default=True) – if False raise an error when syntax errors

  • parser_kw – extra args passed to the parser

  • analysis (str, default=’auto’) – if ‘static’, only static analysis is used to parse call definitions. If ‘auto’, uses dynamic analysis for compiled python extensions, but static analysis elsewhere, if ‘dynamic’, then dynamic analysis is used to parse all calldefs.

Yields:

xdoctest.doctest_example.DocTest – parsed doctest example objects

CommandLine

python -m xdoctest.core parse_doctestables

Example

>>> module_identifier = 'xdoctest.core'
>>> testables = list(parse_doctestables(module_identifier))
>>> this_example = None
>>> for example in testables:
>>>     # print(example)
>>>     if example.callname == 'parse_doctestables':
>>>         this_example = example
>>> assert this_example is not None
>>> assert this_example.callname == 'parse_doctestables'

Example

>>> from xdoctest import utils
>>> docstr = utils.codeblock(
...    '''
...    >>> 1 + 1  # xdoctest: +SKIP
...    2
...    >>> 2 + 2
...    4
...    ''')
>>> temp = utils.TempDoctest(docstr, 'test_modfile')
>>> modpath = temp.modpath
>>> examples = list(parse_doctestables(modpath, style='freeform'))
>>> print(len(examples))
1