xdoctest.parser module

The XDoctest Parser

This parses a docstring into one or more “doctest part” after the docstrings have been extracted from the source code by either static or dynamic means.

Terms and definitions:

logical block:

a snippet of code that can be executed by itself if given the correct global / local variable context.

PS1:

The original meaning is “Prompt String 1”. For details see: [SE32096] [BashPS1] [CustomPrompt] [GeekPrompt]. In the context of xdoctest, instead of referring to the prompt prefix, we use PS1 to refer to a line that starts a “logical block” of code. In the original doctest module these all had to be prefixed with “>>>”. In xdoctest the prefix is used to simply denote the code is part of a doctest. It does not necessarily mean a new “logical block” is starting.

PS2:

The original meaning is “Prompt String 2”. In the context of xdoctest, instead of referring to the prompt prefix, we use PS2 to refer to a line that continues a “logical block” of code. In the original doctest module these all had to be prefixed with “…”. However, xdoctest uses parsing to automatically determine this.

want statement:

Lines directly after a logical block of code in a doctest indicating the desired result of executing the previous block.

While I do believe this AST-based code is a significant improvement over the RE-based builtin doctest parser, I acknowledge that I’m not an AST expert and there is room for improvement here.

References

class xdoctest.parser.DoctestParser(simulate_repl=False)[source]

Bases: object

Breaks docstrings into parts using the parse method.

Example

>>> from xdoctest.parser import *  # NOQA
>>> parser = DoctestParser()
>>> doctest_parts = parser.parse(
>>>     '''
>>>     >>> j = 0
>>>     >>> for i in range(10):
>>>     >>>     j += 1
>>>     >>> print(j)
>>>     10
>>>     '''.lstrip('\n'))
>>> print('\n'.join(list(map(str, doctest_parts))))
<DoctestPart(ln 0, src="j = 0...", want=None)>
<DoctestPart(ln 3, src="print(j)...", want="10...")>

Example

>>> # Having multiline strings in doctests can be nice
>>> string = utils.codeblock(
        '''
        >>> name = 'name'
        'anything'
        ''')
>>> self = DoctestParser()
>>> doctest_parts = self.parse(string)
>>> print('\n'.join(list(map(str, doctest_parts))))
Parameters:

simulate_repl (bool) – if True each line will be treated as its own doctest. This more closely mimics the original doctest module. Defaults to False.

parse(string, info=None)[source]

Divide the given string into examples and interleaving text.

Parameters:

  • string (str) – string representing the doctest

  • info (dict | None) – info about where the string came from in case of an error

Returns:

a list of DoctestPart objects

Return type:

List[xdoctest.doctest_part.DoctestPart]

CommandLine

python -m xdoctest.parser DoctestParser.parse

Example

>>> s = 'I am a dummy example with two parts'
>>> x = 10
>>> print(s)
I am a dummy example with two parts
>>> s = 'My purpose it so demonstrate how wants work here'
>>> print('The new want applies ONLY to stdout')
>>> print('given before the last want')
>>> '''
    this wont hurt the test at all
    even though its multiline '''
>>> y = 20
The new want applies ONLY to stdout
given before the last want
>>> # Parts from previous examples are executed in the same context
>>> print(x + y)
30

this is simply text, and doesnt apply to the previous doctest the <BLANKLINE> directive is still in effect.

Example

>>> from xdoctest.parser import *  # NOQA
>>> from xdoctest import parser
>>> from xdoctest.docstr import docscrape_google
>>> from xdoctest import core
>>> self = parser.DoctestParser()
>>> docstr = self.parse.__doc__
>>> blocks = docscrape_google.split_google_docblocks(docstr)
>>> doclineno = self.parse.__func__.__code__.co_firstlineno
>>> key, (string, offset) = blocks[-2]
>>> self._label_docsrc_lines(string)
>>> doctest_parts = self.parse(string)
>>> # each part with a want-string needs to be broken in two
>>> assert len(doctest_parts) == 6
>>> len(doctest_parts)