xdoctest.static_analysis module

The core logic that allows for xdoctest to parse source statically

class xdoctest.static_analysis.CallDefNode(callname, lineno, docstr, doclineno, doclineno_end, args=None)

Bases: object

Variables:

lineno_end (None | int) – the line number the docstring ends on (if known)

Parameters:

• callname (str) – the name of the item containing the docstring.

• lineno (int) – the line number the item containing the docstring.

• docstr (str) – the docstring itself

• doclineno (int) – the line number (1 based) the docstring begins on

• doclineno_end (int) – the line number (1 based) the docstring ends on

• args (None | ast.arguments) – arguments from static analysis TopLevelVisitor.

class xdoctest.static_analysis.TopLevelVisitor(source=None)

Bases: NodeVisitor

Parses top-level function names and docstrings

For other visit_<classname> values see [MeetTheNodes].

References

[MeetTheNodes]

http://greentreesnakes.readthedocs.io/en/latest/nodes.html

CommandLine

python -m xdoctest.static_analysis TopLevelVisitor

Variables:

• calldefs (OrderedDict) –

• source (None | str) –

• sourcelines (None | List[str]) –

• assignments (list) –

Example

>>> from xdoctest.static_analysis import *  # NOQA
>>> from xdoctest import utils
>>> source = utils.codeblock(
        '''
        def foo():
            """ my docstring """
            def subfunc():
                pass
        def bar():
            pass
        class Spam(object):
            def eggs(self):
                pass
            @staticmethod
            def hams():
                pass
            @property
            def jams(self):
                return 3
            @jams.setter
            def jams2(self, x):
                print('ignoring')
            @jams.deleter
            def jams(self, x):
                print('ignoring')
        ''')
>>> self = TopLevelVisitor.parse(source)
>>> callnames = set(self.calldefs.keys())
>>> assert callnames == {
>>>     'foo', 'bar', 'Spam', 'Spam.eggs', 'Spam.hams',
>>>     'Spam.jams'}
>>> assert self.calldefs['foo'].docstr.strip() == 'my docstring'
>>> assert 'subfunc' not in self.calldefs

Parameters:

source (None | str)

classmethod parse(source)

main entry point

executes parsing algorithm and populates self.calldefs

Parameters:

source (str)

syntax_tree()

creates the abstract syntax tree

Return type:

ast.Module

process_finished(node)

process (get ending lineno) for everything marked as finished

Parameters:

node (ast.AST)

visit(node)

Parameters:

node (ast.AST)

visit_FunctionDef(node)

Parameters:

node (ast.FunctionDef)

visit_ClassDef(node)

Parameters:

node (ast.ClassDef)

visit_Module(node)

Parameters:

node (ast.Module)

visit_Assign(node)

Parameters:

node (ast.Assign)

visit_If(node)

Parameters:

node (ast.If)

xdoctest.static_analysis.parse_static_calldefs(source=None, fpath=None)

Statically finds top-level callable functions and methods in python source

Parameters:

• source (str) – python text

• fpath (str) – filepath to read if source is not specified

Returns:

mapping from callnames to CallDefNodes, which contain

info about the item with the doctest.

Return type:

Dict[str, CallDefNode]

Example

>>> from xdoctest import static_analysis
>>> fpath = static_analysis.__file__.replace('.pyc', '.py')
>>> calldefs = parse_static_calldefs(fpath=fpath)
>>> assert 'parse_static_calldefs' in calldefs

xdoctest.static_analysis.parse_calldefs(source=None, fpath=None)

xdoctest.static_analysis.parse_static_value(key, source=None, fpath=None)

Statically parse a constant variable’s value from python code.

TODO: This does not belong here. Move this to an external static analysis library.

Parameters:

• key (str) – name of the variable

• source (str) – python text

• fpath (str) – filepath to read if source is not specified

Returns:

object

Example

>>> from xdoctest.static_analysis import parse_static_value
>>> key = 'foo'
>>> source = 'foo = 123'
>>> assert parse_static_value(key, source=source) == 123
>>> source = 'foo = "123"'
>>> assert parse_static_value(key, source=source) == '123'
>>> source = 'foo = [1, 2, 3]'
>>> assert parse_static_value(key, source=source) == [1, 2, 3]
>>> source = 'foo = (1, 2, "3")'
>>> assert parse_static_value(key, source=source) == (1, 2, "3")
>>> source = 'foo = {1: 2, 3: 4}'
>>> assert parse_static_value(key, source=source) == {1: 2, 3: 4}
>>> source = 'foo = None'
>>> assert parse_static_value(key, source=source) == None
>>> #parse_static_value('bar', source=source)
>>> #parse_static_value('bar', source='foo=1; bar = [1, foo]')

xdoctest.static_analysis.package_modpaths(pkgpath, with_pkg=False, with_mod=True, followlinks=True, recursive=True, with_libs=False, check=True)

Finds sub-packages and sub-modules belonging to a package.

Parameters:

• pkgpath (str) – path to a module or package

• with_pkg (bool) – if True includes package __init__ files (default = False)

• with_mod (bool) – if True includes module files (default = True)

• exclude (list) – ignores any module that matches any of these patterns

• recursive (bool) – if False, then only child modules are included

• with_libs (bool) – if True then compiled shared libs will be returned as well

• check (bool) – if False, then then pkgpath is considered a module even if it does not contain an __init__ file.

Yields:

str – module names belonging to the package

References

http://stackoverflow.com/questions/1707709/list-modules-in-py-package

Example

>>> from xdoctest.static_analysis import *
>>> pkgpath = modname_to_modpath('xdoctest')
>>> paths = list(package_modpaths(pkgpath))
>>> print('\n'.join(paths))
>>> names = list(map(modpath_to_modname, paths))
>>> assert 'xdoctest.core' in names
>>> assert 'xdoctest.__main__' in names
>>> assert 'xdoctest' not in names
>>> print('\n'.join(names))

xdoctest.static_analysis.is_balanced_statement(lines, only_tokens=False, reraise=0)

Checks if the lines have balanced braces and quotes.

Parameters:

lines (List[str]) – list of strings, one for each line

Returns:

True if the statement is balanced, otherwise False

Return type:

bool

CommandLine

xdoctest -m xdoctest.static_analysis is_balanced_statement:0

References

https://stackoverflow.com/questions/46061949/parse-until-complete

Example

>>> from xdoctest.static_analysis import *  # NOQA
>>> assert is_balanced_statement(['print(foobar)'])
>>> assert is_balanced_statement(['foo = bar']) is True
>>> assert is_balanced_statement(['foo = (']) is False
>>> assert is_balanced_statement(['foo = (', "')(')"]) is True
>>> assert is_balanced_statement(
...     ['foo = (', "'''", ")]'''", ')']) is True
>>> assert is_balanced_statement(
...     ['foo = ', "'''", ")]'''", ')']) is False
>>> #assert is_balanced_statement(['foo = ']) is False
>>> #assert is_balanced_statement(['== ']) is False
>>> lines = ['def foo():', '', '    x = 1', 'assert True', '']
>>> assert is_balanced_statement(lines)

Example

>>> from xdoctest.static_analysis import *
>>> source_parts = [
>>>     'setup(',
>>>     "    name='extension',",
>>>     '    ext_modules=[',
>>>     '        CppExtension(',
>>>     "            name='extension',",
>>>     "            sources=['extension.cpp'],",
>>>     "            extra_compile_args=['-g'])),",
>>>     '    ],',
>>> ]
>>> print('\n'.join(source_parts))
>>> assert not is_balanced_statement(source_parts)
>>> source_parts = [
>>>     'setup(',
>>>     "    name='extension',",
>>>     '    ext_modules=[',
>>>     '        CppExtension(',
>>>     "            name='extension',",
>>>     "            sources=['extension.cpp'],",
>>>     "            extra_compile_args=['-g']),",
>>>     '    ],',
>>>     '        cmdclass={',
>>>     "            'build_ext': BuildExtension",
>>>     '        })',
>>> ]
>>> print('\n'.join(source_parts))
>>> assert is_balanced_statement(source_parts)

Example

>>> lines = ['try: raise Exception']
>>> is_balanced_statement(lines, only_tokens=1)
True
>>> is_balanced_statement(lines, only_tokens=0)
False

Example

>>> # Cause a failure case on 3.12
>>> from xdoctest.static_analysis import *
>>> lines = ['3, 4]', 'print(len(x))']
>>> is_balanced_statement(lines, only_tokens=1)
False

xdoctest.static_analysis.extract_comments(source)

Returns the text in each comment in a block of python code. Uses tokenize to account for quotations.

Parameters:

source (str | List[str])

CommandLine

python -m xdoctest.static_analysis extract_comments

Example

>>> from xdoctest import utils
>>> source = utils.codeblock(
>>>    '''
       # comment 1
       a = '# not a comment'  # comment 2
       c = 3
       ''')
>>> comments = list(extract_comments(source))
>>> assert comments == ['# comment 1', '# comment 2']
>>> comments = list(extract_comments(source.splitlines()))
>>> assert comments == ['# comment 1', '# comment 2']

xdoctest.static_analysis.six_axt_parse(source_block, filename='<source_block>', compatible=True)

Python 2/3 compatible replacement for ast.parse(source_block, filename=’<source_block>’)

Parameters:

• source (str)

• filename (str)

• compatible (bool)

Returns:

ast.Module | types.CodeType