Documentation Decorators: Arguments

I'm going to start with defining a data-structure for actually housing the metadata that the decorators will be generating. At what is probably the lowest-common-denominator level, it could be handled by a dict (of other dicts, mostly), following the structure shown above. Using a simple dict would be... well... simple, really, and it's tempting to keep it simple. Going back to my earlier principles, though, a dict is potentially very easy to misuse, which flies in the face of:

2. It should be as hard to misuse as I can make it;

The output of the data-structure should be a dict, or should be accessible as one, but the mutability of the data that drives that result should be tightly controlled (which goes back to the idea of Restrict public mutability-access (set, delete) until there is a demonstrable need for it.). In general, access to a documentation-metadata structure should be read-only apart from the decorators that set data-values within it. In order to achieve all of these, I'm going to design a class instead: api_documentation. The basic pattern that I'm going to at least try to follow involves:

• Instances of api_documentation will have formal properties for each collection of metadata ([Documentation-Item] notations below) they keep track of.
• Each instance will have an Add[Documentation-Item] method for each type of item that documentation-metadata can be generated for, so:
  • AddArgument to add argument documentation metadata;
  • AddArgList to add argument-list documentation metadata;
  • AddKeywordArg to add keyword-argument documentation metadata;
  • and so on
• Each Add[Documentation-Item] item will have a corresponding _create[Documentation-Item]Metadata method that is responsible for generating consistent metadata-structures for each item-type;

I'm going to start with the structure and mechanisms for handling function- and method-argument documentation-metadata.

The arguments property

#####################################
# Class attributes (and instance-   #
# attribute default values)         #
#####################################

_argSpecs = None                    # Keeps track, internally, of 
                                    # inspect.getargspec results for 
                                    # functions and methods.

_arguments = {}                     # Internal default attribute for 
                                    # arguments.

_defaults = None                    # Keeps track, internally, of 
                                    # argument default values for 
                                    # functions and methods.

#####################################
# Instance property-getter methods  #
#####################################

def _GetArguments( self ):
    """
Gets the dictionary of arguments that the instance has been used to 
document/describe:
{
    <str|unicode name>:
    {
        'default':<None|object>,
        'description':<str|unicode>,
        'expects':<tuple<types>>,
        'haDefault':<bool>,
        'name':<str|unicode>,
    }
}"""
    return self._arguments

#####################################
# Instance Properties               #
#####################################

arguments = property( _GetArguments, None, None, 
    _GetArguments.__doc__.strip() )

The _createArgumentMetadata method

def _createArgumentMetadata( self, callable, name, description, *expects ):
    """
Creates and returns a metadata-dictionary for a function or method argument."""
    # Type-check callable
    if not inspect.isfunction( callable ) and \
        not inspect.isgenerator( callable ) and \
        not inspect.ismethod( callable ):
        raise TypeError( '%s._createArgumentMetadata expects a callable '
            '(function, generator or method) in its "callable" argument, '
            'but was passed "%s" (%s)' % ( self.__name__, callable, 
                type( callable ).__name__ ) )
    # Get argument-specs for the callable, saving them if they don't 
    # already exist. These will be used for checking argument names and 
    # determining defaults.
    if not self._argSpecs:
        self._argSpecs = inspect.getargspec( callable )
    # Similarly, fetch/store default values for the callable
    if self._defaults == None:
        self._defaults = {}
        try:
            argSpecDefaults = list( self._argSpecs.defaults )
            argSpecNames = self._argSpecs.args
        except TypeError:
            # Raised if defaults is None
            argSpecDefaults = []
        if len( argSpecDefaults ):
            argSpecNames = [ name for name in self._argSpecs.args ]
            argSpecDefaults.reverse()
            argSpecNames.reverse()
            argSpecNames = argSpecNames[ 0:len( argSpecDefaults ) ]
            pos = 0
            for argSpecName in argSpecNames:
                self._defaults[ argSpecName ] = argSpecDefaults[ pos ]
                pos += 1
    # Set up the default structure:
    results = {
        'name':None,
        'description':None,
        'hasDefault':False,
        'expects':( object, ),
        }
    # Type- and value-check name
    if type( name ) not in ( str, unicode ):
        raise TypeError( '%s._createArgumentMetadata expects a string or '
            'unicode text-value that is the name of an argument in the '
            'callable for its "name" argument, but was passed '
            '"%s" (%s)' % ( self.__class__.__name__, name, 
            type( name ).__name__ )
        )
    if name not in self._argSpecs.args:
        raise TypeError( '%s._createArgumentMetadata expects a string or '
            'unicode text-value that is the name of an argument in the '
            'callable for its "name" argument, but was passed "%s" (%s), '
            'which does not appear in the list of known arguments for '
            '%s (%s)' % ( self.__class__.__name__, name, 
                type( name ).__name__, callable.__name__, 
                ', '.join( self._argSpecs.args )
            )
        )
    # Name checks out, so add it to the structure.
    results[ 'name' ] = name
    # Type- and value-check description. Any non-empty value is kosher, 
    # since a description should, in theory, be unrestricted.
    if type( description ) not in ( str, unicode ):
        raise TypeError( '%s._createArgumentMetadata expects a non-empty '
            'string or unicode text-value for its "description" argument, '
            'but was passed "%s" (%s)' % ( self.__name__, description, 
                type( description ).__name__ ) )
    if not description.strip():
        raise ValueError( '%s._createArgumentMetadata expects a non-empty '
            'string or unicode text-value for its "description" argument, '
            'but was passed "%s" (%s)' % ( self.__name__, description, 
                type( description ).__name__ ) )
    # Description checks out, so add it to the structure.
    results[ 'description' ] = description
    # Check for default values, and if there is one specified, store it as 
    # well.
    if name in self._defaults.keys():
        results[ 'hasDefault' ] = True
        results[ 'defaultValue' ] = self._defaults[ name ]
    if expects:
        # Type-check members of expects
        results[ 'expects' ] = expects
    # Return the structure
    return results

The AddArgument method

def AddArgument( self, callable, name, description, *expects ):
    """
Adds argument-metadata for the specified argument to the instance's 
documentation-metadata."""
    self._arguments[ name ] = self._createArgumentMetadata( 
        callable, name, description, *expects )

All of the actual decorators themselves I'm going to define as class-methods of a static class named describe. I thought about just building out a set of functions sitting in a module at first, but it seems to me that importing a single class that has all of them as members is much easier to manage in-code than having to remember all their names, and more efficient than importing everything from the module.

The argument decorator-method

@classmethod
def argument( cls, name, description, *expects ):
    """
Decorates a function or method by attaching documentation-metadata about a 
specific named argument, including a description of the argument, and an 
optional type-specification. Also keeps track of default argument values."""
    # Type-check name it must be a string or unicode value
    if type( name ) not in ( str, unicode ):
        raise TypeError( 'describe.argument expects a non-empty string or '
            'unicode text-value containing an argument-name for the item '
            'being decorated, but was passed "%s" (%s)' % ( 
                name, type( name ).__name__ ) )
    # Type- and value-check description
    if type( description ) not in ( str, unicode ):
        raise TypeError( '%s.argument expects a non-empty '
            'string or unicode text-value for its "description" argument, '
            'but was passed "%s" (%s)' % ( cls.__name__, description, 
                type( description ).__name__ ) )
    if not description.strip():
        raise ValueError( '%s.argument expects a non-empty '
            'string or unicode text-value for its "description" argument, '
            'but was passed "%s" (%s)' % ( cls.__name__, description, 
                type( description ).__name__ ) )
    # Type- and value-check expects, if it exists, or set it to ( object, )
    def _argumentDecorator( decoratedItem ):
        """
Performs the actual argument decoration"""
        # Make sure the decorated item has a _documentation attribute, and 
        # if it doesn't, create and attach one:
        try:
            _documentation = decoratedItem._documentation
        except AttributeError:
            decoratedItem._documentation = api_documentation( decoratedItem )
            _documentation = decoratedItem._documentation
        # If we reach this point, then we should add the item to the 
        # metadata, but first we have to create the metadata entry...
        _documentation.AddArgument( decoratedItem, name, description, *expects )
        # Return the decorated item!
        return decoratedItem
    return _argumentDecorator

An example function with argument decoration

@describe.argument( 'arg1', 'Spam (function) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Spam (function) arg2 description', str, unicode, type( None ) )
def Spam( arg1, arg2=None, *args, **kwargs ):
"""Spam (function) original doc-string."""
    pass

Working backwards from the definition and decoration of the Spam function, here's what happens during the parsing phase of execution:

• The describe.argument decorator for the arg2 argument fires (remember that decorators fire from the inside out):
  • The name, description and expected types (expects) are checked, and if any are invalid, and error is raised, ending the decoration execution;
  • The _argumentDecorator function, with the name, description and expects values from the surrounding method-call is generated and returned;
• The _argumentDecorator function returned by the decorator fires, with the Spam function as its implicit argument:
  • It checks for (and if necessary creates) an api_documentation instance attached to the Spam function;
  • It calls the AddArgument method of the attached api_documentation instance, passing the Spam function, and the original name, description and expects values:
  • AddArgument calls _createArgumentMetadata with the passed arguments; and
  • attaches the results to the _arguments of the instance;
  • The original Spam function (with the attached and partially-populated api_documentation instance) is returned
• The describe.argument decorator for the arg1 argument fires, repeating the process above, except that:
  • The returned _argumentDecorator function will have the name, description and expects values provided by the arg1 values;
  • The first decoration-pass will have created the api_documentation instance attached to the Spam function, so that instance will be used instead of creating a new one during the execution of the current _argumentDecorator function returned;

The resulting structure of the arguments metadata after all decoration is complete is:

{
'arg1':{
    'description': 'Spam (function) arg1 description',
    'expects': (<type 'int'>, <type 'long'>, <type 'float'>),
    'hasDefault': False,
    'name': 'arg1'
    },
'arg2':{
    'defaultValue': None,
    'description': 'Spam (function) arg2 description',
    'expects': (<type 'str'>, <type 'unicode'>, <type 'NoneType'>),
    'hasDefault': True,
    'name': 'arg2'
    }
}

This is already a good match for the arguments section of the full structure noted near the beginning of this post. It even (already) includes the expected argument-types, and a default of ( object, ) is already set up to indicate expectation of any type if none is explicitly provided by the decorator-call.

It is, however, missing one thing. I mentioned earlier that I wanted to keep write-access to the metadata's data-structure read-only, in order to make it more difficult to mis-use. Ideally that would mean that alteration of existing dictionary-elements and creation of new elements would not be allowed — raising an error, ideally. That is, given the following code:

# An example of documentation decorators on a function
@describe.argument( 'arg1', 'Description of arg1' )
@describe.argument( 'arg2', 'Description of arg2' )
@describe.argument( 'arg3', 'Description of arg3' )
def MyFunction( arg1, arg2, arg3=None ):
    """
Description of function (original docstring)"""
    # TODO: Generate actual implementation here...
    raise NotImplementedError( 'MyFunction is not yet implemented' )

MyFunction._documentation.arguments[ 'arg1' ] = {}

I'd prefer that the over-write of the arg1 dict-element raise an error right there, even if the alteration made keeps a legitimate structure. As things stand right now, it's possible for the code shown above to be executed without raising an error until much later down the line, which I really don't like. Honestly, I'm not quire sure right this second how I'm going to deal with that, though I have some ideas.