Apologies...
This post turned out to be a lot longer
than I'd anticipated it would be when I started writing it. I struggled for a
while with trying to find one or more good break-points, in order to split it
into two or more smaller posts (easier to digest), but there wasn't anything
that felt like a good spot, so this is long...
So, picking up where I left of with my last post, today I'm planning to
tackle the documentation-metadata structure for argument lists. Like
arguments, arglist documentation should have a description, and should
allow but not require the specification of expected value-types. Since
only one arglist is allowed on any callable in Python, there's no need
to specify the name the way the describe.argument decorator
does for arguments, but the name should still be stored as part of the
metadata structure, if only for consistency. The data-structure should,
then, be somewhat familiar, looking much like the structure for arguments,
but would be stored as a separate metadata-item in the data-structure.
By way of example, consider the Ook test-class defined last
time, with describe.arglist decorators set up for its
methods:
class Ook( object ):
"""
Test-class."""
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', str, unicode )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
@classmethod
@describe.argument( 'arg1', 'Ook.Bleep (classmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Bleep (classmethod) arg2 description' )
@describe.arglist( 'Ook.Bleep (classmethod) arglist description' )
def Bleep( cls, arg1, arg2=None, *args, **kwargs ):
"""Ook.Bleep (classmethod) original doc-string"""
return None
@staticmethod
@describe.argument( 'arg1', 'Ook.Flup (staticmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Flup (staticmethod) arg2 description' )
@describe.arglist( 'Ook.Flup (staticmethod) arglist description', str, unicode )
def Flup( arg1, arg2, *args, **kwargs ):
"""Ook.Flup (staticmethod) original doc-string"""
return None
In its most basic form, the metadata for the args arglist
would be very simple. In context, with the metadata for the arguments, the
data-structure would look something like this:
{
'arg1': {
'description': 'Ook.Fnord (method) arg1 description',
'expects': ( <type 'int'>, <type 'long'>, <type 'float'>),
'hasDefault': False,
'name': 'arg1'
},
'arg2': {
'description': 'Ook.Fnord (method) arg2 description',
'expects': (<type 'object'>,),
'hasDefault': False,
'name': 'arg2'
},
'arglist': {
'description': 'Ook.Fnord (method) arglist description',
'expects': (<type 'str'>, <type 'unicode'>),
'name': 'args'
}
}
That would cover the most basic expected arglist usage pretty nicely.
However, there are some additional wrinkles to consider. One is that
the generally-accepted way to define functions that have variable but
specifically-sequenced arguments is to use arglists and assign values
based on the existence and position of the value in the argument-list
sequence. That explanation got away from me, so let me provide an
example. Say I want a function that might be documented as MyFunction(
[spam, [eggs, [beans]] ). That is, MyFunction
accepts any of the following:
MyFunction()MyFunction( 1 )MyFunction( 1, 2 )MyFunction( 1, 2, 3 )
where the first value is
spam
, the second is
eggs
and the
third
beans
. If any given value is provided, all of its
predecessors in the function's signature must be provided as well. So
beans
require spam and eggs. That function might look something
like this:
def MyFunction( *arguments ):
"""
An example of a function that can accept zero-to-many arguments with
specific meanings based on their position in the argument-list."""
print '#' + '-'*78 + '#'
print 'MyFunction%s called' % ( str( arguments ) )
# Get the first three items from the arguments list *plus* three
# None values so that zero-to-three values are populated (with
# None as a default if not provided)
spam, eggs, beans = tuple( ( list( arguments[ 0:3 ] ) + [ None, None, None ] )[0:3] )
if arguments:
if spam:
print 'spam .... %s' % spam
if eggs:
print 'eggs .... %s' % eggs
if beans:
print 'beans ... %s' % beans
else:
print 'No spam, eggs or beans specified'
If MyFunction is executed in any of several ways:
MyFunction()
MyFunction( 1 )
MyFunction( 1, 2 )
MyFunction( 1, 2, 3 )
MyFunction( 'spam', 'spam' )
then the results, I hope, make the idea more clear:
#------------------------------------------------------------------------------#
MyFunction() called
No spam, eggs or beans specified
#------------------------------------------------------------------------------#
MyFunction(1,) called
spam .... 1
#------------------------------------------------------------------------------#
MyFunction(1, 2) called
spam .... 1
eggs .... 2
#------------------------------------------------------------------------------#
MyFunction(1, 2, 3) called
spam .... 1
eggs .... 2
beans ... 3
#------------------------------------------------------------------------------#
MyFunction('spam', 'spam') called
spam .... spam
eggs .... spam
Given that it's possible (and even probable) that a function
or method will be defined using a variable number of specific-purpose
arguments, the question that should, hopefully, come to mind is What
if those arguments have specific descriptions or type-expectations?
The metadata structure shown so far would most emphatically not
handle that case. It would require something with more detail, along the
lines of:
{
# arguments removed for brevity...
'arglist': {
'description': 'Ook.Fnord (method) arglist description',
'expects': (<type 'str'>, <type 'unicode'>),
'name': 'args',
'sequence':[
{
'name':'spam',
'defaultValue':None,
'description':'The "spam" value to use',
'expects':(<type 'str'>, )
},
{
'name':'eggs',
'defaultValue':None,
'description':'The "eggs" value to use',
'expects':(<type 'int'>, <type 'long'>)
},
{
'name':'beans',
'defaultValue':None,
'description':'The "beans" value to use',
'expects':(<type 'bool'>, None)
},
]
}
}
Ideally, the decoration that generates that structure would look
something like this, then:
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', int, long )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', str )
@describe.arglistitem( 0, 'spam', None, 'The "spam" value to use', str, unicode )
@describe.arglistitem( 1, 'eggs', None, 'The "eggs" value to use', int, long )
@describe.arglistitem( 2, 'beans', None, 'The "beans" value to use', bool, None )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
The arguments for the describe.arglistitem decorator are:
- A sequence value: The position of the sequence-item in the
argument list's
sequence
- A name. The name is not as formal as an argument's name, since
it's really just there for programmer reference;
- A default value. There are some considerations about providing
a default value that I want to discuss, so more on that in a
bit;
- A description; and
- Zero-to-many expected types for the sequence-item. I expect this
to behave pretty much exactly like the
expects
describe.argument.
The end goal for the documentation generated should, then, look something
like this (as plain text):
--------------------------------------------------------------------------------
Fnord( self, arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Fnord (method) original doc-string
ARGUMENTS:
self .............. (instance, required): The object-instance that the method will bind to for execution.
arg1 .............. (int|long|float, required): Ook.Fnord (method) arg1 description
arg2 .............. (any, required): Ook.Fnord (method) arg2 description
args .............. (str, optional): Ook.Fnord (method) arglist description. Specific values accepted, in order, are:
spam ............ (str|unicode, optional, defaults to None): The "spam" value to use
eggs ............ (int|long, optional, defaults to None): The "eggs" value to use
beans ........... (bool|None, optional, defaults to None): The "beans" value to use
--------------------------------------------------------------------------------
There are a few caveats and items of note that I think arise from
this set-up. First and foremost is that if sequence-items are provided,
they must, eventually, resolve to a contiguous list. That is, if the
spam
and beans
items in the example above are specified,
then the sequence-item between them (eggs
) must also
be specified somewhere along the line for the documentation to be accurate
and meaningful. If eggs
was not specified as an argument
sequence item, and there was no generated display as a result, then the
documentation would indicate (inaccurately) that spam
and
beans
were the only meaningful items in the argument-sequence,
and anyone relying on the documentation would not expect their beans
values to appear as eggs
. That would be a problem. To prevent that
from becoming an issue, I'm planning to put a default metadata-structure
in place in any blank
spots in the sequence, probably as part
of the process of creating or updating the sequence
Another consideration stems from the fact that a Python argument-list
is, for all practical purposes, allowed to be arbitrarily long. Even if,
as in this case, some number of arguments are documented as having specific
meanings in the sequence, *args can have additional
items after those. So, for example, a call like
Fnord( 'spam', 1, True, 'these', 'are', 'additional', 'values' )
is perfectly valid from the standpoint of the syntax of the language,
even if it makes no sense in the context of the function.
How should that kind of scenario be handled? Honestly, I don't know. In
cases where the function is only
expecting those first three
(or however-many) sequenced items, I'd implement sequence-length checking
and raise an error if too many arguments were supplied. But that wouldn't
help in cases where there's a legitimate need for a potentially-infinite
remaining set of sequence-items, and there are legitimate uses for that
sort of structure. I have in mind a variant of a
min or
max function that supplies one or more cut-off values –
something that is equivalent to
give me the minimum/maximum value of all
supplied values that is also greater than x and optionally less than y,
though that case might be better handled with an optional standard argument
for y. In any case, the potential for legitimate, continuous sequences
of values after the ones that have specific meaning exists, and should
be represented in the documentation somehow. My gut feeling is to allow
a special sequence-number, -1, for example, that indicates that the
documentation-structure applies to all remaining values. Something like
this, maybe:
@describe.arglistitem( -1, 'values', None, 'The values to process', int, long )
def Fnord( self, arg1, arg2, *args, **kwargs ):
# ...
Another wrinkle is that if there are specific meanings associated with
each argument in the sequence and the sequence is limited, it's
possible (probable, even, maybe?) that the normal
type-specification
of the argument-list is meaningless. That is, using the same Fnord
function above, but constrained in the function so that it
only accepts the spam,
eggs,
and beans
items,
the general (str, optional)
description of the argument-list as a
whole is unnecessary as well as being inaccurate. I'm almost
certain that it would be possible to write code that would handle all
of the various logically-legitimate variants of an argument-list as far
as documentation-decoration were concerned, but it seems to me that doing
so would complicate things more than I'd like — the premise of
these decorators is Documentation Made Easy,
after all, not
Documentation Made Complicated.
I think, in order to keep things
easy,
a reasonable balance between simplicity and hard-and-fast
structure probably requires a rule-set about like this one:
- Argument-lists with no associated sequence simply use the
baseline decorator, with results that look like a decorated
argument:
args .............. (str, optional): Ook.Fnord (method) arglist description.
- Argument-lists that specify one-to-many sequence items, but
that do not have an
all-remaining-items
sequence-item specified do not show type-information
in documentation except those associated with the sequence-items:
args .............. Ook.Fnord (method) arglist description. Specific values accepted, in order, are:
spam ............ (str|unicode, optional, defaults to None): The "spam" value to use
eggs ............ (int|long, optional, defaults to None): The "eggs" value to use
- Argument-lists that specify one-to-many sequence-items and
an
all-remaining-items
at the end of the sequence show the
type-specification of the argument-list as a whole for those,
remaining items, and use the all-remaining-items
description. Using the example -1
describe.arglistitem
decorator-call above, that would yield:
args .............. Ook.Fnord (method) arglist description. Specific values accepted, in order, are:
spam ............ (str|unicode, optional, defaults to None): The "spam" value to use
eggs ............ (int|long, optional, defaults to None): The "eggs" value to use
values* ......... (str, optional, defaults to None): The values to process.
The complete decoration for that last case would look like this, then:
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', int, long )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description' )
@describe.arglistitem( 0, 'spam', None, 'The "spam" value to use', str, unicode )
@describe.arglistitem( 1, 'eggs', None, 'The "eggs" value to use', int, long )
@describe.arglistitem( -1, 'values', None, 'The values to process', str )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
So: Implementation time! There are a few things that need to be done:
- The existing
api_documentation class needs to be
altered to set up default argument-list metadata structures;
- The
describe.arglist method needs to be built;
- The
describe.arglistitem method needs to be built;
- The
__str__ method of the api_documentation
class needs to be altered to output the argument-list metadata;
The changes to
api_documentation are pretty straightforward,
given the pattern established with the various argument-specific
functionality already in place:
class api_documentation( object ):
"""
Provides a common collection-point for all API documentation metadata for a
programmatic element (class, function, method, whatever)."""
# ...
#####################################
# Instance property-getter methods #
#####################################
# ...
def _GetArglist( self ):
"""
Gets the dictionary of the argument-list that the instance has been used to
document/describe:
'arglist': {
'description':<str|unicode>,
'expects':<tuple<types>>,
'name':<str|unicode>,
'sequence':[
{
'name':<str|unicode>,
'defaultValue':<any>,
'description':<str|unicode>,
'expects':<tuple<types>>,
},
]
}"""
return self._arglist
# ...
#####################################
# Instance Properties #
#####################################
# ...
arglist = property( _GetArglist, None, None,
_GetArglist.__doc__.strip() )
# ...
#####################################
# Instance Methods #
#####################################
# ...
def _createArglistMetadata( self, callable, description, *expects ):
"""
Creates and returns a metadata-dictionary for a function or method arglist.
RETURNS
- dict of metadata values
ARGUMENTS:
callable ......... (callable, required) The function or method whose arglist-
metadata is being created
description ...... (str|unicode, required) The description of the arglist.
expects .......... (tuple of types, optional, if not specified, stores
( object, ) as the value) The expected type(s) of the
arglist.
RAISES:
- TypeError:
+ If the supplied callable is not a function, method, or generator
+ If a type for an argument is invalid
- ValueError:
+ If a value for an argument is invalid
"""
# Type-check callable
if not inspect.isfunction( callable ) and \
not inspect.isgenerator( callable ) and \
not inspect.ismethod( callable ):
raise TypeError( '%s._createArglistMetadata expects a callable '
'(function, generator or method) in its "callable" argument, '
'but was passed "%s" (%s)' % ( self.__name__, callable,
type( callable ).__name__ ) )
# Get the argspecs of the callable if necessary
if not self._argSpecs:
self._argSpecs = inspect.getargspec( callable )
# Set up the default structure:
results = {
'name':self._argSpecs.varargs,
'description':None,
'expects':( object, ),
'sequence':[]
}
# Type- and value-check description. Any non-empty value is kosher,
# since a description should, in theory, be unrestricted.
if type( description ) not in ( str, unicode ):
raise TypeError( '%s._createArgumentMetadata expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( self.__name__, description,
type( description ).__name__ ) )
if not description.strip():
raise ValueError( '%s._createArgumentMetadata expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( self.__name__, description,
type( description ).__name__ ) )
# Description checks out, so add it to the structure.
results[ 'description' ] = description
if expects:
results[ 'expects' ] = expects
# Return the structure
return results
# ...
def AddArglist( self, callable, description, *expects ):
"""
Adds arglist-metadata for the specified argument to the instance's
documentation-metadata."""
self._arglist = self._createArglistMetadata(
callable, description, *expects )
As I was working through this chunk of changes, it occurred to me
that I've never actually shown the __str__ method of the
api_documentation class, which is what is generating the
output that I have shown for documentation thus far. I'm going
to hold off on going into that in any great detail for a while, though,
because while it's functional at present, and sufficient for demonstrating
that the metadata-generation is working, it's pretty clunky code and it
doesn't generate what I really want it to, ultimately. It's missing
hanging indentation and variable-length dot-leaders that I'd like to
have in pplace to keep printed
documentation well-formatted and
with a maximum line-length of 80 characters. I'll plan to hit that after
all of the documentation-generation for arguments is complete.
In the meantime, looking at this last chunk of code: The
AddArglist method will have to be revisited once the
describe.arglistitem decorator is implemented. While it
works fine with our Ook test-class (above) so far after
adding metadata-decoration for the methods' argument-lists, once
argument-list items are added into the mix, it'll overwrite an
existing argument-list metadata structure that may well have
valid documentation-metadata if the sequence of decorators isn't very
specific, and I don't want that.
The decoration on the Ook class' methods, for proving
that the decoration works, looks like this now:
class Ook( object ):
"""
Test-class."""
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', bool, None )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', int, long, float )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
@classmethod
@describe.argument( 'arg1', 'Ook.Bleep (classmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Bleep (classmethod) arg2 description' )
@describe.arglist( 'Ook.Bleep (classmethod) arglist description' )
def Bleep( cls, arg1, arg2=None, *args, **kwargs ):
"""Ook.Bleep (classmethod) original doc-string"""
return None
@staticmethod
@describe.argument( 'arg1', 'Ook.Flup (staticmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Flup (staticmethod) arg2 description' )
@describe.arglist( 'Ook.Flup (staticmethod) arglist description' )
def Flup( arg1, arg2, *args, **kwargs ):
"""Ook.Flup (staticmethod) original doc-string"""
return None
And the related decorator method, describe.arglist, is:
@classmethod
def arglist( cls, description='No description provided', *expects ):
"""
Decorates a function or method by attaching documentation-metadata about its
argument-list, including a description of the argument-list, and an optional
type-specification set."""
# Type- and value-check description
if type( description ) not in ( str, unicode ):
raise TypeError( '%s.argument expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, description,
type( description ).__name__ ) )
if not description.strip():
raise ValueError( '%s.argument expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, description,
type( description ).__name__ ) )
# Type- and value-check expects, if it exists, or set it to ( object, )
def _arglistDecorator( decoratedItem ):
"""
Performs the actual arglist decoration"""
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = api_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should add the item to the
# metadata, but first we have to create the metadata entry...
_documentation.AddArglist( decoratedItem, description, *expects )
# Return the decorated item!
return decoratedItem
return _arglistDecorator
This yields the following documentation on Ook, when printed:
--------------------------------------------------------------------------------
Fnord( self, arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Fnord (method) original doc-string
ARGUMENTS:
self .............. (instance, required): The object-instance that the method will bind to for execution.
arg1 .............. (bool|None, required): Ook.Fnord (method) arg1 description
arg2 .............. (any, required): Ook.Fnord (method) arg2 description
args .............. (int|long|float): Ook.Fnord (method) arglist description
--------------------------------------------------------------------------------
Bleep( cls, arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Bleep (classmethod) original doc-string
ARGUMENTS:
cls ............... (class, required): The class that the method will bind to for execution.
arg1 .............. (int|long|float, required): Ook.Bleep (classmethod) arg1 description
arg2 .............. (any, optional, defaults to None): Ook.Bleep (classmethod) arg2 description
args .............. (any): Ook.Bleep (classmethod) arglist description
--------------------------------------------------------------------------------
Flup( arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Flup (staticmethod) original doc-string
ARGUMENTS:
arg1 .............. (int|long|float, required): Ook.Flup (staticmethod) arg1 description
arg2 .............. (any, required): Ook.Flup (staticmethod) arg2 description
args .............. (any): Ook.Flup (staticmethod) arglist description
--------------------------------------------------------------------------------
I mentioned earlier that the logic in the AddArglist method
of api_documentation would need to be altered, and I hinted
at why that would be necessary. Here's a more detailed breakdown
of that issue. Consider the following alterations to Ook.Fnord
in the test-class:
class Ook( object ):
"""
Test-class."""
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', bool, None )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', int, long, float )
@describe.arglistitem( 0, 'arg1', 'Ook.Fnord.args[0] description', float )
@describe.arglistitem( 1, 'arg2', 'Ook.Fnord.args[1] description', int, long )
@describe.arglistitem( 2, 'arg3', 'Ook.Fnord.args[2] description', str, unicode )
@describe.arglistitem( -1, 'values', 'Ook.Fnord.args[3] (values) description', str, unicode )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
The decorator-sequence, I hope, makes good, solid sense to a developer.
Document the argument list, then document the expected items in that list,
in the order they are going to appear in a call. The documentation decorators
also include an all remaining values
item, as discussed before. The
problem is that while the describe decorator-methods
are called in the order specified, the underlying real
decorator
methods are executed in the opposite order. After dropping some print
statements into a bare-bones/stub arglistitem method and the current
version of arglist, this becomes pretty apparent:
- Calling describe.arglist( Ook.Fnord (method) arglist description, (<type 'int'>, <type 'long'>, <type 'float'>) )
- Calling describe.arglistitem( 0, arg1, Ook.Fnord.args[0] description, (<type 'float'>,) )
- Calling describe.arglistitem( 1, arg2, Ook.Fnord.args[1] description, (<type 'int'>, <type 'long'>) )
- Calling describe.arglistitem( 2, arg3, Ook.Fnord.args[2] description, (<type 'str'>, <type 'unicode'>) )
- Calling describe.arglistitem( -1, values, Ook.Fnord.args[3] (values) description, (<type 'str'>, <type 'unicode'>) )
+ Calling _arglistitemDecorator( <function Fnord at 0x7f3a58c0ec08> )
* sequence ...... -1
* name .......... values
* description ... Ook.Fnord.args[3] (values) description
* expects ....... (<type 'str'>, <type 'unicode'>)
+ Calling _arglistitemDecorator( <function Fnord at 0x7f3a58c0ec08> )
* sequence ...... 2
* name .......... arg3
* description ... Ook.Fnord.args[2] description
* expects ....... (<type 'str'>, <type 'unicode'>)
+ Calling _arglistitemDecorator( <function Fnord at 0x7f3a58c0ec08> )
* sequence ...... 1
* name .......... arg2
* description ... Ook.Fnord.args[1] description
* expects ....... (<type 'int'>, <type 'long'>)
+ Calling _arglistitemDecorator( <function Fnord at 0x7f3a58c0ec08> )
* sequence ...... 0
* name .......... arg1
* description ... Ook.Fnord.args[0] description
* expects ....... (<type 'float'>,)
+ Calling _arglistDecorator( <function Fnord at 0x7f3a58c0ec08> )
* description ... Ook.Fnord (method) arglist description
* expects ....... (<type 'int'>, <type 'long'>, <type 'float'>)
Since the arglistitem decorators execute before the
arglist decorator that they relate to, the method needs to establish
the underlying bare-minimum argument-list metadata structure in order to be able to
do what it needs to do. Following the pattern that's been established thus
far, that means that AddArglistItem needs to create it:
def AddArglistItem( self, callable, description, *expects ):
"""
Adds arglist-metadata for the specified argument to the instance's
documentation-metadata."""
if not self._arglist:
self._arglist = self._createArglistMetadata(
callable )
That also means that AddArglist needs to check for an
existing metadata-structure, and modify it if it already exists, rather
than just creating a new one every single time:
def AddArglist( self, callable, description, *expects ):
"""
Adds arglist-metadata for the specified argument to the instance's
documentation-metadata."""
if not self._arglist:
# If it doesn't exist, create it
self._arglist = self._createArglistMetadata(
callable, description, *expects )
else:
# If it DOES exist, update it (partially)
self._arglist[ 'description' ] = description
self._arglist[ 'expects' ] = expects
That should take care of the potential collisions, and yields documentation
results for Ook.Fnord, Ook.Bleep and Ook.Flup
methods that are identical to those before the modification, even with the
arglistitem decorations in place.
It occurs to me that arglistitem documentation would probably
be at least slightly easier to implement if the sequence were
only a list of non-final
argument-list items, rather than
allowing (or requiring) a final
list-item to be tracked in the same
list-structure. The process of building the list of argument-list items would
not have to contend with a length that fluctuates based on more than one
condition in that case. The decoration-call wouldn't need to change at all,
still using -1 as an indicator-value for the final
items
in the argument-list, just the handling of that indicator during decoration.
That would require a minor change to the data-structure, though, looking
something like so:
{
# arguments removed for brevity...
'arglist': {
'description':<str|unicode>,
'expects':<tuple<types>>,
'name':<str|unicode>,
'sequence':[
{
'name':<str|unicode>,
'defaultValue':<any>,
'description':<str|unicode>,
'expects':<tuple<types>>,
},
# ...
],
'final': {
'name':<str|unicode>,
'defaultValue':<any>,
'description':<str|unicode>,
'expects':<tuple<types>>,
}
}
}
Structurally, a final item is identical to any one of the
sequence items, so the process for generating the metadata
should be easily adaptable, if not actually identical across the board.
Another possibility that occurs to me is that maybe the sequence
items (possibly including the final item) could be stored in
the metadata-structure not as a list, but as a dictionary. The advantage to
that approach is that there would be less need, perhaps no need at all, for
rebuilding the sequence items list every time a new item is added.
Looking at the Ook.Fnord method with the sequence and final items
above, and the processing-sequence that occurred, the steps that would need
to happen during execution are (in the order they were reported above):
_arglistitemDecorator( -1, values, ... is called:
arglist: does not exist, so create it with minimal data;
final: does not exist, so create it and populate it;
sequence: does not exist, but isn't being populated, so leave it be;
_arglistitemDecorator( 2, arg3, ... is called:
arglist: exists, use as-is;
final: exists, but isn't being populated, so leave it be;
sequence: does not exist, so create it and add the item to it;
_arglistitemDecorator( 1, arg2, ... is called:
arglist: exists, use as-is;
final: exists, but isn't being populated, so leave it be;
sequence: exists, so add the item to it;
_arglistitemDecorator( 0, arg1, ... is called:
arglist: exists, use as-is;
final: exists, but isn't being populated, so leave it be;
sequence: exists, so add the item to it;
_arglistDecorator( ... is called:
arglist: exists, populate with description and expects as needed;
final: exists, but isn't being populated, so leave it be;
sequence: exists, but isn't being populated, so leave it be;
If
sequence is a list, then each time that it is being touched, it
would, at least potentially, need to be analyzed and rebuilt from its current
state with the addition of the new item. The same sequence of events, then, from
the perspective of the
sequence list, would look like so:
_arglistitemDecorator( -1, values, ... is called:
sequence: does not exist, but isn't being populated, so leave it be;
Nothing to do here
_arglistitemDecorator( 2, arg3, ... is called:
sequence: does not exist, so create it and add the item to it;
At this point, we know that there will eventually be three items
[0..2], so we should create a list 3 elements in length, and
insert the new element at index 2;
_arglistitemDecorator( 1, arg2, ... is called:
sequence: exists, so add the item to it;
At this point, we know that the list exists, and that
the index of the item is within its range of indexed values, so we can
just insert it.
_arglistitemDecorator( 0, arg1, ... is called:
sequence: exists, so add the item to it;
At this point, we know that the list exists, and that
the index of the item is within its range of indexed values, so we can
just insert it.
_arglistDecorator( ... is called:
sequence: exists, but isn't being populated, so leave it be;
Nothing to do here
If, however, the decorator sequence is out of kilter, so that the sequence
of sequences is not in order (say, 1, 0, 2),
each call has to check
and reconcile the current sequence against the incoming index:
_arglistitemDecorator( -1, values, ... is called:
Nothing to do here
_arglistitemDecorator( 1, arg2, ... is called:
sequence: exists, so add the item to it;
At this point, we know that there will eventually be two items
[0..1], so we should create a list 2 elements in length, and
insert the new element at index 1;
_arglistitemDecorator( 0, arg1, ... is called:
sequence: exists, so add the item to it;
At this point, we know that the list exists, and that
the index of the item is within its range of indexed values, so we can
just insert it.
_arglistitemDecorator( 2, arg3, ... is called:
sequence: does not exist, so create it and add the item to it;
At this point, we know that there will eventually be three items
[0..2], so we should create a list 3 elements in length,
copy the existing sequence into the new sequence, and insert the new
element appropriately.
_arglistDecorator( ... is called:
sequence: exists, but isn't being populated, so leave it be;
Nothing to do here
It would get even more complicated to implement to allow for gaps in the provided
decorator-sequence values. Imagine the process-steps for (2, 0, 1, 3, 5, 4) for
example: The first three items in the sequence would be fine, then sequence-items
3, 5 and 4 would
each require the creation of a new list to replace
the old one, population with old values and insertion/replacement of the new
missing
value at the appropriate index-position.
That feels messy and brittle to me, but it's worth confirming or
disproving that feeling, so I'm going to write some code that uses that basic
process, and feed it some simple values using the sequence I had concerns
about (2, 0, 1, 3, 5, 4):
def addToList( theList, theIndex, theItem ):
# First, check the index to see if the current list is long enough
if len( theList ) -1 < theIndex:
# The current list is too short to accommodate the supplied
# index-value, so create a new list that's long enough and
# populate it with the elements of the original list.
theList = [
theList[ index ]
if len( theList ) > index and theList[ index ]
else None
for index in range( 0, theIndex + 1 )
]
# THEN set the appropriate index-location to the value
theList[ theIndex ] = theItem
else:
# Otherwise, just set the appropriate index-location to the
# value
theList[ theIndex ] = theItem
# Return the list, since we cannot pass it by reference and modify
# the original list in place.
return theList
myList = []
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 2, 'two' )
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 0, 'zero' )
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 1, 'one' )
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 3, 'three' )
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 5, 'five' )
print '#' + '-'*78 + '#'
print myList
myList = addToList( myList, 4, 'four' )
print '#' + '-'*78 + '#'
print myList
I'll admit that I'm surprised that it's not as bad as I'd expected.
This code may not be as optimal as it could be (though I was able to use
a conditional list-comprehension to generate the new list, and that feels
pretty good). I haven't run across anything in the standard Python libraries
(yet) that does exactly what's needed: simultaneously growing and populating
a list where the sequence is important. And it actually works,
pretty much first try out of the gate, which I'm very happy about:
#------------------------------------------------------------------------------#
[]
#------------------------------------------------------------------------------#
[None, None, 'two']
#------------------------------------------------------------------------------#
['zero', None, 'two']
#------------------------------------------------------------------------------#
['zero', 'one', 'two']
#------------------------------------------------------------------------------#
['zero', 'one', 'two', 'three']
#------------------------------------------------------------------------------#
['zero', 'one', 'two', 'three', None, 'five']
#------------------------------------------------------------------------------#
['zero', 'one', 'two', 'three', 'four', 'five']
Using this proof-of-concept function as a baseline, and dropping
default undefined
argument-list items into undefined positions
in the sequence (instead of None as used in the example)
actually feels pretty good, the more I think on it.
With all of these thought out, the relevant implementations are:
class api_documentation( object ):
"""
Provides a common collection-point for all API documentation metadata for a
programmatic element (class, function, method, whatever)."""
#####################################
# Class attributes (and instance- #
# attribute default values) #
#####################################
# ...
#####################################
# Instance Methods #
#####################################
# ...
def AddArglistItem( self, callable, sequence, name, description, *expects ):
"""
Adds arglist-metadata for the specified argument to the instance's
documentation-metadata."""
if not self._arglist:
self._arglist = self._createArglistMetadata(
callable )
# First, check the index to see if the current list is long enough
if sequence >= 0:
if len( self._arglist[ 'sequence' ] ) -1 < sequence:
# The current list is too short to accommodate the supplied
# index-value, so create a new list that's long enough and
# populate it with the elements of the original list.
self._arglist[ 'sequence' ] = [
self._arglist[ 'sequence' ][ index ]
if len( self._arglist[ 'sequence' ] ) > index and self._arglist[ 'sequence' ][ index ]
else {
'name':'unnamed',
'description':'No description provided',
'expects':( object, ),
}
for index in range( 0, sequence + 1 )
]
# THEN set the appropriate index-location to the value
self._arglist[ 'sequence' ][ sequence ] = {
'name':name,
'description':description,
}
if expects:
self._arglist[ 'sequence' ][ sequence ][ 'expects' ] = expects
else:
self._arglist[ 'sequence' ][ sequence ][ 'expects' ] = ( object, )
else:
# Otherwise, just set the appropriate index-location to the
# value
self._arglist[ 'sequence' ][ sequence ] = {
'name':name,
'description':description,
}
if expects:
self._arglist[ 'sequence' ][ sequence ][ 'expects' ] = expects
else:
self._arglist[ 'sequence' ][ sequence ][ 'expects' ] = ( object, )
elif sequence == -1:
self._arglist[ 'final' ] = {
'name':name,
'description':description,
}
if expects:
self._arglist[ 'final' ][ 'expects' ] = expects
else:
self._arglist[ 'final' ][ 'expects' ] = ( object, )
else:
raise RuntimeError( 'AddArglistItem expects a sequence value of -1 or greater.' )
class describe( object ):
"""
Nominally-static class (not intended to be instantiated) that provides the
actual functionality for generating documentation-metadata structures."""
# ...
@classmethod
def arglistitem( cls, sequence, name, description='No description provided', *expects ):
"""
Decorates a function or method by attaching documentation-metadata about an
argument-list item at a specific position/index in the sequence of values
expected/accepted by the argument-list."""
# Type-check sequence it must be a long or int value
if type( sequence ) not in ( int, long ):
raise TypeError( 'describe.argument expects an integer or long '
'numeric value containing an index-location for the item '
'being decorated, greater than or equal to -1, but was passed '
'"%s" (%s)' % ( sequence, type( sequence ).__name__ ) )
# Value-check sequence it must be >= -1
if sequence < -1:
raise ValueError( 'describe.argument expects an integer or long '
'numeric value containing an index-location for the item '
'being decorated, greater than or equal to -1, but was passed '
'"%s" (%s)' % ( sequence, type( sequence ).__name__ ) )
# Type-check name it must be a string or unicode value
if type( name ) not in ( str, unicode ):
raise TypeError( 'describe.argument expects a non-empty string or '
'unicode text-value containing an argument-name for the item '
'being decorated, but was passed "%s" (%s)' % (
name, type( name ).__name__ ) )
# Type- and value-check description
if type( description ) not in ( str, unicode ):
raise TypeError( '%s.argument expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, description,
type( description ).__name__ ) )
if not description.strip():
raise ValueError( '%s.argument expects a non-empty '
'string or unicode text-value for its "description" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, description,
type( description ).__name__ ) )
def _arglistitemDecorator( decoratedItem ):
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = api_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should add the item to the
# metadata, but first we have to create the metadata entry...
_documentation.AddArglistItem( decoratedItem, sequence, name, description, *expects )
# Return the decorated item!
return decoratedItem
return _arglistitemDecorator
So, with the updated Ook class:
class Ook( object ):
"""
Test-class."""
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', bool, None )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', int, long, float )
@describe.arglistitem( 0, 'arg1', 'Ook.Fnord.args[0] description', float )
@describe.arglistitem( 1, 'arg2', 'Ook.Fnord.args[1] description', int, long )
@describe.arglistitem( 2, 'arg3', 'Ook.Fnord.args[2] description', bool )
@describe.arglistitem( -1, 'values', 'Ook.Fnord.args[3] (values) description', str, unicode )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
@classmethod
@describe.argument( 'arg1', 'Ook.Bleep (classmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Bleep (classmethod) arg2 description' )
@describe.arglist( 'Ook.Bleep (classmethod) arglist description' )
def Bleep( cls, arg1, arg2=None, *args, **kwargs ):
"""Ook.Bleep (classmethod) original doc-string"""
return None
@staticmethod
@describe.argument( 'arg1', 'Ook.Flup (staticmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Flup (staticmethod) arg2 description' )
@describe.arglist( 'Ook.Flup (staticmethod) arglist description' )
def Flup( arg1, arg2, *args, **kwargs ):
"""Ook.Flup (staticmethod) original doc-string"""
return None
the resulting documentation output is:
--------------------------------------------------------------------------------
Fnord( self, arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Fnord (method) original doc-string
ARGUMENTS:
self .............. (instance, required): The object-instance that the method
will bind to for execution.
arg1 .............. (bool|None, required): Ook.Fnord (method) arg1 description
arg2 .............. (any, required): Ook.Fnord (method) arg2 description
*args ............. (int|long|float): Ook.Fnord (method) arglist description
- arg1 .......... (float): Ook.Fnord.args[0] description
- arg2 .......... (int|long): Ook.Fnord.args[1] description
- arg3 .......... (bool): Ook.Fnord.args[2] description
- values ........ (str|unicode): Ook.Fnord.args[3] (values) description
--------------------------------------------------------------------------------
Bleep( cls, arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Bleep (classmethod) original doc-string
ARGUMENTS:
cls ............... (class, required): The class that the method will bind to
for execution.
arg1 .............. (int|long|float, required): Ook.Bleep (classmethod) arg1
description
arg2 .............. (any, optional, defaults to None): Ook.Bleep (classmethod)
arg2 description
*args ............. (any): Ook.Bleep (classmethod) arglist description
--------------------------------------------------------------------------------
Flup( arg1, arg2, *args, **kwargs ) [instancemethod]
Ook.Flup (staticmethod) original doc-string
ARGUMENTS:
arg1 .............. (int|long|float, required): Ook.Flup (staticmethod) arg1
description
arg2 .............. (any, required): Ook.Flup (staticmethod) arg2 description
*args ............. (any): Ook.Flup (staticmethod) arglist description
--------------------------------------------------------------------------------
Argument-lists and their related (child?
) items are probably the
most complicated item to document. Certainly they involved a substantial
volume of code, as well as the discussion of the implementation needs and
the strategies and tactics that arose from them. The next (and final)
argument-documentation item is keyword arguments, which I expect to be
less complex, and to follow a pattern similar to the argument-list items
covered in this post.
So come back next time and I'll start digging into that... Oh, wait.
I said earlier that I'd discuss the odd decoration-structure that I used in
decorating the decorators themselves, didn't I? OK, then: As things stand
right now, the decoration of all the decorators written so far looks like
this:
######################################################
# Because the actual decorators themselves aren't #
# available to decorate themselves until this point, #
# decorate the decorators. O.o :-) #
######################################################
# describe.arglist
describe.argument( 'description',
'The description of the argument-list',
str, unicode )( describe.arglist )
describe.arglist( 'The types expected by the argument-list.',
type )( describe.arglist )
# describe.arglistitem
describe.argument( 'sequence',
'The sequence of the named item in the argument-list (zero-index)',
int, long )( describe.arglistitem )
describe.argument( 'name',
'The name of the argument-list item',
str, unicode )( describe.arglistitem )
describe.argument( 'description',
'The description of the argument-list item',
str, unicode )( describe.arglistitem )
describe.arglist( 'The types expected by the argument-list sequence-item.',
type )( describe.arglistitem )
# describe.argument
describe.argument( 'name',
'The name of the argument',
str, unicode )( describe.argument )
describe.argument( 'description',
'The description of the argument',
str, unicode )( describe.argument )
describe.arglist( 'The types expected in the argument.',
type )( describe.argument )
Remember that the decorator methods themselves return a function.
In normal
" usage, where a @decoratorName( args )-style
is set up before a decorated item, Python knows to apply the resulting function
to the next item. The automatic chaining of those decorator calls ultimately
applies every decorator to the final decorated item — a function or
method in all the cases so far. But those functions in need of decoration don't
actually exist at the point where a normal
decorator-call would
be placed to decorate them. That is, looking at arglist, for example,
the argument decorator hasn't even been defined by the
point where it would be applied as a normal decorator for arglist.
That, then, means that we have to wait until all of the decorators' definitions
are complete, and their functionality ready to use, before we can decorate
the decorators themselves.
Ultimately, that's all the free-standing
decoration is doing. Stepping
through what happens during the execution of describe.argument( ... )
( describe.arglist ):
describe.argument is called, returning a function;- that function is then called with
( describe.arglist ),
which passes describe.arglist as the decoratedItem;
- The decoration happens, returning the now-decorated
describe.arglist
It's really that simple. Admittedly, it
looks kind of strange, but that's
all that's happening under the hood.
OK, now I'll sign off...
