Another long post...
I'm hoping this isn't a pattern for all my posts, but at the same time, I'm trying to make sure that I keep all the stuff I'm writing at least somewhat logically grouped (even if I can't seem to keep it short, sweet, and to the point).
So, having given it some thought since the post before last, I think I'm going to
attack what I believe to be the simpler set of choices that I left off with in
my last post: Decorators for to-do,
fix-me,
and deprecated
items. I believe these to be the simpler choice because:
- Each of them feels to me like they should be little more than a list of string or unicode values at most; and
- The sequence of those values doesn't strike me as being significant,
at least not within the context of all
to-do
items, or all of any of the other types.
to-door
fix-meitems that should be documented. Chances are good that if there are any critical details for either, they will (should?) be documented either in the code as comments, or in some completely external document or system. All that is really relevant from an API documentation perspective is some indication that they exist, and some indication of their scope or effects, either now or as expected in the future. Any single given item, of either type, may have a lot of information associated with it, and it may even be scattered around and about several places in the code that needs the attention, but each individual item is, for all practical purposes, a single
to-door
fix-meas far as the documentation itself is concerned.
Is the sequence of those items relevant from the perspective of someone reading the documentation? It might be. But I suspect that most of the time (nearly always in my experience), it won't be. If there is some urgent need for a to-do or fix-me to be resolved, it will be communicated to the developer(s) as needed, but there is no reason to commit within the documentation to any specific resolution sequence.
So, structurally, to-do and fix-me items need not be any more complicated than a couple of lists of strings in the overall metadata-structure. Something like this:
{
'fixmes':<list<str|unicode>>,
'todos':<list<str|unicode>>,
}Deprecated items are even more simple, I think. Functionality that is being deprecated is going to fall into one of two categories that I can recall personally:
- It's going to be removed because something else does the job better
- It's going to be removed because it's no longer useful.
Here's what I expect these three decorations to look like in use, added to the
Ook class that I've been using for examples for the last few posts:
class Ook( object ):
"""
Test-class."""
# argument decorators removed for brevity
@describe.deprecated( 'Use new_Fnord instead.' )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
# argument decorators removed for brevity
def new_Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
return None
@classmethod
# argument decorators removed for brevity
@describe.deprecated( 'Will be removed by version X.YY.ZZ' )
def Bleep( cls, arg1, arg2=None, *args, **kwargs ):
"""Ook.Bleep (classmethod) original doc-string"""
return None
@staticmethod
# argument decorators removed for brevity
@describe.todo( 'Clean up output to remove empty members' )
@describe.todo( 'Change output to class with the same interface' )
@describe.fixme( 'Magic _parameters value needs to be removed' )
@describe.fixme( 'Rewrite list-loops to perform the same operations in fewer passes' )
def Flup( arg1, arg2, *args, **kwargs ):
"""Ook.Flup (staticmethod) original doc-string"""
return NoneIn the data-structure, they would look like this:
{
# argument metadata removed for brevity
'deprecated':<str|unicode>,
'fixmes':<list<str|unicode>>,
'todos':<list<str|unicode>>,
}...and that, I believe, will make their implementation easy.
Before I dive into those implementations, though it occurred to me that
the returns
metadata probably also falls neatly into one of these
models. This realization stemmed from thinking out the answer to the question
what can functions retrurn, really, when it comes right down to it?
The answer is pretty much anything, really. None is the
default if there is no explicit return defined. Strings and other text-values,
numbers, booleans, dictionaries, sequences, and objects can all be
returned. The number of permutations allowed is mathematically infinite, I
think, even if the reality is much more restricted. Given that, I asked
myself if it made any sense to even try to generate a decoration
process that could capture all of those possibilities, when writing True
if [some condition], False otherwise,
or something equally difficult is
so simple?
I think not. That, fortunately or not, falls squarely in the realm of
expecting or requiring a certain amount of discipline
in writing
documentation. That same certain amount of discipline
is already
a requirement for providing any documentation for what a function
returns in Python anyway, since there is no indication in the code itself what
(if anything) will be returned. It leaves that particular part of the documentation
in the same semi-nebulous state that pure doc-string documentation is in, but
at least it provides a ready means of identifying what part of the documentation
states what the return-value is. That, while it may not be much, is worth
something to my thinking.
I'm only going to show the detailed code for one of the two variations of these constructs (one each for a list-of-strings and single-string metadata model). Apart from some name-changes, and, of course, where they are stored in the metadata data-structure, they function identically. The same set of changes need to be made in all four cases, though:
- The existing
api_documentationclass needs to be altered to set up default storage; - A
describe.[whatever]method needs to be built; - The
__str__method of the api_documentation class needs to be altered to output the argument-list metadata; and (since I'm documenting these decorators with themselves) - The documentation-decorators need to be created for anything new that's being added to the mix.
api_documentation class:
class api_documentation( object ):
"""
Provides a common collection-point for all API documentation metadata for a
programmatic element (class, function, method, whatever)."""
# ...
#####################################
# Instance property-getter methods #
#####################################
# ...
def _GetDeprecated( self ):
"""
Gets the deprecation-information, if any, of the item that the instance has
been used to document/describe."""
return self._deprecated
# ...
def _GetReturns( self ):
"""
Gets the returns-information, if any, of the item that the instance has
been used to document/describe."""
return self._returns
# ...
#####################################
# Instance Properties #
#####################################
# ...
deprecated = property( _GetDeprecated, None, None,
_GetDeprecated.__doc__)
# ...
returns = property( _GetReturns, None, None,
_GetReturns.__doc__)
# ...Then the additions to describe:
class describe( object ):
"""
Nominally-static class (not intended to be instantiated) that provides the
actual functionality for generating documentation-metadata structures."""
# ...
@classmethod
def deprecated( cls, information ):
"""
Decorates a function or method by attaching documentation-metadata about its
deprecation-state to it."""
# Type- and value-check information
if type( information ) not in ( str, unicode ):
raise TypeError( '%s.keywordargs expects a non-empty '
'string or unicode text-value for its "information" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, information,
type( information ).__name__ ) )
if not information.strip():
raise ValueError( '%s.keywordargs expects a non-empty '
'string or unicode text-value for its "information" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, information,
type( information ).__name__ ) )
def _deprecatedDecorator( decoratedItem ):
"""
Performs the actual deprecated-state decoration"""
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = api_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should set the deprecated value
# to the information provided.
_documentation._deprecated = information
# Return the decorated item!
return decoratedItem
return _deprecatedDecoratorAt this point, the documentation for the newly-decorated Ook
class members looks like this (bearing in mind that describe.fixme
and describe.todo are not implemented yet):
-------------------------------------------------------------------------------- Fnord( self, arg1, arg2, *args, **kwargs ) [function] Ook.Fnord (method) original doc-string DEPRECATED: Use new_Fnord instead. RETURNS: None (at least until the method is implemented) ARGUMENTS: self .............. (instance, required): The object-instance that the method will bind to for execution. arg1 .............. (bool|None, required): Ook.Fnord (method) arg1 description arg2 .............. (any, required): Ook.Fnord (method) arg2 description *args ............. (int|long|float): Ook.Fnord (method) arglist description - arg1 .......... (float): Ook.Fnord.args[0] description - arg2 .......... (int|long): Ook.Fnord.args[1] description - arg3 .......... (bool): Ook.Fnord.args[2] description - values ........ (str|unicode): Ook.Fnord.args[3] (values) description **kwargs .......... Ook.Fnord keyword-arguments list description - keyword1 ...... (int|long|float, required): Ook.Fnord (method) "keyword1" description - keyword2 ...... (None|str|unicode, defaults to None): Ook.Fnord (method) "keyword2" description - keyword3 ...... (None|str|unicode): Ook.Fnord (method) "keyword3" description -------------------------------------------------------------------------------- Bleep( cls, arg1, arg2, *args, **kwargs ) [function] Ook.Bleep (classmethod) original doc-string DEPRECATED: Will be removed by version X.YY.ZZ ARGUMENTS: cls ............... (class, required): The class that the method will bind to for execution. arg1 .............. (int|long|float, required): Ook.Bleep (classmethod) arg1 description arg2 .............. (any, optional, defaults to None): Ook.Bleep (classmethod) arg2 description *args ............. (any): Ook.Bleep (classmethod) arglist description --------------------------------------------------------------------------------
The documentation output for Ook.Bleep shows exactly
the effects of not exerting that certain amount of discipline
mentioned
repeatedly earlier: It shows nothing for a return, not even the actual
None that would be returned. I'm goiong to let that sit and ferment
for a while, though — I have some ideas about how to deal with that
situation, but I want to think out the ramifications of them before I commit to
any of them.
On, then, to the list-of-strings items. Again, first the changes and additions
to api_documentation:
class api_documentation( object ):
"""
Provides a common collection-point for all API documentation metadata for a
programmatic element (class, function, method, whatever)."""
# ...
#####################################
# Instance property-getter methods #
#####################################
# ...
def _GetFixMes( self ):
"""
Gets the "FixMe" items, if any, of the item that the instance has
been used to document/describe."""
return self._fixmes
# ...
#####################################
# Instance Properties #
#####################################
# ...
fixmes = property( _GetFixMes, None, None,
_GetFixMes.__doc__ )And the changes and additions to describe:
class describe( object ):
"""
Nominally-static class (not intended to be instantiated) that provides the
actual functionality for generating documentation-metadata structures."""
# ...
#####################################
# Class Methods #
#####################################
# ...
@classmethod
def fixme( cls, information ):
"""
Decorates a function or method by attaching a "fixme" item to it."""
# Type- and value-check information
if type( information ) not in ( str, unicode ):
raise TypeError( '%s.keywordargs expects a non-empty '
'string or unicode text-value for its "information" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, information,
type( information ).__name__ ) )
if not information.strip():
raise ValueError( '%s.keywordargs expects a non-empty '
'string or unicode text-value for its "information" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, information,
type( information ).__name__ ) )
def _fixmeDecorator( decoratedItem ):
"""
Performs the actual fixme-item decoration"""
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = api_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should set the returns value
# to the information provided.
_documentation._fixmes.append( information )
# Return the decorated item!
return decoratedItem
return _fixmeDecorator
# ...And the resulting output:
-------------------------------------------------------------------------------- Flup( arg1, arg2, *args, **kwargs ) [function] Ook.Flup (staticmethod) original doc-string FIX ME: - Rewrite list-loops to perform the same operations in fewer passes - Magic _parameters value needs to be removed ARGUMENTS: arg1 .............. (int|long|float, required): Ook.Flup (staticmethod) arg1 description arg2 .............. (any, required): Ook.Flup (staticmethod) arg2 description *args ............. (any): Ook.Flup (staticmethod) arglist description TO DO: - Change output to class with the same interface - Clean up output to remove empty members --------------------------------------------------------------------------------
All of these implementations were relatively painless, and none required any helper-methods like the ones created for the arguments — ultimately, they were all either concerned with either just setting a single value, or with appending something to an existing one, so there was no need to complicate things with the decorator-helper structure that's been the pattern so far.
As far as function- and method-API documentation is concerned, then, the only remaining item is documenting what errors/exceptions a callable explicitly raises.
When actually documenting exceptions that a callable raises, there are two important pieces of information:
- What the error is; and
- Why the error happens.
TypeError if a certain argument is passed a non-text value, then the
developer knows not to pass non-text values in that argument. Since it's possible
(perhaps even likely) that multiple conditions can raise the same error-type, the
internal metadata-structure should probably be built out as a dictionary of lists:
{
'raises':<dict <Exception>:<list <str|unicode>>>,
}The keys of that dict-structure are Exception-derived
classes, which would allow the decorator-call to use the actual error-classes in
their calls:
class Ook( object ):
"""
Test-class."""
# argument decorators removed for brevity
@describe.deprecated( 'Use new_Fnord instead.' )
@describe.raises( NotImplementedError, 'if executed.' )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
raise NotImplementedError( '%s.Fnord is not yet implemented' %
self.__class__.__name )The implementation of related api_docuumentation is pretty typical
of the other items in this post, so I won't reproduce yet another variant of it and
waste your time. The decorator implementation is also pretty typical in many ways,
but this is the first time that I've used this sort of dictionary structure, so it
probably bears showing:
@classmethod
def raises( cls, errorType, errorCondition ):
"""
Decorates a function or method by attaching documentation-metadata about its
deprecation-state to it."""
# Type- and value-check errorType
if not issubclass( errorType, Exception ):
raise TypeError( '%s.raises expects a non-empty '
'string or unicode text-value for its "errorCondition" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, errorCondition,
type( errorCondition ).__name__ ) )
# Type- and value-check errorCondition
if type( errorCondition ) not in ( str, unicode ):
raise TypeError( '%s.raises expects a non-empty '
'string or unicode text-value for its "errorCondition" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, errorCondition,
type( errorCondition ).__name__ ) )
if not errorCondition.strip():
raise ValueError( '%s.raises expects a non-empty '
'string or unicode text-value for its "information" argument, '
'but was passed "%s" (%s)' % ( cls.__name__, errorCondition,
type( errorCondition ).__name__ ) )
def _raisesDecorator( decoratedItem ):
"""
Performs the actual raises-state decoration"""
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = api_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should set the raises value
# to the information provided.
try:
_documentation._raises[ errorType ].append( errorCondition )
except KeyError:
_documentation._raises[ errorType ] = [ errorCondition ]
# Return the decorated item!
return decoratedItem
return _raisesDecorator
With all of these decorators in place, here is the decorated code and resulting documentation for the Oook.Fnord method:
class Ook( object ):
"""
Test-class."""
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', bool, None )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', int, long, float )
@describe.arglistitem( 0, 'arg1', 'Ook.Fnord.args[0] description', float )
@describe.arglistitem( 1, 'arg2', 'Ook.Fnord.args[1] description', int, long )
@describe.arglistitem( 2, 'arg3', 'Ook.Fnord.args[2] description', bool )
@describe.arglistitem( -1, 'values', 'Ook.Fnord.args[3] (values) description', str, unicode )
@describe.keywordargs( 'Ook.Fnord keyword-arguments list description' )
@describe.keyword( 'keyword1', 'Ook.Fnord (method) "keyword1" description', int, long, float, required=True )
@describe.keyword( 'keyword2', 'Ook.Fnord (method) "keyword2" description',None, str, unicode, default=None )
@describe.keyword( 'keyword3', 'Ook.Fnord (method) "keyword3" description',None, str, unicode )
@describe.deprecated( 'Use new_Fnord instead.' )
@describe.returns( 'None (at least until the method is implemented)' )
@describe.raises( NotImplementedError, 'if called' )
@describe.todo( 'Clean up output to remove empty members' )
@describe.todo( 'Change output to class with the same interface' )
@describe.fixme( 'Magic _parameters value needs to be removed' )
@describe.fixme( 'Rewrite list-loops to perform the same operations in fewer passes' )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
raise NotImplementedError( '%s.Fnord is not yet implemented' %
self.__class__.__name__ )-------------------------------------------------------------------------------- Fnord( self, arg1, arg2, *args, **kwargs ) [function] Ook.Fnord (method) original doc-string DEPRECATED: Use new_Fnord instead. RETURNS: None (at least until the method is implemented) FIX ME: - Rewrite list-loops to perform the same operations in fewer passes - Magic _parameters value needs to be removed ARGUMENTS: self .............. (instance, required): The object-instance that the method will bind to for execution. arg1 .............. (bool|None, required): Ook.Fnord (method) arg1 description arg2 .............. (any, required): Ook.Fnord (method) arg2 description *args ............. (int|long|float): Ook.Fnord (method) arglist description - arg1 .......... (float): Ook.Fnord.args[0] description - arg2 .......... (int|long): Ook.Fnord.args[1] description - arg3 .......... (bool): Ook.Fnord.args[2] description - values ........ (str|unicode): Ook.Fnord.args[3] (values) description **kwargs .......... Ook.Fnord keyword-arguments list description - keyword1 ...... (int|long|float, required): Ook.Fnord (method) "keyword1" description - keyword2 ...... (None|str|unicode, defaults to None): Ook.Fnord (method) "keyword2" description - keyword3 ...... (None|str|unicode): Ook.Fnord (method) "keyword3" description RAISES: - NotImplementedError + if called TO DO: - Change output to class with the same interface - Clean up output to remove empty members
There are a few other things that I want to do before I call this complete.
First and foremost, is a name-change for api_documentation. That
class is not a full API documentation, but merely the collection of
documentation-metadata for a single function or method (a callable), so I'm
going to refactor-rename it to callable_documentation.
The next thing I'm going to do is work up an HTML-savvy documentation-output mechanism for it. The text-only documentation is important, and I'll come back to it shortly, but for presentation-purposes here, I'd really like to be able to generate something that looks better.
Like, say, this:
- Rewrite list-loops to perform the same operations in fewer passes
- Magic _parameters value needs to be removed
- self
- (instance, required): The object-instance that the method will bind to for execution.
- arg1
- (bool|None, required): Ook.Fnord (method) arg1 description
- arg2
- (any, required): Ook.Fnord (method) arg2 description
- *args
- (int|long|float): Ook.Fnord (method) arglist description
- The following values are specified by position:
- arg1
- (float): Ook.Fnord.args[0] description
- arg2
- (int|long): Ook.Fnord.args[1] description
- arg3
- (bool): Ook.Fnord.args[2] description
- values
- (str|unicode): Ook.Fnord.args[3] (values) description
- **kwargs
- Ook.Fnord keyword-arguments list description
- keyword1
- (int|long|float, required): Ook.Fnord (method) "keyword1" description
- keyword2
- (None|str|unicode, defaults to None): Ook.Fnord (method) "keyword2" description
- keyword3
- (None|str|unicode): Ook.Fnord (method) "keyword3" description
- NotImplementedError
- if called
- Change output to class with the same interface
- Clean up output to remove empty members
- cls
- (class, required): The class that the method will bind to for execution.
- arg1
- (int|long|float, required): Ook.Bleep (classmethod) arg1 description
- arg2
- (any, optional, defaults to None): Ook.Bleep (classmethod) arg2 description
- *args
- (any): Ook.Bleep (classmethod) arglist description
- Rewrite list-loops to perform the same operations in fewer passes
- Magic _parameters value needs to be removed
- arg1
- (int|long|float, required): Ook.Flup (staticmethod) arg1 description
- arg2
- (any, required): Ook.Flup (staticmethod) arg2 description
- *args
- (any): Ook.Flup (staticmethod) arglist description
- Change output to class with the same interface
- Clean up output to remove empty members
This is a combination of a toHTML method created in
callable_documentation to generate the basic documentation
markup...
def toHTML( self ):
"""
Creates and returns an HTML representation of the documentation of the item."""
moduleName = self.DecoratedItem.__module__
className = None
for name, cls in inspect.getmembers( inspect.getmodule( self.DecoratedItem ),
inspect.isclass ):
classMembers = dict( inspect.getmembers( cls ) )
for classMember in classMembers.values():
try:
if classMember.__name__ == self.DecoratedItem.__name__:
className = name
except AttributeError:
pass
methodName = self.DecoratedItem.__name__
itemId = '.'.join( [ item for item in
[ moduleName, className, methodName ] if item ] )
results = '<div id="%s" class="callable documentation">\n' % itemId
results += """ <div class="heading">
<div class="api_type">[%s]</div>
<div class="signature"><span class="api_name">%s</span>(""" % (
type( self.DecoratedItem ).__name__, self.DecoratedItem.__name__ )
if self.arguments or self.arglist or self.kwargs:
argItems = []
if self._argSpecs.args:
argItems += self._argSpecs.args
if self._argSpecs.varargs:
argItems.append( '*%s' % self._argSpecs.varargs )
if self._argSpecs.keywords:
argItems.append( '**%s' % self._argSpecs.keywords )
results += ', '.join( argItems )
results += """)</div>\n </div>\n"""
if self._originalDocstring:
results += """ <div>%s</div>\n""" % (
self._originalDocstring.strip().replace( '\n', ' ' ).replace(
' ', ' ' ).replace( ' ', ' ' ) )
if self.deprecated:
results += """ <div class="deprecated"><strong>Deprecated:<"""
"""/strong> %s</div>\n""" % self.deprecated
if self.returns:
results += """ <div class="returns"><strong>Returns:</strong> """
"""%s</div>\n""" % self.returns
if self.fixmes:
results += """ <div class="fixme"><strong>Fix Me:</strong>\n <ul>\n"""
for fixme in self.fixmes:
results += ' <li>%s</li>\n' % fixme
results += """ </ul>\n </div>\n"""
results += '\n'
if self.arguments or self.arglist or self.kwargs:
results += """ <div><div class="subhead">Arguments</div>\n"""
results += """ <dl class="arguments">\n"""
for argName in self._argSpecs.args:
results += """ <dt>%s</dt>\n""" % ( argName )
results += """ <dd>"""
if argName not in ( 'self', 'cls' ):
results += '('
if len( self.arguments[ argName ][ 'expects' ] ) > 1:
results += ( '|'.join(
[ item.__name__ if hasattr( item, '__name__' )
else str( item )
for item in self.arguments[ argName ][ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if self.arguments[ argName ][ 'expects' ] != ( object, ):
results += self.arguments[ argName ][ 'expects' ][ 0 ].__name__
else:
results += 'any'
if not self.arguments[ argName ][ 'hasDefault' ]:
results += ', required'
else:
if self.arguments[ argName ][ 'defaultValue' ]:
results += ', optional, defaults to "%s" [%s]' % (
self.arguments[ argName ][ 'defaultValue' ],
type( self.arguments[ argName ][ 'defaultValue' ]
).__name__ )
else:
results += ', optional, defaults to %s' % (
self.arguments[ argName ][ 'defaultValue' ] )
results += '): '
results += self.arguments[ argName ][ 'description' ]
elif argName == 'self':
results += '(instance, required): The object-instance that the '
'method will bind to for execution.'
elif argName == 'cls':
results += '(class, required): The class that the method will '
'bind to for execution.'
results += """</dd>\n"""
if self.arglist:
results += """ <dt>*%s</dt>\n""" % (
self.arglist[ 'name' ] )
results += """ <dd>("""
if len( self.arglist[ 'expects' ] ) > 1:
results += ( '|'.join(
[ item.__name__ if hasattr( item, '__name__' )
else str( item ) for item
in self.arglist[ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if self.arglist[ 'expects' ] != ( object, ):
results += self.arglist[ 'expects' ][ 0 ].__name__
else:
results += 'any'
results += '): ' + self.arglist[ 'description' ] + '</dd>\n'
if self.arglist[ 'sequence' ]:
results += """ <dd>The following values are """
"""specified by position:</dd>\n"""
results += """ <dd><dl>\n"""
for argListItem in self.arglist[ 'sequence' ]:
results += """ <dt>%s</dt>\n""" % (
argListItem[ 'name' ] )
results += """ <dd>("""
if len( argListItem[ 'expects' ] ) > 1:
results += ( '|'.join(
[ item.__name__ if hasattr( item, '__name__' )
else str( item ) for item
in argListItem[ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if argListItem[ 'expects' ] != ( object, ):
results += argListItem[ 'expects' ][ 0 ].__name__
else:
results += 'any'
results += '): ' + argListItem[ 'description' ]
results += """</dd>\n"""
if self.arglist.get( 'final' ):
argListItem = self.arglist[ 'final' ]
results += """ <dt>%s</dt>\n""" % (
argListItem[ 'name' ] )
results += """ <dd>("""
if len( argListItem[ 'expects' ] ) > 1:
results += ( '|'.join(
[ item.__name__ if hasattr( item, '__name__' )
else str( item ) for item
in argListItem[ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if argListItem[ 'expects' ] != ( object, ):
results += argListItem[ 'expects' ][ 0 ].__name__
else:
results += 'any'
results += '): ' + argListItem[ 'description' ]
results += """</dd>\n"""
results += """ </dl></dd>\n"""
results += """ </dd>\n"""
if self.keywordargs:
results += """ <dt>**%s</dt>\n""" % (
self.keywordargs[ 'name' ] )
results += """ <dd>%s</dd>\n""" % (
self.keywordargs[ 'description' ] )
if self.keywordargs[ 'keywords' ]:
results += """ <dd><dl>\n"""
for keyword in sorted( self.keywordargs[ 'keywords' ] ):
keywordItem = self.keywordargs[ 'keywords' ][ keyword ]
results += """ <dt>%s</dt>\n""" % (
keywordItem[ 'name' ] )
results += """ <dd>("""
if len( keywordItem[ 'expects' ] ) > 1:
results += ( '|'.join(
[ item.__name__ if hasattr( item, '__name__' )
else str( item ) for item
in keywordItem[ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if keywordItem[ 'expects' ] != ( object, ):
results += keywordItem[ 'expects' ][ 0 ].__name__
else:
results += 'any'
if keywordItem[ 'required' ]:
results += ', required'
if keywordItem[ 'hasDefault' ]:
results += ', defaults to %s' % keywordItem[ 'defaultValue' ]
results += '): ' + keywordItem[ 'description' ]
results += """</dd>\n"""
results += """ </dl></dd>\n"""
results += """ </dl>\n"""
results += """ </div>\n"""
if self.raises:
results += """ <div><div class="subhead">Exceptions</div>\n"""
results += """ <dl class="exceptions">\n"""
for errorClass in sorted( self.raises, key=lambda c: c.__name__ ):
results += """ <dt>%s</dt>\n""" % ( errorClass.__name__ )
for line in self.raises[ errorClass ]:
results += """ <dd>%s</dd>\n""" % ( line )
results += """ </dl>\n"""
results += """ </div>\n"""
if self.todos:
results += """ <div class="todo"><strong>To-Do:</strong>\n <ul>\n"""
for todo in self.todos:
results += ' <li>%s</li>\n' % todo
results += """ </ul>\n </div>\n"""
results += '</div>\n'
return results...and a relatively basic style-sheet in CSS:
.callable
{}
.documentation
{ font-size: 10pt; margin-top:12pt; font-family: sans-serif; }
.api_type
{ float:right; font-size: 10pt; padding-top:2pt; }
.signature
{ font-family: monospace; margin-bottom:6pt; }
.api_name
{ font-weight:bold; }
.documentation .heading
{ clear:left; font-size: 12pt; margin:12pt 0 6pt 0; padding-top:6px; border-top:1px solid black; border-bottom: 1px solid black; }
.documentation .subhead
{ clear:left; font-weight: bold; font-size: 10pt; margin:6pt 0 3pt 0; }
.documentation dl, .documentation ul
{ margin: 0px; }
.documentation dl dt
{ clear:left; float:left; width:10em; text-align:right; margin-right:0.5em; font-weight:bold; }
.documentation dl dt:after
{ content: ':';}
.documentation dl.arguments dt
{ font-family: monospace; font-size: 9pt; }
.documentation dl dd
{ margin-left: 9.5em; }
.documentation dl dd dl
{ margin-left:-7em; }Finally, and coming back to the plain-text documentation as promised,
I'd like to be able to take the documentation-string that's generated by the
callable_documentation class-instances, and stuff that into
the __doc__ of the decorated functions/methods. There's some
basic formatting that will need to be done as well, to keep the resulting
__doc__ within an 80-character width, to provide intelligent
dot-leaders and hanging indentation on the text, and maybe a few other
light-weight formatting items. The main, or at least visible part,
though is one more decorator-method on describe:
@classmethod
def AttachDocumentation( cls ):
"""
Decorates a function or method by attaching a "TODO" item to it."""
def _AttachDocumentationDecorator( decoratedItem ):
"""
Performs the actual __doc__ attachment decoration"""
# Get the documentation metadata
_documentation = decoratedItem._documentation
# Create the formatted docstring
newDocLines = []
line = decoratedItem.__name__
if _documentation.arguments or _documentation.arglist or \
_documentation.keywords:
# It's a function or method, so generate a series of arguments, etc.
line += '( '
argItems = []
argSpecs = inspect.getargspec( decoratedItem )
if argSpecs.args:
argItems = argSpecs.args
if argSpecs.varargs:
argItems.append( '*%s' % argSpecs.varargs )
if argSpecs.keywords:
argItems.append( '**%s' % argSpecs.keywords )
line += ', '.join( argItems )
line += ' )'
line = line.replace( '( )', '()' )
newDocLines.append( FormatLine( line, 4 ) )
if _documentation.deprecated:
newDocLines.append( '' )
newDocLines.append( FormatLine( 'DEPRECATED: %s' %
_documentation.deprecated, 4 ) )
else:
newDocLines.append( '' )
newDocLines.append( FormatLine( _documentation._originalDocstring, 4 ) )
if _documentation.returns:
newDocLines.append( '' )
newDocLines.append( FormatLine( 'RETURNS: %s' %
_documentation.returns, 4 ) )
if _documentation.fixmes:
newDocLines.append( '' )
newDocLines.append( FormatLine( 'FIX ME:' ) )
for fixme in _documentation.fixmes:
newDocLines.append( FormatLine( ' - %s\n' % fixme, 4 ) )
if _documentation.arguments or _documentation.arglist or \
_documentation.keywordargs:
newDocLines.append( '' )
newDocLines.append( FormatLine( 'ARGUMENTS:' ) )
# Determine the dot-leader length for all argument items in the
#docstring
argNames = _documentation.arguments.keys()
if _documentation.arglist[ 'sequence' ]:
argNames += [ ' - %s' % item[ 'name' ] for item in
_documentation.arglist[ 'sequence' ] ]
if _documentation.arglist[ 'final' ]:
argNames.append( ' - %s' % _documentation.arglist[ 'final'
][ 'name' ] )
if _documentation.keywordargs.get( 'keywords' ):
argNames += [ ' - %s' % _documentation.keywordargs[ 'keywords'
][ item ][ 'name' ]
for item in _documentation.keywordargs[ 'keywords' ] ]
dotLeadLen = max( [ len( item ) for item in argNames ] ) + 3
hang = dotLeadLen + 6
if _documentation.arguments:
# print argSpecs
for argName in [ name for name in argSpecs.args
if name[ 0 ] != '*' ]:
if argName not in ( 'self', 'cls' ):
arg = _documentation.arguments[ argName ]
line = ( '%s ' % ( arg[ 'name' ] ) ).ljust(
hang - 1, '.' ) + ' '
line += '('
if len( arg[ 'expects' ] ) > 1:
line += ( '|'.join( [ item.__name__
if hasattr( item, '__name__' )
else str( item ) for item in arg[
'expects' ] ] ) ).replace(
'NoneType', 'None' )
else:
if arg[ 'expects' ] != ( object, ):
line += arg[ 'expects' ][ 0 ].__name__
else:
line += 'any'
if not arg[ 'hasDefault' ]:
line += ', required'
else:
if arg[ 'defaultValue' ]:
line += ', optional, defaults to '
'"%s" [%s]' % ( arg[ 'defaultValue' ],
type( arg[ 'defaultValue' ] ).__name__ )
else:
line += ', optional, defaults to %s' % (
arg[ 'defaultValue' ] )
line += '): '
line += arg[ 'description' ]
elif argName == 'self':
line = ( 'self %s (instance, required): The object-'
'instance that the method will bind to at '
'execution.' % ( '.'*dotLeadLen ) )
elif argName == 'cls':
line = ( 'self %s (class, required): The class '
'that the method will bind to at '
'execution.' % ( '.'*dotLeadLen ) )
else:
raise RuntimeError( 'oops, hahaha!')
newDocLines.append( FormatLine( line, hang ) )
if _documentation.arglist:
line = ( '*%s ' % ( _documentation.arglist[ 'name' ] )
).ljust( hang - 1, '.' ) + ' %s' % (
_documentation.arglist[ 'description' ] )
newDocLines.append( FormatLine( line, hang ) )
arglist = _documentation.arglist
if arglist[ 'sequence' ]:
for arg in arglist[ 'sequence' ]:
line = ( ' - %s ' % ( arg[ 'name' ] ) ).ljust(
hang - 1, '.' ) + ' '
line += '('
if len( arg[ 'expects' ] ) > 1:
line += ( '|'.join( [ item.__name__
if hasattr( item, '__name__' )
else str( item ) for item in arg[
'expects' ] ] ) ).replace(
'NoneType', 'None' )
else:
if arg[ 'expects' ] != ( object, ):
line += arg[ 'expects' ][ 0 ].__name__
else:
line += 'any'
line += '): '
line += arg[ 'description' ]
newDocLines.append( FormatLine( line, hang ) )
if arglist.get( 'final' ):
arg = arglist[ 'final' ]
line = ( ' - %s ' % ( arg[ 'name' ] ) ).ljust(
hang - 1, '.' ) + ' '
line += '('
if len( arg[ 'expects' ] ) > 1:
line += ( '|'.join( [ item.__name__
if hasattr( item, '__name__' )
else str( item ) for item in arg[ 'expects' ]
] ) ).replace( 'NoneType', 'None' )
else:
if arg[ 'expects' ] != ( object, ):
line += arg[ 'expects' ][ 0 ].__name__
else:
line += 'any'
line += '): '
line += arg[ 'description' ]
newDocLines.append( FormatLine( line, hang ) )
if _documentation.keywordargs:
line = ( '*%s ' % ( _documentation.keywordargs[ 'name' ]
) ).ljust( hang - 1, '.' ) + ' %s' % (
_documentation.keywordargs[ 'description' ] )
newDocLines.append( FormatLine( line, hang ) )
if _documentation.keywordargs[ 'keywords' ]:
for keywordItem in sorted( _documentation.keywordargs[
'keywords' ] ):
arg = _documentation.keywordargs[ 'keywords' ][
keywordItem ]
line = ( ' - %s ' % ( arg[ 'name' ] ) ).ljust(
hang - 1, '.' ) + ' '
line += '('
if len( arg[ 'expects' ] ) > 1:
line += ( '|'.join( [ item.__name__
if hasattr( item, '__name__' )
else str( item ) for item in arg[
'expects' ] ] ) ).replace(
'NoneType', 'None' )
else:
if arg[ 'expects' ] != ( object, ):
line += arg[ 'expects' ][ 0 ].__name__
else:
line += 'any'
if not arg[ 'hasDefault' ]:
line += ', required'
else:
if arg[ 'defaultValue' ]:
line += ( ', optional, defaults to "%s" '
'[%s]' % ( arg[ 'defaultValue' ],
type( arg[ 'defaultValue' ]
).__name__ ) )
else:
line += ( ', optional, defaults to %s' % (
arg[ 'defaultValue' ] ) )
line += '): '
line += arg[ 'description' ]
newDocLines.append( FormatLine( line, hang ) )
if _documentation.raises:
newDocLines.append( FormatLine( '' ) )
newDocLines.append( FormatLine( 'RAISES:' ) )
for errorClass in sorted( _documentation.raises,
key=lambda err: err.__name__ ):
newDocLines.append( FormatLine( ' - %s' %
errorClass.__name__ ) )
for line in _documentation.raises[ errorClass ]:
newDocLines.append( FormatLine( ' + %s' % line,
5 ) )
if _documentation.todos:
newDocLines.append( FormatLine( '' ) )
newDocLines.append( FormatLine( 'TO-DO:' ) )
for line in _documentation.todos:
newDocLines.append( FormatLine( ' - %s' % line ) )
# Try to replace the current __doc__ with the new doc-string
try:
decoratedItem.__doc__ = ( '\n'.join( newDocLines ) ).strip()
except:
pass
# Return the decorated item!
return decoratedItem
return _AttachDocumentationDecoratorAs more documentation-decoration efforts are undertaken, I'm expecting that I'll have to come back to this method to add type-based detection to it. The reason behind that is that different documented items will have different metadata structures associated with them. Classes and properties, for example, will not have arguments of any kind. Ideally, though, I'd like to be able to apply this same decorator to any documentation-decorated item and at least not have it raise errors. Whether that will be realized is to be determined (though I already know that it won't matter for classes unless something's changed since the last time I checked).
Ultimately, all the AttachDocumentation method is doing
is gathering the documentation-metadata, formatting it, and trying to
attach it to the original decorated item in the existing __doc__
property. In generating the final format, it's making an attempt to
stick to official
(if, maybe outdated) Python conventions of an
80-character line-width, and providing some basic hanging-indentation
structure. That's what the global FormatLine function's
purpose is:
#####################################
# Defined functions. #
#####################################
def FormatLine( line, hang=0, width=80 ):
"""
Formats the provided line into one-to-many lines constrained to the width (in
spaces), with a hanging indent (also in spaces), returning those lines."""
# First, make sure that the incoming line is just that: ONE line
if '\n' in line or '\r' in line:
line = line.replace( '\n', ' ' ).replace( '\r', ' ' )
# Reduce extraneous spaces
while ' ' in line:
line = line.replace( ' ', ' ' )
# set up second- and subsequent-line indent
if hang:
newLineStart =' ' * hang
else:
newLineStart = ''
results = ''
currentLine = ''
tokens = line.split( ' ' )
for token in tokens:
if len( currentLine ) + len( token ) + 1 <= width:
currentLine += token + ' '
else:
results += currentLine
currentLine = '\n%s' % newLineStart + token + ' '
results += currentLine
return results.rstrip()
__all__.append( 'FormatLine' )Printing the __doc__ of Ook.Fnord,
Ook.Bleep and Ook.Flup with AttachDocumentation
called on each yields:
Fnord( self, arg1, arg2, *args, **kwargs )
DEPRECATED: Use new_Fnord instead.
Ook.Fnord (method) original doc-string
RETURNS: None (at least until the method is implemented)
FIX ME:
- Rewrite list-loops to perform the same operations in fewer passes
- Magic _parameters value needs to be removed
ARGUMENTS:
self .............. (instance, required): The object-instance that the method
will bind to at execution.
arg1 .............. (bool|None, required): Ook.Fnord (method) arg1 description
arg2 .............. (any, required): Ook.Fnord (method) arg2 description
*args ............. Ook.Fnord (method) arglist description
- argitem1 ....... (float): Ook.Fnord.args[0] description
- argitem2 ....... (int|long): Ook.Fnord.args[1] description
- argitem3 ....... (bool): Ook.Fnord.args[2] description
- values ......... (str|unicode): Ook.Fnord.args[3] (values) description
*kwargs ........... Ook.Fnord keyword-arguments list description
- keyword1 ....... (int|long|float, required): Ook.Fnord (method) "keyword1"
description
- keyword2 ....... (None|str|unicode, optional, defaults to None): Ook.Fnord
(method) "keyword2" description
- keyword3 ....... (None|str|unicode, required): Ook.Fnord (method) "keyword3"
description
RAISES:
- NotImplementedError
+ if called
TO-DO:
- Change output to class with the same interface
- Clean up output to remove empty members
--------------------------------------------------------------------------------
Bleep( cls, arg1, arg2, *args, **kwargs )
DEPRECATED: Will be removed by version X.YY.ZZ
Ook.Bleep (classmethod) original doc-string
ARGUMENTS:
self ....... (class, required): The class that the method will bind to at
execution.
arg1 ....... (int|long|float, required): Ook.Bleep (classmethod) arg1
description
arg2 ....... (any, optional, defaults to None): Ook.Bleep (classmethod) arg2
description
*args ...... Ook.Bleep (classmethod) arglist description
--------------------------------------------------------------------------------
Flup( arg1, arg2, *args, **kwargs )
Ook.Flup (staticmethod) original doc-string
FIX ME:
- Rewrite list-loops to perform the same operations in fewer passes
- Magic _parameters value needs to be removed
ARGUMENTS:
arg1 ....... (int|long|float, required): Ook.Flup (staticmethod) arg1
description
arg2 ....... (any, required): Ook.Flup (staticmethod) arg2 description
*args ...... Ook.Flup (staticmethod) arglist description
TO-DO:
- Change output to class with the same interface
- Clean up output to remove empty members
With that in place, the API-documentation decorators, at least for functions and methods, is complete for now. As I start working on actual project code, I'm expecting that I'll want to come back and revisit it to deal with things like configuration-file tie-ins, and maybe other items that I'm not anticipating just yet. For now, though, it's complete.
With so many decorators in play, even just on the throw-away Ook
class, it feels like it might be time to re-visit the earlier concern about
performance impact. To test that, I captured the time it took to define/compile
Ook, decorated as above, and the time it took to define/compile
another class, Eek, that was identical except for the decoration.
The bad news it that the decorated class took upwards of ten times longer to
complete its definition/compilation. The good news is that even with all the
decoration in place, that ten times longer
is still topping
out at about 0.0007 seconds, and has gotten as short as 0.0004 seconds with
some frequency when run directly.
Next up will be documenting class property-members, and classes themselves, using the same sort of decoration process. See you then!
No comments:
Post a Comment