Long Post
Today I'm going to work out a process for documenting class properties
and classes themselves. I'm going to start with documentation for properties.
The goal for that would be to end up with documentation that looks something
like this, at a minimum:
----------------------------------------------------------------------------
PropertyName ..... (int, float) Gets, sets, or deletes the PropertyName
value of the instance. PropertyName description.
----------------------------------------------------------------------------
A few posts back, I noted that one of my key development principles was to
manage/control all public interface entities/members, and
that one aspect of that was to create property getter-, setter- and deleter-methods
(even if they weren't ever actually attached to a property definition)
and create property elements as members of the class.
There are a couple of ways to create properties in Python classes.
One is a decorator-based structure that would look like so:
class Example( object ):
@property
def PropertyName( self ):
return self._propertyName
@PropertyName.setter
def PropertyName( self, value ):
self._propertyName = value
@PropertyName.deleter
def PropertyName( self ):
del self._propertyName
x = Example()
x.PropertyName = 'Spam'
print x.PropertyName
del x.PropertyName
try:
print x.PropertyName
except AttributeError:
print 'PropertyName deletion performed as expected'
This approach executes just fine, but it doesn't appear to provide
any way of attaching any documentation to the resulting property, so that
won't meet my goal. That's too bad, because it'd result in a bit less code.
That discovery, which I made a while back, was what prompted the original
approach shown in that previous post. The example code-structure from that
post, with the appropriate documentation-decorators on those methods would
look like this:
class Ook( object ):
#####################################
# Instance property-getter methods #
#####################################
@describe.AttachDocumentation()
def _GetPropertyName( self ):
"""
Gets the PropertyName property-value of the instance."""
return self._propertyName
#####################################
# Instance property-setter methods #
#####################################
@describe.AttachDocumentation()
@describe.argument( 'value', 'The value to set as the instance\'s '
'PropertyName value', int, float )
@describe.raises( TypeError, 'if passed an invalid value-type' )
@describe.raises( ValueError, 'if passed an invalid value' )
def _SetPropertyName( self, value ):
"""
Sets the PropertyName property-value of the instance."""
# TODO: Type-check the value argument, and raise an error on a
# failure, or remove this comment.
# TODO: Value-check the value argument, and raise an error on a
# failure, or remove this comment.
self._propertyName = value
#####################################
# Instance property-deleter methods #
#####################################
@describe.AttachDocumentation()
def _DelPropertyName( self ):
"""
"Deletes"" the PropertyName property-value of the instance by setting it to
the default value specified in the ClassName attributes."""
try:
del self._propertyName
except AttributeError:
pass
#####################################
# Instance Properties #
#####################################
PropertyName = property( _GetPropertyName, _SetPropertyName,
_DelPropertyName,
'Gets the PropertyName value of the instance.' )
#####################################
# Instance Initializer #
#####################################
@describe.AttachDocumentation()
def __init__( self ):
"""
Instance initializer"""
# Call parent initializers, if applicable.
# Set default instance property-values with _Del... methods as needed.
self._DelPropertyName()
# Set instance property values from arguments if applicable.
The drawback to this structure is that a simple decoration approach simply won't work. This is easily demonstrated:
def mydecorator():
def innerDecorator( decoratedItem ):
return decoratedItem
return innerDecorator
class Example2( object ):
@mydecorator
eggs = property()
Trying to execute this code-snippet raises a SyntaxError.
Trying the other
decorator-execution structure (eggs = mydecorator()(
property() ) raises a TypeError ('property' object
is not callable
). This is probably just as well, since that sort of
decorator-application feels... awkward, I think, at a minimum, and it's
certainly not as readable as the decorators for callables that
are already established. it doesn't appear that there's a decorator-based
approach that would work and meet my needs.
All that said, though, the idea of using a function or method to perform
the same kinds of tasks as a decorator isn't beyond reach. Consider the following
code:
def makeProperty( getter, setter, deleter, description, *expects ):
if not description:
description = 'the property value'
actions = []
if getter:
actions.append( 'gets' )
if setter:
actions.append( 'sets' )
if deleter:
actions.append( 'deletes' )
actions[ 0 ] = actions[ 0 ].title()
if expects:
expectedTypes = ( '|'.join(
[
item.__name__ if hasattr( item, '__name__' )
else str( item )
for item in expects
] ) ).replace( 'NoneType', 'None' )
else:
expectedTypes = 'any'
if len( actions ) > 1:
description = ', '.join( actions[0:-1] ) + ' or %s %s (%s)' % (
actions[ -1 ], description, expectedTypes )
else:
description = '%s %s (%s)' % ( actions[ 0 ], description, expectedTypes )
return property( getter, setter, deleter, description )
class Eek( object ):
def _Getter( self ): pass
@describe.AttachDocumentation()
@describe.raises( TypeError, 'if set to an invalid value-type' )
@describe.raises( ValueError, 'if set to an invalid value' )
def _Setter( self, value ): pass
def _Deleter( self ): pass
x = makeProperty( _Getter, _Setter, _Deleter,
'the example property',
int, float, None )
After execution, calling print Eek.x.__doc__ yields:
Gets, sets or deletes the example property (int|float|None)
Apart from the name of the property itself, which isn't always needed,
this matches the need pretty well. It's just about perfect, I think, for a
print [propertyName].__doc__ call, which I've already mentioned
that I use on occasion to read documentation from a command-line. The missing
property-name is easily supplied in the context of a class for generation of
class-level, summary-level documentation, I suspect. It might well be useful to
include any of the raises decorations from the provided setter-method,
if the setter exists and has been decorated with any describe.raises
calls. That's not difficult to achieve, it just requires the addition of a few lines
of code:
if setter:
try:
exceptions = setter._documentation.raises
for exception in sorted( exceptions, key=lambda e: e.__name__ ):
description += ( )'; Raises %s ' % exception.__name__ +
', '.join( exceptions[ exception ] ) )
except:
pass
return property( getter, setter, deleter, description )
The same property-wrapping call, with that code in place, yields:
Gets, sets or deletes the example property (int|float|None);
Raises TypeError if set to an invalid value-type;
Raises ValueError if set to an invalid value
Ideally, any item that's been documented with the sort of decorator approach
that I'm creating should have a reasonably similar underlying data-structure, and
properties are no exception to that. However, once a property has been created,
there's no way that I've found thus far to actually attach new attributes to
them, which would be necessary if I was to create, for example, a
property_documentation class to hold on to all of the items actually
used to create the documentation. By way of example, I'll create a class
with a single property on it, and try to attach another attribute to it:
class TestClass( object ):
def _getter( self ):
return self._property
def _setter( self, value):
self._property = value
def _deleter( self ):
self._property = None
propertyName = property( _getter, _setter, _deleter, 'This is my property' )
def __init__( self, value ):
self._deleter()
self._setter( value )
try:
print 'Trying ..._newAttribute = None'
TestClass.propertyName._newAttribute = None
print 'YES! This worked!'
except Exception, error:
print 'NOPE: %s: %s' % ( error.__class__.__name__, error )
print
try:
print 'Trying ...__dict__[ "_newAttribute" ] = None'
TestClass.propertyName.__dict__[ '_newAttribute' ] = None
print 'YES! This worked!'
except Exception, error:
print 'NOPE: %s: %s' % ( error.__class__.__name__, error )
print
try:
print 'Trying setattr( propertyName, "_newAttribute", None )'
setattr( TestClass.propertyName, '_newAttribute', None )
print 'YES! This worked!'
except Exception, error:
print 'NOPE: %s: %s' % ( error.__class__.__name__, error )
print
Running this code results in:
Trying ..._newAttribute = None
NOPE: AttributeError: 'property' object has no attribute '_newAttribute'
Trying ...__dict__[ "_newAttribute" ] = None
NOPE: AttributeError: 'property' object has no attribute '__dict__'
Trying setattr( propertyName, "_newAttribute", None )
NOPE: AttributeError: 'property' object has no attribute '_newAttribute'
If there are any other ways to even try to attach new attributes
to a property during or after its definition, I can't think of them offhand.
Certainly nothing else jumps readily to mind that'd be at least reasonably
simple and straightforward. All that said, though, when I examine the uses
I'd have or expect for the documentation of a property, they start me wondering
if that's really all that much of an impediment. What I'd want or need would include:
- Printing the property's documentation from a command-line:
do-able. Though the doc-string itself might not have
any reference back to the actual property-name, it's immaterial because
I'd know the name of the property anyway, having called something like
print propertyName.__doc__.
- Including the property's documentation in the entire documentation for
the class that the property is a member of: do-able, I
think, since the properties themselves would contain all of the information
needed to re-create the doc-string if it weren't actually generated
on
the fly.
The probable lack of a property-name association in the
doc-strings is irrelevant here as well, because I'd almost certainly be
iterating through a series of class-members, and would be able to get
their names during that iteration.
- Creating documentation-output, particularly of entire class' interfaces,
in formats other than plain text: probably do-able.
The same name-relationship as noted just above would be usable, at least
for generating documentation for an entire class and its members. The main
consideration here is probably that once the doc-string has been
created by the pseudo-decoration/wrapper-method process, the original
doc-string, the one passed in the
description in that
method-call, is irrevocably altered.
Of those, only the last item represents any potential major stumbling-block, I think.
But I also think that so long as the
structure of the resulting doc-string
generated by the pseudo-decorator call is consistent, extracting those individual items
during a documentation-generation run for, say, L
AT
EX would not be
terribly difficult either. It might be better served (and maybe make more sense in
documentation-output anyway) to re-structure those to something more like:
[name, from context] ... ([expected types]) [Actions] [description]. [Exceptions]
That structure, once stored in the doc-string of a documented property, would be
pretty easy to extract the relevant items from for whatever re-sequencing or
formatting was desired for output in other than plain text, while still remaining
useful in plain text as well:
[name, from context] would be supplied outside the doc-string,
by whatever process was determining which property the documentation is
being retrieved for;([expected types]) would always be the first item in the
doc-string, and wrapped in ();[Actions] and [description] make sense to keep
together, and would always end with a .
and start right
after the closing )
of the expected types.[Exceptions], if they exist, would be the balance of the
doc-string, and would be separated by .
or ;
characters
If the separators for major blocks are always periods, a single
split call
will take care of identifying the exceptions and non-exceptions items in one pass, and
a second
split, on
)
would suffice to separate expected types from
the actions/description text. The extraction-process wouldn't even require regular
expressions to implement. Even the exception-classes in the list of exceptions would be
easily identified, since they would always be the second word in the individual
exceptions-lists, and the actual conditions wouldn't be hard to extract either,
should it me needed. It's not as clean an implementation as having a consistent underlying
data-structure to refer to, but it's pretty easily managed.
The net result of all of this is that while I'd rather have a data-structure
for property-documentation elements, something like the callable_documentation
class I created for function- and method-documentation, it's just not possible. It's
also not a major road-block, though, since the structure of the resulting
documentation is relatively simple, and can be defined with consistency for retrieving
whatever might be needed later on. So, with all of that said, the process for creating
consistently-documented properties will be provided by describe.makeProperty:
@classmethod
def makeProperty( cls, getter, setter, deleter, description, *expects ):
"""
Creates and returns a property object, using the built-in property method,
building/creating a detailed documentation-string on it in the process."""
if not description:
description = 'the property value'
actions = []
if getter:
actions.append( 'gets' )
if setter:
actions.append( 'sets' )
if deleter:
actions.append( 'deletes' )
actions[ 0 ] = actions[ 0 ].title()
if expects:
expectedTypes = ( '|'.join(
[
item.__name__ if hasattr( item, '__name__' )
else str( item )
for item in expects
] ) ).replace( 'NoneType', 'None' )
else:
expectedTypes = 'any'
if len( actions ) > 1:
description = '(%s) %s or %s %s. ' % ( expectedTypes, ', '.join( actions[0:-1] ), actions[ -1 ], description.strip() )
else:
description = '(%s) %s %s. ' % ( expectedTypes, actions[ 0 ], description )
if setter:
try:
exceptions = setter._documentation.raises
for exception in sorted( exceptions, key=lambda e: e.__name__ ):
description += 'Raises %s ' % exception.__name__ + '; '.join( exceptions[ exception ] ) + '. '
except:
pass
return property( getter, setter, deleter, description.strip() )
While I was working out the documentation-decoration for the makeProperty
method, I chanced across an... interesting... discovery: The decoration-effort that I'd
been making on the decorators themselves is, for most practical purposes, useless. I'm not
sure exactly why this is the case, but it's apparently not possible to overwrite the
__doc__ of an instance method once that instance method has been defined,
so the decoration metadata, while it exists in the callable_documentation
instance attached to each method, is not being attached to those methods by
the final AttachDocumentation call. After a brief panic that I'd spent
quite some time writing code that would turn out to be useless (and moving some code
around in the process) I re-checked the documentation on the Ook test-class,
and it still worked as expected. But the documentation on the decorators themselves will
have to be manually maintained. That's mildly annoying, but so long as similar issues
don't arise later on, it should be fine. While I was thrashing through all of that
double-checking, I altered the AttachDocumentation method so that it simply
retrieves the formatted doc-string text from callable_documentation through
a new method (_createDocString), instead of building it itself. I probably
should've done that from the start, if only in keeping with basic encapsulation, and maybe
the Single
Responsibility Principle, but it simply didn't occur to me until now.
So: On to class-level documentation. The first thing I wanted to do was to make
sure that my expectation of the execution-sequence of decorators on classes and
decorators on their members was correct. I expected that, given a decorated class,
with decorated members (methods, in this case) that the method decorators
would execute to completion before the class-level decorators. Additionally,
I wanted to check to confirm whether or not a class' __doc__ could be
modified/set by a decorator on the class. Here's the code I spun up to test that:
def classdec():
print 'Calling classdec()'
def _classdec( decoratedItem ):
print 'calling _classdec( %s )' % decoratedItem.__name__
try:
decoratedItem.__doc__ = 'This has been decorated'
except Exception, error:
print ' + - %s: decorating %s: %s' % ( error.__class__.__name__, decoratedItem.__name__, error )
decoratedItem._documentation = { 'documentation_added':True }
return decoratedItem
return _classdec
def methoddec():
print 'Calling methoddec()'
def _methoddec( decoratedItem ):
print ' + calling _methoddec( %s )' % decoratedItem.__name__
return decoratedItem
return _methoddec
@classdec()
class decorated_class( object ):
@methoddec()
def instance_method1( self ):
pass
@methoddec()
def instance_method2( self ):
pass
print decorated_class._documentation
When this code is run, it generates the following output:
Calling classdec()
Calling methoddec()
+ calling _methoddec( instance_method1 )
Calling methoddec()
+ calling _methoddec( instance_method2 )
calling _classdec( decorated_class )
+ - AttributeError: decorating decorated_class: attribute '__doc__' of 'type' objects is not writable
{'documentation_added': True}
That seems to me to provide confirmation that decorators execute in the order I'd
expected. It also confirms, unfortunately, that the __doc__ of a class
cannot be modified by a decorator. As with properties, I'm not sure precisely
why this is the case, but given that I rarely need to access class-level documentation
at the level of detail that I do documentation of functions, methods or properties,
I'm OK with that. I do want to be able to access whatever underlying
data-structure I can attach to the class in order to generate external
documentation — HTML output, or LATEX, for example —
but the addition of an arbitrary data-structure (decoratedItem._documentation = {}
in the classdec method above) is shown in the results as doing exactly
what I need it to do.
With that discovery out of the way, it's time to figure out what needs to
be documented in a class. The complete list (that I can think of, at any rate)
is:
- Class-level attributes (i.e., non-property value-attributes, whether they
are expected to be constant or not);
- Instance properties (already accounted for);
- Instance methods (already accounted for);
- Whether the class is abstract or not;
That doesn't really leave a lot to be implemented: Just documentation of class
attributes and abstraction, really. it seems likely to me that a call to
AttachDocumentation will also be desirable, if only to aggregate all
of the relevant property- and method-documentation information into the class-level
documentation-structure.
Before I start digging into the implementation details for class-level
documentation, I'm going to add a few items to the Ook class to
be documented. As part of that, I'm going to make it an abstract class
with the abc.__ABCMeta__ meta-class. I'm also going to add in my
standard template comments, if only because Ook is getting pretty
long now, even with no real implementation in any of its methods. In the process,
I'll add calls for the various decorators as I hope they will shake out:
@AttachDocumentation()
@describe.attribute( '__PrivateAttribute', 'Private attribute description' )
@describe.attribute( '_ProtectedAttribute', 'Protected attribute description' )
@describe.attribute( 'PublicAttribute', 'Public attribute description' )
class Ook( object ):
"""
Test-class."""
#####################################
# Abstraction through abc.ABCMeta #
#####################################
__metaclass__ = abc.ABCMeta
#####################################
# Class attributes (and instance- #
# attribute default values) #
#####################################
__PrivateAttribute = None
_ProtectedAttribe = None
PublicAttribute = None
#####################################
# Instance property-getter methods #
#####################################
def _GetPropertyName( self ):
return self._propertyName
#####################################
# Instance property-setter methods #
#####################################
@describe.raises( TypeError, 'if set to an invalid value-type' )
@describe.raises( ValueError, 'if set to an invalid value' )
def _SetPropertyName( self, value ):
self._propertyName = value
#####################################
# Instance property-deleter methods #
#####################################
def _DelPropertyName( self ):
self._propertyName = None
#####################################
# Instance Properties #
#####################################
propertyName = describe.makeProperty( _GetPropertyName,
_SetPropertyName, _DelPropertyName,
'the PropertyName property of the instance',
int, float )
#####################################
# Instance Initializer #
#####################################
def __init__( self ):
"""
Object initializer"""
self._DelPropertyName()
@describe.AttachDocumentation()
@abc.abstractmethod()
@describe.argument( 'arg1', 'Ook.Fnord (method) arg1 description', bool, None )
@describe.argument( 'arg2', 'Ook.Fnord (method) arg2 description' )
@describe.arglist( 'Ook.Fnord (method) arglist description', int, long, float )
@describe.arglistitem( 0, 'argitem1', 'Ook.Fnord.args[0] description', float )
@describe.arglistitem( 1, 'argitem2', 'Ook.Fnord.args[1] description', int, long )
@describe.arglistitem( 2, 'argitem3', 'Ook.Fnord.args[2] description', bool )
@describe.arglistitem( -1, 'values', 'Ook.Fnord.args[3] (values) description', str, unicode )
@describe.keywordargs( 'Ook.Fnord keyword-arguments list description' )
@describe.keyword( 'keyword1', 'Ook.Fnord (method) "keyword1" description',int, long, float, required=True )
@describe.keyword( 'keyword2', 'Ook.Fnord (method) "keyword2" description',None, str, unicode, default=None )
@describe.keyword( 'keyword3', 'Ook.Fnord (method) "keyword3" description',None, str, unicode )
@describe.deprecated( 'Use new_Fnord instead.' )
@describe.returns( 'None (at least until the method is implemented)' )
@describe.raises( NotImplementedError, 'if called' )
@describe.todo( 'Clean up output to remove empty members' )
@describe.todo( 'Change output to class with the same interface' )
@describe.fixme( 'Magic _parameters value needs to be removed' )
@describe.fixme( 'Rewrite list-loops to perform the same operations in fewer passes' )
def Fnord( self, arg1, arg2, *args, **kwargs ):
"""Ook.Fnord (method) original doc-string"""
raise NotImplementedError( '%s.Fnord is not yet implemented' %
self.__class__.__name__ )
@classmethod
@describe.AttachDocumentation()
@describe.argument( 'arg1', 'Ook.Bleep (classmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Bleep (classmethod) arg2 description' )
@describe.arglist( 'Ook.Bleep (classmethod) arglist description' )
@describe.deprecated( 'Will be removed by version X.YY.ZZ' )
def Bleep( cls, arg1, arg2=None, *args, **kwargs ):
"""Ook.Bleep (classmethod) original doc-string"""
return None
@staticmethod
@describe.AttachDocumentation()
@describe.argument( 'arg1', 'Ook.Flup (staticmethod) arg1 description', int, long, float )
@describe.argument( 'arg2', 'Ook.Flup (staticmethod) arg2 description' )
@describe.arglist( 'Ook.Flup (staticmethod) arglist description' )
@describe.todo( 'Clean up output to remove empty members' )
@describe.todo( 'Change output to class with the same interface' )
@describe.fixme( 'Magic _parameters value needs to be removed' )
@describe.fixme( 'Rewrite list-loops to perform the same operations in fewer passes' )
def Flup( arg1, arg2, *args, **kwargs ):
"""Ook.Flup (staticmethod) original doc-string"""
return None
The new items added to Ook's definition are:
- It has an
@AttachDocumentation() call, which should gather
up all of the documentation items from other decorators as well as from
documented properties and methods.
- It has
describe.attribute decorator-calls for three class-level
attributes:
__PrivateAttribute, a private attribute;_ProtectedAttribe, a protected attribute; andPublicAttribute, a public sttributeNone.
- It has been made capable of supporting abstract properties and
methods through the
__metaclass__ = abc.ABCMeta addition.
- It has one abstract method,
Fnord.
Documentation metadata for class attributes should probably be stored in much
the same fashion as the metadata for callables, and for much the same reasons:
There will be internal data-structures that need to be read-only to prevent
accidental overwriting or corruption of their values, probably some checking
for duplicate names, and all of the other factors that led to the decision to
create the callable_documentation class. Expressed as a
dict, I expect that structure to look something like this, for an
entire class:
{
'attributes':{
'name':{ # <str|unicode>
'description':<str|unicode>,
'name':<str|unicode>,
'value':<object>,
},
# ...
},
'isAbstract':<bool>,
'methods':{
'name':<callable_documentation>,
# ...
}
},
'originalDocstring':<str|unicode>,
'properties':{
'name':{ # <str|unicode>
'description':<str|unicode>,
'expects':<type*>
'name':<str|unicode>,
'raises':{
<Exception>:<str|unicode>,
# ...
}
},
}
}
So, the first thing I'll tackle is documenting class attributes. I'm going
to assemble the class_documentation class that actually stores and
manages class documentation as I go. The attribute documentation is pretty
simple, I think. The decorators in question on the Ook class
(and the attributes that relate to them) are:
@AttachDocumentation()
@describe.attribute( '__PrivateAttribute', 'Private attribute description' )
@describe.attribute( '_ProtectedAttribute', 'Protected attribute description' )
@describe.attribute( 'PublicAttribute', 'Public attribute description' )
class Ook( object ):
"""
Test-class."""
#####################################
# Abstraction through abc.ABCMeta #
#####################################
__metaclass__ = abc.ABCMeta
#####################################
# Class attributes (and instance- #
# attribute default values) #
#####################################
__PrivateAttribute = None
_ProtectedAttribe = None
PublicAttribute = None
Which requires an attribute decorator in the describe
class:
@classmethod
def attribute( cls, name, description=None ):
"""
Decorates a class by attaching documentation-metadata about a single
class-attribute to it."""
# Type- and value-check incoming arguments
if type( name ) not in ( str, unicode ):
raise TypeError( '%s.attribute expects a non-empty str or '
'unicode value whose value is the name of an attribute of '
'the decorated class for its "name" argument, but was '
'passed "%s" (%s)' % ( cls.__name__, name,
type( name ).__name__ ) )
if description != None:
if type( description ) not in ( str, unicode ):
raise TypeError( '%s.attribute expects a non-empty str or '
'unicode value, or None, but was passed "%s" (%s)' % (
cls.__name__, description,
type( description ).__name__ ) )
def _attributeDecorator( decoratedItem ):
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem._documentation = class_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# If we reach this point, then we should set the deprecated value
# to the information provided.
value = getattr( decoratedItem, name )
_documentation.AddAttribute( name, value, description )
# Return the decorated item!
return decoratedItem
return _attributeDecorator
And an AddAttribute method in class_documentation:
def AddAttribute( self, name, value, description=None ):
"""
Adds attribute for the specified attribute to the instance's
documentation-metadata."""
# Check to assure that self.DecoratedItem has an attribute with the
# supplied name
if not name in self.DecoratedItem.__dict__.keys():
raise AttributeError( '%s does not have an attribute with the '
'name "%s"' % ( self.DecoratedItem.__name__, name ) )
# Make sure the decoration-attempt isn't going to override an existing
# method- or property-documentation
if self.methods.get( name ):
raise NameError( '%s.%s is already documented as a method' % (
self.DecoratedItem.__name__, name ) )
if self.properties.get( name ):
raise NameError( '%s.%s is already documented as a property' % (
self.DecoratedItem.__name__, name ) )
if not description:
description = 'Not documented'
# Create the dictionary entry
self.attributes[ name ] = {
'description':description,
'name':name,
'value':value,
}
With the exception of the decorator for the __PrivateAttribute,
this behaves exactly as expected, populating Ook._documentation.attributes
with:
{
'PublicAttribute': {
'description': 'Public attribute description',
'name': 'PublicAttribute',
'value': None
},
'_ProtectedAttribute': {
'description': 'Protected attribute description',
'name': '_ProtectedAttribute',
'value': None}
}
The __PrivateAttribute bears some discussion, perhaps. Since it is
nominally private, it shouldn't really be accessed outside the class it's a
member of — that's what private
means in an OO context. Since
private
class-members in Python are only private by convention,
it would be feasible to modify the decoration process (in AddAttribute)
to check for the mangled
name of a private class-attribute, but I see
no real need for that, so I'll leave it as-is, and remove the decorator for
__PrivateAttribute. The process automatically acquires the values of
the decorated attributes, so all of that feels pretty tidy, overall.
The initialization of class_documentation also bears some
examination, I think. At this point, it's pretty simple, though there are some
aspects to it that I'd like to find better ways of handling:
def __init__( self, decoratedItem ):
"""
Instance initializer"""
# Call parent initializers, if applicable.
# Set default instance property-values with _Del... methods as needed.
self._DelDecoratedItem()
self._attributes = {}
self._decoratedItem = None
self._isAbstract = inspect.isabstract( decoratedItem )
self._methods = {}
self._originalDocstring = None
self._properties = {}
# Set instance property values from arguments if applicable.
self._SetDecoratedItem( decoratedItem )
if decoratedItem.__doc__:
self._originalDocstring = decoratedItem.__doc__
else:
self._originalDocstring = """
No original doc-string provided for %s""" % ( decoratedItem.__name__ )
The potential issue I see (and would like to work out) resides in the
_SetDecoratedItem method:
def _SetDecoratedItem( self, value ):
if not type( value ).__name__ in ( 'type', 'classobj', 'ABCMeta' ):
raise TypeError( '%s.DecoratedItem expects a class (new- or old-'
'style), but was passed "%s" (%s).' % (
self.__class__.__name__, value, type( value ).__name__ ) )
self._decoratedItem = value
It arises in the type-check performed first thing in the method, where I
discovered that classes defined with a __metaclass__ (like the
abc.ABCMeta used to define abstract classes) may not be standard
type or classobj types. In the case of an
abstract class, the type of the class returns as ABCMeta. The
__metaclass__ declaration, as I understand it, allows a developer
to override what base class a new class is derived from, and removes the original
(implied?) type (or classobj?) from the ancestry of
the new class. Completely. That's based on some examples I've read out and around
on the web (here
and here,
as starting-points). Maybe I'm just not Pythonic enough
in my development
efforts to date, but apart from using ABCMeta, I've never used
__metaclass__. Maybe I just don't really understand __metaclass__.
I don't know. At any rate, the practical implication of using __metaclass__
in classes that are being documented with class_documentation is that
any __metaclass__ used in any class must be accounted for in
class_documentation._SetDecoratedItem. An additional (if hopefully
minor) item of note is that the check is using magic strings to identify the
accepted types. I could convert those to real
types, type and
ABCMeta, but doing so would be at the cost of not supporting
classic
Python classes. The underlying cause of that is that classic
classes, report back as being <type 'classobj'>,
but
there is no classobj type available without importing the
types module and comparing against types.ClassType.
Whether classic
classes are formally deprecated or not, the new class
is the officially-recommended way to create classes in Python in Python 3.x (as
noted here), and
while I use the new class
structure in Python 2.x code I write, I cannot rule
out the possibility that I'll want to use classic
-style code even before I
convert over to Python 3.x.
For now, I think I'll leave things be. I may come back and add types.ClassType
and remove the magic strings at some point later, or I may even come back and
significantly alter the process if other changes warrant it. As things stand right
now, though, it's sufficient until proven otherwise, even if I cringe a little bit
at the thought.
Next up is abstraction. When I performed a test-run of the documentation-generation
on the final
version of the Ook class was that even though I'd
made the Fnord method abstract, there was no indication of that in the output:
[function]
Fnord(self, arg1, arg2, *args, **kwargs)
Ook.Fnord (method) original doc-string
Deprecated: Use new_Fnord instead.
Returns: None (at least until the method is implemented)
...
This is complicated by the fact that while Python's
inspect module
does
provide an
isabstract method, that method only operates against abstract
classes, not abstract methods
within those classes. At first blush,
that made it seem like there was no good way to determine if a given method was
abstract at all. However, after creating two new
Ook methods, one
abstract and one not, and examining their respective
__dict__s, I noticed
a difference: The abstract method had an
__isabstractmethod__ attribute
while the non-abstract one did not. My initial plan was to add an
isabstract flag to the
callable_documentation class,
defaulting to
False, and set it to
True if the results
if
inspect.isabstract indicated that the method was abstract. Changing
the approach to a check for the
__isabstractmethod__ instead still
allows that approach to be viable.
That, however, raised another interesting facet: Since a method's abstraction
is controlled by a decorator rather than by a direct declaration on the method as in
other languages, a method's abstraction isn't necessarily detectable at any given
point during the interpretation of the class. Looking at Ook.Fnord as an
example, and remembering that decorators fire from last to first:
- The first
fixmedecorator fires, creating the callable_documentation
instance. At this point, the method has not been defined as abstract.
- The remaining
describe decorators up to the argument
decoration for arg1 execute. At no point through these
decorators' execution is the method abstract.
- The
abc.abstractmethod decorator executes. Now the
method is abstract.
describe.AttachDocumentation executes, and the method is still
abstract.
Basically, the method will not be recognizable as abstract until the
abc.abstractmethod
call is executed, and since that's another decorator, that could happen
anywhere.
So, what I did was to create a helper method on
callable_documentation:
_checkForAbstraction that looks for the
__isabstractmethod__
attribute in the decorated item's
__dict__, (but only if it hasn't
already been flagged as abstract, since that's not removable after the fact):
def _checkForAbstraction( self ):
if self._decoratedItem and not self.isabstract:
if self.DecoratedItem.__dict__.get( '__isabstractmethod__' ):
self._isAbstract = True
That method is then called in every instance in the various method- and
function-decorators where a callable_documentation is either
retrieved or created:
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
except AttributeError:
decoratedItem.__dict__[ '_documentation' ] = callable_documentation( decoratedItem )
_documentation = decoratedItem._documentation
_documentation._checkForAbstraction()
That's a little wasteful of cycles, maybe, but it guarantees that
detection of abstraction is handled correctly so long as the abc.abstractmethod
decoration executes before at least one of the relevant describe
decorators (including describe.AttachDocumentation). Essentially,
the only rule
that has to be followed for it to work is that the
abc.abstractmethod decorator has to live after the
describe.AttachDocumentation decorator in the code, which shouldn't
be too difficult to do. With a couple of minor tweaks here and there, the
documentation accommodates the abstract method just fine:
[abstract function]
Fnord(self, arg1, arg2, *args, **kwargs)
Ook.Fnord (method) original doc-string
Deprecated: Use new_Fnord instead.
Returns: None (at least until the method is implemented)
...
Interestingly, inspect.isabstract, when used to check whether a
class is abstract or not, suffers from the same limitation: Until a member
is declared as an abstract method (or, I presume, an abstract property),
it does not recognize that the class is abstract. Fortunately, by the time
the first class-level decorator fires, any such abstract decorations will already
have completed, so it will not be an issue.
On to properties and methods!
Acquisition of a class' property-documentation can happen during the
creation of the related class_documentation object:
def __init__( self, decoratedItem ):
"""
Instance initializer"""
# Call parent initializers, if applicable.
# ...
self._properties = {}
# Populate the property documentation here and now...
for name in dir( decoratedItem ):
item = getattr( decoratedItem, name )
if inspect.isdatadescriptor( item ):
try:
expects, main = item.__doc__.split( ')', 1 )
expects += ')'
description, allRaises = main.split( '. ', 1 )
description += '.'
description = description.strip()
raises = [ item.strip() + '.' for item in allRaises.split( '.' ) if item ]
self._properties[ name ] = {
'description':description,
'expects':expects,
'name':name,
'raises':raises,
}
except:
# __doc__ doesn't follow our standard structure, so it
# presumably isn't one of ours. Ignore it.
pass
# Set instance property values from arguments if applicable.
While that process doesn't actually yield the exception-keyed dict
that I'd originally wanted, it does keep the data that I actually want in a
usable form. Dropping pprint( self._properties[ name ] ) after
setting self._properties[ name ] shows this:
{'description': 'Gets, sets or deletes the PropertyName property of the instance.',
'expects': '(int|float)',
'name': 'PropertyName',
'raises': ['Raises TypeError if set to an invalid value-type.',
'Raises ValueError if set to an invalid value.']}
While I'm reasonably certain I could figure out a way to get the exception
names out of the raises items, and get the actual exception class
from those names to use as keys, the more I thought about it, the less it seemed
needful to go that far — particularly since any process that serialized
the documentation metadata would have to undo that effort anyway.
With that caveat in mind, the class-level methods structure
should arguably use a plain-vanilla dict as well, rather than
the original callable_documentation object. That would require
either a substantial rework of the class, or the addition of a method that
can generate the sort of dict representation needed —
call it toDict() — and method-documentation can be acquired
in the same pass as property-documentation:
def toDict( self ):
results = {
'arglist':self.arglist,
'arguments':self.arguments,
'deprecated':self.deprecated,
'fixmes':self.fixmes,
'isabstract':self.isabstract,
'keywordargs':self.keywordargs,
'raises':self.raises,
'returns':self.returns,
'todos':self.todos,
}
# TODO: Deal with the type-objects in various "expects" in argument-
# related items?
return results
I'm not sure, but I suspect that somewhere down the line, I'll want or need
to convert the real
types (int, float, etc.)
that are used in the "expects" values of all of the argument-documentation
items into string values. I'll leave that alone for now, though. With
toDict in place, acquiring method-documentation in the
class_documentation object is simple:
# Populate the property documentation here and now...
for name in dir( decoratedItem ):
item = getattr( decoratedItem, name )
if inspect.isdatadescriptor( item ):
# ...
elif inspect.ismethod( item ):
try:
self._methods[ name ] = item._documentation.toDict()
except AttributeError:
# No _documentation item, so not a method with documentation
pass
# Set instance property values from arguments if applicable.
With access to all of the callable_documentation items in a
documented class and/or it's dictionary representation, generating complete
HTML documentation-output for an entire class is pretty straightforward. It's
mostly iterating over collections of metadata-items and wrapping some
markup-structure around those values. With some tweaks to the structures of
methods, utilization of some HTML 5 tags, and some changes to the style-sheet
that controls what they look like, the documentation for our Ook
class looks like so:
Test-class.
-
PublicAttribute
- Public attribute description
-
_ProtectedAttribute
- Protected attribute description
-
PropertyName
- (int|float) Gets, sets or deletes the PropertyName property of the instance.
[function]
Bleep(cls, arg1, arg2, *args, **kwargs)
Ook.Bleep (classmethod) original doc-string
Deprecated: Will be removed by version X.YY.ZZ
Arguments
- cls
- (class, required): The class that the method will bind to for execution.
- arg1
- (int|long|float, required): Ook.Bleep (classmethod) arg1 description
- arg2
- (any, optional, defaults to None): Ook.Bleep (classmethod) arg2 description
- *args
- (any): Ook.Bleep (classmethod) arglist description
[abstract function]
Fnord(self, arg1, arg2, *args, **kwargs)
Ook.Fnord (method) original doc-string
Deprecated: Use new_Fnord instead.
Returns: None (at least until the method is implemented)
Fix Me:
- Rewrite list-loops to perform the same operations in fewer passes
- Magic _parameters value needs to be removed
Arguments
- self
- (instance, required): The object-instance that the method will bind to for execution.
- arg1
- (bool|None, required): Ook.Fnord (method) arg1 description
- arg2
- (any, required): Ook.Fnord (method) arg2 description
- *args
- (int|long|float): Ook.Fnord (method) arglist description
- The following values are specified by position:
- argitem1
- (float): Ook.Fnord.args[0] description
- argitem2
- (int|long): Ook.Fnord.args[1] description
- argitem3
- (bool): Ook.Fnord.args[2] description
- values
- (str|unicode): Ook.Fnord.args[3] (values) description
- **kwargs
- Ook.Fnord keyword-arguments list description
- keyword1
- (int|long|float, required): Ook.Fnord (method) "keyword1" description
- keyword2
- (None|str|unicode, defaults to None): Ook.Fnord (method) "keyword2" description
- keyword3
- (None|str|unicode): Ook.Fnord (method) "keyword3" description
Exceptions
- NotImplementedError
- if called
To-Do:
- Change output to class with the same interface
- Clean up output to remove empty members
The code that generates this output is:
def toHTML( self ):
"""
Creates and returns an HTML representation of the documentation of the item."""
moduleName = self.DecoratedItem.__module__
for name, cls in inspect.getmembers( inspect.getmodule( self.DecoratedItem ), inspect.isclass ):
classMembers = dict( inspect.getmembers( cls ) )
for classMember in classMembers.values():
try:
if classMember.__name__ == self.DecoratedItem.__name__:
property_documentation = name
except AttributeError:
pass
itemId = '%s.%s' % ( moduleName, self.DecoratedItem.__name__ )
documentation = self.DecoratedItem._documentation.toDict()
results = '<section id="%s" class="class documentation">\n' % itemId
results += """ <header class="heading">
<div class="api_type">[%s]</div>
<div class="class_name"><span class="api_name">%s</span></div>
</header>\n""" % ( type( self.DecoratedItem ).__name__, self.DecoratedItem.__name__ )
results += """ <details>
<summary>%s</summary>\n""" % ( documentation[ 'originalDocstring' ].strip() )
if documentation[ 'attributes' ]:
attributes = documentation[ 'attributes' ]
results += """ <section>
<header class="heading">Class Attributes</header>
<dl>\n"""
for attribute in sorted( attributes.values(), key=lambda item: item[ 'name' ] ):
results += """ <dt>
<span class="attribute name">%s</span>
</dt>\n""" % ( attribute[ 'name' ] )
results += """ <dd class="attribute description">%s</dd>\n""" % ( attribute[ 'description' ] )
results += """ </dl>
</section>\n"""
if documentation[ 'properties' ]:
properties = documentation[ 'properties' ]
results += """ <section>
<header class="heading">Properties</header>
<dl>\n"""
for property in sorted( properties.values(), key=lambda item: item[ 'name' ] ):
results += """ <dt>
<span class="property name">%s</span>
</dt>\n""" % ( property[ 'name' ] )
results += """ <dd class="property description">%s %s</dd>\n""" % (
str( property[ 'expects' ] ), property[ 'description' ] )
results += """ </dl>
</section>\n"""
if self.DecoratedItem._documentation.methods:
methods = self.DecoratedItem._documentation.methods
results += """ <section>
<header class="heading">Methods</header>\n"""
pprint( methods )
for name in sorted( methods ):
if name[ 0 ] == '_':
continue
method = getattr( self.DecoratedItem, name )
results += method._documentation.toHTML()
results += """ </section>\n"""
results += """ </details>\n"""
results += '</section>\n'
return results
As I was working on material for the next post, I realized that while I'd created
a method in the describe class to initialize class-level documentation,
I never actually showed it here. On top of that, there was a minor bug that
was causing the documentation of classes derived from other documented classes to
acquire the wrong class_documentation object. The complete (and fixed)
code for the describe.InitClass method is:
@classmethod
def InitClass( cls ):
"""
Decorates a class by attaching documentation-metadata to it."""
# Type- and value-check incoming arguments
def _InitClassDecorator( decoratedItem ):
# Make sure the decorated item has a _documentation attribute, and
# if it doesn't, create and attach one:
try:
_documentation = decoratedItem._documentation
# Make sure we're looking at the proper decorated item. If
# _documentation is resolved, but doesn't have the
# decoratedItem provided to the decorator, we need to create a
# *new* documentation-instance!
if _documentation.DecoratedItem != decoratedItem:
decoratedItem._documentation = class_documentation( decoratedItem )
except AttributeError:
decoratedItem._documentation = class_documentation( decoratedItem )
_documentation = decoratedItem._documentation
# Return the decorated item!
return decoratedItem
return _InitClassDecorator
The issue noted, which was remedied by the if _documentation.DecoratedItem
!= decoratedItem check, boiled down to the _documentation being
returned acquiring the class_documentation instance of the first
super-class encountered that had one. The check forces the creation of a new
class_documentation instance if the applicable decorated item (the class
being decorated) doesn't match the decorated item of the found
class_documentation instance. Pretty straightforward, I think.
I'm reasonably happy with this code for the time being, but once I start
working out how to generate markup in a more DOM-object-oriented fashion,
I suspect that I'll want to come back and re-address this. Right now, the
code that generates this output is so laden with magic
strings
(if only mostly for tag-names) that alteration of it would be painful. On top
of that, keeping track of where things actually live
in the markup
structure is somewhat tedious.
That, I think, pretty much wraps up the documentation, though (finally)!
It took six posts (not quite 300k), over 2,000 lines of code (at just about
100k), and I suspect that it was pretty dry stuff, so I'll give some thought
to what I'm going to hit next, if only to see if I can come up with something
a bit more exciting.
