Showing posts with label planning. Show all posts
Showing posts with label planning. Show all posts

Tuesday, April 25, 2017

Generating and Parsing Markup in Python [2]

With one interface defined, and most of the module design and DOM-compliance properties and methods figured out, today's post will continue with some concrete implementation and more interface definition. I'm going to get as far through all of the non-concrete implementations as I can, since the foundations they provide, while critically important in the long run, aren't as demonstrable as the concrete markup implementations.

Defining the BaseNode Abstract Class

BaseNode is the first abstract class I'm going to tackle in the markup module, and the first definition of any concrete functionality there. Since it's intended to provide some concrete implementation while just passing some of the abstraction from IsNode on to the concrete classes that will derive from it, there isn't a lot of concrete implementation, though.

Implementing and Testing the Concrete Properties

Since I've noted in some detail in my coding standards exactly how I prefer to implement instance properties, I'll be focusing more on how the properties get their jobs done than what the code underneath the public interface really looks like. BaseNode will provide concrete implementations for six properties that are required by the IsNode interface:

  • nextElementSibling;
  • nextSibling;
  • parentElement;
  • parentNode;
  • previousElementSibling; and
  • previousSibling
Four of those properties, the next* and previous* items, rely on their corresponding parent* property — if the instance in question doesn't have a parent of the appropriate type in the parent* property, then there can't be a next* or previous* value. Those parent* properties, then, need to be worked out first.

What Constitutes a parent Anyway? 

The short and obvious answer is probably best expressed as an IsNode instance that has the various *child* properties and *Child methods. That's something that I haven't really addressed in any detail yet. A fairly complete list of the properties and methods that involve child nodes, taken from the big list presented in the last post, would include:

  markup Module Equivalent Class
Member Name Comment Tag Text
Property Members
childElementCount n/a number n/a
childNodes object object object
children n/a object n/a
firstChild null object null
firstElementChild n/a null n/a
lastChild null object null
lastElementChild n/a null n/a
Method Members
appendChild function function function
contains function function function
getElementsByClassName n/a function n/a
getElementsByTagName n/a function n/a
hasChildNodes function function function
insertBefore function function function
replaceChild function function function

At present, in the current class-relationships diagram, there really isn't any single interface, abstract class or class that feels to me like the right place to set those up: That said, there's only one concrete class so far that actually needs any of those members: Tag — although down the line, any concrete document-classes that derive from BaseDocument (which in turn derives from Tag) will need those as well. From an architectural standpoint, that feels to me like a need for an interface (call it IsElement for now) at a minimum that Tag will implement, and that ties in to the various *child*-related members listed above.

I'll plan on working out IsElement right after I finish with BaseNode, then.

Another item for consideration: In the JavaScript DOM functionality that I'm trying to keep consistent with, there are two distinct parent-types: Elements (tags) and nodes (everything else). I've already established that non-tags really can't have children in at least one major browser-engine branch (webkit, used by Safari, Chrome and Chromium). In Firefox (mozilla), it's not much different — the specifics of the error-messages are different, but the fundamental DOM-object relationship is the same: text-nodes can't have children. That, then, begs the question: Why is there a parentElement and a parentNode method? Particularly since running code like this:

tag = document.createElement( 'tag' );
text = document.createTextNode( 'this is a text-node' );
tag.appendChild( text );
console.log( 'text.parentElement ..................... ' + 
    text.parentElement );
console.log( 'text.parentElement == tag .............. ' + 
    ( text.parentElement == tag ) );
console.log( 'text.parentElement.isSameNode( tag ) ... ' + 
    text.parentElement.isSameNode( tag ) );
console.log( 'text.parentNode ........................ ' + 
    text.parentNode );
console.log( 'text.parentNode == tag ................. ' + 
    ( text.parentNode == tag ) );
console.log( 'text.parentNode.isSameNode( tag ) ...... ' + 
    text.parentNode.isSameNode( tag ) );
yields results showing that parentNode and parentElement return the same tag-element:
text.parentElement ..................... [object HTMLUnknownElement]
text.parentElement == tag .............. true
text.parentElement.isSameNode( tag ) ... true
text.parentNode ........................ [object HTMLUnknownElement]
text.parentNode == tag ................. true
text.parentNode.isSameNode( tag ) ...... true
It just seems... weird, I guess. I hope that it's some sort of concession made for backwards compatibility, but I can't be sure that's the case. On top of that, I can't think of a use-case at all where parentNode wouldn't return the same thing as parentElement. Non-element nodes can't have children, and thus can't be parents, bu the naming convention in the JavaScript DOM-methods seems to be pretty consistent in *Element* methods returning elements (tags) only, while *Node* methods can return any node-type, including elements.

However, in the interests of preserving the DOM-object consistency that I want to preserve, I guess I'll have to keep both those properties. That doesn't mean, though, that I need to have separate property-getters for each, though!

The parent, parentElement and parentNode Properties

It may be simpler to just show the code in this case, then note the differences from my usual patterns:

#-----------------------------------#
# Instance property-getter methods  #
#-----------------------------------#

@describe.AttachDocumentation()
@describe.returns( 'IsElement instance or None' )
def _GetParent( self ):
    """
Returns the IsElement object that the instance is a child of, or None if there is 
no parent-child relationship available for the instance."""
    return self._parent

#-----------------------------------#
# Instance property-setter methods  #
#-----------------------------------#

@describe.AttachDocumentation()
@describe.argument( 'value', 
    'the object to set as the parent of the instance',
    IsElement
)
def _SetParent( self, value ):
    """
Sets the instance's parent to the supplied IsElement object."""
    if not isinstance( IsElement, value ):
        raise TypeError( '%s.parent expects an instance of a class '
            'derived from IsElement, but was passed "%s" (%s), '
            'which is not one' % ( 
                self.__class__.__name__, value.__repr__(), 
                type( value ).__name__
            )
        )
    self._parent = value

#-----------------------------------#
# Instance property-deleter methods #
#-----------------------------------#

def _DelParent( self ):
    """
Deletes the instance's parent relationship by setting it to None"""
    self._parent = None

#-----------------------------------#
# Instance Properties (abstract OR  #
# concrete!)                        #
#-----------------------------------#

parent = describe.makeProperty(
    _GetParent, None, None, 
    'the IsElement object that the instance is a child of', 
    IsElement, None
)
parentElement = describe.makeProperty(
    _GetParent, None, None, 
    'the IsElement object that the instance is a child of', 
    IsElement, None
)
parentNode = describe.makeProperty(
    _GetParent, None, None, 
    'the IsElement object that the instance is a child of', 
    IsElement, None
)
What all of this provides is a set of three different properties (parent, parentElement and parentNode) that are all pointed at the same property-getter method (_GetParent). If, in the future, there's a demonstrable need to separate those out for some reason, it should be a relatively simple matter of creating a new property-getter, -setter and -deleter method-set, then re-assigning the methods in whichever property-declaration need to point to the new method(s). The one concern that I think would come up in that sort of scenario is what would have to happen to the parent property. Right now, the three are completely interchangeable, but if an actual difference between parentElement and parentNode ever surfaces, the parent property may well need to be deprecated or even removed, rather than linger there being confusing.

The nextElementSibling, nextSibling, previousElementSibling and previousSibling Properties

There is a common theme that runs across these four properties, all based around the idea that if the instance has a parent, then that parent has children and childNodes properties that are a sequence of IsElement- and IsNode-derived objects, respectively. Given that, all of these methods need to look at all of the instance's parent's childNodes (which will always include all IsNode-derived types), find the position of the instance whose sibling is being sought in that sequence, then return the previous or next node or element before or after that position, respectively. The first step, finding the position of the instance in its parent's childNodes is common across all four methods.

The _GetnextElementSibling and _GetnextSibling getter-methods show the determination of the index of the instance in its parent's collection of children (parentIndex in parent.childNodes) and the slicing of those childNodes to retrieve everything after the instance in that collection. _GetnextElementSibling also shows filtering of that slice, so that only IsElement items will be considered as candidates for the return value.

@describe.AttachDocumentation()
@describe.returns( 'IsElement object or None' )
def _GetnextElementSibling( self ):
    """
Gets the next IsElement element in the instance's parent's children after the 
instance's position in that sequence of objects"""
    if self.parent:
        # Get the index of the instance in its parent's collection 
        # of children. If this fails, there's an issue with adding 
        # children somewhere else...
        parentIndex = self.parent.childNodes.index( self )
        # Get a slice of the parent's children that captures all the 
        # children *after* the index
        nodesAfter = self.parent.childNodes[ parentIndex + 1: ]
        # Since this is an "element" property, filter those down to just 
        # IsElement members
        nodesAfter = [ 
            node for node in nodesAfter 
            if isinstance( IsElement, node )
        ]
        if len( nodesAfter ) > 0:
            # If there's at least two items, return the first one in 
            # the list
            return nodesAfter[ 0 ]
        else:
            # Otherwise, there aren't any *elements* after the instance, 
            # so return None
            return None
    else:
        # The instance has no parent, and thus there are no siblings.
        return None

@describe.AttachDocumentation()
@describe.returns( 'IsNode object or None' )
def _GetnextSibling( self ):
    """
Gets the next IsNode element in the instance's parent's children after the 
instance's position in that sequence of objects"""
    if self.parent:
        # Get the index of the instance in its parent's collection 
        # of children. If this fails, there's an issue with adding 
        # children somewhere else...
        parentIndex = self.parent.childNodes.index( self )
        # Get a slice of the parent's children that captures all the 
        # children *after* the index
        nodesAfter = self.parent.childNodes[ parentIndex + 1: ]
        if len( nodesAfter ) > 0:
            # If there's at least two items, return the first one in 
            # the list
            return nodesAfter[ 0 ]
        else:
            # Otherwise, there aren't any *elements* after the instance, 
            # so return None
            return None
    else:
        # The instance has no parent, and thus there are no siblings.
        return None
Really, the only major difference between _GetnextElementSibling and _GetnextSibling is whether the intermediate list (nodesAfter) is filtered.

The same basic pattern is used in _GetpreviousElementSibling and _GetpreviousSibling, including the filtering or non-filtering of the intermediate results (nodesBefore). The major difference between either _Getprevious* method and its _Getnext* counterpart is the initial slice of the instance's parent.childNodes:

            # Get a slice of the parent's children that captures all the 
            # children *before* the index
            nodesBefore = self.parent.childNodes[ 0:parentIndex - 1 ]
The filtering aspect in _GetpreviousElementSibling is identical to the code above for _GetnextElementSibling, and doesn't exist at all in _GetpreviousSibling.

Implementing and Testing the Concrete Methods

I'd originally expected to implement only two of the abstract methods of IsNode in BaseNode: IsEqualNode and isSameNode. With the addition of the IsElement interface to the markup class-zoo, though, any concrete implementation of isEqualNode will, I think, have to be moved out to the concrete classes — since those are the most-shallow points in the inheritance structure where all of the various properties that the method needs will actually exist.

That leaves isSameNode as the only concrete method-implementation of BaseNode:

@describe.AttachDocumentation()
@describe.argument( 'node', 
    'the node-object to compare to the instance to '
    'see if they are the same',
    IsNode
)
@describe.raises( TypeError,
    'if passed a node value that is not an IsNode instance'
)
@describe.returns( 
    'True if the supplied node is the same node-object as '
    'the instance, False otherwise'
)
def isSameNode( self, node ):
    """
Determines if a supplied node is the same node-object as 
the instance"""
    if not isinstance( node, IsNode ):
        raise TypeError( '%s.isSameNode expects an instance '
            'of IsNode for comparison, but was passed '
            '"%s" (%s)' % ( 
                self.__class__.__name__, 
                node, type( node ).__name__
            )
        )
    # If the node is the same object, it will 
    # have the same id, so:
    return id( self ) == id( node )

Normally, I'd also be looking to implement unit-tests of BaseNode, now that all of its concrete implementation is complete. In this case, because all of the *Sibling properties require participation in a node-tree structure that won't be available until I have both IsElement and a concrete class that derives from it implemented (Tag in this case), I only went as far as getting the test-method requirements stubbed out, along the lines of:

def testpreviousSibling(self):
    """Unit-tests the previousSibling property of a BaseNode instance."""
    self.fail( 'testpreviousSibling is not implemented' )
and
def testisSameNode(self):
    """Unit-tests the isSameNode method of a BaseNode instance."""
    self.fail( 'testisSameNode is not implemented' )
That means that I'll have several test-failures for a while:
########################################
Unit-test results
########################################
Tests were successful ... False
Number of tests run ..... 37
 + Tests ran in ......... 0.01 seconds
Number of errors ........ 0
Number of failures ...... 15
########################################
I could implement a dummy class in the test-module that derives from BaseNode, and test against that class, and if there weren't a concrete class expected that would serve that purpose, that's exactly what I'd do. Since I will have one, eventually, that just feels... wasteful, I guess, so I'd rather get Tag operational and then come back to these tests. Until then, I'll just have to live with these test-failures.

Defining the IsElement Interface

Between the previous post and the breakdown of members needed in IsElement above, there's really not much discussion needed, I think, nor a whole lot of code to show and explain.

The Abstract Properties of IsElement

On basic principle, I did do another run through the members of an element listed at the w3schools site, just to ensure that I didn't miss any. What I netted out with for properties in IsElement was:

childElementCount = abc.abstractproperty()
childNodes = abc.abstractproperty()
children = abc.abstractproperty()
firstChild = abc.abstractproperty()
firstElementChild = abc.abstractproperty()
lastChild = abc.abstractproperty()
lastElementChild = abc.abstractproperty()

There were other properties that I had to think about too, though — accessKey (which may well be globally available at the implementation-level of a Tag), and attributes (which absolutely is a Tag property). When push came to shove, though I opted to implement those as concrete members of Tag rather than drop them into the IsElement interface. The rationale for that decision was mostly based on the realization that the only other elements that I'm expecting to be concerned with are documents, and my current plan is to derive a BaseDocument abstract class from Tag anyway. In that scenario, all of the other properties and methods would be implemented by Tag, and available to documents through their derivation from BaseDocument anyway. Those members include all tag-level properties that are also attributes in markup, as well as any properties that aren't specifically related in some way to having, working with, or altering a parent-child relationship between a Tag and any other IsNode instance.

The Abstract Methods of IsElement

The same criteria noted above for properties was also used to cull down the list of methods that would be required by IsElement, for pretty much te same reasons. The resulting abstract methods are:

@abc.abstractmethod
def appendChild( self, child ):
    raise NotImplementedError( '%s.appendChild is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def hasChildNodes( arg1, arg2=None, *args, **kwargs ):
    raise NotImplementedError( '%s.hasChildNodes is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def insertBefore( self, newChild, existingChild ):
    raise NotImplementedError( '%s.insertBefore is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def insertChildAt( self, newChild, index ):
    raise NotImplementedError( '%s.insertChildAt is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def removeChild( self, child ):
    raise NotImplementedError( '%s.removeChild is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def removeChildAt( self, index ):
    raise NotImplementedError( '%s.removeChildAt is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def removeSelf( self ):
    raise NotImplementedError( '%s.removeSelf is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

@abc.abstractmethod
def replaceChild( self, newChild, existingChild ):
    raise NotImplementedError( '%s.RequiredMethod is not implemented as '
        'required by IsElement' % self.__class__.__name__ )

Testing the Abstract Members of IsElement

The unit-testing of the abstract members of IsElement follows the pattern established by the testing of IsNode members shown in my previous post, with the hopefully-obvious change of class-name being tested:

def testPROPERTYNAME(self):
    """Unit-tests the PROPERTYNAME property of an IsElement instance."""
    try:
        testInstance = markup.IsElement()
    except TypeError, error:
        actual = 'PROPERTYNAME' in str( error )
        self.assertTrue( actual, 'The TypeError raised by trying to '
            'instantiate IsElement should include the "PROPERTYNAME" '
            'abstract method-name' )
    except Exception, error:
        self.fail( 'testPROPERTYNAME expected a TypeError, '
            'but %s was raised instead:\n  - %s' % ( 
                error.__class__.__name__, error
            )
        )
def testMETHODNAME(self):
    """Unit-tests the METHODNAME method of an IsElement instance."""
    try:
        testInstance = markup.IsElement()
    except TypeError, error:
        actual = 'METHODNAME' in str( error )
        self.assertTrue( actual, 'The TypeError raised by trying to '
            'instantiate IsElement should include the "METHODNAME" '
            'abstract method-name' )
    except Exception, error:
        self.fail( 'testMETHODNAME expected a TypeError, '
            'but %s was raised instead:\n  - %s' % ( 
                error.__class__.__name__, error
            )
        )
Since that pattern is simple and established, I won't go into the details of their implementation here, but I'll get them in place and make sure that they run as expected. Bearing in mind that there are still fifteen failures from the still-pending tests of BaseNode, those same failures should still appear, but the number of tests run and passed should increase:
########################################
Unit-test results
########################################
Tests were successful ... False
Number of tests run ..... 56
 + Tests ran in ......... 0.01 seconds
Number of errors ........ 0
Number of failures ...... 15
########################################

With the change made to the markup module's class-zoo (the addition of IsElement in the upper right of the diagram), I didn't get quite as far long in this post as I'd hoped before hitting my post-length cut-off, but I feel like I made solid progress.

There's one more abstract class that I'm going to define in my next post before I can start some actual concrete implementations: HasTextData. With that done, I'll be able to knock out three concrete classes pretty quickly, I think: CDATA, Comment and Text.

The completion of those will also require me to give some thought to exactly how I plan for rendered markup to be issued back out to a browser, so there will be at least some discussion around that as well.

Thursday, April 20, 2017

Generating and Parsing Markup in Python [1]

The first thing that I'm going to do in building out the markup module's class-structure is to figure out where all of the various members of those classes originate, and at what point they are concrete. One of my priorities, as mentioned before, is to try and keep as much similarity between the classes and their members in the markup module and the equivalent DOM objects in typical JavaScript implementations on the client side.

Conforming to DOM Conventions

I can't really meet that goal, conforming to the interfaces of DOM elements (tags, text-nodes, comments and CDATA sections) until I know what members they expose in a browser context. What I did, then, to determine that was write a chunk of JavaScript living in a bare-bones HTML page (download below) that iterates over the list of properties and methods listed on the w3schools.com site, checked an instance of each node-type (except CDATA sections, more on that in a bit) for each property- and method-member that might be available, and reported what came back in that check-process. If a given element did not report that it had the member, then the equivalent class-member in the markup module could be skipped. If the check returned an expected type, like a function for a method, that member should be kept. Anything else that came back will require some additional discovery.

I'd originally included CDATA sections in my collection of objects to examine, but the browser that I ran the page against (Chromium) wouldn't actually allow the creation of a CDATA section, even though it has a document.createCDATASection method. Creation of CDATA sections is not supported for HTML documents according to the error-message I got back. The closest to an actual CDATA that I could get was a comment that contained all of the CDATA's original content, plus the [CDATA[ start and ]] end text. As a result, I don't really know what a CDATA's members look like without doing more digging around. For the time being, I'm willing to leave that be, though — the Comment, Tag and Text classes will likely suffice for my needs for the time being.

The breakdown I got back from that analysis-script was:

  markup Module Equivalent Class
Member Name Comment Tag Text
Property Members
accessKey n/a string n/a
attributes n/a object n/a
childElementCount n/a number n/a
childNodes object object object
children n/a object n/a
classList n/a object n/a
className n/a string n/a
clientHeight n/a number n/a
clientLeft n/a number n/a
clientTop n/a number n/a
clientWidth n/a number n/a
contentEditable n/a string n/a
dir n/a string n/a
firstChild null object null
firstElementChild n/a null n/a
id n/a string n/a
innerHTML n/a string n/a
isContentEditable n/a boolean n/a
lang n/a string n/a
lastChild null object null
lastElementChild n/a null n/a
namespaceURI n/a string n/a
nextElementSibling null null null
nextSibling null null null
nodeName string string string
nodeType number number number
nodeValue string null string
offsetHeight n/a number n/a
offsetLeft n/a number n/a
offsetParent n/a null n/a
offsetTop n/a number n/a
offsetWidth n/a number n/a
ownerDocument object object object
parentElement null null object
parentNode null null object
previousElementSibling null null null
previousSibling null null null
scrollHeight n/a number n/a
scrollLeft n/a number n/a
scrollTop n/a number n/a
scrollWidth n/a number n/a
style n/a object n/a
tabIndex n/a number n/a
tagName n/a string n/a
textContent string string string
title n/a string n/a
Method Members
addEventListener function function function
appendChild function function function
blur n/a function n/a
click n/a function n/a
cloneNode function function function
compareDocumentPosition function function function
contains function function function
focus n/a function n/a
getAttribute n/a function n/a
getAttributeNode n/a function n/a
getElementsByClassName n/a function n/a
getElementsByTagName n/a function n/a
getFeature n/a n/a n/a
hasAttribute n/a function n/a
hasAttributes n/a function n/a
hasChildNodes function function function
insertBefore function function function
isDefaultNamespace function function function
isEqualNode function function function
isSameNode function function function
isSupported n/a n/a n/a
nodelist.item n/a n/a n/a
normalize function function function
querySelector n/a function n/a
querySelectorAll n/a function n/a
removeAttribute n/a function n/a
removeAttributeNode n/a function n/a
removeChild function function function
removeEventListener function function function
replaceChild function function function
scrollIntoView n/a function n/a
setAttribute n/a function n/a
setAttributeNode n/a function n/a
toString function function function
This gives me enough information to at least start making decisions about where various member-properties and -methods need to be defined, and how. Given the class-relationships already defined: Any keeper item from the table above that exists in all the class-types should be required by the IsNode interface, at least as a default consideration. The same consideration should also be given to any items that return the same values across all the class-types, even if they haven't been flagged as a keeper. The logic behind that statement boils down to the fact that while I checked each node-type in the original JavaScript script, I did not populate a large-enough node- and element-sample in that script to feel confident that I captured every valid low-level member. If possible, those same items should also have a concrete implementation in the BaseNode abstract class. There will probably be a few items that, even though they fall into that category, just don't make sense in those locations, but I'll note those as I go along.

Defining the IsNode interface

Starting, then, with the items in the table that are keepers, or that returned identical values across all the different node-types, the following are either directly valid or need to be looked at in more detail for requirement in IsNode:

  markup Module Equivalent Class
Member Name Comment Tag Text
Property Members
childNodes object object object
nextElementSibling null null null
nextSibling null null null
nodeName string string string
nodeType number number number
nodeValue string null string
parentElement null null object
parentNode null null object
previousElementSibling null null null
previousSibling null null null
textContent string string string
Method Members
addEventListener function function function
appendChild function function function
cloneNode function function function
compareDocumentPosition function function function
contains function function function
hasChildNodes function function function
insertBefore function function function
isDefaultNamespace function function function
isEqualNode function function function
isSameNode function function function
normalize function function function
removeChild function function function
removeEventListener function function function
replaceChild function function function
toString function function function

While I was stripping down the list, I noticed that parentElement and parentNode didn't get flagged in such a way to be considered for inclusion in IsNode, but it's a basic fact of markup-languages that all nodes should have those properties — if they aren't populated, that simply means that the node doesn't have a parent currently, but they might well later after some manipulation. The nodeValue property

Looking over that list of remining members, there are a few that don't make any sense to include in IsNode already:

  • Any members that involve child nodes — Those are aspects of a Tag, certainly, but since Comment and Text will also derive from IsNode and they don't have child nodes (and can't?), those should go away. That removes:
    • The childNodes property;
    • The appendChild method;
    • The hasChildNodes method;
    • The insertBefore method;
    • The removeChild method; and
    • The replaceChild method;
  • Any members relating to manipulation of event-listeners — On the server side, where all of the markup module's functionality is actually running, there is no browser context available, so no event-handling processes, and so none of these members are useful. That removes:
    • The addEventListener method; and
    • The removeEventListener method;
The rest will need to be exmined in more detail, one by one, so let me just jump into that now...

Implementing and Testing the Abstract Properties

Since IsNode is only an interface, there are no concrete implementations of properties to define, only abstract property requirements that will be picked up by derived classes. That makes the definition of those property requirements very simple, and the testing of them pretty straightforward. The real trick is determining where the concrete implementations of them is going to occur. Going through the list of properties:

nextElementSibling
Returns the next element at the same node tree level — w3schools
Abstract property in IsNode
Implement in BaseNode
nextSibling
Returns the next node at the same node tree level — w3schools
Abstract property in IsNode
Implement in BaseNode
nodeName
Returns the name of a node — w3schools
Returns the tag-name for Tags, and magic-string constants for other node-types (#comment for a Comment, #document for a document, #text for a Text object, and #cdata for a CDATA if the pattern is maintained).
Abstract property in IsNode
Implement in CDATA, Comment, Tag and Text classes
nodeType
Returns the node type of a node — w3schools
Returns 8 for Comments, 4 for CDATAs, 1 for Tags and 3 for Texts
Abstract property in IsNode
Implement in CDATA, Comment, Tag and Text classes
nodeValue
Sets or returns the value of a node w3schools
It appears that this method returns the first text-node child of an element, rather than the entire set of text-node values, at least in Chromium. At any rate, it's dependent on the presence of child nodes, so...
Skip
Implement in Tag
parentElement
Returns the parent element node of an element — w3schools
Abstract property in IsNode
Implement in BaseNode
parentNode
Returns the parent node of an element — w3schools
Abstract property in IsNode
Implement in BaseNode
previousElementSibling
Returns the previous element at the same node tree level — w3schools
Abstract property in IsNode
Implement in BaseNode
previousSibling
Returns the previous node at the same node tree level — w3schools
Abstract property in IsNode
Implement in BaseNode
textContent
Sets or returns the textual content of a node and its descendants — w3schools
The return value is, essentially, a concatenation of all child Text nodes in a Tag, or the data value (the content) of a Comment or Text instance. If CDATA is assumed to behave like a Comment, then it would also return the inner content of the instance.
Abstract property in IsNode
Implement in HasTextData and Tag
The abstraction of these properties in isNode is just a few lines of code:
#-----------------------------------#
# Abstract Properties               #
#-----------------------------------#

nextElementSibling = abc.abstractproperty()
nextSibling = abc.abstractproperty()
nodeName = abc.abstractproperty()
nodeType = abc.abstractproperty()
parentElement = abc.abstractproperty()
parentNode = abc.abstractproperty()
previousElementSibling = abc.abstractproperty()
previousSibling = abc.abstractproperty()
textContent = abc.abstractproperty()
The test-methods for each property will follow this pattern:
def testPROPERTYNAME(self):
    """Unit-tests the PROPERTYNAME property of an IsNode instance."""
    try:
        testInstance = markup.IsNode()
    except TypeError, error:
        actual = 'PROPERTYNAME' in str( error )
        self.assertTrue( actual, 'The TypeError raised by trying to '
            'instantiate IsNode should include the "PROPERTYNAME" '
            'abstract method-name' )
    except Exception, error:
        self.fail( 'testPROPERTYNAME expected a TypeError, '
            'but %s was raised instead:\n  - %s' % ( 
                error.__class__.__name__, error
            )
        )
In a nutshell, what this does is ensures that the abstract properties appear in the TypeError that is raised by trying to instantiate IsNode, ensuring that the property being tested is abstract.

Implementing and Testing the Abstract Methods

The same basic rule, that member-definitions need only exist in the IsNode interface, applies to the method members as well. The main decisions that need to be made are also similar: where does a given method-requirement and -definition belong, and yields a similar list as the properties noted above:

cloneNode
Clones an element — w3schools
Since this is capable of making shallow or deep copies, and the mechanism for making those copies will vary, it'll have to be implemented in the concrete classes.
Abstract method in IsNode
Implement in CDATA, Comment, Tag and Text
compareDocumentPosition
Compares the document position of two elements — w3schools
The description of the method on the w3schools site, frankly, has me wondering if there's even any point to implementing this on the server side. I've never seen this method used in the wild, though that doen't mean that it isn't used. I can't think of a use-case for it that isn't better served (at least on the server side) by local Python code, particularly since all the real method returns is a bit-mask number-value that indicates relative position between the owner element and the element provided.
Skip
contains
Returns true if a node is a descendant of a node, otherwise false — w3schools
The contains method applies only to objects that have children, really. That hasn't stopped it from being callable on DOM node where it doesn't really make sense, though. For example, executing this JavaScript:
ook = document.createTextNode( 'ook' );
eek = document.createTextNode( 'eek' );
ook.contains( eek );
in several browsers yields
false
That result kind of makes sense — neither of the created text-nodes is a parent of the other, nor can either be appended to the other (calling ook.appendChild( eek ) throws an error).
I'm going to skip this method for now, but there's some discussion around that decision that I'll dig into shortly.
isDefaultNamespace
Returns true if a specified namespaceURI is the default, otherwise false — w3schools
Text-nodes don't have a namespace — it's not a defined member of that node-type at all. Nor do comments, and I presume that the same would hold true for CDATA sections.
Skip
Implement in Tag
isEqualNode
Checks if two elements are equal — w3schools
The complete criteria for testing equality on the client side is listed at the w3schools link above, but since those criteria are dependent on properties that won't exist across all IsNode instances, the usefulness of that list is, perhaps, questionable. Still, being able to perform a comparison is useful. Then the real question is how is that going to be done? I'll work out more details on that later, but for now:
Abstract method in IsNode
Implement in BaseNode
isSameNode
Checks if two elements are the same node — w3schools
Abstract method in IsNode
Implement in BaseNode
normalize
Joins adjacent text nodes and removes empty text nodes in an element — w3schools
This feels like it's something that shuoldn't exist ouside of a Tag, and that seems to be borne out by the fact that it's not possible to usefully call normalize on a text- or comment-node in the browser.
Skip
Implement in Tag
toString
Converts an element to a string — w3schools
Abstract method in IsNode
Implement in CDATA, Comment, Tag and Text

The contains discussion

Also like the abstract-property definitions, abstract methods don't need much in IsNode:

    @abc.abstractmethod
    def METHODNAME( arg1, arg2=None, *args, **kwargs ):
        raise NotImplementedError( '%s.METHODNAME is not implemented as '
            'required by IsNode' % self.__class__.__name__ )
And the unit-tests, since they're really just checking the same sort of relationship between methods and the IsNode interface-class as the property-tests did, is almost identical:
def testMETHODNAME(self):
    """Unit-tests the METHODNAME method of an IsNode instance."""
    try:
        testInstance = markup.IsNode()
    except TypeError, error:
        actual = 'METHODNAME' in str( error )
        self.assertTrue( actual, 'The TypeError raised by trying to '
            'instantiate IsNode should include the "METHODNAME" '
            'abstract method-name' )
    except Exception, error:
        self.fail( 'testMETHODNAME expected a TypeError, '
            'but %s was raised instead:\n  - %s' % ( 
                error.__class__.__name__, error
            )
        )

With those tests in place for IsNode in the test_markup.py unit-test module, the test-results come back clean:

########################################
Unit-test Results: idic.markup
#--------------------------------------#
Tests were SUCCESSFUL
Number of tests run ... 18
Tests ran in .......... 0.001 seconds
########################################
IsNode, then, is done — written and tested.

Dealing with Enumerations in Python 

Python doesn't really have a formal enumeration-type like several other languages do, but there are a number of ways to work around that. My personal favorite uses namedtuple from the collections module, based on some observations I've made about how an enumeration behaves:

  • An enumeration is a constant;
  • An enumeration is immutable — its values cannot be changed at run-time;
  • An enumeration's members are individually accessible by name; and
  • An enumeration is a container, with members that can be used for comparison purposes. That is, given an enumeration of nodeTypes, with presumably-distinct CDATA, Comment, Tag and Text values:
    nodeTypes.Tag in nodeTypes          # == True
    nodeTypes.Text in nodeTypes         # == True
    nodeTypes.CDATASection in nodeTypes # == True
    nodeTypes.Comment in nodeTypes      # == True
    
There are probably a few more aspects to the behavior of an enumeration, but those three are the main ones, at least that I can think of at this point.

Using a namedtuple, it's actually pretty easy to generate a constant value that exhibits all of those behaviors. The basic code required looks something like this, using the nodeType values from the w3schools site and generating an enumeration-equivalent named nodeTypes that could be added to the markup module:

from collections import namedtuple

nodeTypes = namedtuple(
    'enumNodeTypes', 
    [ 'Tag', 'Text', 'CDATASection', 'Comment' ],
    )(
        Tag=1,
        Text=3,
        CDATASection=4,
        Comment=8,
    )

__all__.append( 'nodeTypes' )
  • nodeTypes is a constant because namedtuple returns a class, and the code then creates an instance of that class;
  • It's immutable because it's not possible to add values to, remove values from, or alter existing values of the named items except by altering the definition of those members in the code;
  • Its members are individually accessible by name because that's a basic capability of a namedtuple-generated class; and
  • It's a container that allows the use of someValue in nodeTypes.
That last item is best explained with a quick demonstration: Printing nodeTypes and all of the nodeTypes.NAME in nodeTypes examples in the container criteria above yields:
All entries in nodeTypes
  + enumNodeTypes( Tag=1, Text=3, CDATASection=4, Comment=8 )
nodeTypes.Tag in nodeTypes ............ True
nodeTypes.Text in nodeTypes ........... True
nodeTypes.CDATASection in nodeTypes ... True
nodeTypes.Comment in nodeTypes ........ True
nodeTypes.CDATASection in nodeTypes ... True
nodeTypes.Comment in nodeTypes ........ True
12 in nodeTypes ....................... False
"ook" in nodeTypes .................... False
So, while this approach may not be a real enumeration, it provides all of the functionality of one that I think I'll need.

It occurs to me that I don't really have a unit-testing strategy or policy for module-level constants, but frankly I'm not sure that one is really needed, at least not yet. I say not yet now because at some level, there simply has to be some trust in the underlying language. Even with nodeTypes being a non-simple value, it's still a value that is tightly tied to core language structures and functionality, and it shouldn't be possible to break that without altering the code itself.

There's been a fair chunk of analysis in this post, but some code too, and the next logical piece to work out would probably push the length of this post past where I'd like, so I'm going to stop here for now. The next few items that I'm going to tackle include the BaseNode and HasTextContent abstract classes, I think, then I'll have enough of the foundational abstraction written to be able to take a swing at the CDATA, Comment and Text concrete classes. I promised to include the analysis JavaScript-page, though, so here's that

Thursday, April 13, 2017

Where to Go From Here?

Having spent the last several weeks now on stuff that, while necessary, just isn't fun or sexy from a coding perspective, I'm looking to start getting into some parts of the framework that are actually usable for writing web-applications. As I see it, there are several moving parts that are common to most web-apps, even setting aside the integration with a web-server. I'm not certain which I'll pick up next, though, so I figured I'd write about some of them and see if anything jumped to mind as a logical next step.

Generation (and Parsing) of Markup

Web-applications are all about pages, ultimately, even if those pages are very nearly empty to start with and populated with an AJAX call or whatever other process might surface in the future. In turn, pages are all about markup, whether it's HTML5, XHTML, or XML.

Having spent most of the last several years writing application-code for a creative agency, I firmly believe the idea of separation of markup/structure from functionality/logic has merit — to the point that one of my goals for this framework is to make it as easy as possible to keep that separation, while still allowing as much designer-level control as possible over the structure and appearance of pages. If you're thinking that sounds like a tall order... well, you may be right. Still, I've worked out ways and means of getting a fair part of the way to that goal, and I think it's quite do-able, though it might take some work.

If I'm brutally honest about it, I suspect that my preference for that sort of separation probably stems, at least in part, from several years of working with web CMSs, where the separation of content from markup was a key concept. There have been other influences that push me in that direction, though.

Setting that aside for the moment, I have to say also: I've looked at several Python markup libraries over the years, and have yet to find one that I really like. My perfect solution would include:

  • The ability to represent all of the different and varied node-types in any of the markup languages that I'm interested in or expect to use. Right now, that would include:
    • Tags (of course);
    • Documents (which are, more or less, just extensions of Tags);
    • Text-nodes (also of course);
    • Comments; and
    • CDATA nodes (mostly for the sake of a complete set of node-types).
  • The ability to extend Tags out into specific types of tags in specific languages. This would include things like HTML <form>-elements and all the various <input>-tags within those forms, at a minimum. Part of the reasoning behind that desire is the potential for being able to attach server-side validation to individual user interaction elements on a page, so that, for example, submission of a bad date-value through an <input type="date"> field will automatically flag the submission as invalid.
  • That ability to extend Tags is also something that I hope to use to generate page components — Items that can be included in a page through markup and by non-programmers to allow folks with more of a graphic-design kind of focus to still be able to work with pages that use the framework with little or no technical assistance.
  • A reasonably complete set of DOM functions. More specifically, I'd like programmatic manipulation of markup-objects to be do-able and I'd like it to mirror the set of capabilities that a lot of web-developers already know through JavaScript. There are some additions that I'd like to make as well (being able to tell an element to remove itself from its parent, for example), but on the whole, if it's do-able with JavaScript on the client side of the exchange, I want it to also be do-able server-side, and I want it to use the same property, function- or method-names unless there's a compelling reason that it can't.
  • The ability to parse static markup, whether in the form of a template for an entire document, or some chunk of markup that's used for templating at a smaller scale, into a server-side object-set. That capability allows the server-side application-logic to make changes to some or all of a document before it gets rendered and sent back to the browser client. That capability opens up a lot of flexibility in creating web-applications, I believe.
There may be other considerations that surface later, but these are the ones that, at present, are my high-priority criteria — must-haves, mostly, with a few should-haves sprinkled here and there through the hypothetical stack.

I expect that to be a pretty daunting effort. Without planning out any internal detail or functionality, and before dealing with any of the variations between different markup languages, I believe that all of this requires a pretty substantial class-structure:

Then, too, in order for this to actually be usable, I'd also need to start digging in to the process(es) of connecting a web-server request to a markup-generation response. There are a couple of options to pick from, as I mentioned some time ago, but pursuing any of them will take some time as well.

But the results would be visible. Not a small consideration, perhaps.

A DAL and Data-Objects Implementation

Most web-applications also have some sort of data-access layer too. That's what makes them applications, in many cases — the ability to interact with data that persists across users and sessions. While Python's got a solid set of database-connection modules: there are several for MySQL and PostgreSQL, at least one for ODBC connections, and I'm sure there are several for the various NoSQL data-stores. All of them that I'm aware of conform to the Python Database API Specification, which makes data-access across those different engines pretty consistent.

What I'd like to see on top of that is some consistency in how database calls are actually made. In particular, I feel that if there are objects in an application's structure whose state-data is persisted in a back-end data-store, I'd very much like for those data-objects to be able to manage their own state, without the code having to explicitly check to see if they need to be created, updated or deleted in the data-store. There's also some common access-functionality that I believe would be useful to attach to all instances of data-objects, or, in some cases, to the classes that define those data-objects in the context of an application.

Data-access can be relatively expensive in an application — a connection to the database has to be made, queries executed, and results gathered and returned, and maybe formatted before the data is really ready for the application to make use of it. The data-object strategy may help streamline those data-access processes, at least somewhat, but something else that could help would be allowing the data-access layer to be able to cache queries' results, at least for the duration of a page request/response cycle. Another possible gain would center around a more lazy loading sort of data-access — one where queries aren't actually executed until their results are needed. I'm not sure that is viable (or even desirable — I'll want to think on that more), but it's something that I'd like to at least consider in a DAL.

Since data-stores are going to have different properties from one another (server and credentials at the minimum), but different instances of data-stores that connect to the same server with the same credentials is possible, that strikes me as a good justification to make them configurable. If I pursue that line of thinking, that means that I'll have to start thinking about how I want to set up configuration, then.

Even with that, I think it'll be less complex, structurally, than the markup module, though — the number of classes is about the same, but there's more items that derive from a single, common base class than in the markup structure, and I'm not expecting the base classes in the DAL space to have nearly the same method-member count as I think will exist in the markup base classes.

Though that ignores the configuration aspects of individual data-stores.

So, Where Next?

After all this, I think I've arrived at a path to continue with:

  • Creating the markup module — There's at least the potential that writing all of that out will occupy a lot of time, if only because there's a lot of functionality that will surface, but it will lead nicely, I think, into...
  • Working out details on how to connect parsed-, generated- and application-manipulated markup to a web-server, so that it can be displayed.
  • Creating the data_access module feels like a solid next step after those — with this in place, the beginnings of real application functionality can, I think, start to take shape, leading to...
  • Design and implementation of a page-component model that will facilitate the sort of content/logic separation that I'm aiming for.
  • That, in turn, will allow me to take a serious look at creation of the interaction elements I mentioned earlier, though I may have to think out details on server-side validation before that, or at least enough of the fundamentals of validation to be able to work on validation of interaction elements concurrently.