The Inverse Extension Design Pattern

In an inheritance hierarchy, permit each parent class's method to extend its child class's method so it can act as a decorator for its child class's behavior.


In object-oriented programming, it is common for a child class's method to
extend the parent class's method. However, inverting this pattern is useful
as well. Although most object-oriented programming languages support the
former, I have never encountered one that supported the latter. Nonetheless,
it is possible to implement this pattern without explicit support from
the programming language, just as it was possible to implement inheritance
in C before C++ was created.

A few interesting features of normal extension should be kept in mind. First,
a child class can call its parent class's method anywhere within its own method.
Hence, the child class is responsible for determining if the parent gets to
go first, last or somewhere in the middle. Secondly, the child class gets to
choose what to pass to the parent class's method and what to do with
whatever, if anything, the parent class's method returns. Naturally, these
properties hold for each step in the entire class hierarchy. When inverting
this pattern, these properties are inverted as well. Hence, the parent class
is responsible for determining if the child gets to go first, last or somewhere
in the middle. Furthermore, the parent class gets to choose what to pass to
the child class's method and what to do with whatever, if anything, the child
class's method returns (see Figure 1).


____________
| A        |        ^                   |
|__________|        |                   |
| method() |        |                   |
|__________|        |                   |
     |              |                   |
    /_\             |                   |
_____|______        |                   |
| B        |        |                   |
|__________|        |                   |
| method() |        |                   |
|__________|        |                   |
     |              |                   |
    /_\             |                   |
_____|______        |                   |
| C        |        |                   |
|__________|        |                   |
| method() |        |                   |
|__________|        |                   v

Class Hierarchy     Normal Extension    Inverse Extension

                              Method Invocation

Figure 1. Compare Extension with Inverse Extension

The fact that the parent class's method "wraps" the child class's method rather than
the other way around is the whole motivation for this pattern. At first glance, it
is natural to suggest using the Template Method pattern: in the parent class A's
method "a", call the child class B's method "b". This indeed implements the
wrapping behavior, but it is not scalable. For instance, if a grandchild class C
is added, if b is to wrap C's method, C must add a method c. Creating a new name
for each step in the hierarchy is nowhere near as elegant as simply
calling "super", which you can do with normal extension. Furthermore, creating
such names is not flexible. Suppose you wish to move C to be a subclass of A.
Aside from changing the C's parent class, you also must rename its c
method b. As your class hierarchies get larger, this irritation becomes worse.

In traditional HTML, that is, in HTML where tables are used for layout, this
pattern is quite helpful. The parent class is called the Layout. It has a
method that lays out the page, deciding where the navigation should go and
where the main content should go. The child class is called the Screen. It
has a method that outputs the main content. Although the Screen inherits
from the Layout, it is the Layout's method that takes control first. The
Layout then passes control to the Screen's method--the opposite of
"super"--when it is time for the Screen's method to do its work. To change
the navigation used on a particular Screen--login pages and help pages usually
look quite different from normal pages--simply change that Screen's parent class.
To have all of the Screens in a section have their own sub-navigations, subclass
the existing Layout with a new Layout and have the Screens subclass the new Layout.
Applicability
Inverse Extension can be used when the behavior of the parent's method should decorate
the behavior of the child's method instead of the other way around, especially
if this is true for more than two layers of the class hierarchy. You
also can use inverse extension when a parent class method needs to control
access to the method of its child class.
Structure, Participants and Collaborations
The structure of the class hierarchy is the same as it is for normal
extension. One participant is the Parent (Layout), whose method wraps the
child class's method. The other participant is the Child (Screen), whose
method does the main task at hand, without having to worry about many of
the details that the parent class's method handles for it.

The Parent class's method decides when and if to forward the flow of
control to the Child class's method. It optionally may perform additional
operations before and after forwarding the flow of control. It may pass
any argument it wants to the Child class's method and do anything it wants
with whatever, if anything, the Child class's method returns.
Consequences
The Inverse Extension pattern has the following benefits and liabilities:

  • Changing the child class's parent class easily changes all of the code that
    wraps the child class's method. In the example above, changing a normal
    Screen's look-and-feel to match that of a help page involves simply
    changing the Screen's parent (a Layout). No code within the Screen itself
    needs to be changed.
  • The hierarchy is necessarily fixed. This pattern is not appropriate if you
    need to change the decorator dynamically. The Decorator pattern is more
    appropriate in that case.
  • It is hard to implement. If this pattern is not implemented by the
    programming language, it may be challenging to implement manually.
    Specifically, it requires a bit of meta-programming--navigating the
    inheritance hierarchy and dynamically looking up methods, through
    reflection.

Implementation
As mentioned above, implementing this pattern manually requires a bit of
meta-programming:

  1. You must be able to iterate over the class hierarchy. In languages
    such as Python that support multiple inheritance, I have found it
    helpful to constrain the iteration to the "leftmost", that is, the most
    primary, classes in the hierarchy, assuming that the others
    probably would be mixins.
  2. While iterating over the class hierarchy, you must be able to save a
    reference to a method of a given name. That is, given an instance "obj"
    and a method "m", the goal is to loop over the classes in obj's class
    hierarchy and find all the classes that define m, keeping a list of
    references to those methods.
  3. You must have a way for the parent class's method to call the child
    class's method. Having created the list of methods named m above, the
    parent class's method must be able to call the next m in the
    list.
  4. It may be irritating to have a child class's method signature be
    constrained by a parent class's method signature. Suppose a parent
    class, A, has two child classes, B and C. A is generic, whereas B and C are more specific. Let's apply Inverse Extension to
    the method m. B.m may wish to receive one argument whereas C.m
    wishes to receive two arguments. In normal extension, calling B.m or
    C.m with a differing number of arguments is no problem. To maintain
    this flexibility for Inverse Extension, it is important that B.m and
    C.m be able to have signatures that are different from each other
    and from A.m. To implement this, a "varargs" feature in the language is helpful. It
    often is appropriate for the parent class's method to accept an arbitrary
    number of arguments and simply pass them unmodified to the child class's
    method.
  5. Should the caller know Inverse Extension is happening? A coworker,
    David Veach, noticed that in normal extension, calling obj.m() hides
    whether extension is happening in m. It often makes sense to observe
    this constraint when applying Inverse Extension.

Sample Code
Python's dynamic nature makes implementing Inverse Extension straightforward.
In consideration of the points above, it isn't difficult to loop over an object's
class hierarchy, thanks to the __bases__ attribute. Nor is it hard to look up
a method of a given name in each class, thanks to "getattr".

When calling a parent class's method m, a function named callNext is passed as
the first argument. callNext can be invoked anywhere within the method to
transfer control to the child class. callNext is implemented using a
closure, containing references to the inheritance hierarchy. In Java, an Iterator
offers a similar mechanism. Python also supports a varargs feature, hence the
child class's method need not be constrained by the parent class's method.

In the sample code I have provided, in the interest of simplicity, I have not
tried to hide the Inverse Extension pattern from the caller. Nonetheless,
hiding the implementation of a private method that uses Inverse Extension
behind the API of a public method is a trivial matter.

Listing 1 contains the sample implementation in Python. Anthony Eden helped
me to create a similar Java implementation, InverseExtend.java, which
is shown as Listing 3 at the end of this article.

Listing 1. Sample Code in Python


def inverseExtend(boundMethod, *args, **kargs):

    """Iterate downward through a hierarchy calling a method at each step.

    boundMethod -- This is the bound method of the object you're interested in.
    args, kargs -- The arguments and keyword arguments to pass to the
        top-level method.

    You can call this method via something like this:

        inverseExtend(object.method, myArg, myOtherArg)

    When calling the method at each step, I'll call it like this: 

        Class.method(object, callNext, *args, **kargs)

    However, the lowest level class's method has no callNext parameter,
    since it has no one else to call:

        Class.method(object, *args, **kargs)

    In the method:
    
        callNext(*args, **kargs) 
        
    should be called when it is time to transfer control to the subclass.  This
    may even be in the middle of the method.  Naturally, you don't have to pass
    *args, **kargs, but a common idiom is for the parent class to just receive
    *args and **kargs and pass them on unmodified.

    """

    # Build all the necessary data structures.

    obj = boundMethod.im_self
    methodName = boundMethod.im_func.__name__

    # Figure out the classes in the class hierarchy.  "classes" will
    # contain the most senior classes first.

    Class = obj.__class__
    classes = [Class]
    while Class.__bases__:
        Class = Class.__bases__[0]
        classes.insert(0, Class)

    # Skip classes that don't define the method.  Be careful with getattr
    # since it automatically looks in parent classes.  

    last = None
    methods = []
    for Class in classes:
        if (hasattr(Class, methodName) and 
            getattr(Class, methodName) != last):
            last = getattr(Class, methodName)
            methods.insert(0, last)

    def callNext(*args, **kargs):
        """This closure is like super(), but it calls the subclass's method."""
        method = methods.pop()
        if len(methods):
            return method(obj, callNext, *args, **kargs)
        else:
            return method(obj, *args, **kargs)

    return callNext(*args, **kargs)


# Test out the code.
if __name__ == "__main__":

    from cStringIO import StringIO

    class A:
        def f(self, callNext, count):
            buf.write('<A count="%s">\n' % count)
            callNext(count + 1)
            buf.write('</A>')

    class B(A):
        # I don't have an f method, so you can skip me.
        pass

    class C(B):
        def f(self, callNext, count):
            buf.write('  <C count="%s">\n' % count)
            callNext(count + 1)
            buf.write('  </C>\n')

    class D(C):
        def f(self, count):
            buf.write('    <D count="%s" />\n' % count)

    expected = """\
<A count="0">
  <C count="1">
    <D count="2" />
  </C>
</A>"""

    buf = StringIO()
    d = D()
    inverseExtend(d.f, 0)
    assert buf.getvalue() == expected
    buf.close()

Sidebar: Implementing the Pattern Using Python's New Generators
For fans of functional programming, serious Python hackers and other
hardcore engineers who think the callNext parameter is inelegant, I
offer an alternate Python implementation of this pattern using Python's new
generators. Listing 2 uses generators for non-local flow of control. No
callNext parameter is needed. Each parent class method does a
yield *args, **kargs when it is time to call the child class's method. In fact,
this implementation flattens the recursion down to a loop, despite the fact
that the code is not tail recursive. In this way, it is similar to a
continuation. The one drawback of this implementation is it is not
possible to maintain the return value semantics of the earlier
implementation. That is, it is not possible for each child class method to return a value to its
parent class method because of the nature of generators. Nonetheless, I
offer Listing 2 as an intriguing use of generators.

Listing 2. Generator Code for Non-Local Flow of
Control


import types


def inverseExtend(boundMethod, *args, **kargs):

    """Iterate downward through a hierarchy calling a method at each step.

    boundMethod -- This is the bound method of the object you're interested in.
    args, kargs -- The arguments and keyword arguments to pass to the
        top-level method.

    You can call this method via something like this:

        inverseExtend(object.method, myArg, myOtherArg)

    When calling the method at each step, I'll call it like this: 

        Class.method(object, *args, **kargs)

    Each parent class method *must* be a generator with exactly one yield
    statement (even if the yield statement never actually gets called), but the
    lowest level class method must *not* be a generator.  In the parent class:
    
        yield args, kargs

    should be called when it is time to transfer control to the subclass.  This
    may be in the middle of the method or not at all if the parent class does
    not wish for the child class's method to get a chance to run.  

    """

    # Build all the necessary data structures.

    obj = boundMethod.im_self
    methodName = boundMethod.im_func.__name__

    # Figure out the classes in the class hierarchy.  "classes" will
    # contain the most senior classes first.

    Class = obj.__class__
    classes = [Class]
    while Class.__bases__:
        Class = Class.__bases__[0]
        classes.insert(0, Class)

    # Skip classes that don't define the method.  Be careful with getattr
    # since it automatically looks in parent classes.  

    last = None
    methods = []
    for Class in classes:
        if (hasattr(Class, methodName) and 
            getattr(Class, methodName) != last):
            last = getattr(Class, methodName)
            methods.append(last)

    # Traverse down the class hierarchy.  Watch out for StopIteration's which
    # signify that the parent does not wish to call the child class's method.
    # generatorMethods maps generators to methods which we'll need for nice
    # error messages.

    generators = []
    generatorMethods = {}
    for method in methods[:-1]:
        generator = method(obj, *args, **kargs)
        assert isinstance(generator, types.GeneratorType), \
            "%s must be a generator" % `method`
        try:
            (args, kargs) = generator.next()
        except StopIteration:
            break
        generators.insert(0, generator)
        generatorMethods[generator] = method

    # If we didn't have to break, then the lowest level class's method gets to
    # run.

    else:
        method = methods[-1]
        ret = method(obj, *args, **kargs)
        assert not isinstance(ret, types.GeneratorType), \
            "%s must not be a generator" % method

    # Traverse back up the class hierarchy.  We should get StopIteration's at
    # every step.

    for generator in generators:
        try:
            generator.next()
            raise AssertionError("%s has more than one yield statement" %
                                 `generatorMethods[generator]`)
        except StopIteration:
            pass


# Test out the code.
if __name__ == "__main__":

    from cStringIO import StringIO

    class A:
        def f(self, count):
            buf.write('<A count="%s">\n' % count)
            yield (count + 1,), {}
            buf.write('</A>')

    class B(A):
        # I don't have an f method, so you can skip me.
        pass

    class C(B):
        def f(self, count):
            buf.write('  <C count="%s">\n' % count)
            yield (count + 1,), {}
            buf.write('  </C>\n')

    class D(C):
        def f(self, count):
            buf.write('    <D count="%s" />\n' % count)

    expected = """\
<A count="0">
  <C count="1">
    <D count="2" />
  </C>
</A>"""

    buf = StringIO()
    d = D()
    inverseExtend(d.f, 0)
    assert buf.getvalue() == expected
    buf.close()

Known Uses
Perl Mason uses the Inverse Extension
pattern as a key component of its templating capabilities. A parent template
can provide a common look-and-feel for each of its child templates. Naturally,
a child template has no idea what part of the layout it belongs in, so it is
necessary for the parent template to call the child template with the call
$m->callnext when it is time to produce its output. Perl Mason was the inspiration
for the same feature in my Python Web application framework
Aquarium, as well
as for this article.
Related Patterns
The Template Method pattern is similar to the Inverse Extension pattern
when there are only two levels in the class hierarchy. Each involves a
parent class method calling a child class method. However, as mentioned
above, the Inverse Extension pattern is more appropriate when the Template
Method pattern needs to be applied recursively--when there is an
arbitrary and varying number of classes in the class hierarchy. If the
class hierarchy is deep and volatile, creating a new method name at each
step, which is required by the Template Method pattern, is not scalable.

The Decorator pattern is similar to the Inverse Extension pattern, because
the decorating class's method calls the decorated class's method. However,
the Decorator pattern is applied at runtime instead of being based on the
class hierarchy. If you need to wait until runtime in order to associate
which objects get to decorate which other objects, use the Decorator
pattern to apply the decorators dynamically. If you want the decorators to
be applied automatically based on the class hierarchy, use the Inverse
Extension pattern.

Listing 3. Sample Code in Java


Listing 3.
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;

/**
 * @author Anthony Eden, Shannon Behrens
 */

/**
 * This class has a static method, <code>inverseExtend</code>, that implements
 * the Inverse Extension design pattern.
 */
public class InverseExtend {

    private Object obj;
    private String methodName;
    private Object arg;
    private List methods;

    /** Just accept the parameters. */
    private InverseExtend(Object obj, String methodName, Object arg) {
        this.obj = obj;
        this.methodName = methodName;
        this.arg = arg;
    }
    
    /** 
     * Iterate downward through a hierarchy calling a method at each step.
     *
     * @param obj the object to pass to the method
     * @param methodName the name of the method to call
     * @param arg the argument to pass to the method
     *
     * The method should have a signature something like:
     *
     * <code>public static Object f(InverseExtend inverseExtend, 
     *           Object arg)</code>
     * 
     * (The method must be static because non-static methods are always virtual
     * and it's not possible to call an overriden virtual method.  If you wish
     * to refer to a particular instance, you must pass it using the arg.)
     *
     * Within the method:
     *
     * <code>inverseExtend.next(arg)</code>
     *
     * should be called when it is time to transfer control to the subclass.  
     * This may even be in the middle of the method.  
     */
    public static Object inverseExtend(Object obj, String methodName, 
        Object arg) {

        InverseExtend me = new InverseExtend(obj, methodName, arg);

        // Figure out the classes in the class hierarchy.  "classes" will
        // contain the most senior classes first.
        List classes = new ArrayList();
        Class c = obj.getClass();
        classes.add(c);
        while ((c = c.getSuperclass()) != null) {
            classes.add(0, c);
        }

        // Skip classes that don't define the method.  Be careful--getMethod()
        // will search parent classes for the method.
        me.methods = new ArrayList();
        Method last = null;
        Class[] signature = { me.getClass(), (new Object()).getClass() };
        for (int i = 0; i < classes.size(); i++) {
            c = (Class) classes.get(i);
            try {
                Method m = c.getMethod(me.methodName, signature);
                if (!m.equals(last)) {
                    last = m;
                    me.methods.add(0, last);
                }
            } catch (NoSuchMethodException e) {
                // Don't worry yet.  Someone else might have the method.
            }
        }

        // Now it's time to worry if me.methods is still empty.
        if (me.methods.size() == 0) {
            throw new RuntimeException(
                new NoSuchMethodException(me.methodName));
        }

        return me.next(arg);
    }

    /** 
     * Call the subclass's method.  If there is no subclass or if any other
     * exception is thrown, raise a RuntimeException.
     *
     * @param arg pass this to the subclass's method
     * @return whatever it returns
     */
    public Object next(Object arg) {
        if (methods.size() == 0) {
            throw new RuntimeException("Stack underflow.");
        }
        Method method = (Method) methods.get(methods.size() - 1);
        methods.remove(methods.size() - 1);
        Object args[] = { this, arg };
        try {
            return method.invoke(null, args);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    /* 
     * Everything below is used to show the code in action.  Please excuse the
     * sloppiness and redundancy ;)  I'm using static inner classes so that the
     * test cases are near the code being tested.
     */

    public static void main(String[] args) {
        String expected = 
            "<A count=\"0\">\n" +
            "  <C count=\"1\">\n" +
            "    <D count=\"2\" />\n" +
            "  </C>\n" +
            "</A>\n";
        Argument argument = new Argument();
        argument.buf = new StringBuffer();
        argument.count = 0;
        inverseExtend(new D(), "f", argument);
        if (argument.buf.toString().equals(expected)) {
            System.out.println("InverseExtend test passed.");
        } else {
            System.out.println("InverseExtend test failed.");
            System.out.println("Expected:\n" + expected);
            System.out.println("Got:\n" + argument.buf.toString());
        }
    }

    private static class Argument {
        public StringBuffer buf;
        public int count;
    }

    private static class A {
        public static Object f(InverseExtend inverseExtend, Object arg) {
            Argument argument = (Argument) arg;
            argument.buf.append("<A count=\"" + argument.count + "\">\n");
            argument.count++;
            inverseExtend.next(argument);
            argument.buf.append("</A>\n");
            return null;
        }
    }

    private static class B extends A {
        /* I don't have an f method, so you can skip me. */
    }

    private static class C extends B {
        public static Object f(InverseExtend inverseExtend, Object arg) {
            Argument argument = (Argument) arg;
            argument.buf.append("  <C count=\"" + argument.count + "\">\n");
            argument.count++;
            inverseExtend.next(argument);
            argument.buf.append("  </C>\n");
            return null;
        }
    }

    private static class D extends C {
        public static Object f(InverseExtend inverseExtend, Object arg) {
            Argument argument = (Argument) arg;
            argument.buf.append("    <D count=\"" + argument.count + "\" />\n");
            return null;
        }
    }
}

Acknowledgments
Thanks go to Anthony Eden, Brandon L. Golm and Kyle VanderBeek for reviewing
the article and/or source code.
Resources
Gamma, Erich, et. al. Design Patterns: Elements of Reusable Object-Oriented
Software
. Addison-Wesley, ISBN 0-201-63361-2.

Shannon Behrens is a self-professed language lawyer who works for Iron
Port Systems in San Bruno, California. His eventual goal is to implement
a Python-like systems language and then develop a practice kernel in
that language.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

inner keyword

Shannon -jj Behrens's picture

Guido van Rossum said:

Regarding BETA having the same feature: it probably inherits this from
Simula-67; ISTR that it has "inner" as a keyword to indicate that a
subclass's code should run. See e.g.
http://staff.um.edu.mt/jskl1/talk.html and search for Inner. But this
appears to be before methods were invented. (Being Scandinavian
there's no way that BETA would *not* be a descendant of Simula-67. :-)

the world turned upside down

zyxtberk's picture

Thanks for an informative and interesting take on class hierarchies. At Open List, we rely heavily on a pipeline architecture for data processing. We used to use a pretty standard, 3-layer class hierarchy, where each type of object to be fed through the pipeline was instantiated at the lowest level of the chain, selectively calling up to higher versions of methods. What we realized, however, was that in order to add new types of objects to the pipeline, we had to define bottom-level classes that would either a) defer to parent methods, b) stand in lieu of them, or c) do their own work in combination with calls to SUPER (parent) methods. The problem with this approach is that every new source required adding code at the bottom layer, even if it defered completely to higher-level classes to do the work. In other words, the traditional class heiracrhy expected us to labor in the realm of the specific even if all we needed was a call to the general. To rectify it, we developed a base class that walks down, as opposed to up the inheritance tree. Generic (higher-level) classes are called first in the pipeline. These look up to two layers down, defering to any more specific classes that exist. And, because these lower-level classes inherit from their higher level classes, we can still take advantage of a) overriding methods in parent classes, and b) calling parent methods by way of SUPER. The difference is that we can now process any new type of data without having to write stub classes for it. The order of preference is to start with parents, defer to children only when they exist, and then to let the children selectively leverage their parent's abilities. We can now add data sources in about 1/10 the time, still take advantage of using hierarchy to handle conditional pipeline processing, and we have also greatly reduced the number of bugs, volume of code, etc. I realize this is only a partial analogue to what's detailed in the article, but seeing the world turned upside down can have its advantages!

If it weren't for the "Design Pattern" wording...

Mabooka-Mabooka's picture

..., I wouldn't comment on this.

However (putting aside many other issues with this article):

- it is a language-dependent hack;
- it suggests a hard way for solving a non-existing problem.

That’s why I wouldn’t be so bold as to call it “pattern

If it weren't for the "Design Pattern" wording...

Shannon -jj Behrens's picture

> If it weren't for the "Design Pattern" wording...
>
> ..., I wouldn't comment on this.
>
> However (putting aside many other issues with this article):
>
> - it is a language-dependent hack;

If we were C programmers and this were 30 years ago, I think you could
say the same thing about OOP in general ;) (and in particular, vtables
in C). The basic idea is sound even if it is difficult to implement in
various languages.

Furthermore, to quote: "The first time, it's a hack. The second time
it's a trick. The third time, it's a well established principle."
Considering the number of places I've used and seen this pattern, I'm
definitely at the "well established principle stage" ;)

> - it suggests a hard way for solving a non-existing problem.

Well, I'm not arguing that you have this problem, but I definitely do.

> That’s why I wouldn’t be so bold as to call it “pattern

Inheritance gone horribly wrong

Anonymous's picture

The message doesn't seem to go through to you. Sorry to say that the initial design is badly broken. "Inverse Extension" "fixes" this broken design. Hence Mabooka-Mabooka's "hack" wording. It is an example of fixing a problem by adding additional complexity rather than rectifying the starting point.

Ask yourself the questions "Is an application a kind of a layout?". (Likewise for the other classes.) The answer is no. Application should thus not inherit from Layout. An application *has* pages. A page *has* a layout. Note the *has*, no *is a refinement of* there.. *has*. Hence composition should be used, not inheritance. Think of something like the following (Content being a (base)class that encapsulates the different content types (html, xml etc):

abstract class Page
{
// ..
Content getContent()
{
// ..
Layout layout = this.getLayout();
// ..
}

protected Layout getLayout()
{
// The default layout is CompleteLayout();
return CompleteLayout();
}

// ..

}

Need a page that has a layout that is different layout? Override getLayout():

final class LoginPage extends Page
{
// ..

protected Layout getLayout()
{
return SparseLayout();
}

// ..

}

No need for a "deep class hierarchy." No need for a "volatile class hierarchy": override getLayout() to achieve a certain layout for XYZPage, rather than shoving XYZPage around in the class hierarchy. No need for inverse extension.

Your class relationships are inside out, upside down. Hence your need for a bizarre, upside down, construct like having a baseclass extend its subclasses.

"Inverse Extension" maybe used quite often. That does not imply that it qualifies as a design pattern. Authors that need it should fix their designs.

has-a rather than is-a

Shannon -jj Behrens's picture

[Hmm, I replied earlier, but my reply got dropped. I wonder if there is a limit to how deep threads can go.]

Ok, let's be nice to one another :) It'll make this conversation more enjoyable for both of us, and it'll stimulate open minds.

I do agree that it is a nice idea for a screen to have-a layout instead of be-a layout. I've implemented it this way before. It has the benefit of additional flexibility--you can choose the layout dynamically. However, I have a couple questions:

If one layout is a specialization of another, does it extend the other layout (is-a) or delegate (has-a) to another layout? It makes a lot of sense for one layout to say, "I'm just like the Sparse layout, however, in the center, I also output the tabs." In the example shown below (see HEADS UP below), I have multiple layers. Does each layer have-a or is-a?

Also, I find it very useful for a screen to be able to override methods defined in any of the parent class layouts. I do this all the time. If I switch to has-a, and perhaps I have a chain of has-a (a screen has a layout named Cluster which has a layout named Complete which has a layout named Sparse...), how do I override methods? Specifically, I want the screen to override Sparse.doHelpBar, CssAndJavaScript.getTitle, Bare.getHeaders, etc. Explicitly writing delegation methods is a no-go because there are too many of them. Some languages (such as C#, ObjC, and Python) support "implicit" delegation, but not all languages do. What do you recommend? How can a screen override arbitrary methods as arbitrary layers?

Best Regards,
-jj

Re: has-a rather than is-a

Anonymous's picture

> Ok, let's be nice to one another :) It'll make this conversation more enjoyable for both of us, and it'll stimulate open minds.

No offense intended. Don't take this personally, we are discussing the merits of a technical solution, not you.

>
> I do agree that it is a nice idea for a screen to have-a layout instead of be-a layout. I've implemented it this way before. It has the benefit of additional flexibility--you can choose the layout dynamically. However, I have a couple questions:
>
> If one layout is a specialization of another, does it extend the other layout (is-a) or delegate (has-a) to another layout? It makes a lot of sense for one layout to say, "I'm just like the Sparse layout, however, in the center, I also output the tabs." In the example shown

Sounds like subclassing.

> below (see HEADS UP below), I have multiple layers. Does each layer have-a or is-a?

Why not have a LayerLayout class, an instance of which accepts the stuff (Components, see below) construction time and puts the stuff into layers?

>
> Also, I find it very useful for a screen to be able to override methods defined in any of the parent class layouts. I do this all the
> time. If I switch to has-a, and perhaps I have a chain of has-a (a screen has a layout named Cluster which has a layout named Complete which has a layout named Sparse...), how do I override methods? Specifically, I want the screen to override Sparse.doHelpBar, CssAndJavaScript.getTitle, Bare.getHeaders, etc. Explicitly writing delegation methods is a no-go because there are too many of them. Some languages (such as C#, ObjC, and Python) support "implicit" delegation, but not all languages do. What do you recommend? How can a screen override arbitrary methods as arbitrary layers?

Unfortunately, you phrased your questions in terms of changes you make to your code, rather than in terms of what you want to achieve, so I'll have to do a little guessing as to what functionality you need.

BTW, this begs the question what you do when you have two screens, say a news page and a faq page, in your application that have identical layout. Do you introduce two classes NewsScreen and FAQScreen, the first overriding CssAndJavaScript.getTitle to return "News" and the second overriding CssAndJavaScript.getTitle to return "FAQ"?

Do you need to override all these methods to finetune the layout of a page? Seems so. The getTitle thing to me does sound like an atribute of a screen: 'Screen = new SomeScreen("This is the page's title");'.

The Sparse.doHelpBar does sound like an boolean attribute to the Sparse class that tells in instance of the class to display the HelpBar or not. Might also be separate layout subclasses, one taking a HelpBar, the other not. Can't tell at this point.

Why do you want to override Bare.getHeaders and other methods? Finetuning? What do they do?

You seem to have a need to recursively wrap parts that make up a page into html; have tables inserted into table cells, into layers etc, etc. Am I right? This sounds like the Composite Design Pattern. Why not introduce a class that represents the parts that make up a screen/page: abstract class Component, say, having two, concrete subclasses, SimpleComponent (the actual content) and CompositeComponent. The latter *having* Components. A Component does have a Layout. Problem solved. No need for 'many delegation methods': simple recursion does the trick.

Has-a rather than is-a indeed, I think.

Correction..

Anonymous's picture

..should read 'return new CompleteLayout();' and 'return new SparseLayout();'

Interesting and neat code.

Anonymous's picture

Interesting and neat code. It took me a little bit to get into the article. I only gave it a shot because I was curious how you did this with Python, but I wasn't sure at first why I should WANT to do this. The topic is a little abstract. You should lead with a practical example for motivation. The HTML example was a little fuzzy. I still had trouble disambiguating this from the Template pattern.

Interesting and neat code.

Shannon -jj Behrens's picture

> I wasn't sure at first why I should WANT to do this.

Yes, I definitely failed to explain this. See below where I try to do better.

Thanks!

see below

Shannon -jj Behrens's picture

See the "HEADS UP" below. :)

What class of problems does Inverse Extension solve?

Anonymous's picture

It is unclear to me what class of problems are solved by the Inverse Extension pattern. I fully appreciate the fact that Inverse Extension is like the Template Pattern. In the Template Pattern one has subsequently refining methods that have different names and, in general, varying signatures as one goes down the inheritance hierarchy.

The article seems to imply that the Inverse Extension pattern apllies to a class hierarchy in which the refining methods, so te speak, all have the same name and signature. I do not understand what class of problems would have this look.

What I miss in the article is the following

- An example of a class hierarchy implemented without Inverse Extension.
- An exposition of the problems it has.
- An refactoring of the hierarchy with Inverse Extension.
- An explanation of how Inverse Extension fixes the problems.

Could you give such an example?

What class of problems does Inverse Extension solve?

Shannon -jj Behrens's picture

Hmm, sorry, I guess the paragraph that starts with "In traditional HTML, that is," didn't come out as clearly as I had hoped. As I mentioned, I first saw this pattern in Mason (the Perl templating engine) where it is a key component. I ported it to Python in my project, Aquarium, and it is a key part of Aquarium.

Imagine you have a common look and feel for all the Web apps at a company. Each application at the company subclasses the Layout class from the common look and feel and adds its own stuff. Then each section within that app might add its own stuff. Then each individual page adds its own stuff. You have four layers in the inheritance hierarchy. Suppose at each layer, you're using a table to wrap the lower layer's content. It's like an onion. Now, you could create a new method at each layer: doCommonLookAndFeel, doApp, doSection, doScreen, where each method in the parent class calls the child class's method. However, what happens if you suddenly need to shove in a layer, for instance, you need to introduce a layer that is a parent of the sections. You end up renaming methods, etc., and it all gets really sticky. Consider what happens if you want some pages to subclass layers that are higher up. Suddenly, you have a page defining the doSection method in order to call its own doScreen method. Ugly. With inverseExtend, you can add as many layers to the onion as you want, and you can shove in new layers in the middle of the onion if you want. It all works out. In my case, each layer has a __call__ method (which is Pythonic), and you can rearrange things all over the place just by changing the parent class.

By the way, this isn't some academic exercise. I'm actually the "UI Platform Engineer" at my company, and I maintain the shared look and feel as well as the framework across all the Web UI's at my company. The whole onion idea is something I really deal with.

Well, I hope that helps.

Thanks for your reply. It

Anonymous's picture

Thanks for your reply. It gives me some idea of what you are getting at. However, on the basis of your reply I wouldn't be able to write down an example of the class of problems that Inverse Extension solves. I'm still puzzled as witnessed by the following questions.

Does the Layout class encapsulate the common look and feel?

Do I understand you correctly that the hierarchy is as follows, :- indacting subcalssing, Layout :- Application :- Section :- Page?

Are Screens and Pages synonyms? If not, where does Screen fit into this model?

What is a Section ? I.e. what does it represent?

How do these classes relate? What is the behaviour? I.e. what methods are involved, what do they do and how do they call eachother? What is its public interface? What is its context? How does a client use an Layout instance? How is a page ulimatally displayed?

What do the doCommonLookAndFeel, doApp, doSection, doScreen do? Return html?

How many sections does an application have?

How many pages does a section have?

Why would it be needed to suddenly shove in a layer?

Why would one need some pages to subclass layers that are higher up?

Where does navigation fit into this model?

HEADS UP: big, concrete example of my usage of this pattern

Shannon -jj Behrens's picture

Ok, I replied, but it was too big, and the system won't let me post it. Hence, I've posted it on my blog. Please see comment 2 here. I lay out the concrete class hierarchy of how I've actually used this pattern. I hope everyone who still has questions reads this. Thanks!

Thanks for your reply. It

Shannon -jj Behrens's picture

> Thanks for your reply.

Thanks for your patience as I try to explain what I'm talking about.

> It gives me some idea of what you are getting at. However, on the basis of your reply I wouldn't be able to write down an example of the class of problems that Inverse Extension solves. I'm still puzzled as witnessed by the following questions.
>
> Does the Layout class encapsulate the common look and feel?

Yes, a Layout does encapsulate the common look and feel. By "common look and
feel", I mean that all of our apps look and behave pretty much the same. In my
code, I've used the term Layout pretty much as an interface.

> Do I understand you correctly that the hierarchy is as follows, :- indacting subcalssing, Layout :- Application :- Section :- Page?

Well let me step away from the hypothetical and talk about an actual class
hierarchy (top to bottom). Screens are the main content. Layouts are pretty
much any parent of a screen. I'll start with the layouts, and at the bottom
are the screens.

In Aquarium, which is my open source framework:

Bare -- This class understands content types, HTTP headers (including turning
on or off HTTP caching), etc.

Some subclasses in your app might include a screen that generates an XML
report or a screen that generates a dynamic image containing a graph.

HTML -- This is mostly for people who want to type the HEAD, HTML, BODY,
etc. tags themselves. It knows that it has content type text/html.

CssAndJavaScript -- This class knows about the basic layout of HTML. It has
methods for the title, CSS, JavaScript, etc. that can be overriden. It uses a
bunch of defaults that are defined in a properties file. It's here so that I
don't have to duplicate boilerplate.

Simple screens supclass from CssAndJavaScript when I'm trying to get a
message out without caring about look and feel. Consider the screens that
Apache uses for 404, etc.

Now, from within my common look and feel code, which is used by multiple apps
at my company:

Sparse -- This is the layout used for content where you don't want to show the
tabs.

The login screen, the help screens, the app-specific 404 screen, etc. all
extend from here.

Complete -- This layout contains the complete navigation, including tabs.

Most of the screens in the application extend from this. Most of apps
that use my common look and feel use this as their main layout.

Cluster -- This layout knows about extra navigation needed for clustering
(which we've unfortunately used as a misnomer for centralized management).

Only some of my apps need to worry about clustering. In those apps,
screens use this layout instead of the Complete layout.

Now, for actual screens:

various screens -- At the lowest level are the various screens. The basic idea
is that you start a screen by picking how much you want your parent class to do
for you automatically. You can always override methods defined in your parent
classes that it sets up for you to override, such as getTitle. You usually try
to pick the most specialized layout (e.g. Cluster or Complete depending on
whether or not your app supports clustering), however, certain screens need to
assert more control. For instance, it makes no sense for an XML report to
subclass the Cluster layout.

> Are Screens and Pages synonyms?

Yes, sorry about that. Screen is the actual "interface". I used page as a
synonym although there's nothing actually called page in my code.

> If not, where does Screen fit into this model?
>
> What is a Section ? I.e. what does it represent?

Now that I've laid out a concrete class hierarchy, let's just drop the
hypothetical stuff.

> How do these classes relate?

All those classes above extend from one another.

> What is the behaviour?

Based on inverse extend, each higher level class's __call__ method gets called.
Whenever it's ready, it passes off control to the child class's __call__
method. Hence, the parent classes "wrap" the child classes. This really makes
sense in HTML.

(It's also useful in a couple other contexts. Imagine a parent class that
checks access restrictions beforing allowing access to the child class. Note,
the parent class can prevent the child class's __call__ method from running at
all by simply refusing to call callNext! I have found this to be a useful
technique.)

> I.e. what methods are involved,

Everyone has a __call__ method (which is a Pythonism).

> what do they do and how do they call eachother?

Layouts look like (pseudocode):

class CssAndJavaScript(HTML):

def __call__(self, callNext):
# do stuff, like include the navigation
callNext(*args, **kargs) # Let the subclass do its stuff.
# do more stuff, like include the copyright footer

Screens look like (pseudocode):

class MyScreen(CssAndJavaScript):

def __call__(self):
# Do my main content

> What is its public interface?

In Aquarium, it's easy. Everyone has a __call__ method.

> What is its context?

I have the notion of a context, but I'm pretty sure it doesn't match what
you're asking about. The important thing is that the child class's __call__
method is called by the parent class's __call__ method in order to do its work
at just the right time.

> How does a client use an Layout instance? How is a page ulimatally displayed?

Well, Aquarium uses Cheetah for this. Here's a simple screen:

| #extend aquarium.layout.Cluster
| #implements __call__
|
| Hi! I'm a simple looking screen, but my parent automatically takes care of
| outputing all the common look and feel!

When you load a URL in the browser, Aquarium maps the URL to a particular
screen. Then it calls inverseExtend on that screen's __call__ method. Hence,
the parent class gets to do work, and then it passes control down the
inheritance hierarchy, until finally the screen gets to output its main
content.

> What do the doCommonLookAndFeel, doApp, doSection, doScreen do?

Dropping the hypothetical names.

> Return html?

Yes, actually they do. Hence, the parent class can take that HTML and do
whatever it wants with it.

> How many sections does an application have?

You saw how many layouts I have. In practice, each application might add maybe
one additional layout on an as needed basis. Each app might have up to about
100 screens.

> How many pages does a section have?

Dropping the hypothetical names.

> Why would it be needed to suddenly shove in a layer?

I've had to do this twice so far.

1. My buddy told me that the CssAndJavaScript class was stupid. He wanted to
output the HTML boilerplate manually. Hence, I slipped in the HTML class
between Bare and CssAndJavaScript to stop his complaining.

2. When we added centralized management, I added a subclass of Complete called
Cluster. I had to change all the screens to subclass Cluster instead of
Complete. Note, because I was using inverse extend instead of the template
design pattern, I only had to change one line, the name of the superclass,
instead of having to change method names as well (you need multiple method
names for multiple layers of the template design pattern). It's nice because
they all have the one method, __call__, and the only thing that matters is your
parent class.

> Why would one need some pages to subclass layers that are higher up?

Basically, a screen picks the parent class that provides the behavior it wants.
An XML report doesn't want any HTML at all. The login page doesn't need
navigational tabs. The generic 404 page that comes with Aquarium subclasses
CssAndJavaScript because it doesn't know anything about your app-specific
layouts.

> Where does navigation fit into this model?

Layout classes do something like a server side include at the appropriate place
to incorporate the navigation. A layout class could have the navigation
inline, but I don't do that because I may need to reuse the same navigation
class from two different layouts. Hence, a layout may "include" a bunch of
navigation classes and call its screen subclass (i.e. callNext), and it "lays
out" how all these things relate to each other in the HTML.

In summary, I use inverse extend to apply the "don't repeat yourself" rule to
HTML. Just because it isn't real code doesn't mean it's a good idea to
duplicate it all over the place :-D

Heard about Common Lisp?

Jon Haugsand's picture

What actually makes this different from Common Lisp's CLOS system? There you have :before, :after and :around methods. These are used to force som actions onto methods defined in subclasses.

If I have misunderstood your article, I apoligize. If not, I think this is worth mentioning, to put it mildly.

Regards.

Heard about Common Lisp?

Shannon -jj Behrens's picture

By the way, the fact that this design pattern is actually a feature in Beta as well as possibly Common Lisp does not detract from its usefulness as a design pattern. In fact, the opposite is true. It's a way of organizing your code that is helpful for solving a certain class of problems. Sometimes your language supports this technique directly, sometimes it doesn't. Either way, you must use the technique delibrately. By writing a design pattern spec as I have done, we now have the freedom to call this technique by name. Newbies can hear the name and go look it up if they need to. Furthermore, the name transcends programming languages. I.e. a Java programmer and a Beta programmer can now more easily understand each other when suggesting this technique as a solution to a problem. I got the feeling when I read your post that you were saying, "Shoot, we've been doing that for years in Common Lisp!" Exactly ;) That's the point of a design pattern--to document and name techniques that experienced programmers have been doing for years. In fact, if no one had been using this trick, it would not qualify as a design pattern!

Heard about Common Lisp?

Shannon -jj Behrens's picture

Yes, I know about Common Lisp ;)

Do :before, :after, and :around allow you to march all the way down the inheritance hierarchy (i.e. for more than two levels) keeping the same method name? It sounds to me that maybe what you're suggesting is that each child class method can use :around to pass control to the parent class method, and have the parent class method call back to the child class method at the appropriate time. Is that right? Can you post an example using one of my test cases from above?

More on Common Lisp

Jon Haugsand's picture

An example/introduction to this feature can be found
A Brief Guide to CLOS. And yes, you can combine this feature all the way, but no, the behaviour is probably not what you have in mind.

Actually, if you have a class hierarchy, the most specific :before and :after methods are run ourtermost. Here is a simple example:

(defclass food () ())
[defclass food nil nil [0 0 0] nil nil nil [] nil nil nil ...]
(defmethod cook :BEFORE ((f food)) (print "A food is about to be cooked."))

(defmethod cook :AFTER ((f food)) (print "A food has been cooked."))

(defclass pie (food)
((filling :accessor pie-filling
:initarg :filling
:initform 'apple)))

(defmethod cook ((p pie))
(print "Cooking a pie.")
(oset p filling (list 'cooked (pie-filling p))))

(defmethod cook :BEFORE ((p pie)) (print "A pie is about to be cooked."))

(defmethod cook :AFTER ((p pie)) (print "A pie has been cooked."))

(defclass lutefisk-pie (pie) ())
(defmethod cook :BEFORE ((p lutefisk-pie)) (print "Lutefisk in oven"))
(defmethod cook :AFTER ((p lutefisk-pie)) (print "Lutefisk ready for you."))

(setq lute-1 (make-instance 'lutefisk-pie :filling 'stockfish))

(cook lute-1)

"Lutefisk in oven"

"A pie is about to be cooked."

"A food is about to be cooked."

"Cooking a pie."

"A food has been cooked."

"A pie has been cooked."

"Lutefisk ready for you."
(cooked stockfish)
--------------
So, I was a little bit to quick. This is not what you want with your decoration example. However, you can use the meta-object protocol (MOP) to define other kind of behaviour.

Thanks for the article anyway.

Recards, Jon

More on Common Lisp

Jon Haugsand's picture

As far as I understand, you can actually quite easy change this behaviour through define-method-combination calls. Common lisp is extremely powerful for these kind of things.

And, if I forgot, yes, you are right. My comment was not an attempt to disregard your main ideas. (However, one should expect some reference to languages that actually do implement this.)

Merry Christmas. :-)

-- Jon

thanks!

Shannon -jj Behrens's picture

Wow! Thanks for your helpful comments! Although, as you said, you were too quick the first time, I'm confident that you could do this with the CLOS.

This only serves to humorously prove my theory that no matter what you write about, someone somewhere has already done it in Lisp or it already exists as a feature in Common Lisp. I've joked that you could make a living writing about features that already in exist in Common Lisp, reimplementing them in languages like Python and Ruby. ;)

At least this time, it wasn't *an obvious ripoff* in that Common Lisp doesn't already have a special form for it (that we know of) ;)

The Beta programming language

Shannon -jj Behrens's picture

Doug Landauer was helpful enough to point out that this design pattern was actually implemented originally as a feature of the Beta programming language:

http://faqs.org.ru/en/programming/beta-language-faq-2.htm
http://sern.ucalgary.ca/courses/SENG/609.03/W98/jyzhu2/research.html

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState