Grok Configuration Phase and its Extension

~~ DRAFT: Updated 20th Jan 13:53 GMT ~~

Prerequisites:
Martian PyPi Documentation
Understand 'grokking' and 'grokkers'.
setuptools
zc.buildout

Audience:
Developers

To understand how Grok is organised and how to extend Grok, we must first understand how Grok locates, registers and configures components. At startup, Grok discovers utilities and adapters so they can be registered the Zope Component Architecture for later retrieval by interface. The registration is done by Martian, and the process of scanning packages and grokking them is known as configuration.

A Martian Refresher
A Grok project consists of a number of components (predominantly classes) that have different roles, such as views, containers and models, characterised by an interface, that are grokked during the configuration stage. Grokking is made possible by the definition of three things:

• A child class inheriting from martian.*Grokker with an execute method
  
• Some baseclass from which Grokkable components will inherit
• A directive class

Let's take a look at a quick example all defined in a single test module:

class Person(object): # A baseclass for grokkable classes
  pass

class name(martian.Directive):
  scope = CLASS

class PersonGrokker(martian.ClassGrokker):
  martian.component(Person)
  martian.directive(name)

  def execute(self, class_, name, **kw):
      register_people(class_, name)

As you can see, the name and PersonGrokker classes both inherit from Martian classes, and this allows Martian to register our PersonGrokker using something known as a meta-grokker. A meta-grokker is responsible for registering grokkers into Martian's own registry, which is constructed as modules are scanned during the application's configuration stage. A meta-grokker is conceptually the same as regular grokking, except that it extends Martian. Naturally, before classes can be registered, Martian must know about PersonGrokker. So, Martian must scan modules that contain Grokker definitions, conventionally named meta.py, before classes that inherit from Person.

Martian can then go onto scan subclasses of Person:

class Santa(Person):
  name('Santa Claus')

If any of this is confusing, then I suggest you read the Martian documentation and write your own grokkers. Otherwise, the important thing to remember is that we must first register a grokker before Martian scans our class.

The ZCML Configuration Chain
How does Martian discover the various components and directives during configuration, and how does Grok ensure that meta-grokking occurs before regular grokking? Both are achieved using a extensions to Zope's XML based configuration process; ZCML. ZCML is based on extensible directives that, in our examples, are completely separate from Martian directives.

When the configuration stage begins (during startup) for a grokproject created package (which we'll call myproject), it first looks at the

parts/app/site.zcml

file and executes the configuration there. This file contains the ZCML directive:

<include package="myproject" />

This instructs the Zope configuration machinery to visit the specified package and include/process the directives found within the configure.zcml file. (The include directive can also take an additional file parameter that overrides which file it includes from the specified package.) Our configure.zcml file looks like:

<configure xmlns="http://namespaces.zope.org/zope"
  xmlns:grok="http://namespaces.zope.org/grok">
<include package="grok" />
<includeDependencies package="." />
<grok:grok package="." />
</configure>

These directives are processed sequentially - so let's look at them in turn. First, it includes the Grok package's configure.zcml. This is quite a large configuration file which we'll mostly ignore; the important directive is:

<include package="grok" file="meta.zcml" />

We now process meta.zcml, which contains the following three relevant directives:
<br /> <include package="z3c.autoinclude" file="meta.zcml" /><br /> <include package="grokcore.component" file="meta.zcml" /><br /> <grok:grok package=".meta" /><br />
The z3c.autoinclude and grokcore.component lines enable two new ZCML configuration directives to be used from here:

• grok
• includeDependencies (and includePlugins)

The 'grok' ZCML Directive
The 'grok' directive allows us to specify packages to grok for configurable components. If you use the following:

<grok:grok package=".meta" />

Grok will, using the Martian system described above, scan the target package or module, (meta.py in the example given) registering anything that subclasses a known grokkable baseclass. After the grokcore.component configuration has registered this new 'grok' directive, it instructs itself to meta-grok its own meta.py that contains various grokkers for all the basic Zope building blocks - such as grok.Adapter, grok.GlobalUtility. So, from here files grokked containing subclasses of any of these will be registered in the Zope Component Architecture.

The 'includeDependencies' ZCML Directive
The includeDependencies directive, used as follows:

<includeDependencies package="." />

will iterate (in order) over the given package's setup.py for entries in the "install_requires" section and include any meta.zcml and configure.zcml files automatically (in that order) from those. Since meta.zcml will conventionally contain the directive to scan meta.py, our grokkers will be discovered before configure.zcml instructs Martian to grok any configured components.

---

Thus - returning to the meta.zcml in the Grok package - once Grok's meta.py is scanned, the system will know how to go on and grok classes that inherit from grok.Site, grok.Application, grok.REST etc, and all the other central Grok baseclasses that we use to construct our applications. Any packages that are scanned from now onwards can register these types of components.

Moving back to our package's configure.zcml, the next directive is:

<includeDependencies package="." />

So, our setup.py's "install_requires" is scanned for packages. We have no new entries in here at the moment, but this mechanism will be later used to include custom grokkers as independent and separate packages.

Finally, the last directive is:

<grok:grok package="." />

This gives the order to scan through the current package (all .py files), grokking anything it knows how to. So, this is how our main application is registered (it inherits from grok.Application), how all our views are registered (grok.View) and indeed all other grok.* subclasses.

The Truth about Configuration
Until now, you'll probably be under the impression that the Grok machinery is registering components as it goes, and that when a global utility is found, for example, it immediately adds it to the Zope component registry. This is only partly true: Martian is indeed registering grokkers, but when these grokkers grok classes they are not immediately placed into the component registry.

Instead a function/position args/keyword args set, known as an action, is passed to the configuration machinery to be invoked later. Along with this set a 'discriminator' is passed, which if not None, will raise a configuration conflict exception if two actions contain the same discriminator. This allows Grok/Zope to enforce configuration polices, such as "we cannot allow two or more adapters with the same name, adapting the same context and providing the same interface".

Additionally, an order parameter can be specified optionally which dictates the order in which actions are called. Once configuration is completed, all the actions are processed in the order they were created unless order values dictate otherwise.

Extending Grok with Custom Grokkers
We now know everything to equip our Grok project with our own grokkers as a separate package, so that custom components can be created and registered at startup. For the purposes of example, we will define some class grokkers that register according to some 'flavour'.

The first step is to create a package somewhere outside of our grokproject with a setup.py. Let's assume it's called customgrokkers and the source is in src/customgrokkers/.
In the src/customgrokkers/ directory of this package we'll need a components.py, a meta.py, directives.py and meta.zcml.

components.py will contain the baseclass that our custom components will inherit from, meta.py will contain the ClassGrokker, directives.py will contain a single directive class that we'd like our custom components to contain. The meta.zcml file will contain a directive to grok our package:

So, components.py:

import grok

class CustomComponent(object):
    grok.baseclass()

directives.py

import martian

class flavour(martian.Directive):
    scope = martian.CLASS
    store = martian.ONCE
    default = 'banana'

__init__.py:

registry = {}

meta.py:

import martian
from customgrokkers.component import CustomComponent
from customgrokkers.directive import flavour

def register_custom_class(class_, flavour):
    customgrokkers.registry[flavour] = class_

class CustomComponentGrokker(martian.ClassGrokker):
    martian.component(CustomComponent)
    martian.directive(flavour)

    def execute(self, class_, config, flavour, **kwds):
        config.action( # defer actual registration
            discriminator = None,
            callable = register_custom_class,
            args = (class_, flavour),
            order = 5 
        )   
        return True

meta.zcml

<configure grok="http://namespaces.zope.org/grok">

<grok:grok package=".meta" />
</configure>

We can now add our customgrokkers package to our myproject's setup.py file and rerun buildout. (Of course, you'd have to install customgrokkers into your python path before doing so using setuptool's develop command).

Packages that actually define compatible components (i.e. inherit from customgrokkers.CustomComponent in this case) should have a configure.zcml that looks something like this in their package directory:

<configure grok="http://namespaces.zope.org/grok">
<include package="customgrokkers" />

<grok:grok package="." />
</configure>

Define your subclasses of customgrokkers.CustomComponent in any .py file in the package and they will be registered. Again, this package needs a mention in the setup.py too. Alternatively such python scripts and ZCML could live safely in your customgrokkers package thanks to precedence of meta.zcml over configure.zcml in z3c.autoinclude.

Conclusion
Once the underlying concepts described have been understood, you can understand how Grok fits together internally (making reading and naviagting the source code easier) and you can build flexible and decoupled systems with declarative python directives for your own extensions.

Further Reading

• z3c.autoinclude - Explanation of the includeDependencies ZCML directive


• Grok SVN Repository - Contains the ZCML files we've talked about and Grok specific grokkers.


• The grokcore.component SVN Repository - Contains the definition of the grok directive and grokkers for adapter/utilities/multiadapters etc...