Continuous Documentation with Sphinx + AppEngine

Bitbucket is a great choice when you need to host a private code repository and have a small number of collaborators. Sphinx is excellent for developers who want to manage documentation, it is used by many Python projects including the Plone community and has been adopted by some in the Ruby and Java communities.

There are workflows that support public documentation of projects using these tools, but what if you want to also have a private documentation site?

I decided to see what was possible if my needs were as follows:

• Private documentation
• Continuous documentation
• Needs to work with bitbucket

For public facing documentation I recommend Readthedocs.org. However, because I wanted to keep a private repository and private documentation, I decided to host my documentation on Google App Engine. There's a way to do something similar on Heroku, but I like what I was able to "stitch" together with App Engine.

Assumptions

I'll start with the assumption that you know enough about git and bitbucket to create a git repository at bitbucket and do checkouts, commits and pushes to the repository. If you're unfamiliar with this then you'll need to go and learn that first, then come back here.

I also assume that you have the python app  engine SDK installed and available in your path (meaning that it you can run the 'appcfg.py' command as a standalone command on the terminal.

The Pieces

Step 1 - Setup Sphinx

In your git checkout create your Sphinx documentation using the following command:

sphinx-quickstart

Accept the defaults for the first 3 options (Root path, Seperate source and build directories, Name prefix) presented:

Step 2 - Configure the repository for app engine deployment

Add the following app.yaml file to your repository (change the app engine id to match your own).

Note: If you don't want to force logins then change the lines that say 'login: admin' to 'login: optional'

You can now try a deployment by running the following command:

appcfg.py update . --oauth2 --oauth2_refresh_token=`cat .token`

If all went well, add that command to the .git/hooks/post-commit file in your repository ( I use post-commit because it is a client side hook, bitbucket isn't going to work for server side hooks in this use case)

appcfg.py update . --oauth2 --oauth2_refresh_token=`cat .token`

This will ensure that after each commit the documentation will be pushed to app engine, just before you do your next commit ensure that the command is executable:

chmod +x .git/hooks/post-commit

Step 3 - Enjoy

After your next commit you should be able to visit your shiny new app engine powered documentation site located at <your-app-id>.appspot.com.

Invite Collaborators

Now that you know that it's working, go to your app engine dashboard (http://appengine.google.com) and visit the app that you're using for documentation. Then in the permissions section invite new collaborators using their gmail or google apps email address.

They will receive a link to an authorization page that looks something like this:

Once they accept they will be able to view the documentation.

Possible Improvements

I'd like to make this into a script so that it is even faster to get going. Using the admin permissions and forcing is a bit of a work around. There are cases when this is not ideal, for example if the documentation needs to be shared with less trusted users.

Progress on the Plone Theming Guide

TL;DR; I'm working on a document that helps designers get started with Plone theming. It fits into the idea of making Plone more approachable. At the bottom of this post is a checklist of things needed to fuel Plone adoption.

Warning! this post will use terms like dogfooding, flexslider and Plone. Don't panic, it is mostly harmless and, apart from the current sentence, is written in a manner accessible enough for the general public.

Plone's Diazo system is, to date, the most elegant approach to theming among content management systems. In an effort to spread the Plone and Diazo love I started writing a document on Plone theming.

This really wasn't supposed to happen. It all started because I wanted to "rub together" a few quick pointers to help to get a designer up to speed with Plone theming. I assumed that there was enough Diazo theming documentation on the web that I could simply point at the correct details and they'd be on their way. Out of that exercise, this happened.

It's actually a side project, being done in my nearly non-existent spare time. After spending about two and a half weeks, the "few quick pointers" now span more than 40 pages. The document fills many of the gaps that I discovered. I'm not finished, but it's definitely taking shape and at least it provides a more structured approach to getting started.

Sliders: The Best Part

The theming document is at the stage where a motivated reader should be able to create a theme for Plone. I'm especially pleased with the section that covers rules for custom sliders and Plone carousels.

It guides the reader through the rules needed to transform the output of the common Plone add-ons collective.carousel and Products.Carousel into markup compatible with Flexslider. The principles are applicable to other sliders.

More to do

Some gaps still exist, for example there are not yet specific details on how to package a finished theme. Also I'd like to explore in more detail the best practices for bringing in Plone's dynamic functionality (JS and CSS) without "clobbering" the theme's JS and CSS.

The Vision

Plone is an extremely user friendly content management system yet it can be very overwhelming to the uninitiated. The big picture is about making Plone more approachable to the self learner. This ties in nicely with ideas from Mikko Ohtamaa, who captures his thoughts on how to stimulate Plone adoption in the diagram below.

As I tweeted recently:

... I see a core doc plus a http://patterntap.com type facility (built with Plone if we're dogfooding*) and low barrier to contribution

* Look up dogfooding (I was once shot down in class for suggesting that "dogging" was a verb :) I hope some of my former English and Communication teachers are reading this).

When this core document is completed I'd like to see a facility that makes it easy for persons to contribute snippets/recipes. Good examples are useful when you're just starting out.

My other recent side project focuses on getting started with Plone really quickly and is inline with this vision of making Plone approachable.

The Checklist

If the Plone community can be excellent in these areas, Plone adoption will take off.

• Improved theming documentation and ready made themes
• Make getting started with development easier (I'd like to hear more about Plone IDE)
• Product Bundles (distributions of Plone that address different markets, ftw.inflator is looking good for this.)
• An easy way to deploy finished code into production 

Plone Visual Documentation Experiment

Adding Screenshots to Documentation

I've believed for a while that the Plone documentation in general could benefit from some useful screenshots. To test my theory I headed over to developer.plone.org for a short experiment to see what screenshots might do for a document.

As a proof of concept I wanted to see the extent to which adding a few strategic screenshots to a tutorial might improve the experience of learning Plone. I decided to use a tutorial that I thought would do well with screenshots, check out the screenshot version at creating forms with PloneFormGen.

Before screenshots

Here is the before picture of the entire document.

After screenshots

Here's a screenshot of a section where I added a supporting screenshot:

Methodology

The general approach was to follow the tutorial, identify where to put supporting screenshots, create the screenshots then add them.

I decided to read through the documentation and note where a screenshot or diagram would be appropriate with an SSS or DDD. Then I would replace all the SSSs with appropriate screenshots and DDDs will be replaced with diagrams.

In the first step my file has little SSS notes.

A replaced SSS looks like this:

Outcome and Observations

I ended up only using screenshots this time around (no DDD diagrams) but I'll definitely have scenarios where diagrams are more appropriate in the future.  I used animated gifs instead of flash or something else because they will be viewable when creating printed versions of the documentation.  I think the simple animations help to illuminate some of the explanations in the documentation.

Future and Possible Improvements

I can see this approach being very helpful at documentation sprints and it would give a newbie experience with specific Plone features while they create screenshots as well as gaining experience with revision management procedures.

It would be good to review the screenshots and see how they can be optimized so they best support the documentation. I'd love to look at options for streamlining the process of creating the gifs as it was quite cumbersome.