Continuous Documentation with Sphinx + AppEngine

Bitbucket is a great choice when you need to host a private code repository and have a small number of collaborators. Sphinx is excellent for developers who want to manage documentation, it is used by many Python projects including the Plone community and has been adopted by some in the Ruby and Java communities.

There are workflows that support public documentation of projects using these tools, but what if you want to also have a private documentation site?

I decided to see what was possible if my needs were as follows:

• Private documentation
• Continuous documentation
• Needs to work with bitbucket

For public facing documentation I recommend Readthedocs.org. However, because I wanted to keep a private repository and private documentation, I decided to host my documentation on Google App Engine. There's a way to do something similar on Heroku, but I like what I was able to "stitch" together with App Engine.

Assumptions

I'll start with the assumption that you know enough about git and bitbucket to create a git repository at bitbucket and do checkouts, commits and pushes to the repository. If you're unfamiliar with this then you'll need to go and learn that first, then come back here.

I also assume that you have the python app  engine SDK installed and available in your path (meaning that it you can run the 'appcfg.py' command as a standalone command on the terminal.

The Pieces

Step 1 - Setup Sphinx

In your git checkout create your Sphinx documentation using the following command:

sphinx-quickstart

Accept the defaults for the first 3 options (Root path, Seperate source and build directories, Name prefix) presented:

Step 2 - Configure the repository for app engine deployment

Add the following app.yaml file to your repository (change the app engine id to match your own).

Note: If you don't want to force logins then change the lines that say 'login: admin' to 'login: optional'

You can now try a deployment by running the following command:

appcfg.py update . --oauth2 --oauth2_refresh_token=`cat .token`

If all went well, add that command to the .git/hooks/post-commit file in your repository ( I use post-commit because it is a client side hook, bitbucket isn't going to work for server side hooks in this use case)

appcfg.py update . --oauth2 --oauth2_refresh_token=`cat .token`

This will ensure that after each commit the documentation will be pushed to app engine, just before you do your next commit ensure that the command is executable:

chmod +x .git/hooks/post-commit

Step 3 - Enjoy

After your next commit you should be able to visit your shiny new app engine powered documentation site located at <your-app-id>.appspot.com.

Invite Collaborators

Now that you know that it's working, go to your app engine dashboard (http://appengine.google.com) and visit the app that you're using for documentation. Then in the permissions section invite new collaborators using their gmail or google apps email address.

They will receive a link to an authorization page that looks something like this:

Once they accept they will be able to view the documentation.

Possible Improvements

I'd like to make this into a script so that it is even faster to get going. Using the admin permissions and forcing is a bit of a work around. There are cases when this is not ideal, for example if the documentation needs to be shared with less trusted users.

Security Matters

Every now and then, I take a peek at the landscape of the blogging world. In case I decide to change my platform. I run this blog on Blogger to save me the headache of worrying about updates and security for my blogging platform.

One of my explorations has me looking at "static" blogging platforms which "bake" your entries into static HTML files. This led me to a static blogging system called "Jeykll" and an article describing a way to run a Jekyll blog on Google AppEngine using DryDrop. That's not the point of this post, I ended up on a tangent due to the fact that the author of the article, though a Wordpress lover, had choosen to leave Wordpress due to security issues.

In his own words:

"I hosted about 25 wordpress blogs on my mosso account for various friends. I kept most of them up to date, but a lot of them were for friends and were not under my control. 100% of them got owned. hah. It was just something they did. no matter how fast or often i updated the wordpress software - it would be owned at least one time. My personal blog was safe for some reason. Maybe it was because I always ran the bleeding edge version from SVN. I will not miss the constant updates and the attacks. The wordpress community does a good job of handling this issue. I, however, was tired of it."

Apart from the Plone sites that I manage, I am responsible for two Wordpress sites (I'm aware of all the cautionary tales so I'm keeping those WP sites up to date, backed up, plus my fingers crossed).

Bottom-line, security matters!

update:
If you're seriously looking into static blogging you may want to check out Ruhoh.