root / PyCon07 / slides.txt

==============
PYCon Tutorial
==============

How to document an open source Project ?

<tarek@ziade.org>

.. contents:

How am I ?
==========

- Python developer since 2000

- Core developer in Nuxeo CPS (Zope) for 3 years

- Involved in Zope sprints

- Author of a french book "Programmation Python" (Eyrolles, 2006)

- Creator of AFPY (french speaking PUG)

- CTO at Emencia, for a Python e-commerce framework


What are the goals of this tutorial ?
=====================================

- Understand the place of documentation in software projects

- Learn a few patterns to document your project

- Practice them !

How this tutorial is organized ?
================================

- Each part comes with:

 - a few slides
 - exercises *you* do.

The plan
========

- A few definitions
- Part 1: Writing a document
- Part 2: Writing documents for Python
- BREAK - Yepeee
- Part 3: Using documents for TDD
- Part 4: Writing documents for a Python project
- LUNCH - Yeeahhaa

A few definitions
=================

What kind of documentation are we talking about ?

A few definitions
=================

Technical documentation that helps to understand how a codebase
works and how it can be used.

This can be:

- a document describing a package, a module or a serie of package
- a glossary
- a tutorial
- a recipe

A few definitions
=================

What is the use and who is the target ?

A few definitions
=================

- Project developers, to...

  - build a shared knowledge
  - improve code quality

- Community, to...

  - facilitate participation (e.g. non developers)
  - facilitate adoption and comprehension

Part 1
======

Part 1 - Writing a document

This part is composed of three sections:

- Mastering reStructuredText
- Writing techniques ans tips: the seven laws
- Team writing

Mastering reSTructuredText
==========================

reSTructuredText is

- a plaintext markup syntax and parser system
- easy to read as-is, yet powerful
- the Pythonistas laTeX

Mastering reSTructuredText
==========================

- example (Tarek, switch on)
- cheatsheet
- see http://docutils.sf.net

Mastering reSTructuredText
==========================

|smile| let's do exercise #1 and #2

Mastering reSTructuredText
==========================

reST can be:

- used for all documentation needs

- manipulated like code (versioning, etc.)

- easily transformed into various shapes

Python developers |love| reST

Writing techniques and tips
===========================

The seven *laws* for technical writing:

- Two-step writing process
- Simple style
- Targeted readership
- Focused information
- Realistic examples
- "Light but sufficient" approach
- Structured documents

The Two-step writing process
============================

Writing should be done in two stages (Elbow, 1980):

- a free writing where ideas are written on the paper no matter the shape.
- a review stage where things are structured and reviewed.

Each stage should take 50% of the time.

|important| Split you writing time in two phases, no one can write it right
the first time (except Mozart)

Simple style
============

- Use short sentences and simple writing style.

- Use simple vocabulary

Targeted readership
===================

Focus on your target when you write a document (RÃŒping, 2003).

Assume their background knowledge to restrict the scope of the documentation.

|important| A *prerequest* section can help a lot.

Focused information
===================

A document is about a clear focus.

A precise title for each section and the document itself, and a summary
can help.

|important| If you cannot easily name the document or one of its section,
there's a problem

Realistic examples
==================

Drop the *foo* and *bar* habits. Examples must be real-life examples:

Neh::

    >>> from package import graph
    >>> foo = graph.calculateSquare(1, 1, 1, 1)
    >>> bar = graph.renderSquare(foo)

Better::

    >>> from package import graph
    >>> square = graph.calculateSquare(1, 7, 1, 10)
    >>> square_view = graph.renderSquare(square)

"Light but sufficient" approach
===============================

A working software is more important than the best documentation in the world
(Ambler, 2002).

*Quality over Quantity* is the best rule.

|important| Spending too much time to find something in the documentation is
a bad sign.

|important| Think documents like code. Always limit the size of its sections,
examples, etc.

|wink| Modularized documentation is the key.

Structured documents
====================

Use a clear document portfolio to facilitate the reading. Document should
use:

- an abstract with a readers guideline
- a toc when there are more than one section
- references
- glossary
- tables and diagrams

|important| Never write a document that doesn't have a template

Writing techniques and tips
===========================

Distribution of tipsheet, let's do exercise #3, and apply the laws |smile|

Team writing
============

Pair-writing ensures:

- understandable documents
- increased and homogeneous quality

|important| The best pair is made with the end-user

|important| Each document should have a champion, though.

Team writing
============

Let's do exercise #4 to feel it |glasses|

Topic ideas:

    - cooking recipe
    - how to play a board or card game
    - how to fish shrimps


Writing a document
==================

Conclusions for Part 1:

- reST is perfect to write documents
- Anyone can write good documentation, with the seven laws in mind
- Team writing provides to documents what pair-programming provides to code

Pause
=====


.. figure:: media/sleeping.jpg
   :alt: Sleeper
   :align: center
   :height: 350
   :width: 450

   Developer that does not practice documentation

Pause
=====

.. figure:: media/winner.png
   :alt: Winner
   :align: center
   :height: 350
   :width: 250

   Developer that write documentation (=winner)

Part 2
======

Part 2 - Writing documents in Python

This part specializes the documenting process to Python software

Writing documents in Python
===========================

Python projects are organized into:

- code: classes, functions
- module
- package
- groups of package (let's call it a *bundle*)

Documenting code
================

The code is documented with:

- inline comments
- functions, classes and modules docstrings

Inline comments
===============

- the less, the best
- needed only if code is not self-explanatory
- used for local design notes (XXX, TODO and friends)

Functions, classes and modules
==============================

- a function or method docstring should be composed of
    - a single, short sentence
    - an extra text when needed
    - sometimes, explanation on in and out variables

- classes and module docstrings resumes the role of each element

|important| read PEP8 and PEP256

Documenting modules
===================

- Let's create a text file for each important module in a package

- The text file describes what the module does

- example: utils.txt and utils.py

- Let's practice: exercise #5 |glasses|


Documenting a package
=====================

A package comes with a few mandatory files:

- **README.txt**: a global file that presents the package
- **LICENCE** or **COPYING**: the licence used
- **CHANGELOG**: Lists of changes for each version of the package
- **INSTALL.txt**: how to install the package
- **TODO.txt**: Lists all things to do (sometimes replaced with a
  link to a tracker or a website in README.txt)

Documenting a package
=====================

A package comes with a few mandatory folders:

- doc: collects all documentation file, except README.txt
- tests: contains unit and functional tests

Documenting a package
=====================

Let's add mandatory files to `pycommunity` (exercise #6)

Using documents for Test-Driven Development (TDD)
=================================================

- Tests can be added within documents

- They look like prompt sessions

- example: doctest.txt

Using documents for TDD
=======================

Let's call it *DDD* (Documentation-Driven Development)

Why should you use DDD ? |smile|

Using documents for TDD
=======================

How is managed documentation in projects, most of the time ?

Using documents for TDD
=======================

.. figure:: media/classical_cycle.jpg
   :alt: Sleeper
   :align: center

Using documents for TDD
=======================

|sad| fast obsolescence of documentation

|sad| you need to fight with developers so they write doc

|sad| your project website is always obsolete

Using documents for TDD
=======================

What brings DDD to projects ?

Using documents for TDD
=======================

.. figure:: media/ddd_cycle.jpg
   :alt: Sleeper
   :align: center


*(fruit may vary depending on the season)*

Using documents for TDD
=======================

|smile| A big part of the documentation is growing together with code

|smile| developers use documentation to write code (and they |love| it)

|smile| your project website upgrades can be partly automated

Doctesting
==========

Let's doctest

Doctesting
==========

|important| A doctest is still a document: do not drawn it with test fixtures

|important| The doctest shows usage examples of the **public** code

|important| The unit tests tests everything else

|wink| Keep your doctests code blocks short and simple

Doctesting
==========

|glasses| Let's practice with exercise #8

Writing documents in Python
===========================

Conclusion for Part 2:

- There's 3 level of documentation:

  - inline comments
  - docstrings
  - separated docstrings
- Packages comes with mandatory documentation
- We can do TDD with doctests, and deal with doc. like code

Pause
=====

.. figure:: media/magnum.jpg
   :alt: "Doctests are so cool" -- Magnum
   :align: center
   :height: 300
   :width: 400

   "Doctests are so cool" -- Magnum

Part 3
======

Part 3 - Writing document for Python Projects

This part is composed of four sections:

- Overview
- Writing tutorials
- Writing cookbook and recipes
- Writing a glossary

Writing document for Python Projects
====================================

- Python projects are *bundles*
- *bundles* have high level documentation:

  - Tutorials
  - Recipes, gathered in a cookbook
  - a Glossary

Tutorial
========

A tutorial is a very high-level document that explains a functionality of a
bundle, how to use it or to change it.

Tutorial
========

A tutorial...

- ..should not contain any code, or very little

- ..makes a heavy use of diagrams, tables

- ..should be understandable by non-developers

- ..need to focus on a single topic, regardless the number of
  packages involved.

- ..should be less than 8000 signs e.g. two screens

Tutorial
========

Possible template:

- title & abstract (two sentences max)
- prerequest: level needed, other documents to read
- toc (when # sections > 1)
- body
- references
- a list of other tutorials to take on next
  (*a graph of tutorials can make a good tutorials listing*)

Tutorial
========

|glasses| let's do exercise 9

Recipes
=======

- Recipes are the technical parts of tutorials.

- A recipe answer to one question, that should be its title.

- Each question is linked to the sentences founded in tutorials.

- A recipe is short (2 screens)

Recipes
=======

Examples:

- How to code a RSS view ?
- How to extract the content of a web page ?

Recipes
=======

Possible template:

- title & abstract (two sentences max)
- prerequest: level needed, other documents to read
- toc (when # sections > 1)
- body
- references
- a list of related recipes

Recipes
=======

Let's do exercice 10: "How to use ConfigParser"

Glossary
========

A glossary builds a shared knowledge by helping to..

- ..avoid using two terms for the same definition
- ..removes approximacies in documents:
    not sure on how to explain a concept with simple words ?
    look in the glossary |wink|

Glossary
========

|important| If I remove all terms founded in the glossary from tutorials,
they should be made of plain english.

Linking everything
==================

- A tutorial links to recipes.

- Tutorials and recipes makes heavy use of the glossary

- A bundle is fully documented with:

  - tutorials
  - recipes
  - modules doctests
  - inline comments and doctests

PyCommunity
===========

A tool that:

- reads a bundle

- extract its documentation

- generates a website with it

- is not released yet (soon |wink|)

PyCommunity
===========

- can be hooked in the subversion post-commit hook

- can be configured to inject its generated page in an existing site

|smile| let's try it #exercise 11

Conclusions
===========

- Using `DDD` is a good way to enhance the QA

- It is enjoyed by most developers

- It increases both writing and programming skills

The end
=======

- Thank you !

- Questions ? Answers ?

References
==========

Useful books:

- "Writing with Power" - Peter Elbow
- "Agile Documentation" - Andreas RÃŒping

Useful websites:

- http://emenciacommerce.com/en: Emencia Commerce Suite - My company software
- http://tarekziade.wordpress.com : My blog

Incoming Events
===============

You all go to Rupy in Poznan, Polan, 14/15 April |wink|

.. |wink| image:: media/wink.png
.. |smile| image:: media/smile.png
.. |important| image:: media/important.png
.. |love| image:: media/love.png
.. |glasses| image:: media/glasses.png
.. |sad| image:: media/sad.png