root / RuPy / t_and_d__dd.txt

.. include:: <s5defs.txt>

===========
TDD and DDD
===========

:Authors: Olivier Grisel <olivier.grisel@ensta.org>
:Date:    $Date: 2007-02-21$

.. This document is published under the CC-By-SA 2 license

.. class:: small

  * A few words on Extreme Programming

  * Writing TestCases and test_suites in python

  * Documentation Driven Development with the doctest module

  * Additional tools (continuous integration testing, coverage)


.. container:: handout

    The first part of the presentation introduces Test Driven Development
    in Python with the unittest module. TDD is a common eXtreme Programming
    strategy to dramatically enhance the QA of a python project.

    The second part of the presentation focuses on the python specific
    doctest module that pushes TDD to the next level: DDD. We will
    show how to write runnable documentation that is used at all stages
    of the life of a python project:

     - technical specifications drafting of new features
     - testing ongoing implementation
     - introducing living examples to new developers
     - ensuring the documentation stays up-to-date when refactoring


.. contents::
   :class: handout

.. |bullet| unicode:: U+02022
.. |mode| unicode:: U+00D8 .. capital o with stroke

.. footer:: Location |bullet| Date


Extreme Programming
===================

.. class:: incremental

* Fashionable methodology to drive software development projects

* 3 fundamental concepts:

  .. class:: incremental

  - Peer programming

  - Short iterations (the customer is in the inner loop)

  - Tests, tests, tests -> automate them!


Testing principles
==================

.. class:: incremental

* 1 test == sample use case for each method in each class in each module

* reproducible and organised in a "test suite"

* run them as often as you can

* think: "If it's not tested, then it's broken"

* tool to find bugs early != formal proof of correctness


More principles
================

.. class:: incremental

* Compare test results to expected results

* Test common use cases

* Test weird cases: what if arg = 0, None, ... ?

* Test that the right exception is raised


How to do that?
===============

.. class:: incremental

* "Python comes with batteries included"

* Use the ``unittest`` module

* Derive the ``TestCase`` class

* Add ``setUp / tearDown`` methods

* Implement tests as ``test_something`` methods

* Use ``assert_``, ``assertEquals``, ``assertRaises``, ...

* Collect the tests and feed them to a runner


.. container:: handout

  Here is an example test setup to test file creation, reading and writing.

  ::

    from os.path import join, exists
    import unittest, tempfile, shutil

    class FileTestCase(unittest.TestCase):

        def setUp(self):
            self.test_folder = tempfile.mkdtemp()

        def test_text_file(self):

            content = "some text\nto write\nin a file"

            filename = join(self.test_folder, "my_file.txt")

            f = file(filename, 'wb')
            f.write(content)
            f.close()

            self.assert_(exists(filename))

            lines = file(filename, 'r').readlines()
            self.assertEquals("some text\n", lines[0])
            self.assertEquals("to write\n", lines[1])
            self.assertEquals("in a file", lines[2])

        def test_binary_file(self):
            data = '234wgfge43rv34rvq3w5q3v54q3245q'

            filename = join(self.test_folder, "my_file.bin")

            f = file(filename, 'wb')
            f.write(data)
            f.close()

            self.assert_(exists(filename))

            f2 = file(filename, 'rb')
            self.assertEquals("23", f2.read(2))
            self.assertEquals("4wgf", f2.read(4))
            self.assertEquals("ge43rv34rvq3w5q3v54q3245q", f2.read())

        def tearDown(self):
            shutil.rmtree(self.test_folder)

    def test_suite():
        suite = unittest.TestSuite()
        suite.addTests(unittest.makeSuite(FileTestCase))
        return suite

    if __name__ == '__main__':
        unittest.main(defaultTest='test_suite')
        from os.path import join, exists
        import unittest, tempfile, shutil



Testing hierarchy
=================

.. class:: incremental

* Unit tests: each method of each module independently

* Integration tests: several interacting modules

* Functional tests: black box / UI tests

* Fast to run / numerous -> Slow to run / fewer of them

* All of them serve also as regression tests

* Refactoring harness


Testing methodology
===================

.. class:: incremental

* T. *D*. D. means *driven*

* Write the test first and then the implementation

* A matter of point of view -> cleaner APIs

* When the tests pass, stop writing code, you are done!

* For each bug found, reproduce it with a test and *then* fix it


Documentation Driven Development
================================

.. class:: incremental

* Most tests show good sample usage

* Let's use them as developer documentation

* Documentation is a pedagogical story with samples in it

* Early stage: used as brainstorming sandbox

* Then: evolves as a technical specification of your API

* Then: guaranteed up-to-date user friendly documentation


Modules: ``doctest`` and ``docutils``
=====================================

.. class:: incremental

* original python

* tell a story with the reST syntax

* insert examples as if run in a console

* use ``docutils`` to generate a nice doc (xHTML, PDF, S5, ...)

* use ``doctest`` to check the examples actually work


doctest examples
================

.. class:: incremental

  ::

    >>> def tell_the_truth():
    ...     return "doctests rock!"
    ...
    >>> tell_the_truth()
    'doctests rock!'

  * Now this presentation is doctestable!

  ::

     import doctest
     import unittest

     def test_suite():
         return doctest.DocFileSuite('t_and_d__dd.txt')

     if __name__ == '__main__':
         unittest.main(defaultTest='test_suite')


Writing techniques and tips
===========================

.. class:: incremental

The seven *laws* for technical writing (by Tarek Ziadé and others):

  .. class:: incremental

  - Two-step writing process

  - Simple style

  - Targeted readership

  - Focused information

  - Realistic examples (no foo/bar stuff)

  - "Light but sufficient" approach

  - Structured documents



Tool #1: Test collectors / runners
==================================

.. class:: incremental

* 2 options to build a test suite:

  .. class:: incremental

  - manually (``unittest.TestSuite().addTest(...)``)

  - use a script that does it for you

* nosetests

* zope testrunner

* others, use google


Tool #2: Continuous integration testing
=======================================

.. class:: incremental

* Robot to launch tests on each VCS checkin

* Python reference tool: buildbot

* Used to test python itself:

  .. class:: incremental

  http://www.python.org/dev/buildbot/all/


Tool #3: Coverage monitoring
============================

.. class:: incremental

* Principle: each executable line of code should be tested

* Remember: not a formal proof, just a way to find bug niches

* coverage.py : text mode reports

* trace2html : HTML coverage reports


Questions ?
===========

.. class:: incremental

Have no idea? Here is one:

  .. class:: incremental

  - how did write such beautiful slides?