DEVELOPERS.txt

Zwiki Developer's Guide
#######################

This guide is to explain current Zwiki development practices and to
collect helpful guidelines.


Zwiki repository policies
=========================
Here are some guidelines for committers to the main Zwiki repository.

- http://zwiki.org/repos/ZWiki is the official Zwiki "trunk" repo,
  used for product releases and for running zwiki.org. 
  A darcsweb interface is available at
  http://joyful.com/darcsweb/darcsweb.cgi?r=ZWiki;a=summary .

- Contributors to this repo should read and sign CONTRIBUTORS.txt,
  which describes this repo's copyright and license policy.



File layout
===========

See README for a list of documentation files.

Directory overview:

================ ==================================================
ZWiki/           main source files
 plugins/        functionality-adding plugins
  tracker/       zwiki issue tracker plugin
  rating/        zwiki page rating plugin
  latexwiki/     page types supporting LaTeX
  mathaction/    page types supporting math tools
 tools/          administrator command-line tools
 pagetypes/      standard zwiki page types
 Extensions/     external methods
 scripts/        example zope/zwiki scripts
 i18n/           translation files
 skins/          skin templates & skin-based content
  zwiki/         the standard zwiki skin for zope & plone
 wikis/          wiki templates
  basic/         the standard pages and scripts for a new wiki
 profiles/       CMF/Plone GenericSetup configuration
================ ==================================================

Currently,

- there is no separate tests directory. Each source file has a companion
  file with _tests appended to the name, in the same directory. This
  sometimes is cluttersome but it makes tests easier to find, grep, and
  makes plugin tests easier.

- similarly, there is no separate docs directory. Documentation files are
  kept in the most appropriate directory, with ALLCAPITALIZED names to
  make them stand out.

- most source files have capitalized names. Some newer files have
  lower-case names.


Documentation guidelines
========================

Most Zwiki documentation is kept in the http://zwiki.org wiki.
Developer-specific docs are under http://zwiki.org/DeveloperNotes.
Developer documentation is to some extent moving to files within the
codebase, like this file.  These typically use restructured text markup.
To browse these filesystem docs online, use http://zwiki.org/repos/ZWiki
or the darcsweb interface.

Filesystem vs wiki docs
-----------------------
Filesystem docs:

- faster for one developer wielding a text editor
- or several devs who are good with darcs
- revision controlled - secure, trackable
- can work offline
- can get out of sync, uncommited changes here and there
- can be auto-generated - epydoc, test output, author stats etc.
- can be published (but not edited directly) online
- less visible and less pretty for readers
- better than wiki for large single documents

current uses:

- CHANGES - release notes, also published at zwiki.org/ReleaseNotes
- CONTRIBUTORS - official contributor list
- GPL, LICENSE - license and copyright status
- REPOPOLICY - copyright and other developer policies for this repo
- README_tests - developer test documentation
- README - basic product info
- TODO - project plans & todos
- other READMEs - basic info for each directory
- plugins/latexwiki/LICENSE.txt,NOTES.txt,README.txt - latexwiki-specific copyright info, dev notes, admin/user notes
- plugins/mathaction/INSTALL.txt,LICENSE.txt,Notes,README.txt,UPGRADE - needs consolidation like the above
- TESTRESULTS - unit test output updated hourly, published at zwiki.org/TestResults
- misc/ - public notes & snippets, published but not committed
- MYNOTES/ - simon's notes & snippets, not published
- MYNOTES/RelNotesTemplate - template for CHANGES

Wiki docs:

- slower to view and slower to edit
- more accessible and more editable for a wide audience
- one master copy used by everyone - simple
- no special tools needed
- email integration - notification and mailin
- good for indexing and searching many documents (local and google search)
- formatting aids - table of contents, headings, tables
- more effort to protect and maintain; more vulnerable to decay
- more expensive to host - every page takes memory & speed
- more public - indexed quickly, for ever
- can organize a large number of documents in a hierarchy

current uses:
 
- zwiki.org, including
- NextReleaseNewsDraft - unused
- GeneralDiscussion
- InternationalisationDiscussion
- IssueTracker
- plone.demo.zwiki.org
- free zwikis, including kpug zope3 docs, leo, etc.
- zopewiki.org, joyful.com etc.

Thoughts:

- use filesystem for developers, wiki for users/admins/general public
- use text files anywhere for fast developer notes
- one requirement, keep them current - but what to do with old/temporary notes ?
- no special docs directory, just upper-case, no suffix
- publish all filesystem docs online, like rdoc or within wiki (they will be googlable ?)
- restructured text and moving docs between FS and wiki encouraged
- wiki needs to be fast

Code docs
---------
We have the following kinds of in-code documentation:

- ALLCAPS documentation files in any directory, like this README

- File docstrings

- Class docstrings

- Method/function docstrings

- Comments

  - comments are sometimes used instead of the docstrings above. This may
    or may not be appropriate, I don't know.

  - a section heading within a file, class or method.

  - a short explanatory comment appended to a line of code. 

  - longer explanatory comments when necessary.

  The Zwiki code has a lot of chatty comments. Nowadays we should try to
  minimise their use eg by making them unnecessary or by using
  intention-revealing names.

- Names

  Names of directories, files, classes, methods, variables are where
  documentation meets code. They are perhaps our most powerful tool. :)

  Use clarifying and intention-revealing names wherever possible to
  minimise the need for other documentation.

  Choosing good names for small chunks of code helps us to build up our
  own mini-language (DSL) for Zwiki-building, which helps us to say more
  with less code.


Conciseness
-----------
Concise, not over-verbose, non-redundant docs are good. To help us write
better docs, we could try to follow this length guideline when possible:

 ================= =================================================
 doc files         any length
 file docstrings   one paragraph
 class docstrings  one paragraph
 method docstrings one sentence, plus one per argument, or as needed
 comments          one sentence or fragment, on one line
 ================= =================================================


Docstrings
----------

The first sentence should be a concise summary of the object's purpose.

Older docstrings follow the PEP guideline to separate the first line from
the rest with a blank line, but we have abandoned this to save vertical
space.

It may sometimes be ok to omit the line breaks at the docstring's start
and end if it doesn't hurt readability.

Describe method arguments and return values when needed. We might want to
come up with a standard haskellish way of noting the expected input and
output types and whether side effects are expected.


Function/method contracts
-------------------------
We are considering ways to document function and method "contracts". 
`Here's <http://www.htdp.org/2003-09-26/Book/curriculum-Z-H-5.html#node_sec_2.5>`_
an example. We are most interested in

- their input (argument) and output (return value) types

- their degree of statefulness/statelessness. Are they

  - pure functions depending only on the arguments ?
  - well-behaved methods which depend on/modify only self ?
  - or do they depend on/modify other pages, the wiki folder, other things ?

Understanding statefulness in particular should help us to identify and
gather together the most stateful code, reduce state dependencies, and
increase the proportion of pure and semi-pure functional code, which
will greatly aid testing, debugging and reliability.  

Here is a possible convention for now: when you touch a method or
function, review the docstring and code and try add a comment describing
the contract. The contract specifies the return type(s), whether it
redirects, state that it depends on other than the arguments, state that
it modifies other than the return value, and any other effects. Here are
some examples::

    def method(args) # -> return type(s) [; redirects] [; depends on: ...] [; modifies: ...] [; other effects: ...]

    def htmlquote(self, text): # -> string

    def asAgeString(self,time): # -> string | empty string; depends on: self (for current time)

    def handleSubtopicsProperty(self,subtopics,REQUEST=None): # -> none ; modifies: self

    def revisionNumberBefore(self, username): # -> revision number | none 
        # depends on: self, revisions

    def expungeEditsEverywhereBy(self, username, REQUEST=None, batch=0): # -> none
        # depends on: all pages, revisions ; modifies: all pages, revisions

    def upgradeAll(self,render=1,batch=0,REQUEST=None): # -> none
        # depends on: wiki
        # modifies: wiki (folder, pages, dtml methods, catalog, outline, revisions..)

A few more notes: 

These don't have to be perfectly correct; even a rough guess could be
useful.  Do review the contract for correctness every time you change some
code. Contracts should ideally be cumulative, ie reflect the contracts of
methods called by this one - looking at code one or two levels deep should
be enough. Very common features like the dependency on self for permission
checking are omitted. It may be better to keep contracts on one line for
easy scanning, even if long. Things we depend on or modify include: self
(usually the current page object), folder (the wiki folder properties and
acquired context), wiki (the folder and all pages in it), catalog,
outline, revisions, request.


Code guidelines
===============

In general, it's best to follow, first, the current guidelines in this
file, second, the style of existing Zwiki code, and third, standard Python
best practices.


Vertical space is precious, use long lines
------------------------------------------

IMHO, vertical space is more valuable than horizontal space. It's more
valuable to see more lines on the screen than to see the full contents of
every line. Keeping lines within 80 characters can obscure the shape of
code constructs, especially with functional coding style, and also means
we are less likely to see the whole method at once. Long lines, even over
80 characters, may sometimes be preferable. 

This assumes you have your editor truncate long lines, not wrap them.  I
do this, and maximize the window when I need more visibility on the line
ends.


Message passing style, functional programming style, etc.
---------------------------------------------------------

The Zwiki codebase is influenced by Smalltalk. Eg: 

- we encapsulate state and behaviour as objects when that makes sense

- we break up code into many small methods with intention-revealing names
  to form an expressive domain-specific language.

- "tell, don't ask", ie just send messages, ignore what's returned. 
  We haven't used this and I don't know how it applies to us; it's
  something to think about.

Recent code is also influenced by functional programming. Eg: 

- we minimise the use of variables and the use of procedural/sequential
  code (do this, then that, then the other). Instead, we compose functions
  (/methods) where possible.

- we prefer referentially-transparent functions where possible (return
  value depends only on the arguments - no side effects).

Think "fewer moving parts". Variables, imperative sequenced instructions,
and side effects are often unnecessary.  When we cut them out, there is
less to break and the code is more reliable and more testable.


More about naming
-----------------

Try to give procedures (functions/methods with side effects) verb names.
Try to give variables and side-effect-free functions noun names.


Security
--------

To specify method permissions, we sometimes use explicit zope security
declarations, sometimes omit the docstring, and sometimes use the _
prefix. The latter two interfere with readability and should be phased
out. We probably want to move to an all methods private by default policy,
and explicity declare permissions on everything that we want to expose.


Imports
-------
Imports are usually at the start of the file and in three groups: python,
then zope, then zwiki imports.


Testing
-------
We currently use pyunit for unit testing and a bit of doctest + mechanize
for functional testing.

http://palladion.com/home/tseaver/obzervationz/2008/unit_testing_notes-20080724
has useful guidelines for improving our unit tests. Excerpt:

 Tests which conform to these rules and guidelines have the following properties:

 - The tests are straightforward to write.
 - The tests yield excellent coverage of the AUT.
 - They reward the developer through predictable feedback (e.g., the growing list of dots for passed tests).
 - They run quickly, and thus encourage the developer to run them frequently.
 - Expected failures confirm missing / incomplete implementations.
 - Unexpected failures are easy to diagnose and repair.
 - When used as regression tests, failures help pinpoint the exact source of the regression (a changed contract, for instance, or an underspecified constraint).
 - Writing such tests clarifies thinking about the contracts of the code they test, as well as the dependencies of that code.