Documentation-Driven Development

So it’s been about 6 months since I last found some time to work on DiaVN, and I finally managed to get at it again. The first usable version, albeit limited (ability to take one Dia diagram and generate RenPY code), will be available soon.

The most difficult thing, when working with little time on big projects, is how to find one’s way in the work again. After six months, I was having a lot of trouble understanding what I did, how I did it, and why I did it, and this is at that point I realized that my documentation was inadequate - not that it was not present, but it was not written in the perspective of only touching the project twice a year.

What was missing exactly ?

• First of all, a non-technical documentation describing the architecture of the project as well as the specificities of the "syntax" used for the diagrams
• But also, a precise list of what are the questions to be resolved, what is still left to be done, and the advancement status of what was in progress

As my project is not on github yet (it will only be there when it will be the first public version), I did not have the possibility to use the Issues as todolists and my only possibility was to leave comments in the code, hoping I would make something out of it again after all this time.

So I decided to rewrite and complete the project documentation as a "welcome back" task, and I used the amazing Sphinx tool for that matter.

I started by writing the non-technical part of the documentation, then I dived in my code and rewrote the technical documentation. I used the "sphinx.ext.todo" plug-in to generate myself an evolving TODO list catching all the TODOs scattered in the code.

Going on this way, I realized that Test-Driven Development is a great thing, sure, but the first step is before, it is to describe in natural language what we want to do, and that is really writing documentation. So it came to me that a better method would be some sort of "Documentation-Driven Development", where I would first write what I want to do (with normal words), then I would code the tests to verify the described scenarios, and finally I would write the code to implement the idea.

It so happens that Documentation-Driven Development is a real thing, even if not well-known yet, and there are some resources online (for example this post here). Digging into the subject, I stumbled upon someone who said the concept reminded him of something Donald Knuth (the Stanford professor who publishes "The Art of Computer Programming") has invented, which is called "literate programming." In this method, code and documentation are written together as one document which alternates descriptions and thoughts about a project, and snippets representing how it works. At compile-time, a specialized tool will extract the code from the literate program, and compile it, while another tool will format the documentation.

With this mashup, it is natural to edit the documentation when one needs to edit the code - we usually change code either to correct bugs or to implement new ideas, and this is naturally written first as natural language.

I did not implement literate programming into DiaVN, but I’m sure going to give it a try and become a more literate programmer in my next whereabouts !