I've been working on a new static site generator tailored for managing documentation sites, and I am curious as to what people really want/need for their projects' documentation.
For those of you who keep great documentation:
1) Is it better to keep documentation with your code, or managed separately?
2) How do you keep your documentation up-to-date?
3) What tools/processes would make it easier for you to maintain great documentation?
For those of you who don't write much documentation:
1) What is the main reason why you don't write much documentation?
2) How much of your daily workflow depends on reading documentation, and does this impact how much documentation you write?
3) What tools/processes would make it easier for you to start focusing more on documentation?
Top comments (27)
Maintaining code and documentation separately makes it painful. Also, I'm just bad at documenting things. 😶
Code can't lie, documentation can. It's sooo easy to get them out of sync.
I like how modern languages acknowledge this problem by trying to solve it in the language itself.
Both go & rust address this by encouraging you to add documentation along with your code and include nice tooling to generate documentation from comments in your code.
After adopting Go as my primary back-end language, I found myself applying some of those good habits in other languages I use.
Riiiight :-)
code tells exactly what it does. sometimes it's not what you expect, though
Only when working solely with primitive operators that cannot be overloaded. Otherwise, code can easily lie. There is no shortage of methods like getFoo(), that only retrieves the latest 5 foo (unless it’s a Tuesday), but also removes the oldest foo, sends some random administrator a loosely related email, and might possible do a perfect frontflip and backflip to wrap things up (though not always, and we’ve yet to figure out why it doesn’t do the backflip before the frontflip).
Getting out of sync was the perfect term I was looking for. Thanks, Ben!
I wrote a series of articles about this issue. I think the main problem is that we're not writing documentation for our current ourselves. We're writing it for some abstract user (ourselves in the future or a random user). A bit like the Pomodoro technique, I think we can simulate an emergency effect and become concerned right now by our documentation, by in a very regular way testing how a new developer can handle the project just thanks to written documentation.
This is a really good point to keep in mind, one that I know I need to get better at. The most helpful documentation out there really does focus on showing new users unfamiliar with the full framework how to solve a problem with the framework, rather than just showing off all its many features without much guidance. I'll definitely be checking out those articles later today, but do you have any practical, easily rememberable tips that help you put yourself into that mindset?
The final article in my series especially talks about this process. To sum it up: frequently find a developer that is not familiar with your code, and ask them to 1/ install the project, 2/ dive into the code (understand the structure and how things work).
Then the dev writes an issue on the repository, summarizing every problem they found in the documentation. And if they can't install the project because of a problem, then they can even stop there and directly write an issue.
This process can be called a Bus Factor Review. If done frequently (every 2 weeks?) it can be very fast (because it's incremental) and very useful to 1/ keep the doc in sync and 2/ to easily have a summary of the obscure things in your project.
Of course this applies in the context of a company, but I guess we could even create a community to do so, with personal projects :).
From my experience, documentation tends to get old and outdated pretty quickly and that's, in my opinion, the main problem. When you can't trust your documentation it loses all its value.
We're currently using Guru. It allows us to write small doc snippets (mitigate laziness) and keep them up to date since Guru will notified the authors when the snippet is getting old to check if it's still valid and fresh.
You can also create these snippets from Slack. Most of the time a discussion starts in Slack and at some point someone adds a guru snippet to wrap it up.
That's an interesting tool, I've never heard of it before. I like the idea of incorporating documentation as part of the normal daily discussion, and having a bot checking for potentially out-of-date documentation. Do you find this resource to be sufficient for most of the new developers on your team?
Sure, it's replaced our documentation wiki altogether. People really like it and it's full of useful information. I think it certainly is worth the cost. We have a board for newbies with a lot of "cards" with useful information. Things like, how to start building a service, how to setup IDE, how to connect to different places, how our CI works and so on.
That's true, many of the conversations where devs are looking for a specific piece of knowledge originate in Slack. There are also other tools such as Obie, etc also integrate with Github to help close the loop.
The main reason documentation gets skipped is because it is writing the same code twice. You write it once in a programming language and once in a human language. Then you have two places to maintain. When the pressure is on to deliver something, only programming language really needs to be updated to get the functionality. But when you skip updating the documentation, it becomes ever more daunting to go back and straighten it out.
In a previous team, there was a dedicated documentation person (tech writer) who made sure the documentation stayed up to date with the changes that were happening in the sprint. The tech writer split their time with multiple teams because it was not a full time effort for any individual team.
That was really the only way that I have seen to make sure documentation gets updated consistently. Make a person primarily responsible for it.
I think that's one of the hardest things for me. I really want to believe that my code is clean enough to document itself so I don't need to write as much actual documentation, but it will naturally get messier and harder to read over time that that's when I start to wish I had written that documentation in the first place.
It's a really great idea, having a team member fully-devoted to keeping documentation up-to-date. I've been following Netlify closely for a couple months, and I was really impressed when I found @verythorough does exactly that for Netlify. I'm sure this would be a difficult thing to convince upper management to devote resources to, but what practical benefits have you experienced that would help convince them for a team wanting a full-time documentation writer/engineer?
You first have to consider who the documentation is for. For that team I was on, the documentation was for the user. It described how to accomplish things with our application. It also encompassed release notes. So it was not integrated into code at all. It was stored and updated in a separate system and generated for each release.
If you are documenting the API of your library, the situation is a bit different. Most devs will expect documentation that pops up in their editor when they hover over it (or something similar). Devs typically have to write this embedded-in-code documentation. Then if necessary, more exhaustive documentation, usage guidelines, and further examples might be available on the website. The website could be based on the code comments initially, but these are not typically the best format for users to read through. So then you end up with supplemental documents that are not embedded in code to make for a better documentation UX. Like an Introduction section describing the product at a high level and providing an organization structure to drill into specific topics. Eventually documentation should become a first class product on its own, doubling as a support tool (and maybe a sales aid, so potential users can evaluate the product capabilities for their needs). While the code comments become more of an appendix. A first class documentation product probably won't happen accidentally while the team is really focused on something else. It is going to need a dedicated team in the long run. But perhaps not all library / API products need to go this far.
The value of having a dedicated tech writer depends on the product. If you provide a public API / library and make money from it, I would think a tech writer would not be a hard sell in the long term. If it is an internal tool, that can be difficult to convince people unless it is already in the corporate DNA. The value there is more limited.
Actually, I write tons of documentation. It's kind of a necessity: I jump back and forth across projects within a given contract plus breaking across contracts. While I've got a freakish memory, if I don't at least write breadcrumbs for myself, it's back to the Googs I must go (sometimes finding my own articles).
For me, the most onerous problem is the layers of documentation and the available tools to write/maintain it in. Not only do I have the problem of "keep the dox in sync with the code", I then have the "meta" task of "dumb that documentation down". And the inevitable "we've got to push your stuff out to new audiences: please dumb the dox down some more". Keeping layers of documentation reflective of the current code-based reality is, at best, "a chore".
I strive for "obviousness" in everything that I write - be it code or documentation. In general, I try to make it so that if you have half a clue, you can figure out what's going on just by how things are named, how they're organized, how their exits are trapped/signalled ...and how the blocks they're in are commented. But most people are code-phobic and won't even look in the code to see if something smacks them in the face. My first layer of not-embedded-in-the-code documentation is typically READMEs kept in the code's project. Those tend to stay fairly in sync with the code ...but that's mostly self-discipline that sees to that.
I assume that, with my in-project documentation, I could probably set up triggers that send me "I see you updated tool X but the document that seems to reference it wasn't updated: is that correct" types of messages. Haven't really investigated because we generally don't get funding for the kind of for-fee tools that I could probably leverage for that.
As to the further-abstracted documentation - that are hosted away from the code (in RedMine, Confluence or whatever a given contract's preferred documentation-tool is) - it's beyond a pain to keep it all up to date. I know that everyone loves simplified markup languages ...but there seem to be enough SMLs that simply cutting-and-pasting from one documentation-system to another doesn't really work.
I mean, you can see this just in Dev.To if you've set up RSS-based publishing from your "real" blogs' location. Something that looks great on your preferred platform requires significant massaging to make look the same in Dev.To. And that assumes that Dev.To's Markdown (SML) flavor supports the same feature sets that your article source was written in.
I've tried tools like PanDoc to get around some of these problems, "Markdown is Markdown is Markdown" isn't quite true and you can get surprises when your auto-formatted code doesn't quite render the way you expected i to. While authoring languages like HTML (or even TeX/LaTeX) have steep learning curves (particularly if you want to get fancy with their extended feature-sets), they felt both a lot more reliably portable and controllable.
I wish I knew the solution to the problem. Also wish I could say it seems like things are getting better ...but the opposite feels to be true.
I've never been a part of an organization large enough to have these kinds of issues, but it definitely sounds like automating the docs process only gets more important at larger scales, while the very nature of the available tools severely hinders this. But it gets me thinking: do you think some of these problems would get better if the synchronization worked by pushing out the rendered markup rather than the SML itself? So, for example, having one system that that allows you to write in your preferred SML, and notifies other systems of changes by compiling that markup and just pushing out pure HTML?
I really wish I knew what the answer was. When it comes to writing - be it code or dox - I really prefer "write it once" methods. And, while HTML is pretty well standardized and stable, the richer the use of that HTML/CSS, the harder it is for many SMLs to decompose it into something they can render.
Like @einenlum I also have written an article about how I manage docs for my open source project.
I prefer to keep the documentation colocated; this ensures that the documentation is correct for any given commit, and makes deploying it much easier. I don't know if another static site generator will be able to overcome the home team advantage of GitHub Pages, for all Jekyll wouldn't be my first choice of framework.
Personally, I find myself more in the latter group, but wanting to get into the former. Given that I do not write much documentation:
1) It's hard to justify taking the time to write documentation when I could be fixing bugs or creating new features.
2) I depend almost exclusively on official documentation (as opposed to other resources like Stack Overflow), and I recognize the impact that good documentation has on the community.
3) Most projects would benefit from managing documentation in multiple ways (code API docs, wiki, changelogs, etc.), and I want to manage all these in one place rather than having to use multiple tools for all these. Full disclosure, this answer is loaded as I am developing Orchid for exactly this purpose. Nevertheless, I started creating Orchid precisely because the market for documentation tools are all so isolated and hard to manage.
Short answer: this is why we built Corilla.
Long answer: I lived this pain so much at Red Hat as a technical writer. Solving this first internally and then externally has been a joy. The whole "docs as code" movement is great for devs that work without much collaboration with other teams. But terrible for collaboration. Ditto the "don't use Markdown" rhetoric. I used to believe that too... until I needed to collaborate with support, marketing, UX, PM, etc.
One workflow that makes it easy for people to collaborate. And multiple output possibilities (including direct API access). Documentation in 2018 is so much nicer than the last decade of nonsense we endured (it was hand-coding DocBook XML that broke my sanity in the end... all good startups come from enduring red tape and broken workflows IMHO. Or even average ones like us :P).
Deadlines that have little merit other than keeping the team in a constant state of hurrying.
I've built archbee.io to make it easier for developers to maintain documentation. Check it out, you'll be impressed. The UX is made for developers. Built internal knowledge bases and product docs easily.