The story of the Drupal 7 core help update

Posted in: 

This one's all Drupal folks, cause that's pretty much all I've done for the last two and a half weeks. This is what happened when I asked the question, "Is there some reason we don't just fix it all?" I did not know then what I was getting myself into...

A small inconsistency

It all started in late summer, when I was testing some Drupal 7 core patches for moving fields and image handling into core, and at some point clicked my way into the Help pages. There was a blatant typo on the Node module help, and then a change in language that needed to be made, so on August 1st, 2009 I created an issue for it.

Then someone posted a patch to update the text. And Emma Jane Hogbin posted another...and then she and I started talking on IRC about how it would be much better to have some more clear formatting that could be applied across the board, fans of consistency that we are.  And we went back and forth for a while making little improvements. Thanks to having learned how to apply and create patches several months back, I had just enough experience to make changes and roll new patches.

The patch was marked RTBC (reviewed and tested by the community). Yay! But wait... then Angie, who is the Drupal 7 core maintainer saw the issue (as she is the one who does final reviews of issues marked RTBC before they are committed to the core codebase). And she was like WHOA, this is a major change! A good change, but a major one, which would need to be applied across the board to all the core modules (and eventually all the contributed modules). She also made us aware that if we wanted to make such an overarching change that we would need to be able to prove a higher level of discussion and consensus on the issue and then complete the change in time for the string freeze deadline.

A much bigger task at hand

Various documentation team members went back and for for a while about whether the new standard would be supported, and kept updating the patch getting closer and closer to a format we were happy with. By now it was the end of October, and the patch was finally set back to RTBC. This is when it popped back up on Angie's radar, who was like (paraphrasing)... "So, you guys haven't really addressed the question of whether you are going to make a massive across the board change here." Oh, right, that.

A new patch was created that reverted the format updates, and just updated the text to match Drupal 7 functionality. There was nothing wrong with this patch, and it could have just been applied and nothing else said about it.

But I didn't feel right about it... So I thought to myself, is there an actual reason that we are not making this massive change? It's not exactly challenging, it's just a lot of grunt work. And at the end of it, we would have some much more helpful and readable help references for the core modules in Drupal 7 rather than the difficult to read and horribly out of date information that was currently on the help pages.

Accepting the challenge

I posted back on the issue and asked this very question, and said that I thought it was really important to improve the help. Jennifer Hodgdon, who was another initial supporter, started a thread on the Documentation team mailing list, and it turned out that people agreed on one thing: the help text sucked and we ought to fix it. Happy birthday to me, on November 13th, 18 days before the Drupal 7 string freeze deadline, we made the final decision to overhaul the help pages, and Jennifer posted the template for the change.

Since Jennifer is a more experienced member of the Documentation team, as well as a much more advanced programmer than me, she got us off on the right foot and lead the charge for the first half, with the agreement that when she was going to be away for most of the homestretch I would take over and steer the ship. We'd been working well together and I felt like I was competent enough (and had enough other support available) to take on this responsibility.

The standard was signed off on, and on November 20th (ten days to the deadline), a list of modules was posted so people could start signing up for which updates they would take on. And then the real work began. And oh my...was there a lot of it.

Nose to the grindstone

I had no idea starting this out how much there really was to do. I was only thinking about the template change. But there was also a major need to update the content of the text, and those changes had to be reviewed and tweaked, and reviewed and tweaked some more. And then some more... I learned more about Drupal, the contribution workflow, and writing docs in PHP in that week and a half, than probably in the previous six months. Slowly but surely, myself and the others who were helping out got into a groove, and started cranking out some great updated help text patches. I got a couple of hours of "fun hour" at work to work away at this, but otherwise it was all afterhours; it basically devoured my free-time for almost two weeks.

Maybe it was the timing with American Thanksgiving, or having Jennifer away (thank goodness she was back for about a day and a half before the deadline!), but we definitely lost some momentum after the first five or six days. Feeling a certain level of responsibility, I basically crammed like a crazyperson most evenings and all through the final weekend, reviewing and updating patches. Thankfully a couple other dedicated people stuck with it, tag teaming on patches and reviews. Yes, I totally burnt myself out, but with the help of those great people who stuck it out, we miraculously got all of the patches for the 39 help texts done (44 if you count the Field submodules), updated to Drupal 7 content-wise, reviewed until within an inch of their lives, and marked RTBC (many of them already committed) by the December 1st, 2009 deadline.

The result?

Here is an example of the old help text for the node module:

And here is the updated version:

It might not look like much, but particularly for new site admins, and even more so with some of the more complicated modules, this is going to provide a truly useful resource (rather than a slightly confusing bunch of text that mainly just forces people to click through to the handbook). It was one of those highly neglected and unloved parts of Drupal, and it makes me extremely happy that is got a little much-needed love. On top of that, the first one of my help patches that was committed was my FIRST CORE PATCH! In Drupal-land, this is a momentous event, and I was thrilled to see that two of the others who were helping out both had their first core patches shortly after me while working on this.

(For anyone interested, the help doc standard is here, the main thread for the issue is here and there are several more branched off of it, all tagged with d7help.)

What I learned

This can be summed up in three points.

  1. How much work really goes into Drupal. This was the most time I'd ever spent actually working on Drupal (core), not online documentation, not event organizing. I had done some patch reviews before, but this was the most concentrated amount of time spent in the issue queue, and on IRC, working away alongside all of the developers who have also been cramming for this deadline. Countless hours and energy by people around the world goes into making this an amazing open-source web platform. It's pretty mind-blowing to see how it all comes together.
  2. Technical skills. I can now roll patches in my sleep. The whole process of tracking issues, downloading and applying patches, reviewing and making changes, and posting updates is something I now am completely comfortable with. I also learned some basic PHP coding, and also some of the associated coding standards that the Drupal development community uses. Prior to this, I could at best cut and past PHP into the right place. Now I am beginning to cross the line from memorizing to actually understanding what I am looking at. That is a huge accomplishment for someone who three short years ago was finishing off a Masters in Health Geography, and looking at her first CSS code.
  3. A few determined people can make a big change. The crazy part of this was that despite having to gather a lot of feedback and get consensus on the change, it was really just a handful of dedicated people who made this come to fruition. Several people helped out with reviews, but I have to give extra-special props to the people who did the bulk of the work with me: Jennifer Hodgdon (aka. jhodgdon) from Poplarware, Lisa Rex (aka. lisarex) who is a Freelancer specializing in usability, and Boris Doesborg (aka. batigolix) who from what I could gather works at Erasmus Hogeschool in Brussels. Last but not least Drupal 7 core maintainer, Angie Byron (aka. webchick) from Lullabot, who by golly is one of the most patient and dedicated people I know. Angie fielded a ton of questions about module functionality and what should or shouldn't be included, as well as doing a ton of final reviews and giving great, useful feedback to everyone who was working on this. We never would have made it without her help, and it was great fun some of those late nights on IRC.

All in all, a great learning experience, a great result, and my goodness am I glad it's done (at least until issues start turning up for contrib modules)!

Comments

excellent!

great description of the trials and tribulations of documenting open source software
docs should be separated more from the code IMHO

anyhow, doubt you have energy but it would be good to document what you learned between november 20th and dec 1,2009 - coz others could probably learn faster if you did!

you did an amazing job, ariane

i also spent much of my free time on these d7 help fixes, but i stuck to the easier modules and the smaller patches (cowardly avoiding the issues that aimed for changing 3 help texts at once!). but you were rewriting, improving, fixing *all* help texts in D7. so the main credits go to you and jennifer

i am indeed working for the Erasmushogeschool Brussel (http://www.erasmushogeschool.be/en/erasmus-university-college-brussels) at the communication department. besides that i manage the drupal documentation in dutch at http://drupal.be/documentatie

take some rest and prepare for Drupal 8's fully integrated context-sensitive advanced help system (with built-in jolly help assistant running around in the footer region giving unasked advice :))

Thanks for all the support! :-)

Boris D - If it's any comfort, I completely avoided those huge patches until all the other ones were done, they *were* a little scary! Aha... Dutch doc team, I should have known! I was thinking, where did this guy come from, he certainly knows what he's doing! As Angie can attest, having you and Lisa in the queue with me while Jennifer was away saved my sanity - I was freaking out a bit for a couple days! You did great work, and I hope to work on something with you again in the future!

Angie - It is *really* funny (verging on creepy?) that your first core patch was also on the Help. But dog help me if I am core maintainer in 5 years; I don't think everyone (read: me) is cut out for that! I can't thank you enough for all your encouragement and support, I'd be a lot more reluctant to take on crazy things like this if I didn't know there was such a great mentor at the helm, and I know there are many more who share that sentiment. I'm so, so glad that I could help even a little bit to get your Drupal mojo back. It was a ton of fun getting to work a bit more closely with you during this! ps. w00t greeeeeen!

Ariane! even though I sometimes stuggle with turning on my computer some days, and might not get the exact technical stuff that you *do*, I am so totally impressed and proud of your dedication and how much you've grown! I swear it was just 2 years ago when you were first teaching yourself drupal, and you've grown into someone whose erally helped create something!

Congrats on your achievements Ariane, and fantastic work it is. So much of this stuff slips under the radar underrated all the time.

I was one of the lucky ones to witness your hard work over IRC etc and admired your steadfast approach - it is no mean feat to catapult improvements up into core; it's certainly not something I've done.

People will experience a different and better Drupal thanks to your hard work. Well done!

You did an amazing job getting in there, and ripping up and rearranging core help. You really inspired me to get stuck in to the D7 help issue queue, and stay on it (I was late to the party, but glad I made it and stayed to the end). See you San Fran, hopefully!

Aw thanks Mig! That means a great deal coming from you! :-)

Lisa, like I was saying to Boris D. you two totally made a huge contribution yourselves, we would never have made the deadline without your help. It was a joy to work on this with such super-detail oriented, thorough, and writerly people. Indeed SF, I will be there!

"It might not look like much..." Are you kidding? This looks like a fantastic usability improvement! Well done. It will make Drupal a lot easier for new users to figure out.

It's very inspiring to hear about your journey into the contributor world. I think a lot of people who love Drupal but are not php developers will be inspired by your story and realize there are ways they can help out (that includes me!) Hearing about all the positive support you received makes it much less intimidating.

Wow, thanks LIsa, that really does make it feel worthwhile. :-) I hope it does help people figure things out.

I know there is this reputation that web dev and even more open source can be a hostile environment, and sure there are always bad seeds, but coming in as a non-developer, I have had such great acceptance and support. You just have to find those great patient people who are good mentors to help you figure it out, and then pass on the torch!

Ariane, I agree wholeheartedly with Lisa. My eyes popped when I saw the new format. I've been doing technical editing for 20 years, and you have made a drastic difference in how approachable all aspects of Drupal will be for folks like me. I'm convinced that much of the so-called learning curve of Drupal results from the time required to find the specific information needed. Your improvements to "Help" will help many developers succeed when otherwise the process of figuring out what to use and how it should work would have burned all the time (and energy) they could devote to their project.

Thanks for sharing your experience, too. In doing so, you are showing many others just how they can help make Drupal work even better.

Thanks Cliff - I'm not going to lie, that is really satisfying to hear. I thought something like working on help docs would be a more thankless task. I'm really pleased people seem to be finding the new format to be as much of an improvement as hoped.

Great job Ariane, and thanks so much for writing it up! I was one of those people that "flaked out" ... in there at the beginning of the help formatting marathon but couldn't hang in there. So I really appreciate how you and Jennifer and a few others really hung in and saw it through.

In a world that is broken in so many ways that are so complex to fix, I find comfort in making small fixes to Drupal, one patch at a time. My own approach is incoherent, dropping in when something burns and fading away quickly when the flame subsides. So your taking a big fat task and seeing it all the way through is really impressive to me.

Again, great job.

Add new comment