Improving the Content Development Cycle
When developing a learning platform, teams must work to get enormous quantities of educational content–rich text, audio, videos, and images–into the application on a regular basis.
Getting the content into the hands of users efficiently is mission critical: it’s needed in order to review, revise, and test the app, and ultimately for the magical moment when a learner signs in and gains access to this valuable content. Content creation is never done–making a learning app that is as useful as possible requires continually improving the substance, scope, and design of the content. While software development is often thought of in terms of “product lifecycle,” the people who build content-rich, dynamic learning platforms can also mark our days by where we are at any given time in the content development cycle.
Gargantuan Content Batches For the Win?
Companies like ours typically publish new content in large batches to align with certain times of the school year. This gives us a chance to let customers know to expect upcoming changes in the content. For the authors, working in batches encourages coherence and consistency within a set of new material. The content development cycle is ubiquitous–a fact of life in all of the roles that I’ve had in edtech, including here at Ellevation where I’m a software engineer on our Strategies professional development product.
When I joined the team, the module authoring process was painstaking and repetitive. But other urgent priorities had always been more pressing than improving this process, and our good-natured content team was willing to fight through some manual work, if need be. During the publishing process each module in a batch was:
- Authored in a doc…
- Pasted into a form…
- Converted to JSON…
- Uploaded to a shared folder…
- Downloaded by a dev…
- Ingested…
- Transformed to new JSON…
- Swapped with the old JSON…
- Converted to a database migration…
- Run on a feature environment…
- Content reviewed by editors
This process was repeated approximately once for each module in a batch. For a batch with six modules, that’s * frantically does calculations * …a cool 66-step process. 😱
Unless, of course, the author discovered some kind of issue when they finally saw their content rendered in our feature environment… in which case we had to do it all over again.
Unsurprisingly, the engineers didn’t relish our part of the work, generally preferring more of a “creative problem solving” vibe and less of a “rote steps” vibe to our day. But perhaps most importantly, working in these massive batches stretched out our content production calendars, because every single module took weeks of back and forth as our content experts attended to a slew of eleventh-hour details that made a BIG difference in the quality and polish of the final product.
We made this process work, but I began to wonder if we could do better.
What Could We Learn from Software Dev Cycles?
A foundational tension was coming into view: working in gargantuan batches, although conventional in publishing, is considered very bad in software development. This is because software development requires feedback–both to see if customers actually want what you’re building, but also to see if what you’ve built actually works. If you wrote your code in a giant batch, and only merged and shipped it once a year, you could be certain that you’d hit thousands of horrifying bugs. For this reason, modern software practice has relentlessly pushed towards more frequent deploys of smaller changes.
A big part of the time-sink for us was the latency in our QA step; we were passing our big content batch back and forth like a beach ball, dashing across the sand to catch and throw it, but only perfecting one or two items inside the batch each time. How could we avoid the overhead of this back and forth? Maybe it was time to dev up the content process, and break that batch into more manageable chunks.
It would be easier to move around individual bricks than to move around a building made of bricks…
I Think There Might Be a Better Way
I chatted with my teammates about the idea of building a tool to let us preview pieces of content in a realistic context without needing to deploy it to a database for each round of improvements. I wondered if we could break up the batch for the content team so that they could test out changes and immediately review the outcome. While the final module would still ultimately get published as part of a large batch, it would allow the content team to test smaller chunks of work before we initiated the heavyweight publishing process. I hedged a bit on details when I pitched this idea because, although hopeful, I wasn’t sure how feasible this idea would truly be. My teammates were open to the idea, so I set off to see if we could reuse elements of our existing codebase to create a preview tool.
Emily’s Brief Ode to React 💕
At the outset I found myself thinking, as I often do, about the beauty of React. I love the flexible, component-based architecture that it enables. I love that by using functional-style React, a component is just a javascript function that you can reuse at will for any purpose. Thinking about this simple but powerful capability, I wondered if we could achieve a content preview by reusing our existing learning module code in a new dedicated area of the app, feeding in some draft content directly through a UI. I hoped that this would eliminate tedious work to deploy code, and offer the team a self-service option to preview one module at a time.
Digging into the code for our learner experience, I found that there was some unneeded complexity in how content was getting fed in. I streamlined this, allowing just one “source of truth” content blob to be provided. Now it was time to create a new tool. I put our existing code for displaying a learning module onto a new page. I wrapped it with a new layer that would display an input box at the top of the tool for a new draft module to be pasted in. This was truly fun to build because I was challenging myself to keep the new code to a minimum. I took advantage of an existing “preview mode” we already provide to users where they can see a learning module in a read-only mode that disables saving their progress in the module. This flow would work perfectly for the preview tool.
At this point, I was nervous. Was this really going to work? Was I going to make my team happy or have to break it to them that the content process would never get better? I took a draft content blob and pasted it into the new tool and got… a sea of browser errors.
I won’t sugarcoat history for this blog!
But, sure enough, with some small tweaks to my draft content and the underlying learning module code, the real moment of truth arrived: I pasted in the content one final time and saw the title of my draft module pop up on screen. We were in business.
The Grand Reveal
I was ready to show the tool to the team. I was certain that it improved the engineering experience, but I was less sure how the content team would react. Although I was hopeful the tool would smooth things out for them, it still fell wildly short of a first-class content management system, and there was still some amount of copy-and-pasting. But as I met with them and walked through how the tool worked, I could see a sparkle in their eyes. Wait, so we can see the learning module without the devs deploying it? Yes! And no more passing back and forth JSON files each time? That’s right!
Putting the Tool to the Test
We’ve now had a chance to run through the whole content cycle from start to finish with the content preview tool as part of our toolkit. The effect of having the tool was huge, allowing us to cut down our 66-step publishing process to just 11 steps, with self-service content preview along the way. Publishing itself now takes about three days vs. the several weeks that it took in the past, and the content team reports that they’ve noticed a reduction in human error because there’s a lot less copy-and-pasting now.
The content team even reported some additional benefits of the tool that I didn’t anticipate at all:
- There is a lot more experimentation possible during the content development process now. This helps them explore potential improvements to content much more efficiently and purposefully.
- They’ve been able to establish a much clearer timeline for the final publication of modules that helps make the process more easily scalable.
Epilogue: Bravery, Resourcefulness, and a Bias Toward Small Batch Thinking
What makes me proud of this project is that the team went out on a limb and tried something creative even though we weren’t sure that it would work. This required trust and good communication to make sure everyone knew how the exploration was unfolding. There could have come a point where the lift of building a preview tool was becoming too high and we might have called it off, and I think we proceeded very thoughtfully in that regard.
Another takeaway is that improving an existing process with new tooling might sound like a bigger lift than it actually is if you are able to reuse existing code. React gives us the ability to compose components together to create new functionality that would be much harder to achieve if starting from scratch.
Finally, applying “small batch thinking” to our content publishing process played a real part in helping us expand our module catalog by 120% in the last year. This created space for the dev team to simultaneously add a set of features that admins and teachers have been clamoring for, and enabled the content team to focus on envisioning the next set of impactful content to create. When a small change can provide big business value AND make space for a better use of the team’s creative verve, that is absolute magic.
