"Structure" in Computer-Generated Novels

"Welcome! I think you'll find that writing code that writes a book that is not-boring to read for the first few hundred words is not too difficult.
After those first few hundred words, though... well, all I can suggest is you download one of the completed novels (from this year or from any earlier year) and try to read the whole thing. The word "boring" does not quite do justice to the experience."---Chris Pressey

Every November is "National Novel Generation Month" (NaNoGenMo), a competition inspired by the more famous NaNoWriMo. In NaNoGenMo, the goal is to write computer code that can write a novel of 50,000 words or more. It started in 2013 and the only rules are that you must provide a copy of your novel and the source code to qualify.

On one hand, the task of NaNoGenMo can be seen as pretty easy. Here's Ruby code for a "novel":

puts "meow" * 50000

But, good luck trying to read this "meow" novel without falling asleep. It is very simple to indefinitely scale upwards, but people can end up getting bored with the resulting output. Infinite text generation is boring on its own, and it might just be better to focus on creating an algorithm that can generate a few "great" short stories instead of trying to produce one boring and tedious novel.

But some NaNoGenMo programmers see this state of affairs as a challenge. Can humans write an algorithm that can generate a fully "human-readable" novel? This task can be difficult, but does not seem technically impossible. Many NaNoGenMo programmers have attempted this task, and has produced okay-ish output. Progress can be made in the field of novel generation.

And while novel generation may not be used to generate "NYT best-seller-status" novels in the near-future, knowledge here could then be used to generate interesting short-form literature suitable for use in video games and content mills. The field itself is also interesting for me since it gives me a chance to understand the limitations, dangers, and potentials of AI as a whole (as writing stories is generally assumed to require "intelligence").

I do plan on blogging about the various techniques that can be used to make NaNoGenMo novels interesting. However, I do not want to write a very long and tiresome blog post that will make my "meow" novel look exciting by comparison. So this blog post will only briefly touch on three different techniques, based on the idea of providing "structure" to a novel (in the hopes that said 'structure' makes the story more human-readable).

Human-generated literature tends to have structure that serves to justify the existence of a story. This structure can be as vague as a simple outline to complex frameworks such as the "Hero's Journey". Structure provides "order" to a story, making it more appealing to consume (since the story seems more "meaningful"). Structure can be provided through three techniques: Simulations, Story Compilers, and Frame Stories.

Simulations

The idea behind simulations seem simple. Reality has great stories, so by simulating reality, you can also see great stories arise as well. The game "Dwarf Fortress" tends to be brought up as an example of the success of simulation. To start a game, you first generate an entire world, with every civilization, place, and character having its own simulated history. You can browse those the generated histories to find relationships between the different aspects of the generated world, and be entertained (without ever needing to actually play the game itself).

Other simulations have also been used in NaNoGenMo to generate novels:

Around the World In X Wikipedia Articles is a novel about two people traveling around the world and visiting different locations
Simulationist Fantasy Novels are computer-generated novels about a party of fantasy adventurers on a quest to destroy an evil MacGuffin
A Daring Journey to the Bottom of the Pit is a novel about a hero going on a quest to find a new MacGuffin

However, simulations do have their weaknesses. Other than the sheer difficulty of planning a simulation, there's also the worry that the output of simulations themselves can be dull to read. According to John Ohno:

Simulation is definitely a way to produce 'plot' in the sense of events that follow each other with internal consistency, causality, and logic, but it doesn't in any way filter for what would be interesting to a reader. Most things that happen in reality (or any procedural simulation of a subset of reality) is not very interesting from the perspective of a reader of fiction, even things that are potentially very interesting to watch (it's one thing to watch a kung-fu movie, but reading a pokemon-style transcript of the same film's fight scenes would be incredibly boring).

Dwarf Fortress is yet another example where simulation can fail in "filtering" out uninteresting stuff. For every interesting story about an elf taking over a Dwarf Kingdom, there are thousands of dull stories about one dwarf deciding to change his profession from "fish cleaner" to "blacksmith cleaner".

Possible approaches to improving simulations could be to improve the text that is generated during a simulation (reading beautiful paragraphs explaining why a dwarf has sadly decided to change jobs can make such an event just as engaging as the rare and unique events that can happen in the game), to output as much detail in your simulation as possible (providing detailed job descriptions of "blacksmith cleaners" and "fish cleaners" could make both jobs more interesting to read about) and to narrow the focus of a simulation to only focus on important aspects (if you think stories about the career advancement of a dwarf are boring, then don't model it in your simulation).

Story Compiler

Chris Pressey, in generating the NaNoGenMo novel "A Time For Destiny", used a "story compiler" approach to organize his novel. Essentially, the program would first generate a plausible outline for a short story. Then the computer will use the outline as a basis for further text generation. In this way, the program is able to establish a coherent interesting plot while also filtering out uninteresting ideas. The outline itself would then be further refined it with additional steps. (The generated novel itself would be a series of short stories that are unified by an overarching "plot".)

This approach was also useful for adding additional "authorial techinques" that a simulation-only approach couldn't do. Chris Pressey was able to implement "Chekov's Gun" successfully in the "story compiler" - if a story requires the existence of Chekov's Gun, write the gun, and then insert a description of the gun at the beginning and the end of the story. This type of approach is notably not possible in a simulation, because, according to Chris:

[Y]ou can't just output events as they happen. You have to go back and examine them & edit them selectively. So you have to keep the events around in some kind of data structure. This is what led to the compiler-like architecture.

Chris Pressey's work is likely the most 'successful' computer-generated novel, at least according to Greg Kennedy...

I read three full chapters before skimming. After the first two I was preparing to skip forward, but then Nebulon showed up, and things got interesting again!

...but after that I wondered what other plot events might happen, so I started scrolling...

so I guess that's 2770 words read. Not bad.

For comparison's sake, Chris Pressey had a goal of trying to get people to read ~1500 words. The biggest problem that Chris had to worry about was scaling up the system up to 50,000 words...he had to develop not only the events that the story compiler uses, but also all the 'steps' necessary to process those events into actual text. While it is not difficult, it does require effort, and he could not complete this task in November. Chris commented:

Also, there's a sense in which I should've spent the final week adding as many new subplot choices as I could, so maybe you'd see "4. Stranded in Space" after reading the 3rd chapter, and maybe you'd read even more words, and... yeah, I probably should've ... But I was so tired of reading the escapades of Serenity Starlight by the time I released this (I'm sure I've read 50,000 words of it myself), and being sick didn't exactly help matters either. I will be content at it being a proof-of-concept for this approach to that goal.

Frame Stories

Perhaps the easiest way to provide structure to a story is to "fake it". If you provide a few words to establish a "frame story" that justifies why various passages are right next to each other, then the humans reading the story will believe that there must be some structure in the story and will be able to consume it based on this structure. This is an especially fruitful course of action since it doesn't require much code to be written at all. All that requires is some creativity on the part of the programmer to 'convince' the readers.

This type of approach is used by Martin O' Leary in his NaNoGenMo novel The Deserts of the West, which is a travelogue of a fantasy world. Each entry in the novel is just a description of a city. However, Martin tends to provide clues that implies that the author is actually traveling from one city to another, and also provide procedurally-generated maps of the cities in question. Thus, a person will be tempted to read the content properly to learn more about the various fantasy cities.

It's not just limited to fantasy cities. This can be applied to:

A "frame story" can even be induced by providing an illusion of continuity within the broader work. The Generative {{GIANT}} Who had No Heart In His Body is a series of short stories about a Prince who is fighting against an evil Giant. At the end of each story, the Prince defeats the Giant, only to turn into another Giant. Then a new Prince arise, to fight the new evil Giant. And the cycle repeats endlessly for over 60,000 words.

A "frame story" approach can be promising but it may fail in keeping the reader engaged throughout the whole novel, at least according to Chris Pressey:

I don't think the "many small works" strategy is actually very successful at side-stepping the "boredom problem"... if anything, it might be even harder to read all the way through a list of 500 generated recipes, than through a single generated story of the same length.

In other words, the whole ("the novel") may be less than the sum ("hundreds of short stories"). The short stories may be better off as 'stand-alone pieces' to be consumed one at a time, then to be smashed together and thinly justified with an excuse "frame story". There is, however, a benefit to using "frame stories"...it makes the story generator itself more flexible and avoids possible repetition that can also make a novel boring to read, at least according to John Ohno:

Many small works does, however, counteract the problem of trying to bake in internal consistency and continuity -- and keeping consistency and continuity forces generators to be simplified or constrained in ways that can produce more boring text at novel length. So, having an anthology format can indirectly help in preventing the novel from being boring or formulaic, by explaining away the existence of novelty sources that outside an anthology format might be seen as confusing or an error (in the same way that explicitly nonchronological stories, extremely abstract and vague prose, swapping out narrators, or having unreliable narrators can).

Conclusion

Simulations, Story Compilers, and Frame Stories are very useful approaches to organizing computer-generated novels, making them more "bearable" for human consumption. Obviously, there are other techniques (which I will get to discussing in future blog posts, if I ever get time). And you can always use multiple approaches, in the hopes of producing even more interesting output. More work is necessary though to improve machine capabilities. Machines may not produce literature on-par with humans...yet. But machine capabilities are slowly increasing, as programmers realize the strengths and limits of their tools.

Appendix

By the way, each blog post that I write on this topic of computer-generated literature will be computer-generated, including this one. (Humans have always been writing about AI for so long that it sounds pretty silly. It's better to program AI to write about itself.) You can view the source code used to generate this specific blog post right here, formatted in Markdown. The structure for this specific blog post is simple enough to automate: an introduction section, three bulletpoint sections (each bulletpoint focusing on a different technique), and a final conclusion section to wrap things up. The sections are only unified together through a "frame story" (specifically, the format of a normal blog post). It's a very basic algorithm, and by no means "strong AI". But that's the point. If this is what I can do right now in an hour, imagine what future programmers will do.