View source for NaNoGenMo 2019 - GPT-2 Edition

=The Basics=
My goal this year was to take some random text, character, and location generators and build a basic recursive quest engine.

Let's unpack that.  All I want for now is:
* locations - a small network of places, with the ability for characters to travel between them
* characters - a recurring finite set of characters who travel between the locations and interact when they're in the same place
** stretch goal - interact when they're passing one another!
* recursive - the idea is that we have a world... it has, say, cities, within cities are places (inns, bars, markets), within those may be rooms... or maybe our recursion is deep enough at two levels

Eh, maybe that's even too much.  I work a super lot and in eight days of November so far all I've done is written those bullets.  But let's see what's out there...

====progress...====
So there are some good generators out there that sort of start to do what I want.  I may just daisy chain them together... listed a few below.
TextWorld world generations is almost exactly what I want, it's just single-character.  Hmmm.


==Local laptop==

===gpt-2===
This: https://openai.com/blog/better-language-models/
Made this: https://github.com/openai/gpt-2
Which made this: https://colab.research.google.com/drive/1gB03iSnshYcSzSCrS9gPGcoOFCpMhVq_

hit this bug: https://github.com/openai/gpt-2/issues/178

 <nowiki>
git clone https://github.com/openai/gpt-2.git && cd gpt-2
docker build --tag gpt-2 -f Dockerfile.cpu .
docker run -it gpt-2 bash
export PYTHONIOENCODING=UTF-8</nowiki>

Need to run this on AWS or something:
 <nowiki>
2019-11-09 20:43:18.919955: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-09 20:43:23.418088: W tensorflow/core/framework/allocator.cc:122] Allocation of 154389504 exceeds 10% of system memory.
2019-11-09 20:43:23.722155: W tensorflow/core/framework/allocator.cc:122] Allocation of 154389504 exceeds 10% of system memory.
2019-11-09 20:43:44.915623: W tensorflow/core/framework/allocator.cc:122] Allocation of 18137088 exceeds 10% of system memory.
2019-11-09 20:43:45.006514: W tensorflow/core/framework/allocator.cc:122] Allocation of 18210816 exceeds 10% of system memory.
2019-11-09 20:43:45.102333: W tensorflow/core/framework/allocator.cc:122] Allocation of 18284544 exceeds 10% of system memory.</nowiki>

Although it did actually spit out text, so for now I'm working inside this docker.

===Parl.AI===
 <nowiki>
git clone https://github.com/facebookresearch/ParlAI.git
cd ParlAI; python setup.py develop
</nowiki>

===textworld===
From: https://www.microsoft.com/en-us/research/project/textworld/
Yields: https://github.com/microsoft/textworld

hit this bug: https://github.com/authomatic/chromedriver_installer/issues/11
 <nowiki>
pip install textworld</nowiki>



==Attempt 1==

Textworld is really straighforward and has a nice generator... the example from the README.md is:
 <nowiki>tw-make custom --world-size 5 --nb-objects 10 --quest-length 5 --seed 1234 --output tw_games/custom_game.ulx</nowiki>
Or
 <nowiki>tw-make custom --world-size 15 --nb-objects 20 --quest-length 20 --seed 1729 --output tw_games/game2.ulx</nowiki>


Also, it's a quick modification to the sample "play_gym.py" to the environment https://textworld.readthedocs.io/en/latest/textworld.envs.wrappers.html
Just open an outfile <code>outfile = open('textworld_out.txt', 'w+')</code> and drop the environment to it via text <code>outfile.write(env.render('text'))</code>
So that a playthrough drops a nice quest line with some descriptive information.

After that, it's a fairly easy change to https://github.com/openai/gpt-2/blob/master/src/interactive_conditional_samples.py to read from an input file rather than the command line.
Point that at the output from TextWorld, and GPT-2 will embellish a story based on each line of the TextWorld adventure.

Voila!

[[NaNoGenMo2019 Attempt 1]]


==Next Steps==

GPT-2 is all over the place.  Need to dig in to focus on specific datasets; retrain on gutenberg-fantasy-english.
Also, TextWorld seems broken.  Can't quite figure out how to win even a very simple quest which makes me wonder if it's generating winnable quest trees.
Lastly, still need to add multiple people.

But for now, we have writing!