NaNoGenMo 2019 - GPT-2 Edition - Revision history

Chip at 04:39, 30 November 2019

2019-11-30T04:39:08Z

← Older revision		Revision as of 04:39, 30 November 2019
Line 14:		Line 14:
	TextWorld world generations is almost exactly what I want, it's just single-character. Hmmm.		TextWorld world generations is almost exactly what I want, it's just single-character. Hmmm.

			==Github==
			My repo is here: https://github.com/chipmonkey/ChipMonkeyNaNoGenMo2019
			And my NaNoGenMo repo is here: https://github.com/NaNoGenMo/2019/issues/76

	==Local laptop==		==Local laptop==

Chip: NaNoGenMo Final Text

2019-11-30T04:29:32Z

NaNoGenMo Final Text

← Older revision		Revision as of 04:29, 30 November 2019
Line 84:		Line 84:

	But for now, we have writing!		But for now, we have writing!


			=Final Product=

			Ok, here's what happened. Work got busy. HAHA, no, it's always busy. Anyway.

			What follows here is basically the README.md from the github code here:

			Borrowing heavily from these open source products
			* gpt-2: https://github.com/minimaxir/gpt-2-simple
			* textworld: https://github.com/microsoft/TextWorld

			Steps:
			# Run finetune_gpt2.py to train GPT2 on Grimm's fairy tales
			# Run make_a_novel.py

			make_a_novel.py generates a story with TextWorld and walks through
			a play run (that part isn't fully automated yet).
			It then takes the narrative from the playthrough and passes it through
			gpt2 to add lots of words, which are hopefully embellishments on the story
			thus generating a novel.

			It's not very coherent yet, but ya know.

			As a shortcut, if you have a gpt-2 model, and want to use the example text you should be able to just do:

			<nowiki>
			import make_a_novel
			make_a_novel.makethenovel()
			</nowiki>

			which will load a model available to gpt-2-simple and generate text from a pre-played TextWorld output storyguide.

			The result of doing that, at least one example, is my NaNoGenMo 2019 submission:
			[[NaNoGenMo2019 Final Text]]

Chip: add v1

2019-11-10T23:07:54Z

add v1

@@ Line 53: / Line 53: @@
 hit this bug: https://github.com/authomatic/chromedriver_installer/issues/11
   <nowiki>
-pip install textworld
+pip install textworld</nowiki>
 </nowiki>

Chip: /* gpt-2 */ it worked

2019-11-10T06:54:44Z

gpt-2: it worked

← Older revision		Revision as of 06:54, 10 November 2019
Line 28:		Line 28:
	docker build --tag gpt-2 -f Dockerfile.cpu .		docker build --tag gpt-2 -f Dockerfile.cpu .
	docker run -it gpt-2 bash		docker run -it gpt-2 bash
	export PYTHONIOENCODING=UTF-8		export PYTHONIOENCODING=UTF-8</nowiki>
	</nowiki>

	Need to run this on AWS or something:		Need to run this on AWS or something:
Line 38:		Line 37:
	2019-11-09 20:43:44.915623: W tensorflow/core/framework/allocator.cc:122] Allocation of 18137088 exceeds 10% of system memory.		2019-11-09 20:43:44.915623: W tensorflow/core/framework/allocator.cc:122] Allocation of 18137088 exceeds 10% of system memory.
	2019-11-09 20:43:45.006514: W tensorflow/core/framework/allocator.cc:122] Allocation of 18210816 exceeds 10% of system memory.		2019-11-09 20:43:45.006514: W tensorflow/core/framework/allocator.cc:122] Allocation of 18210816 exceeds 10% of system memory.
	2019-11-09 20:43:45.102333: W tensorflow/core/framework/allocator.cc:122] Allocation of 18284544 exceeds 10% of system memory.		2019-11-09 20:43:45.102333: W tensorflow/core/framework/allocator.cc:122] Allocation of 18284544 exceeds 10% of system memory.</nowiki>
	</nowiki>
			Although it did actually spit out text, so for now I'm working inside this docker.

	===Parl.AI===		===Parl.AI===

Chip at 02:01, 10 November 2019

2019-11-10T02:01:19Z

← Older revision		Revision as of 02:01, 10 November 2019
Line 12:		Line 12:
	====progress...====		====progress...====
	So there are some good generators out there that sort of start to do what I want. I may just daisy chain them together... listed a few below.		So there are some good generators out there that sort of start to do what I want. I may just daisy chain them together... listed a few below.
			TextWorld world generations is almost exactly what I want, it's just single-character. Hmmm.


Line 26:		Line 27:
	git clone https://github.com/openai/gpt-2.git && cd gpt-2		git clone https://github.com/openai/gpt-2.git && cd gpt-2
	docker build --tag gpt-2 -f Dockerfile.cpu .		docker build --tag gpt-2 -f Dockerfile.cpu .
			docker run -it gpt-2 bash
			export PYTHONIOENCODING=UTF-8
	</nowiki>		</nowiki>

Chip: formatting

2019-11-09T20:48:34Z

formatting

← Older revision		Revision as of 20:48, 9 November 2019
Line 1:		Line 1:
	[The Basics]		=The Basics=
	My goal this year was to take some random text, character, and location generators and build a basic recursive quest engine.		My goal this year was to take some random text, character, and location generators and build a basic recursive quest engine.

Line 10:		Line 10:
	Eh, maybe that's even too much. I work a super lot and in eight days of November so far all I've done is written those bullets. But let's see what's out there...		Eh, maybe that's even too much. I work a super lot and in eight days of November so far all I've done is written those bullets. But let's see what's out there...

	~~[[[~~progress...~~]]]~~		====progress...====
	So there are some good generators out there that sort of start to do what I want. I may just daisy chain them together... listed a few below.		So there are some good generators out there that sort of start to do what I want. I may just daisy chain them together... listed a few below.


	[Local laptop]		==Local laptop==

	[[gpt-2]]		===gpt-2===
	This: https://openai.com/blog/better-language-models/		This: https://openai.com/blog/better-language-models/
	Made this: https://github.com/openai/gpt-2		Made this: https://github.com/openai/gpt-2
Line 38:		Line 38:
	</nowiki>		</nowiki>

	[[Parl.AI]]		===Parl.AI===
	<nowiki>		<nowiki>
	git clone https://github.com/facebookresearch/ParlAI.git		git clone https://github.com/facebookresearch/ParlAI.git
Line 44:		Line 44:
	</nowiki>		</nowiki>

	[[textworld]]		===textworld===
	From: https://www.microsoft.com/en-us/research/project/textworld/		From: https://www.microsoft.com/en-us/research/project/textworld/
	Yields: https://github.com/microsoft/textworld		Yields: https://github.com/microsoft/textworld

Chip: added gpt2, Parl.AI, textworld

2019-11-09T20:46:05Z

added gpt2, Parl.AI, textworld

@@ Line 1: / Line 1: @@
 [The Basics]
 This: https://openai.com/blog/better-language-models/
 Made this: https://github.com/openai/gpt-2
-[Local laptop]
+hit this bug: https://github.com/openai/gpt-2/issues/178
   <nowiki>
 git clone https://github.com/openai/gpt-2.git && cd gpt-2
 docker build --tag gpt-2 -f Dockerfile.cpu .
 </nowiki>
@@ Line 15: / Line 42: @@
 git clone https://github.com/facebookresearch/ParlAI.git
 cd ParlAI; python setup.py develop
 </nowiki>

Chip: Starting point

2019-11-09T19:21:32Z

Starting point

New page

[The Basics]
This: https://openai.com/blog/better-language-models/
Made this: https://github.com/openai/gpt-2

[Local laptop]

[[gpt-2]]
<nowiki>
git clone https://github.com/openai/gpt-2.git && cd gpt-2
docker build --tag gpt-2 -f Dockerfile.cpu .
</nowiki>

[[Parl.AI]]
<nowiki>
git clone https://github.com/facebookresearch/ParlAI.git
cd ParlAI; python setup.py develop
</nowiki>