One Thousand And One Neural Network Nights

Samples from the GPT-2 neural network are generally short – a few paragraphs – because it can only write 3 or 4 paragraph of text in a single sample. (This is vastly better than earlier networks like char-rnn).

I wanted to try out GPT-2 creating a single unbroken sample by feeding each sample into the next over and over again, on the vanilla GPT-2, just to see where it went.

I discovered that the bane of this neural network is a list. With the default 345M model almost every single run ended in an infinite list (Bible verses, Roman Numerals, vaguely sequential numbers.) In between there were a few megabytes of climate speeches, but everything ended in numbers staitons. May do a ‘absurdly long lists’ posts later. But if you need to defeat an evil robot powered by the GPT-2 neural network don’t go with the classic approach of “This statement is a lie.” Start a list because once a neural network stats counting IT CAN NOT STOP.

I still wanted to try a longer sample. One Thousand And One Nights is sort of a single story, sort of a series of short stories. Meandering narratives, asides, stories inside stories – story told by design to never end – it already sounds a lot like what you get out of a neural network! So I began with first paragraph of One Thousand and One Nights.

Continue reading “One Thousand And One Neural Network Nights”

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑