Yesterday we introduced you to the latest project at Bitjam: ANNA. A learning algorithm that analyses poetry and audibly delivers it in an old-fashioned regional Potteries dialect, the project has been inspired by the bid for Stoke-on-Trent as the next UK City of Culture. We interviewed Liam Mountford, Senior Developer at Bitjam, about the technical elements of creating artificial intelligence (AI) that is able to independently learn.
Why did you have the idea to create a computerised system that creates poetry and delivers it in a potteries dialect?
The initial idea came about from a discussion between me and Carl. Carl knew I had an interest in machine learning from the work I did for my dissertation, I created a neuro-evolutionary algorithm that was to predict the results of the 2015 general election. I have had a number of discussions about neural networks and similar machine learning and AI algorithms with Carl, where I tended to ramble on about some of the more interesting aspects such as trying to create a Neural Network (NN) that could produce pieces of art. Carl liked the sound of a creative learning machine and formed the idea of using it to create potteries poetry.
From a more technical point of view, how does ANNA actually work?
Once we had our idea we sat down and had a discussion about the possible complexity of the project. A fully fledged bespoke neural network is quite a lot of work so we decided to try to find some existing neural networks to base our work off of. With the help of Jacob we found a Recursive Neural Network (RNN) designed to take text input and after a large number of training cycles we then tried to get ANNA to output some meaningful ‘learned’ poetry.
So, tell us more about neural networks.
In it’s simplest form a NN consists of 3 layers. The input layer accepts various inputs. In theory, as many or as few as you need. Next the hidden layer, this is the heart and soul of a NN and helps to create the correlating links between the various inputs supplied at the input layer. Finally the output layer, is the part of the NN that spits out – hopefully! – some sort of meaningful information. Where a RNN differs is that there is a recursive link between the output and hidden layer, this part is important as it helps the RNN to learn the structure of the language supplied. In turn this means that the particularly interesting part with the RNN is that it is able to learn dialects such as potteries.
How does ANNA learn poetry?
ANNA is a python script based on a simple RNN, we feed in around 200 pages worth of potteries dialect poetry aiming to produce some sort of meaningful poetry. Anna runs through about 500 recursive cycles of the input text per “epoch” of learning for a total of around 30 epochs. An epoch is essentially a single full training cycle.
What were the challenges of this project?
One problem that always rears it’s head is a shortage of data to feed into the RNN. So the next step for us was to source plenty of potteries dialect based poetry from poets past and present. First we tried to source as much poetry as possible from an online source, the main works we used were by Arnold Bennett. We then tried sourcing further poetry from Wilfred Bloor‘s sons Roger & Ian. Wilfred Bloor wrote over 400 Jabez tales in Potteries dialect (the Jabez character is a countryman living in the shadows of industrial Potteries). We also spoke to Alan Barrett, Stoke-born writer, storyteller, poet, and actor. Thanks to the kind contributions of these people we managed to collect plenty of poetry that has been fed into ANNA.
Come and see ANNA in action at the ACAVA Studios: Spode Works open studios event on Saturday 7th / Sunday 8th October from 12pm at Bitjam, studio number 22.