Tensorflow快餐教程(12) - 用机器写莎士比亚的戏剧

0
2
0
1. 云栖社区>
2. 博客>
3. 正文

## Tensorflow快餐教程(12) - 用机器写莎士比亚的戏剧

lusing 2018-05-29 20:34:05 浏览1781

# 高层框架：TFLearn和Keras

## 机器来写莎士比亚的戏剧

from __future__ import absolute_import, division, print_function

import os
import pickle
from six.moves import urllib

import tflearn
from tflearn.data_utils import *

path = "shakespeare_input.txt"
char_idx_file = 'char_idx.pickle'

if not os.path.isfile(path):
urllib.request.urlretrieve("https://raw.githubusercontent.com/tflearn/tflearn.github.io/master/resources/shakespeare_input.txt", path)

maxlen = 25

char_idx = None
if os.path.isfile(char_idx_file):

X, Y, char_idx = \
textfile_to_semi_redundant_sequences(path, seq_maxlen=maxlen, redun_step=3,
pre_defined_char_idx=char_idx)

pickle.dump(char_idx, open(char_idx_file,'wb'))

g = tflearn.input_data([None, maxlen, len(char_idx)])
g = tflearn.lstm(g, 512, return_seq=True)
g = tflearn.dropout(g, 0.5)
g = tflearn.lstm(g, 512, return_seq=True)
g = tflearn.dropout(g, 0.5)
g = tflearn.lstm(g, 512)
g = tflearn.dropout(g, 0.5)
g = tflearn.fully_connected(g, len(char_idx), activation='softmax')
learning_rate=0.001)

m = tflearn.SequenceGenerator(g, dictionary=char_idx,
seq_maxlen=maxlen,
checkpoint_path='model_shakespeare')

for i in range(50):
seed = random_sequence_from_textfile(path, maxlen)
m.fit(X, Y, validation_set=0.1, batch_size=128,
n_epoch=1, run_id='shakespeare')
print("-- TESTING...")
print("-- Test with temperature of 1.0 --")
print(m.generate(600, temperature=1.0, seq_seed=seed))
print("-- Test with temperature of 0.5 --")
print(m.generate(600, temperature=0.5, seq_seed=seed))

pip install tflearn

TFLearn是专门为Tensorflow开发的高层次API框架。

g = tflearn.input_data([None, maxlen, len(char_idx)])
g = tflearn.lstm(g, 512, return_seq=True)
g = tflearn.dropout(g, 0.5)
g = tflearn.lstm(g, 512, return_seq=True)
g = tflearn.dropout(g, 0.5)
g = tflearn.lstm(g, 512)
g = tflearn.dropout(g, 0.5)
g = tflearn.fully_connected(g, len(char_idx), activation='softmax')
learning_rate=0.001)

m = tflearn.SequenceGenerator(g, dictionary=char_idx,
seq_maxlen=maxlen,
checkpoint_path='model_shakespeare')

# Build neural network
net = tflearn.input_data(shape=[None, 6])
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net)

# Define model
model = tflearn.DNN(net)
# Start training (apply gradient descent algorithm)
model.fit(data, labels, n_epoch=10, batch_size=16, show_metric=True)

THAISA:
Why, sir, say if becel; sunthy alot but of
bived with wond I saTt fas,'? You and grigper.

FIENDANS:
By my wordhand!

KING RECENTEN:
Wish sterest expeun The siops so his fuurs,
And emour so, ane stamn.
she wealiwe muke britgie; I dafs tpichicon, bist,
Turch ose be fast wirpest neerenler.

NONTo:
So befac, sels at, Blove and rackity;
The senent stran spard: and, this not you so the wount
hor hould batil's toor wate
What if a poostit's of bust contot;
Whit twetemes, Game ifon I am
Ures the fast to been'd matter:
To and lause. Tiess her jittarss,
Let concertaet ar: and not!
Not fearle her g

PEMBROKE:
There tell the elder pieres,
Would our pestilent shapeing sebaricity. So have partned in me, Project of Yorle
again, and then when you set man
make plash'd of her too sparent
upon this father be dangerous puny or house;
Born is now been left of himself,
This true compary nor no stretches, back that

POLIXENES:
I have unproach the strangest
Or hope not fall-a a cause of banque.

JESSICA:
He that comes to find the just,
And eyes gold, substrovious;
Yea pity a god on a foul rioness, these tebles and purish new head meet again?

20轮之后的结果：

y prison,
Fatal and ominous children and the foot, it will
hear with you: it is my pace comprite
To come my soldiers, if I were dread,
Of breath as what I charge with I well;
Her palace and every tailor, the house of wondrous sweet mark!

STANLEY:
Take that spirit, thou hast
'no whore he did eyes, and what men damned, and
I had evils; by lap, or so,
But wholow'st thy report subject,
And no rassians which he secure
of genslications; when I have move undertake-inward, into Bertounce;
Upon a shift, meet as we are. He beggars thing
Have for it will, but joy with the minute cannot whom we prarem
-- Test with temperature of 0.5 --
y prison,
Fatal and ominous rein here,
The princess have all to prince and the marsh of his company
To prove brother in the world,
And we the forest prove more than the heavens on the false report of the fools,
Depose the body of my wits.

DUKE SENIOR:
The night better appelled my part.

ANGELO:
Care you that may you understand your grace
I may speak of a point, and seems as in the heart
Who be deeds to show and sale for so
unhouses me of her soul, and the heart of them from the corder black to stand about up.

CLAUDIO:
The place of the world shall be married his love.

## 从生成城市名字说起

Zachary
Zafra
Zag
Zahl
Zaleski
Zalma
Zama
Zanesfield
Zanesville
Zap
Zapata
Zarah
Zavalla
Zearing
Zebina
Zebulon
Zeeland
Zeigler
Zela
Zelienople
Zell
Zellwood
Zemple
Zena
Zenda
Zenith
Zephyr
Zephyr Cove
Zephyrhills
Zia Pueblo
Zillah
Zilwaukee
Zim
Zimmerman
Zinc
Zion
Zionsville
Zita
Zoar
Zolfo Springs
Zona
Zumbro Falls
Zumbrota
Zuni
Zurich
Zwingle
Zwolle

from __future__ import absolute_import, division, print_function

import os
from six import moves
import ssl

import tflearn
from tflearn.data_utils import *

path = "US_Cities.txt"
if not os.path.isfile(path):
context = ssl._create_unverified_context()
moves.urllib.request.urlretrieve("https://raw.githubusercontent.com/tflearn/tflearn.github.io/master/resources/US_Cities.txt", path, context=context)

maxlen = 20

X, Y, char_idx = \
string_to_semi_redundant_sequences(string_utf8, seq_maxlen=maxlen, redun_step=3)

g = tflearn.input_data(shape=[None, maxlen, len(char_idx)])
g = tflearn.lstm(g, 512, return_seq=True)
g = tflearn.dropout(g, 0.5)
g = tflearn.lstm(g, 512)
g = tflearn.dropout(g, 0.5)
g = tflearn.fully_connected(g, len(char_idx), activation='softmax')
learning_rate=0.001)

m = tflearn.SequenceGenerator(g, dictionary=char_idx,
seq_maxlen=maxlen,
checkpoint_path='model_us_cities')

for i in range(40):
seed = random_sequence_from_string(string_utf8, maxlen)
m.fit(X, Y, validation_set=0.1, batch_size=128,
n_epoch=1, run_id='us_cities')
print("-- TESTING...")
print("-- Test with temperature of 1.2 --")
print(m.generate(30, temperature=1.2, seq_seed=seed).encode('utf-8'))
print("-- Test with temperature of 1.0 --")
print(m.generate(30, temperature=1.0, seq_seed=seed).encode('utf-8'))
print("-- Test with temperature of 0.5 --")
print(m.generate(30, temperature=0.5, seq_seed=seed).encode('utf-8'))

t and Shoot
Cuthbertd
Lettfrecv
El
Ceoneel Sutd
Sa

stle
Finchford
Finch Dasthond
Wlaycoyarfw

averal
Cape Carteret
Acbiropa Heowar Sor Dittoy
Do

hoenchen
Schofield
Stcojos
Schabell
StcaKnerum Cri

Hill
Cherry Hills Village
Hillfood Pork
Hillbrook

ckitat
Kline
Klondike
Klonsder
Klansburg
Dlandon
D

Branch
Villages of Ocite
Sidaydaton
Sidway
Siddade

Atlasburg
Atmautluak
Attion
Attul
Atta
Aque Creek

tflearn.SequenceGenerator的好处和坏处都是将细节都封装起来了，我们难以看到它的背后发生了什么。

## 温度参数

-- Test with temperature of 1.2 --

Atlasburg
Atmautluak
Attion
Attul
Atta
Aque Creek
-- Test with temperature of 1.0 --

Atlasburg
Atmautluak
Attila
Attaville
Atteville
-- Test with temperature of 0.5 --

Atlasburg
Atmautluak
Attigua
Attinword
Attrove


## 跨后端高层API - Keras，生成尼采的文章

Keras是可以跨Tensorflow，微软的CNTK等多种后端的API。

pip install keras

model = Sequential()

optimizer = RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)

'''Example script to generate text from Nietzsche's writings.
At least 20 epochs are required before the generated text
starts sounding coherent.
It is recommended to run this script on GPU, as recurrent
networks are quite computationally intensive.
If you try this script on new data, make sure your corpus
has at least ~100k characters. ~1M is better.
'''

from __future__ import print_function
from keras.callbacks import LambdaCallback
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import LSTM
from keras.optimizers import RMSprop
from keras.utils.data_utils import get_file
import numpy as np
import random
import sys
import io

path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
with io.open(path, encoding='utf-8') as f:
print('corpus length:', len(text))

chars = sorted(list(set(text)))
print('total chars:', len(chars))
char_indices = dict((c, i) for i, c in enumerate(chars))
indices_char = dict((i, c) for i, c in enumerate(chars))

# cut the text in semi-redundant sequences of maxlen characters
maxlen = 40
step = 3
sentences = []
next_chars = []
for i in range(0, len(text) - maxlen, step):
sentences.append(text[i: i + maxlen])
next_chars.append(text[i + maxlen])
print('nb sequences:', len(sentences))

print('Vectorization...')
x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
for t, char in enumerate(sentence):
x[i, t, char_indices[char]] = 1
y[i, char_indices[next_chars[i]]] = 1

# build the model: a single LSTM
print('Build model...')
model = Sequential()

optimizer = RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)

def sample(preds, temperature=1.0):
# helper function to sample an index from a probability array
preds = np.asarray(preds).astype('float64')
preds = np.log(preds) / temperature
exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)
probas = np.random.multinomial(1, preds, 1)
return np.argmax(probas)

def on_epoch_end(epoch, logs):
# Function invoked at end of each epoch. Prints generated text.
print()
print('----- Generating text after Epoch: %d' % epoch)

start_index = random.randint(0, len(text) - maxlen - 1)
for diversity in [0.2, 0.5, 1.0, 1.2]:
print('----- diversity:', diversity)

generated = ''
sentence = text[start_index: start_index + maxlen]
generated += sentence
print('----- Generating with seed: "' + sentence + '"')
sys.stdout.write(generated)

for i in range(400):
x_pred = np.zeros((1, maxlen, len(chars)))
for t, char in enumerate(sentence):
x_pred[0, t, char_indices[char]] = 1.

preds = model.predict(x_pred, verbose=0)[0]
next_index = sample(preds, diversity)
next_char = indices_char[next_index]

generated += next_char
sentence = sentence[1:] + next_char

sys.stdout.write(next_char)
sys.stdout.flush()
print()

print_callback = LambdaCallback(on_epoch_end=on_epoch_end)

model.fit(x, y,
batch_size=128,
epochs=60,
callbacks=[print_callback])

### 文本生成背后的原理 - 只不过是概率的预测而己

TFLearn的封装做得太好，我们看不到细节。所以我们参看一下Keras的同功能实现的代码:

        for i in range(400):
x_pred = np.zeros((1, maxlen, len(chars)))
for t, char in enumerate(sentence):
x_pred[0, t, char_indices[char]] = 1.

preds = model.predict(x_pred, verbose=0)[0]
next_index = sample(preds, diversity)
next_char = indices_char[next_index]

generated += next_char
sentence = sentence[1:] + next_char

sys.stdout.write(next_char)
sys.stdout.flush()

preds = model.predict(x_pred, verbose=0)[0]

def sample(preds, temperature=1.0):
# helper function to sample an index from a probability array
preds = np.asarray(preds).astype('float64')
preds = np.log(preds) / temperature
exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)
probas = np.random.multinomial(1, preds, 1)
return np.argmax(probas)

## 前情提要

Tensorflow快餐教程(1) - 30行代码搞定手写识别：https://yq.aliyun.com/articles/582122
Tensorflow快餐教程(2) - 标量运算：https://yq.aliyun.com/articles/582490
Tensorflow快餐教程(3) - 向量：https://yq.aliyun.com/articles/584202
Tensorflow快餐教程(4) - 矩阵：https://yq.aliyun.com/articles/584526
Tensorflow快餐教程(5) - 范数：https://yq.aliyun.com/articles/584896
Tensorflow快餐教程(6) - 矩阵分解：https://yq.aliyun.com/articles/585599
Tensorflow快餐教程(7) - 梯度下降：https://yq.aliyun.com/articles/587350
Tensorflow快餐教程(8) - 深度学习简史：https://yq.aliyun.com/articles/588920
Tensorflow快餐教程(9) - 卷积：https://yq.aliyun.com/articles/590233
Tensorflow快餐教程(10) - 循环神经网络: https://yq.aliyun.com/articles/591118
Tensorflow快餐教程(11) - 不懂机器学习就只调API行不行？: https://yq.aliyun.com/articles/594258

lusing
+ 关注