Introduction to Deep Learning with TensorFlow

Topconf Tallinn 2018

Oliver Zeigermann / @DJCordhose

http://bit.ly/topconf-dl

Example: Customer Data - Risk of Accidents

How would you rank me (47) for a car having 100 mph top speed, driving 10k miles per year?

Programmer's approach: Code Rules by Hand


if age < 25:
    if speed > 140:
        return red # young people, fast cars: high risk
    else:
        return yellow # young people: medium risk
            

if age > 75:
    return red # old people: high risk
            

if miles_per_year > 30:
    return red # a lot of driving: high risk
if miles_per_year > 20:
    return yellow #  a bit of driving: medium risk
            

return green # otherwise: low risk
            

https://colab.research.google.com/github/djcordhose/ai/blob/master/notebooks/sklearn/classic-code.ipynb

Different Approach: Supervised Deep Learning

Keras Layers

Sequential Model


model = keras.Sequential()
        

Fully Connected Hidden Layer


model.add(Dense(units=50, input_dim=3))

Softmax Output Layer


model.add(Dense(units=3, activation='softmax'))
        

https://www.tensorflow.org/api_docs/python/tf/keras/layers https://colab.research.google.com/github/djcordhose/ai/blob/master/notebooks/tensorflow/nn-training.ipynb

How does learning work?

This boils down to an optimization problem

The loss to be minimized is calculated from the difference between the softmax output and the known true category


model.compile(loss='sparse_categorical_crossentropy',
             optimizer='adam')
            

What does the neural network learn?

All the weights of a the neurons


model.summary()

_________________________________________________________________
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
hidden1 (Dense)              (None, 50)                200       
_________________________________________________________________
softmax (Dense)              (None, 3)                 153       
=================================================================
Total params: 353
Trainable params: 353
Non-trainable params: 0
_________________________________________________________________

Architectures of Convolutional Neural Networks: VGG

There are a number of specialized neural network layers

MNIST - Using a model already trained

Exploring the different types layers together

https://transcranial.github.io/keras-js/#/mnist-cnn

Keras CNN Layers

Convolution


    model.add(Conv2D(filters=32, padding='same', activation='relu'))
                

Max Pooling


model.add(MaxPooling2D())
                

Flatten 2d to make it accessible to Dense layers


model.add(Flatten())
            

https://www.tensorflow.org/api_docs/python/tf/keras/layers
https://colab.research.google.com/github/djcordhose/ai/blob/master/notebooks/tensorflow/fashion-mnist-keras.ipynb

More complex architecture: Google Inception V3

Paper: Going Deeper with Convolutions
Why two classifiers?

Keras RNN Layers

Embedding - turning Words into numbers


model.add(Embedding(input_dim=vocab_size, 
                    input_length=text_length, 
                    output_dim=latent_dimension))
        

Optimized RNN Nodes


model.add(GRU(units=32))

https://www.tensorflow.org/api_docs/python/tf/keras/layers https://colab.research.google.com/github/djcordhose/ai/blob/master/notebooks/tensorflow/sentiment-gru.ipynb

Instead of a Wrap Up: What's next?

Finding data sets to play with

Introduction to Deep Learning with TensorFlow
Oliver Zeigermann / @DJCordhose
http://bit.ly/topconf-dl