Commit b467aeab authored by Andrej Benjak's avatar Andrej Benjak
Browse files

ML Rmd

parent a12998e8
Pipeline #36670 passed with stage
in 52 seconds
This diff is collapsed.
title: "Basic Keras example"
author: "Panagiotis Papasaikas"
date: "2020-05-29"
css: styles.css
keep_md: true
toc: true
toc_depth: 2
toc_float: false
theme: sandstone
highlight: tango
chunk_output_type: console
```{r setup, include=FALSE, class.source = "rchunk"}
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, eval = TRUE)
options(width = 80)
# Summary
Here we will illustrate how to install `keras` and the required **R** and **Python** package dependencies.
Next we will demonstrate the implementation of a simple Multilayer Perceptron (MLP) model with
both R and Python `keras` including **model definition**, **compilation** and **training**.
Finally we will quickly contrast the **sequential** vs the **functional** `keras` APIs for model definition.
# Keras installation in R
For extensive documentation on the R `keras` installation see:
["Install Keras and the TensorFlow backend"](
!!! **Do not execute the following code. This is only for demontration purposes ** !!
```{r eval=FALSE}
install_keras(method="conda", tensorflow="2.1.1") #
#installation with GPU support:
install_keras(method="conda", tensorflow="gpu") # OR
install_keras(method="conda", tensorflow="2.1.1-gpu")
# (NOTE: only do this if you have an NVIDIA GPU + CUDA!)
If you wish to add additional `PyPI` packages to your *Keras* / *TensorFlow* environment you can either specify the packages in the `extra_packages` argument of `install_keras()`, or alternatively install them into an existing environment using the `reticulate::py_install()` function, eg:
!!! **Do not execute the following code. This is only for demontration purposes ** !!
```{r eval=FALSE}
# Mutilayer Percetron model for MNIST digit classification using the **R** `keras`
At this point all our **R** and `keras` package requirements are in place so we are ready to build a first `keras` model!
First we need to load the `keras` library:
```{r libload}
## check if Python is available and has already been initialized
## dicsover Python that provides "keras"
#reticulate::py_discover_config(required_module = "keras")
## For setup on a renku environment:
## Preparing the data:
```{r MLP}
#loading the keras inbuilt mnist dataset
#separating train and test file
# flattening the 2D featuresinto 1D for feeding into the MLP and normalizing the matrix
train_x <- array(train_x, dim = c(dim(train_x)[1], prod(dim(train_x)[-1]))) / 255
test_x <- array(test_x, dim = c(dim(test_x)[1], prod(dim(test_x)[-1]))) / 255
#converting the target variable to one hot encoded vectors using keras inbuilt function
train_y <- to_categorical(train_y,10)
test_y <- to_categorical(test_y,10)
## Model definition
The sequential model is a linear stack of layers.
You create a sequential model by calling the `keras_model_sequential()` function and then a series of layer functions.
Each layer also needs to know what input shape it should expect. For this reason, the first layer in a sequential model (and only the first, because following layers can do automatic shape inference) needs to receive information about its input shape.
As illustrated in the example below, this is done by passing an input_shape argument to the first layer.
#defining a keras sequential model
model <- keras_model_sequential()
#defining the model with 1 input layer[784 neurons], 1 hidden layer[784 neurons] with dropout rate 0.4 and 1 output layer [10 neurons]
#i.e digits from 0 to 9
model %>%
layer_dense(units = 784, input_shape = 784, activation = 'relu') %>%
layer_dropout(rate=0.4) %>%
layer_dense(units = 10,activation = 'softmax')
## Compilation
We are now going to compile the specified model. There are two required arguments that need to be specified
for model compilation: The **loss function** and the **optimizer**.
Whereas the *loss function* specifies our training objective the *optimizer* specifies the specific algorithmic machinery
During compilation we can also specify other (built-in or custom) function *metrics* we wish to keep track of during training.
#compiling the defined model with metric 'accuracy' and optimiser 'adam'
model %>% compile(
loss = 'categorical_crossentropy',
optimizer = 'adam',
metrics = c('accuracy')
## Training
Training is typically done with the fit function. Fit has a few required arguments:
The (compiled) model to be trained, the training and target data.
Other important (optional) arguments for fit is the number of training epochs, the batch size a list of validation data as well as a list of callbacks (objects that perform different actions at various steps of training ).
We will familiarize ourselves with several of these `fit` arguments in the second, more extensive, exercise.
#fitting the model on the training dataset
history <- model %>% fit(train_x, train_y, epochs = 50, batch_size = 3000, validation_data=list(test_x,test_y))
# Mutilayer Percetron model for MNIST digit classification using the **Python** `keras`
We now demonstrate the implementation of the exact same model in **Python** `keras`.
As you will see everything from model definition to compilation and training looks extremely similar.
```{python, class.source="pythonchunk"}
#importing the required libraries for the MLP model
import keras
from keras.models import Sequential
import numpy as np
#loading the MNIST dataset from keras
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
#reshaping the x_train, y_train, x_test and y_test to conform to MLP input and output dimensions
import pandas as pd
#performing one-hot encoding on target variables for train and test
#defining model with one input layer[784 neurons], 1 hidden layer[784 neurons] with dropout rate 0.4 and 1 output layer [10 #neurons]
from keras.layers import Dense
model.add(Dense(784, input_dim=784, activation='relu'))
#compiling the defined model with metric 'accuracy' and optimiser 'adam'
model.compile(loss='categorical_crossentropy', optimizer="adam", metrics=['accuracy'])
# fitting model and performing validation,y_train,epochs=50,batch_size=1024,validation_data=(x_test,y_test), verbose=0)
import matplotlib.pyplot as plt
# summarize history for accuracy
plt.title('model accuracy')
plt.legend(['train', 'test'], loc='upper left')
# summarize history for loss
plt.title('model loss')
plt.legend(['train', 'test'], loc='upper left')
Plotting the model:
```{python, class.source="pythonchunk", eval=FALSE}
import keras
import pydotplus
from keras.utils.vis_utils import model_to_dot
keras.utils.vis_utils.pydot = pydot
dpi=96 )
# Functional API
The `Keras` functional API is a more flexible/modular way for mode definition which makes it more appropriate for more complex models with non-linear topology such as multi-input/output models or models with shared layers.
Remember that the three key objects of the `Keras`API (but also of the underlying backends such as `Tensorflow`) are *Tensors*, *Layers* and *Models* .
The main idea that a deep learning *model* is usually a directed acyclic graph (DAG) of *layers* connected by *tensor* edges.
The functional API is a more explicit way to construct this DAG by specifying the *layers* nodes of the graph that perform operations
on the *tensors* that "flow" through the graph's edges --> *Tensor*-*flow*! 😊.
This modular specification of the DAG design is exactly what makes it so flexible.
Let's see how we would define the previous MLP model using the Functional API:
# input layer
inputs <- layer_input(shape = c(784))
#The inputs tensor that is returned contains information about the shape and dtype of the input data that you feed to your model. Here's the shape and dtype:
# We now create new nodes in the graph of layers by calling a layer on this inputs object:
predictions <- inputs %>%
layer_dense(units = 784, activation = 'relu') %>%
layer_dropout(rate=0.4) %>%
layer_dense(units = 10, activation = 'softmax')
# Until this point we have not instantiated a model. We have just specified a graph.
# Now we instantiate the model by specifying its inputs and outputs in the graph of layers
model_f <- keras_model(inputs = inputs, outputs = predictions)
With the functional API, it is easy to reuse trained models: you can treat any model as if it were a subgraph that you can reuse
as a building block of another model.
Note that you aren’t just reusing the architecture of the model, you are also reusing its weights.
# Useful links
**General tutorials and collection of examples models:**
**Articles on the functional API:**
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment