Static HTML notebook found here: Pluto
Use the above link for interactive charts!
My task for this research is to explore JuliaLang and Flux.jl through experiments on the ubiquitous data-set known as 'the iris data-set'.
Data Summary
Data set: iris
This data set contains 150 samples iris flower. The features in each sample are the length and width of both the iris petal and sepal, and also the species of iris. data = 150×5
Each feature is recorded as a floating point value except for the species (string). The species identifier acts as the labels for this data set (if used for supervised learning). There are no missing values. The data and header is separated into two different files.
This data could be used for iris classification. This could be useful in an automation task involving these flowers or as a tool for researchers to assist in quick identification. Other, less "real world" applications include use as a data set for ML systems such as supervised learning (NN) and unsupervised learning (K-NN).
Imports
begin
import Pkg;
packages = ["CSV","DataFrames","PlutoUI","Plots","Combinatorics"]
Pkg.add(packages)
using CSV, DataFrames, PlutoUI, Plots, Combinatorics
plotly()
theme(:solarized_light)
end
Loading, cleaning, and manipulating the data
begin
path = "iris/iris.data"
csv_data = CSV.File(path, header=false)
iris_names = ["sepal_len", "sepal_wid", "petal_len", "petal_wid", "class"]
df = DataFrame(csv_data.columns, Symbol.(iris_names))
dropmissing!(df)
end
Splitting the data into three iris classes
As you can see, there is a equal representation of each class:
begin
df_species = groupby(df, :class)
end
Class sizes: (50, 5), (50, 5) (50, 5)
Visualizations
Comparing length vs width of the sepal and petal
begin
scatter(title="len vs wid", xlabel = "length", ylabel="width",
df.sepal_len, df.sepal_wid, color="blue", label="sepal")
scatter!(df.petal_len, df.petal_wid, color="red", label="petal")
end
Comparing all combinations of variables
Column pairs per chart: [sepal_len, sepal_wid, petal_len, petal_wid, class]
-> [1, 2] , [1, 3] , [1, 4]
-> [2, 3] , [2, 4] , [3, 4]
begin
# Get all combinations of colums
combins = collect(combinations(1:4,2))
combos = [(df[x][1], df[x][2]) for x in combins]
# Plot all combinations in sub-plots
scatter(combos, layout=(2,3))
end
Comparing the sepal length vs sepal width vs petal length of all three classes of iris
Restricted to three variables to plot in 3d
begin
setosa, versicolor, virginica = df_species
scatter(setosa[1], setosa[2], setosa[3], label="Setosa", xlabel="d")
scatter!(versicolor[1], versicolor[2], versicolor[3], label="versicolor")
scatter!(virginica[1], virginica[2], virginica[3], label="virginica")
end
[3] Deep Learning
Imports
begin
Pkg.add("Flux")
Pkg.add("CUDA")
Pkg.add("IterTools")
using Flux
using Flux: Data.DataLoader
using Flux: @epochs
using CUDA
using Random
using IterTools: ncycle
Random.seed!(123);
end
The Data
Formating data for training (including onehot conversion)
begin
# Convert df to array
data = convert(Array, df)
# Shuffle
data = data[shuffle(1:end), :]
# train/test split
train_test_ratio = .7
idx = Int(floor(size(df, 1) * train_test_ratio))
data_train = data[1:idx,:]
data_test = data[idx+1:end, :]
# Get feature vectors
get_feat(d) = transpose(convert(Array{Float32},d[:, 1:end-1]))
x_train = get_feat(data_train)
x_test = get_feat(data_test)
# One hot labels
# onehot(d) = [Flux.onehot(v, unique(df.class)) for v in d[:,end]]
onehot(d) = Flux.onehotbatch(d[:,end], unique(df.class))
y_train = onehot(data_train)
y_test = onehot(data_test)
end
Creating DataLoaders for batches
begin
batch_size= 1
train_dl = DataLoader((x_train, y_train), batchsize=batch_size, shuffle=true)
test_dl = DataLoader((x_test, y_test), batchsize=batch_size)
end
The Model
I am going to implement a fully connected neural network to classify by species.
- Layers: Chain(Dense(4, 8, relu), Dense(8, 3), softmax)
- Loss: logit binary crossentropy
- Optimizer: Flux.Optimise.ADAM
- Learning rate: 0.001
- Epochs: 30
- Batch size: 1
Training
begin
### Model ------------------------------
function get_model()
c = Chain(
Dense(4,8,relu),
Dense(8,3),
softmax
)
end
model = get_model()
### Loss ------------------------------
loss(x,y) = Flux.Losses.logitbinarycrossentropy(model(x), y)
train_losses = []
test_losses = []
train_acces = []
test_acces = []
### Optimiser ------------------------------
lr = 0.001
opt = ADAM(lr, (0.9, 0.999))
### Callbacks ------------------------------
function loss_all(data_loader)
sum([loss(x, y) for (x,y) in data_loader]) / length(data_loader)
end
function acc(data_loader)
f(x) = Flux.onecold(cpu(x))
acces = [sum(f(model(x)) .== f(y)) / size(x,2) for (x,y) in data_loader]
sum(acces) / length(data_loader)
end
callbacks = [
() -> push!(train_losses, loss_all(train_dl)),
() -> push!(test_losses, loss_all(test_dl)),
() -> push!(train_acces, acc(train_dl)),
() -> push!(test_acces, acc(test_dl)),
]
# Training ------------------------------
epochs = 30
ps = Flux.params(model)
@epochs epochs Flux.train!(loss, ps, train_dl, opt, cb = callbacks)
@show train_loss = loss_all(train_dl)
@show test_loss = loss_all(test_dl)
@show train_acc = acc(train_dl)
@show test_acc = acc(test_dl)
end
Results
One example prediction:
begin
y = (y_test[:,1])
pred = (model(x_test[:,1]))
end
Prediction: 0.00020066714 , 0.19763687 , 0.8021625
Truth: 0 , 0 , 1
error: 0.395675f0
Confusion Matrix
[4] Conclusion
Tools
I chose to implement a basic feed forward neural network because of the scale of the problem. With the data set containing so few samples with very little features a small network would fit better. I chose a NN because I wanted to evaluate Julia as a suitable tool for me to use with deep learning solutions. Again, because of the size of the problem, shallow ML approaches would have been sufficient. Something to expand on in this research is to compare to such methods.
I wanted to challenge myself and learn an entirely new language and platform for this project. The Julia Programming Language is a high level, dynamically typed language. It comes with its own web-based editor that is much like Python's Jupter notebooks. Because Julia is newer and the community is smaller than Python, the documentation and support were not even close in magnitude. This slowed me down considerably. Despite the setbacks, I learned a lot in this research and I am glad I decided to use Julia.
Results
My model's test accuracy was 95.55%. This is satisfactory for me due to the simplicity of the data set and the model. While one species was linearly seperable, the other two were not. These later species are the main problem for the model to tackle.
As I stated in the beginning of this paper, this model could be used for classification tasks such as automation or as a tool for bio researchers to aid in identification. Furthermore, this model could be used as a pre-trained model for more specific tasks; I understand this statement is a bit of a stretch but I want to account for as many applications as possible.
[5] Related work
Related research: Kaggle
Top comments (0)