Transfer learning with Monk

Monk provides a syntax invariant transfer learning framework that supports Keras, Pytorch and Mxnet in the backend. (Read — Documentation).

Website
Github

Transfer Learning

Transfer learning is one of the most used techniques in training computer vision models. To put it simply, an already trained model is picked and retrained for a different use-case.

The key steps involved are

• Data ingestion
• Model selection
• Setting parameters
• Training
• Evaluating results
• Comparing results with previous training sessions
• Changing the parameters and retraining until we find the best fit model
• ………… Iterating the same

Computer Vision developers have to explore strategies while selecting the correct learning rates, a fitting CNN architecture, use the right optimisers and fine-tune many more parameters to get the best performing models.

Let’s compare steps carried out traditionally vs Monk

1. Write less code to begin the prototyping process

Typically to tackle transfer learning, frameworks like keras, pytorch or mxnet are used

Transfer learning using pytorch — Ref: https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html

The traditional way: The very first step to transfer learning is to understand pytorch and write these many lines of code!!!!

The Monk way: Start with 5 lines of code instead of 40 — Use Monk’s Quick Mode

from monk.pytorch_prototype import prototype
gtf = prototype(verbose=1);
gtf.Prototype("Project-1", "Experiment-1");
gtf.Default(dataset_path="train",
              model_name="resnet18_v1",num_epochs=10);
gtf.Train();

2.Seamlessly switch back-end frameworks

Transfer learning using keras — Ref: https://medium.com/@14prakash/transfer-learning-using-keras-d804b2e04ef8

The traditional way: Learn keras and again write many lines of code.

The Monk way: Import Keras utilities from Monk and write the same code

from monk.keras_prototype import prototype
gtf = prototype(verbose=1);
gtf.Prototype("Project-1", "Experiment-1");
gtf.Default(dataset_path="train",model_name="resnet18_v1",
           num_epochs=10);
gtf.Train()

Monk is Syntax Invariant up-to a certain level of abstraction.

Now, one may ask that with just these 5 lines of code one is losing the capabilities to manipulate parameters.

3. Manipulate parameters in a standardized way

Load your experiment in the quick mode and make changes to parameters at every stage

Dataset updates

gtf.update_input_size(256); - Change input shape
gtf.update_trainval_split(0.6); - Change splits

and many more….. Check out Monk’s Update Mode

Model updates

gtf.update_freeze_layers(10); - freeze layers
gtf.append_linear(final_layer=True); - append layers

and much more. Check out Monk’s Expert Mode

4. Compare all the experiments executed using Monk

The traditional way: Write extra code that creates comparisons, and make a lot of changes to training code in order to generate these metrics.

The Monk way: Invoke comparison capabilities in simple functional format

ctf = compare(verbose=1);
ctf.Comparison("Sample-Comparison-1")# Step 1 - Add experiments
ctf.Add_Experiment("Project-Testing-Optimizers", "SGD");
ctf.Add_Experiment("Project-Testing-Optimizers", "ADAM");
ctf.Add_Experiment("Project-Testing-Optimizers", "ADAGRAD");
ctf.Add_Experiment("Project-Testing-Optimizers", "NAG");
ctf.Add_Experiment("Project-Testing-Optimizers", "NADAM");
ctf.Add_Experiment("Project-Testing-Optimizers", "ADAMAX");# Step 2 - Compare
ctf.Generate_Statistics();

And generate Results

Compare Accuracies

5. Other benefits of using Monk

• Resume training sessions when interrupted from the last epoch
• Run Experimental data analysis — discover class imbalance, missing data, corrupt data
• Copy experiment from one system to another, be it local or cloud
• Estimate training times before actual runs
• Semi-automatically find hyper-parameters by running mini-experiments

Upcoming stories: Tutorials on using MONK

Happy Coding!!!