DLog 4: Machine Learning revisited
Over the last few weeks I have been pretty occupied with my own research project. After what seems like an eternity, I am back to implementing novel neural net architectures and training them. It has been exhilarating to have the liberty to work on my own project without the sword of a conference deadline hanging over my head. As lame as it sounds, working intimately with a challenging problem and getting down in the trenches of implementing a project from scratch has brought me immense satisfaction and peace of mind. My experiments are not yielding expected results yet, but that’s something for another dev log.
One bad habit I’ve had for a long time now is the inertia towards not changing my programming style to adopt development best practices and consequently not growing as a developer. I am actively trying to fix this in my current project. As of now I’ve implemented two good practices and plan to implement to a few more over the course of it.
Code formatting and build automation:
The first change I’ve made is that I am using the code formatter black to keep my codebase nicely formatted as I develop it. I changed the column-width to 80 but other than that I’ve let black do what it does best.
Also, since it is cumbersome to run black repeatedly, I’ve just automated it under make format in my Makefile. A simple make format before commits and my codebase stays pristine (at least formatting-wise)!
Visualizing training:
As ashamed as I am of admitting this, I’ve always been one of those people who logs their ML training metrics like losses and accuracies as console outputs. Until now. I’m now using Tensorboard with my Pytorch code and it has been an incredible change. I can predict with certainity that it is going to be a staple of my machine learning projects.
That’s all for today’s update. Hopefully I will have experimental results for next time!
Key Learnings
Next
I plan to keep working with my research over the summer, and learn experiment management and other tools related to research engineering roles. I’ve already started with Tensorboard. Next, I plan to include Weights & Biases to track my experiments and perform hyperparameter sweeps. Since I also plan to do a code release I will be looking into configuration management as well.