What have I been upto in 2021?

19 Aug, 2021

This is part two of my series on journaling the topics and subjects I’m interested in, in a chronological order. Part 1 covered 2020 and this covers my interests in 2021 (upto the publication date).

To give a quick recap, I spent most of my time in 2020 diving deep into Machine Learning, getting to a point where I was comfortable grokking the latest research papers. I also spent some time on frontend heavy full stack development and platform and infrastructure stuff including Kubernetes and Cloud native technologies. While I still had a major interest in core ML going into 2021, the interest was slowly fading due to the reasons I mentioned in Part 1’s Summary.

This was also the year I was starting my master’s journey in CS at UCLA.

January - March

This was my first quarter at UCLA and I took courses in Machine Learning Algorithms, Natural Language Processing (NLP) and Graph Neural Networks. My favorite course in the quarter was NLP because I wanted to deeply understand transformers and play with them on some projects. I accomplished both these goals and also learned about large scale training of transformers which revolutionized the field of NLP (think BERT). I also got familiar with the HuggingFace transformers library and was blown away by its usability and developer experience. At the same time, I also learnt about strategies for large scale training of deep learning models in another class project and got to experience DeepSpeed. This slightly tilted my interest from core ML to large scale training and distributed systems. I was intrigued by the engineering and networking aspects involved in training such enormous models.

Project Reports

April - June

My second quarter, I took courses in ML architecture, Reinforcement Learning and Blockchain. While the RL course was alright, ML architecture and Blockchain courses were the most interesting ones. Firstly, I had wanted to learn about Blockchain technologies for quite some time and the course was an excellent introduction to it. Secondly, the ML architecture course was very challenging due to my limited architecture and hardware knowledge, but I got to learn about the basics of GPUs and implemented some core ML operations like matrix multiplication and convolutions in CUDA and we also implemented an attention module in CUDA. Towards the end of this quarter, I got even more interested in distributed systems (due to its involvement in all blockchain technologies) and multi-GPU training (now that I had learnt a bit about GPUs).

Project Reports

July - August

This was summer time so I had just one course (which was to meet the requirements). Apart from that, I started auditing MIT’s Distributed Systems course 6.824 along with its labs in Go. I loved learning about different systems and challenges. My interest in distributed systems has only been growing and I am excited about completing the course and its projects. I’m also taking a big data systems course in Fall which is a research oriented course that also goes into the details of different distributed systems and involves working on a research project related to the topic. Moreover, I also plan to take a networking course to learn about the communication and networking involved in large systems with thousands of computers.

Summary

My Master’s degree has been an amazing experience so far, making me fall in love with Computer Science all over again. I developed core interests in Distributed Systems and Large Scale Machine Learning over the year, and I am excited to carry it into the future. At the same time, I’ve also tried keeping myself upto date with the latest in platform, infrastructure, backend and a little bit of frontend. Additionally, I’ve realized that I’m also interested in building tools and libraries for Machine Learning models and techniques, making them accessible to the masses; and the amazing Huggingface libraries are a key factor for this. I’ll soon be writing a post detailing why I’m so interested in these topics and why I think they’ll be my primary interests for a long time to come.

Next, I’ll be continuing the distributed systems course and I am also starting a book titled Designing Data Intensive Applications.

comments powered by Disqus