Ask HN: ML Papers to Implement

Great question!

I seem to be in a similar situation as an experienced software engineer who has jumped into the deep end of ML. It seems most resources either abstract away too much detail or too little. For example, building a toy example that just calls gensim.word2vec doesn't help me transfer that knowledge to other use cases. Yet on the other extreme, most research papers are impenetrable walls of math that obscure the forest for the trees.

Thus far, I would also recommend Andrej Karpathy's Zero to Hero course (https://karpathy.ai/zero-to-hero.html). He assumes a high level of programming knowledge but demystifies the ML side.

P.S. If anyone is, by chance, interested in helping chip away at the literacy crisis (e.g., 40% of US 4th graders can't read even at a basic level), I would love to find a collaborator for evaluating the practical application of results from the ML fields of cognitive modeling and machine teaching. These seemingly simple ML models offer powerful insight into the neural basis for learning but are explained in the most obtuse ways.

great, thanks!

also for your PS, can you give a little more detail? What's your end result, what have you done so far etc

A lot depends on what you're interested in.

Some papers that are runnable on a laptop CPU (so long as you stick to small image sizes/tasks):

1) Generative Adversarial Networks (https://arxiv.org/abs/1406.2661). Good practice to have a custom training loops, different optimisers and networks etc.

2) Neural Style Transfer (https://arxiv.org/abs/1508.06576). Nice to be able to manipulate pretrained networks and intercept intermediate layers.

3) Deep Image Prior (https://arxiv.org/abs/1711.10925). Nice low-data exercise in building out an autoencoder.

4) Physics Informed Neural Networks (https://arxiv.org/abs/1711.10561). If you're interested scientific applications, this might be fun. It's good exercise in calculating higher order derivatives of neural networks and using these in loss functions.

5) Vanilla Policy Gradient (https://arxiv.org/abs/1604.06778) is the easiest reinforcement learning algorithm to implement and can be used as a black-box optimiser in a lot of settings.

6) Deep Q Learning (https://arxiv.org/abs/1312.5602) is also not too hard to implement and was the first time I had heard about DeepMind, as well as being a foundational deep reinforcement learning paper .

Open AI gym (https://github.com/openai/gym) would help get started with the latter two.

Not sure if it's beginner friendly but I found implementing NeRF from scratch a good exercise. Especially since it reveals many details that are not immediately obvious from the paper.

I'd love for someone to do a good quality PyTorch enabled implementation of Sampled AlphaZero/MuZero [1]. RLLib has an AlphaZero, but it doesn't have the parallelized MCTS you really want to have and the "Sampled" part is another twist to it. It does implement a single player variant though, which I needed. This would be amazing for applying MCTS based RL to various hard combinatorial optimization problems. Case in point, AlphaTensor uses their internal implementation of Sampled AlphaZero.

An initial implementation might be doable in 5 hours for someone competent and familiar with RLLib's APIs, but could take much longer to really polish.

[1]: https://arxiv.org/abs/2104.06303

Andrej Karpathy is currently releasing videos for a course [1] that goes from zero to GPT.

1. https://karpathy.ai/zero-to-hero.html

I have implemented YOLO v1 and train/tested it on synthetic images with geometric forms. Implementing the loss function thought me a lot on how backpropagation really works. I used keras/tf.

I would recommend diffusion: try starting with Lilian Weng's blog post and writing up the process for yourself. For all it's abilities, the code for DDPM is surprisingly simple.

This is not exactly what you are looking for but you should browse Papers with Code:

https://paperswithcode.com/

I enjoyed doing this through the Coursera deep learning specialization:

https://www.coursera.org/specializations/deep-learning

The lectures take you through each major paper, then you implement the paper in the homework. Much faster than reading the paper yourself.

hmm, I found that I didn't really like this course but I'll look through it again

You could start with Word2Vec or GloVe, for example. Another option is to to with CV papers and start with Alexnet or first ResNet.

you could maybe write the whole thing in a few hours, but debugging what you wrote to recreate prior results will probably take much longer depending on choice of problem

Soft Actor Critic, a reinforcement learning algorithm.