Multiple Linear Regression from Scratch in Numpy

Linear regression is probably the most simple ‘machine learning’ algorithm. I bet you’ve used it many times, possibly through Scikit-Learn or any other library providing you with an out-of-the-box solution.

But have you ever asked yourself: How does the model actually work behind the scenes?

Sure, in case of simple linear regression ( only one feature ) you can calculate slope and intercept coefficients with a simple formula, but those formulas cannot be transferred to multiple regression. If you don’t know anything about simple linear regression, check out this article:

Today I will focus only on multiple regression and will show you how to calculate the intercept and as many slope coefficients as you need with some linear algebra. There will be a bit of math, but nothing implemented by hand. You should be familiar with the terms like matrix multiplication, matrix inverse, and matrix transpose.

If those sound like science fiction, fear not, I have you covered once again:

At the bottom of that article is a link to the second part, which covers some basic concepts of matrices. Let’s now quickly dive into the structure of this article:

1. Math behind
2. Imports
3. Class definition
4. Declaring helper function
5. Declaring fit() function
6. Declaring predict() function
7. Making predictions
8. Conclusion

A lot of stuff to cover, I know. I’ll try to make it as short as possible, and you should hopefully be able to go through the entire article in less than 10 minutes.

Okay, let’s dive in!