The code is designated for teaching/demonstrating the simplest and most fundamental model of machine learning. The learner should implement the gradient derivation and the update of the bias parameter B of the regression class.
Required packages:
numpy, python-mnist, and matplotlib
Questions:
The learning rate is set to 9e-9 in the code.
Play around with its value and observe the loss value.
Without googling, what is the role of the learning rate based on your observations?
What seems to be a good range of values for the learning rate?
How powerful is linear regression?
Why is the model limited? (Think mathematically and visually (draw a plot))
How can you improve it? (Think about the data and the available features)
A fundamnetal rule has been broken in this code to obtain the learning rate value of 9e-9,
in regards to data handling, what could it be?