Title 04. Solution - Gradient Descent.pdf ✅

pacmann.io © 2022 – Pacmann AI 18 Solution Approach
pacmann.io © 2022 – Pacmann AI 19 Approach ● There are some approaches to solve the linear regression model ○ The least square solution ○ The gradient descent ○ The probabilistic approach
pacmann.io © 2022 – Pacmann AI 20 Gradient Descent Approach
pacmann.io © 2022 – Pacmann AI 21 Gradient Descent Approach ● We want to choose w so as to minimize RSS(w) iteratively. — — — ● To do so, lets use a search algorithm that starts with some “initial guess” for w, ● and then repeatedly changes w to make RSS(w) smaller ● Until we converge to a value of w that minimize RSS(w). ● This is the gradient descent algorithm RSS(w) w Initial guess
pacmann.io © 2022 – Pacmann AI 22 Gradient Descent Approach ● The update algorithm would be ● To implement above algorithm, we need to know the RSS gradient w.r.t w RSS(w) w Initial guess