Title 06. Feature Split Algorithm.pdf ✅

pacmann.io © 2022 – Pacmann AI 38 Feature Split Algorithm
pacmann.io © 2022 – Pacmann AI 39 Feature Split Algorithm ● Given a subset of data M (a node in a tree) ● For each feature in M: ○ Split data of M according to feature-p ○ Compute classification error of split ● Choose feature-p* with highest error gain
pacmann.io © 2022 – Pacmann AI 40 Feature Split Algorithm ● Given a subset of data M (a node in a tree) ● For each feature in M: ○ Split data of M according to feature-p ○ Compute impurity of split ● Choose feature-p* with highest information gain Impurity A measure of uniformity of node’s data to yields information about data. More uncertainty → more impure Information Gain Gain of impurity before & after splitting More gain → more information
pacmann.io © 2022 – Pacmann AI 41 Feature Split Algorithm Impurity A measure of uniformity of node’s data to yields information about data. More uncertainty → more impure Classification ● Entropy ● Gini Index ● Log-Loss (Misclassification Error) Regression ● Variance (MSE) based on slide by Pedro Domingos
pacmann.io © 2022 – Pacmann AI 42 Feature Split Algorithm Impurity Classification ● Entropy ● Gini Index ● Log-Loss (Misclassification Error) from Element of Statistical Learning