Lakhasly

KNN

CLASSIFICATION

AGENDA

o Feature Scaling o KNN CLASSIFICATION

KNN Classification

What is Classification

o Motivation: Prediction o Is a bank loan applicant "safe" or "risky"? o Which treatment is better for patient, "treatment X" or "treatment Y"?o Classification is a data analysis task where a model is constructed to predict classlabels(categories)

The Basics General Approach

A two-step process: cLearning (training)step a construct classification model ?Build classifier for a predetermined set of classes ?Learn from a training dataset (data tuples + their associated classes) a Supervised Learning cClassification step a model is used to predict class labels for given data (testset)

The Basics General Approach

Training data

Class label

Attribute vector

Classification Algorithm

Classification rules

The Basics General Approach

Test data

(Mohammed, youth, medium) Loan decision?Does it best fit the first category, or the second?

KNN

CLASSIFICATION

AGENDA

• Feature Scaling • KNN CLASSIFICATION

KNN
Classification

What is Classification

• Motivation: Prediction
• Is a bank loan applicant “safe” or “risky”?
• Which treatment is better for patient, “treatment X” or “treatment Y”?

• Classification is a data analysis task where a model is constructed to
predict classlabels(categories)

The Basics General Approach

A two-step process:
¢Learning (training)step à construct classification model
⚫Build classifier for a predetermined set of classes
⚫Learn from a training dataset (data tuples + their associated classes) à
Supervised Learning
¢Classification step à model is used to predict class labels for given data
(testset)

The Basics General Approach

Training
data

Class
label

Attribute
vector

Classification
Algorithm

Classification
rules

The Basics General Approach

Test
data

(Mohammed, youth, medium)
Loan decision?

Risky

Estimate classifier accuracy
(to avoid overfitting)

Predict classification of

new data

Classification rules

% test set tuples correctly classified

Instance based
Learning

K-Nearest

Neighbor Classifier

K-NN Classifier algorithm

• k Nearest Neighbor (or KNN ) is a supervised machine learning algorithm useful for
classification problems.

• It calculatesthe distance between the test data and the input and gives the prediction
according.

K-Nearest Neighbor Classifier

• The question is, how do you determine which category
a new data point would belong to? Does it best fit the
first category, or the second?

KNN Classifier Algorithm Steps

• Typically, the Euclidean distance
formula is mostly used.
• Other numbers of neighbors can
be used, and a different distance
formula can be used.
• It’s up the person to decide on
how they want the model built.

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

How to choose factor K

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

KNN Classifier Algorithm Steps

K-Nearest Neighbor Classifier

K-Nearest Neighbor Classifier

How to choose factor K

K nearest
Neighbor for
categorical
features