Finding nearest neighbors isn't an algorithm, it is a goal. There are many different algorithms to accomplish that. This is more about what certain functions do in python.
If you wanted to be totally unambiguous, “the k-nearest neighbor classification algorithm” might be better, but it’s pretty common to abridge that to “nearest neighbors” or “KNN” in a machine learning context.
You are absolutely right that there are many ways to efficiently find those neighbors, but the classification algorithm itself doesn’t care; it’s implementation detail.
This reads weird. Either this is transcribed lecture notes, or the author is not that fluent an English speaker. Either way, I think somone unfamiliar with KNN would struggle to learn anything from this.
The probabilistic interpretation is odd, since it can only produce a few (maybe even one!) distinct values.
$k$ should probably be chosen to avoid ties (e.g., odd for a two-class problem) so you always get exactly one answer, and you should definitely chose $k$ using different data than you use to evaluate the model (e.g., validation set or inside a cross-validation loop).
This is a bit confusing if you're not already familiar with KNN. A simple example using heap or some of the more basic data structure oriented approaches.
Unless I am not understanding something, this appears to have multiple incorrect statements in it, for instance:
> It first identifies the k points in the training data that are closest to the test value and calculates the distance between all those categories. The test value will belong to the category whose distance is the least.
There is no distance measure in deciding a category. The distance measure is used to find the N nearest neighbors. Once those neighbors are found, the class of the new data point is assigned based on the relative number of entities of each class in the N neighbor set (always taken as the most common class in the neighbor set, as far as I am aware).
8 comments
[ 5.3 ms ] story [ 31.9 ms ] threadThe first sentence is "KNN also known as K-nearest neighbor is a supervised and pattern classification learning algorithm"
If someone said 'the triangle rendering algorithm' they would be asked 'which triangle rendering algorithm'.
You are absolutely right that there are many ways to efficiently find those neighbors, but the classification algorithm itself doesn’t care; it’s implementation detail.
The probabilistic interpretation is odd, since it can only produce a few (maybe even one!) distinct values.
$k$ should probably be chosen to avoid ties (e.g., odd for a two-class problem) so you always get exactly one answer, and you should definitely chose $k$ using different data than you use to evaluate the model (e.g., validation set or inside a cross-validation loop).
> It first identifies the k points in the training data that are closest to the test value and calculates the distance between all those categories. The test value will belong to the category whose distance is the least.
There is no distance measure in deciding a category. The distance measure is used to find the N nearest neighbors. Once those neighbors are found, the class of the new data point is assigned based on the relative number of entities of each class in the N neighbor set (always taken as the most common class in the neighbor set, as far as I am aware).