MACHINE
MACHINE LEARNING COMPLETE
CLASS NOTES 2025
What is Machine Learning?
It is the science and art of programming computers so they can learn from data.
We can using Machine Learning in data mining to dig into large amounts of data which
can help us discover patterns that are immediately apparent.
What is Machine Learning great for?
Problems for which existing solutions require a lot of hand-tuning or long list of rules.
(ML can often simplify code and perform better)
Complex problems for which there is no good solution at all use traditional approach.
Fluctuating environments. ( A ML system cab adapt to new data)
Getting insights about complex problems and large amounts of data.
Supervised Learning
The training data you feed to the algorithm includes the desired solutions called labels
Unsupervised learning
The training data is unlabeled. The system tries to learn without a teacher.
One important unsupervised learning task is anomaly detection. ( We will detect
unusual credit transactions, manufacturing defects, and etc.)
Semi supervised learning
Some algorithms can deal with partially labeled training data. It is usually a lot of
unlabeled data and a little bit of labeled data.
Reinforcement learning
is an area of machine learning concerned with how software agents ought to take
actions in an environment so as to maximize some notion of cumulative reward.
MACHINE
,MACHINE
Reinforcement learning is one of three basic machine learning paradigms, alongside
supervised learning and unsupervised learning.
Batch Learning
Another criterion to classify Machine Learning is to tell whether or not the system can
learn incrementally from a stream of incoming data.
In batch learning the system is incapable of learning incrementally. It must be trained
using all the available data. This will generally take a lot of time and computing
resources so it is typically done offline. First the system is trained and then it is
launched into production and runs without learning anymore. It just applies what it
learned. That is called offline learning.
If you want batch learning system to know about new data (such as a new type of
spam), you need to train a new version of the system from scratch on the full dataset.
(Not t just the new data, but also the old data) Then you have to stop the old system
and replace it with the new one).
Online Learning
You train the system incrementally by feeding it data instances, sequentially, either
individually or by small groups called mini batches. Each step is fast and cheap, so the
system can learn about new data on the fly, as it arrives. (Online learning is good for
systems that receive continuous data on a flow. (Stock prices)
One is one important parameter of Online Learning -
how fast they should adapt to changing data> This is called the learning rate. or step
size in machine learning is a hyperparameter which determines to what extent newly
acquired information overrides old information.[1] The learning rate is often denoted by
the character η or α.
If you see a high learning rate, then your system will rapidly adapt to new data, but it will
also tend to quickly forget the old data. (you don't want a spam filter to flag only the
latest kinds of spam it was shown). If you set a low learning rate, the system will have
more inertia, that is. It will learn more slowly. But it will also be less sensitive to noise in
the new data or to sequences of non-representative data points.
MACHINE
,MACHINE
A bug challenge: is that if bad data is fed to the system, the system's performance will
gradually decline. If we are talking about a live system, your clients will notice. For
instance, bad data could come from a malfunctioning sensor on a robot or someone
spamming a search engine to try and rank high in search results. To reduce this risk.
You need to monitor the system closely and promptly switch learning off ( and revert to
the previous state) if you detect a drop in performance. May also want to monitor the
input data and react to abnormal data (using an anomaly detection algorithm)
Anomaly detection algorithm
is a technique used to identify unusual patterns that do not conform to expected
behavior, called outliers.
instance-based learning
It is a family of learning algorithms that, instead of performing explicit generalization,
compares new problem instances with instances seen in training, which have been
stored in memory.
It just about storing the training data instances. Though the training data itself can be
preprocessed in many ways and stored in memory.
is a family of learning algorithms that, instead of performing explicit generalization,
compares new problem instances with instances seen in training, which have been
stored in memory.
It is called instance-based because it constructs hypotheses directly from the training
instances themselves.
This means that the hypothesis complexity can grow with the data:
in the worst case, a hypothesis is a list of n training items and the computational
complexity of classifying a single new instance is O(n). One advantage that instance-
based learning has over other methods of machine learning is its ability to adapt its
model to previously unseen data. Instance-based learners may simply store a new
instance or throw an old instance away.
Examples of instance-based learning algorithm are the k-nearest neighbors algorithm,
kernel machines and RBF networks.
MACHINE
, MACHINE
These store (a subset of) their training set; when predicting a value/class for a new
instance, they compute distances or similarities between this instance and the training
instances to make a decision.
Called that the system learns the examples by heart, then generalizes to new cases
using similarity measure.
Categorizing Machine Learning Algorithms:
How they generalize. Most Machine Learning tasks are about making predictions. This
means that given a number of training examples, the system needs to be able to
generalize to examples, it has never seen before. Having a good performance measure
on the training data is good, insufficient; the true goal is to perform well on new
instances:
Generalization: Instance-Based learning and model-based learning.
Model based learning
Another way to generalize from a set of examples is to build a model of these examples,
then use that model to make predictions.
Challenges in a Machine Learning Project
Insufficient Quantity of Training Data
- it takes a lot of data for most machine learning algorithms to work properly. Even for
such simple problems you typically need thousands of examples, and for complex
problems such as image or speech recognition you may need millions of examples
(unless you can reuse parts of an existing model).
-Non-representative training data -
In order to generalize well, it is crucial that your training data be representative of the
new cases you want to generalize. This is true whether you use instance-based learning
or model based learning
-Poor Quality Data
-Irrelevant Features:
The system must be capable of learning if the training data contains enough relevant
MACHINE
MACHINE LEARNING COMPLETE
CLASS NOTES 2025
What is Machine Learning?
It is the science and art of programming computers so they can learn from data.
We can using Machine Learning in data mining to dig into large amounts of data which
can help us discover patterns that are immediately apparent.
What is Machine Learning great for?
Problems for which existing solutions require a lot of hand-tuning or long list of rules.
(ML can often simplify code and perform better)
Complex problems for which there is no good solution at all use traditional approach.
Fluctuating environments. ( A ML system cab adapt to new data)
Getting insights about complex problems and large amounts of data.
Supervised Learning
The training data you feed to the algorithm includes the desired solutions called labels
Unsupervised learning
The training data is unlabeled. The system tries to learn without a teacher.
One important unsupervised learning task is anomaly detection. ( We will detect
unusual credit transactions, manufacturing defects, and etc.)
Semi supervised learning
Some algorithms can deal with partially labeled training data. It is usually a lot of
unlabeled data and a little bit of labeled data.
Reinforcement learning
is an area of machine learning concerned with how software agents ought to take
actions in an environment so as to maximize some notion of cumulative reward.
MACHINE
,MACHINE
Reinforcement learning is one of three basic machine learning paradigms, alongside
supervised learning and unsupervised learning.
Batch Learning
Another criterion to classify Machine Learning is to tell whether or not the system can
learn incrementally from a stream of incoming data.
In batch learning the system is incapable of learning incrementally. It must be trained
using all the available data. This will generally take a lot of time and computing
resources so it is typically done offline. First the system is trained and then it is
launched into production and runs without learning anymore. It just applies what it
learned. That is called offline learning.
If you want batch learning system to know about new data (such as a new type of
spam), you need to train a new version of the system from scratch on the full dataset.
(Not t just the new data, but also the old data) Then you have to stop the old system
and replace it with the new one).
Online Learning
You train the system incrementally by feeding it data instances, sequentially, either
individually or by small groups called mini batches. Each step is fast and cheap, so the
system can learn about new data on the fly, as it arrives. (Online learning is good for
systems that receive continuous data on a flow. (Stock prices)
One is one important parameter of Online Learning -
how fast they should adapt to changing data> This is called the learning rate. or step
size in machine learning is a hyperparameter which determines to what extent newly
acquired information overrides old information.[1] The learning rate is often denoted by
the character η or α.
If you see a high learning rate, then your system will rapidly adapt to new data, but it will
also tend to quickly forget the old data. (you don't want a spam filter to flag only the
latest kinds of spam it was shown). If you set a low learning rate, the system will have
more inertia, that is. It will learn more slowly. But it will also be less sensitive to noise in
the new data or to sequences of non-representative data points.
MACHINE
,MACHINE
A bug challenge: is that if bad data is fed to the system, the system's performance will
gradually decline. If we are talking about a live system, your clients will notice. For
instance, bad data could come from a malfunctioning sensor on a robot or someone
spamming a search engine to try and rank high in search results. To reduce this risk.
You need to monitor the system closely and promptly switch learning off ( and revert to
the previous state) if you detect a drop in performance. May also want to monitor the
input data and react to abnormal data (using an anomaly detection algorithm)
Anomaly detection algorithm
is a technique used to identify unusual patterns that do not conform to expected
behavior, called outliers.
instance-based learning
It is a family of learning algorithms that, instead of performing explicit generalization,
compares new problem instances with instances seen in training, which have been
stored in memory.
It just about storing the training data instances. Though the training data itself can be
preprocessed in many ways and stored in memory.
is a family of learning algorithms that, instead of performing explicit generalization,
compares new problem instances with instances seen in training, which have been
stored in memory.
It is called instance-based because it constructs hypotheses directly from the training
instances themselves.
This means that the hypothesis complexity can grow with the data:
in the worst case, a hypothesis is a list of n training items and the computational
complexity of classifying a single new instance is O(n). One advantage that instance-
based learning has over other methods of machine learning is its ability to adapt its
model to previously unseen data. Instance-based learners may simply store a new
instance or throw an old instance away.
Examples of instance-based learning algorithm are the k-nearest neighbors algorithm,
kernel machines and RBF networks.
MACHINE
, MACHINE
These store (a subset of) their training set; when predicting a value/class for a new
instance, they compute distances or similarities between this instance and the training
instances to make a decision.
Called that the system learns the examples by heart, then generalizes to new cases
using similarity measure.
Categorizing Machine Learning Algorithms:
How they generalize. Most Machine Learning tasks are about making predictions. This
means that given a number of training examples, the system needs to be able to
generalize to examples, it has never seen before. Having a good performance measure
on the training data is good, insufficient; the true goal is to perform well on new
instances:
Generalization: Instance-Based learning and model-based learning.
Model based learning
Another way to generalize from a set of examples is to build a model of these examples,
then use that model to make predictions.
Challenges in a Machine Learning Project
Insufficient Quantity of Training Data
- it takes a lot of data for most machine learning algorithms to work properly. Even for
such simple problems you typically need thousands of examples, and for complex
problems such as image or speech recognition you may need millions of examples
(unless you can reuse parts of an existing model).
-Non-representative training data -
In order to generalize well, it is crucial that your training data be representative of the
new cases you want to generalize. This is true whether you use instance-based learning
or model based learning
-Poor Quality Data
-Irrelevant Features:
The system must be capable of learning if the training data contains enough relevant
MACHINE
