June 17, 2013, 3:47 pm
Note to learn from blog of Lethic
Link: http://zlethic.com/?p=733
The Relationship Between Data Mining and Machine Learning
It’s been a long time since I get to know about such concepts as: Data Mining, Machine Learning, Artificial Intelligence, Pattern Recognition and etc.. However, they are in a kind of mess and it’s hard to figure out the relationship between them, while it is important for one who wants do some research on an area to have an overview on it. So I searched for some articles and essays on these topics on the Internet and tried to arrange them in order.
It’s been a long time since I get to know about such concepts as: Data Mining, Machine Learning, Artificial Intelligence, Pattern Recognition and etc.. However, they are in a kind of mess and it’s hard to figure out the relationship between them, while it is important for one who wants do some research on an area to have an overview on it. So I searched for some articles and essays on these topics on the Internet and tried to arrange them in order.
Before the discussion let’s check out the Wikipedia for these four phrases: Data Mining, Machine Learning, Artificial Intelligence, Pattern Recognition.
1)Data Mining:
Concept:
The analysis step of the “Knowledge Discovery in Databases” process, or KDD.
Concept:
The analysis step of the “Knowledge Discovery in Databases” process, or KDD.
Methods:
involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. And using methods and algorithms such as Bayes’ theorem (1700s), regression analysis (1800s), neural networks, cluster analysis, genetic algorithms (1950s), decision trees (1960s), and support vector machines (1990s).
involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. And using methods and algorithms such as Bayes’ theorem (1700s), regression analysis (1800s), neural networks, cluster analysis, genetic algorithms (1950s), decision trees (1960s), and support vector machines (1990s).
Purpose:
Discover the hidden pattern from the data sets, automatic or semi-automatic analysis of large quantities of data to extract previously unknown interesting patterns such as groups of data records (cluster analysis), unusual records (anomaly detection) and dependencies (association rule mining).
Discover the hidden pattern from the data sets, automatic or semi-automatic analysis of large quantities of data to extract previously unknown interesting patterns such as groups of data records (cluster analysis), unusual records (anomaly detection) and dependencies (association rule mining).
2)Machine Learning:
Concept:
a branch of artificial intelligence, is about the construction and study of systems that can learn from data.
Concept:
a branch of artificial intelligence, is about the construction and study of systems that can learn from data.
Methods:
Supervised learning, Unsupervised learning, Semi-supervised learning, Reinforcement learning, Learning to learn, Developmental learning. Decision tree learning, Association rule learning, Artificial neural networks, Genetic programming, Inductive logic programming, Support vector machines, Clustering, Bayesian networks, Representation learning, Similarity and metric learning, Sparse Dictionary Learning.
Supervised learning, Unsupervised learning, Semi-supervised learning, Reinforcement learning, Learning to learn, Developmental learning. Decision tree learning, Association rule learning, Artificial neural networks, Genetic programming, Inductive logic programming, Support vector machines, Clustering, Bayesian networks, Representation learning, Similarity and metric learning, Sparse Dictionary Learning.
Purpose:
Gives computers the ability to learn without being explicitly programmed
Gives computers the ability to learn without being explicitly programmed
3)Artificial Intelligence:
Concept:
a branch of computer science that studies and develops intelligent machines and software
Concept:
a branch of computer science that studies and develops intelligent machines and software
Methods:
statistical methods, computational intelligence and traditional symbolic AI
Cybernetics and brain simulation
Symbolic
Sub-symbolic
Statistical
Search and optimization
Logic
Probabilistic methods for uncertain reasoning
Classifiers and statistical learning methods
Neural networks
Control theory
statistical methods, computational intelligence and traditional symbolic AI
Cybernetics and brain simulation
Symbolic
Sub-symbolic
Statistical
Search and optimization
Logic
Probabilistic methods for uncertain reasoning
Classifiers and statistical learning methods
Neural networks
Control theory
Purpose:
Deduction, reasoning, problem solving
Knowledge representation: qualification problem
Planning: Intelligent agents
Learning: Machine learning(Supervised Unsupervised), reinforcement learning
Natural language processing
Motion and manipulation: robotics, navigation, mapping
Perception: Computer vision speech recognition, facial recognition and object recognition.
Creativity
General intelligence
Deduction, reasoning, problem solving
Knowledge representation: qualification problem
Planning: Intelligent agents
Learning: Machine learning(Supervised Unsupervised), reinforcement learning
Natural language processing
Motion and manipulation: robotics, navigation, mapping
Perception: Computer vision speech recognition, facial recognition and object recognition.
Creativity
General intelligence
4)Pattern Recognition:
Concept:
the assignment of a label to a given input value
generally categorized according to the type of learning procedure used to generate the output value
Concept:
the assignment of a label to a given input value
generally categorized according to the type of learning procedure used to generate the output value
Methods:
Categorical sequence labeling algorithms (predicting sequences of categorical labels)
Classification algorithms (supervised algorithms predicting categorical labels)
Clustering algorithms (unsupervised algorithms predicting categorical labels)
Ensemble learning algorithms (supervised meta-algorithms for combining multiple learning algorithms together)
General algorithms for predicting arbitrarily-structured (sets of) labels
Multilinear subspace learning algorithms (predicting labels of multidimensional data using tensor representations)
Parsing algorithms (predicting tree structured labels)
Real-valued sequence labeling algorithms (predicting sequences of real-valued labels)
Regression algorithms (predicting real-valued labels)
Categorical sequence labeling algorithms (predicting sequences of categorical labels)
Classification algorithms (supervised algorithms predicting categorical labels)
Clustering algorithms (unsupervised algorithms predicting categorical labels)
Ensemble learning algorithms (supervised meta-algorithms for combining multiple learning algorithms together)
General algorithms for predicting arbitrarily-structured (sets of) labels
Multilinear subspace learning algorithms (predicting labels of multidimensional data using tensor representations)
Parsing algorithms (predicting tree structured labels)
Real-valued sequence labeling algorithms (predicting sequences of real-valued labels)
Regression algorithms (predicting real-valued labels)
Purpose:
automatic speech recognition, classification of text into several categories (e.g., spam/non-spam email messages), the automatic recognition of handwritten postal codes on postal envelopes, automatic recognition of images of human faces, or handwriting image extraction from medical forms
automatic speech recognition, classification of text into several categories (e.g., spam/non-spam email messages), the automatic recognition of handwritten postal codes on postal envelopes, automatic recognition of images of human faces, or handwriting image extraction from medical forms
From all above we can get a rough conclusion that Artificial Intelligence is a relatively “big” area while Machine Learning and Pattern Recognition are branches of it. Artificial Intelligence also contains other branches such as Robotics, Natural Language Processing.
As for Machine Learning and Data Mining, these two terms are commonly confused. They often employ the same methods and overlap significantly. They can be roughly defined as follows:
Machine learning focuses on prediction, based on known properties learned from the training data. It is concerned with concerned with algorithms whose performance at some task improves as it gains experience at that task
Data mining (which is the analysis step of Knowledge Discovery in Databases) focuses on the discovery of (previously) unknown properties on the data. It analysis data for the purpose of discovering unforeseen patterns or properties.
They both look to data and try to extract some value from it. The difference is ML’s goal is to reproduce the known knowledge or DM’s goal is to discover some unknown knowledge. Obviously they are interwind since some algorithms in ML can also be applied to DM.
Machine learning focuses on prediction, based on known properties learned from the training data. It is concerned with concerned with algorithms whose performance at some task improves as it gains experience at that task
Data mining (which is the analysis step of Knowledge Discovery in Databases) focuses on the discovery of (previously) unknown properties on the data. It analysis data for the purpose of discovering unforeseen patterns or properties.
They both look to data and try to extract some value from it. The difference is ML’s goal is to reproduce the known knowledge or DM’s goal is to discover some unknown knowledge. Obviously they are interwind since some algorithms in ML can also be applied to DM.
I used to made a robot concerning the issues on automatic following and navigation. it can be categorized to robotics which is a branch of Artificial Intelligence, while it also applied some Pattern Recognition methods. And Another program I joined was to build an agent based simulation, in which the ‘intelligent agent’ is also concerned to the AI issues. The field I’m now focusing on is the Recommender System and it is kind of Data Mining application using the machine learning tools.
References:
http://en.wikipedia.org/wiki/Machine_learning
http://en.wikipedia.org/wiki/Data_mining
http://en.wikipedia.org/wiki/Artificial_intelligence
http://en.wikipedia.org/wiki/Pattern_Recognition
https://www.quora.com/Whats-the-relationship-between-data-mining-and-machine-learning
