bayesian classification
Bayesian classification
•The classification problem may be formalized using a-posteriori probabilities:
• P(C|X) = prob. that the sample tuple X=<x1,…,xk> is of class C.
•E.g. P(class=N | outlook=sunny,windy=true,…)
•Idea: assign to sample X the class label C such that P(C|X) is maximal
Estimating a-posteriori probabilities
•Bayes theorem:
P(C|X) = P(X|C)·P(C) / P(X)
•P(X) is constant for all classes
•P(C) = relative freq of class C samples
•C such that P(C|X) is maximum =
C such that P(X|C)·P(C) is maximum
C such that P(X|C)·P(C) is maximum
•Problem: computing P(X|C) is unfeasible!
Naïve Bayesian Classification
•Naïve assumption: attribute independence
P(x1,…,xk|C) = P(x1|C)·…·P(xk|C)
•If i-th attribute is categorical:
P(xi|C) is estimated as the relative freq of samples having value xi as i-th attribute in class C
P(xi|C) is estimated as the relative freq of samples having value xi as i-th attribute in class C
•If i-th attribute is continuous:
P(xi|C) is estimated thru a Gaussian density function
P(xi|C) is estimated thru a Gaussian density function
•Computationally easy in both cases
Play-tennis example: estimating P(xi|C)