Flowers are among the raw materials in many industries including the pharmaceuticals and cosmetics. Manual classification of flowers requires expert judgment of a botanist and can be time consuming and inconsistent. The ability to classify flowers using computers and technology is the right solution to solve this problem. There are two algorithms that are popular in image classification, namely Convolutional Neural Network (CNN) and Support Vector Machine (SVM). CNN is one of deep neural network classification algorithms while SVM is one of machine learning algorithm. This research was an effort to determine the best performer of the two methods in flower image classification. Our observation suggests that CNN outperform SVM in flower image classification. CNN gives an accuracy of 91.6%, precision of 91.6%, recall of 91.6% and F1 Score of 91.6%.