Image classification
Problem description
Classifying images of handwritten digits (8×8 pixels) with machine learning algorithms and a neural network, then building a real image classifier with the camera in Teachable Machine.
Objectives
- Representing an image as a vector of pixels
- Comparing kNN, SVM, Random Forest and MLP
- Trying transfer learning in Teachable Machine
Problem type
Image Classification
Dataset
Tools used
Algorithms
Implementation steps
- In KNIME: File → Import KNIME Workflow… and choose image-classification.knwf, then Execute all.
- Open image-classification.ows in Orange to classify the pixel table.
- Use PCA, then Scatter Plot, to see how the digits cluster in two dimensions.
- In Teachable Machine: Image Project → two or more classes → capture at least 50 images per class → Train → Preview.
- Export the Teachable Machine model (TensorFlow.js) if you want to use it on a web page.
Workflow diagram
Orange
DataEvaluateModelVisualize
↔ Drag the diagram sideways to see all of it.
Connections as text (8)
- File → Data Table (Data)
- File → Test and Score (Data)
- kNN → Test and Score (Learner)
- SVM → Test and Score (Learner)
- Random Forest → Test and Score (Learner)
- Neural Network → Test and Score (Learner)
- Test and Score → Confusion Matrix (Evaluation Results)
- File → Scatter Plot (Data)
KNIME
SourceManipulatorLearnerPredictorOtherData portModel port
↔ Drag the diagram sideways to see all of it.
Connections as text (5)
- CSV Reader → Partitioning
- Partitioning → Random Forest Learner
- Random Forest Learner → Random Forest Predictor
- Random Forest Predictor → Scorer
- Partitioning → Random Forest Predictor
Project files
- CSV datasetCSV260.6 KB
- WEKA ARFF fileWEKA ARFF265.5 KB
- Orange workflow (.ows)Orange7.1 KB
- Data in Orange format (.tab) for the workflowOrange TAB261.3 KB
- Computed reference results (CSV)Results228.0 B
- KNIME workflow (.knwf): ran successfully in the KNIME 5.12 engine (not opened in the interface)KNIME111.6 KB
- Step-by-step implementation guide (PDF)PDF599.6 KB
- Full project package (ZIP)ZIP818.3 KB
The KNIME file (.knwf) was built on KNIME 5.12 and all its nodes ran successfully in the KNIME engine without opening the interface; open it from File → Import KNIME Workflow.
Results
Actual results computed on the attached files
• kNN (k=5): Accuracy 97.9% (± 1.1%), F1-macro 0.979
• SVM (RBF): Accuracy 98.4% (± 0.9%), F1-macro 0.984
• Random Forest: Accuracy 97.9% (± 0.8%), F1-macro 0.979
• MLP (1x64): Accuracy 98.3% (± 0.9%), F1-macro 0.983
Teachable Machine results depend on your own images, so we do not publish numbers for them.
Evaluation metrics
- Accuracy
- Confusion Matrix for ten classes
Interpreting the results
All the models exceed 97% because the images are small, clean and centred. Larger, more varied real images need convolutional networks (CNN) or transfer learning as in Teachable Machine.
Discussion questions
- Which digits are confused with each other most, and why?
- Why do large images need a CNN instead of an MLP?
- How do you make sure the Teachable Machine model has not learned the background instead of the object?
Try it in the Visual Neural Network Lab
