Motivation
I purchased Scott Ross Scarlatti CD box that contains all the Scarlatti keyboard sonatas.
Listened to a few pieces and found they modulate frequently and I'd like to know what key is played on now.
Creating training data for 24 key classifier Linear Support Vector Machine
Rip Scarlatti Scott Ross CDs from K1 to K100 to create K1.flac to K100.flac
The harpsichord is tuned in baroque pitch. I'm not good at inspecting the key of baroque pitch so pitch is increased by semitone to concert pitch using WWArbitraryResampler. And convert them to monaural WAV and store on WAV_Reference directory for AI training.
WWArbitraryResampler: https://sourceforge.net/p/playpcmwin/wiki/WWArbitraryResampler/
Listen to FLAC of concert pitch files from K1 to K100 and roughly write down time and key: https://sourceforge.net/p/playpcmwin/code/HEAD/tree/PlayPcmWin/WWKeyClassifier2/train/Keys_faithful.csv
On K25, Cisdur appears but it is changed to Desdur (to converge number of keys to 24).
Create harpsichord key press data for learning. FFT with 16384 window length and extract 110Hz (A2) to 1318.5Hz (E6) frequency components by semitone step (Fig.1). harpsichod have several lower keys, but lower key press info is blurred and spread to adjacent keys so I decided to exclude them.
Fig.1: harpsichord keys.
On Matlab, run classificationLearner() and input spectral component data of 24 keys music snippets and learn LinearSVM to create linear SVM model and export C# Linear SVM coefficients.
Resulted classifier code:
https://sourceforge.net/p/playpcmwin/code/HEAD/tree/PlayPcmWin/WWKeyClassifier2/KeyClassifierCore.cs
Created app, WWKeyClassifier2:
https://sourceforge.net/p/playpcmwin/wiki/WWKeyClassifier2/
Accuracy of the SVM classifier
There are 24 keys categories to be classified. 12 major keys and 12 minor keys.
Combination(24,2) = 276 one-by-one Binary Classifier is created from the training. For example, Cdur-Cmoll classifier can tell is input music Cdur or Cmoll, there is Cdur-Amoll classifier, Cdur-Gdur classifier and so on.
Prediction accuracy is about 80%. It seems Hdur tends to falsely classified as Edur.
What the AI learned from the training
Following is the`beta' coefficient table of Cdur-Cmoll binary linear SVM classifier. This table means: when E note is appeared, it is probably Cdur. When Es (Dis in the following graph) note is appeared, it is probably Cmoll. This is somewhat understandable.
When the music is found to be either Cdur or Cmoll,
- E note is appeared→ it is probably Cdur.
- Es note is appeared→ it is probably Cmoll.
Table 2: Cdur-Cmoll binary classifier beta coefficients.
Table 3: Cdur-Amoll classifier. G note→Cdur, Gis note→Amoll.
Table 4: Cdur-Gdur classifier. Fis note→Gdur. F note→Cdur. E note→may be Cdur.
Table 5: Cdur-Fdur classifier. H note→Cdur, B note→Fdur. D note→may be Cdur. A note→may be Fdur.
Table 6: Hdur-Edur classifier. A note→Edur, Ais note→Hdur. Cis note→may be Hdur. Gis note→may be Edur. There is no apparent flaw seen on this chart but this classifier tends to falsely classify Hdur music as Edur.
Table 7: Cdur-Fisdur classifier. This is easy job. Computer never mis-classify Cdur music as Fisdur because those two keys are very different.
About Voting
There are Combination(24,2) = 276 binary classifiers, each classifier tells which key is more probable with given two keys. For example, Desdur-Cdur classifier tells inputted data is Desdur or Cdur. When one FFT data is inputted, our Multiclass Classifier runs 276 binary classifiers and choose most voted key as a result.
For example, this process creates something like the following table and "Dmoll" is chosen as a most probable result. From the table, it seems the next candidate is Adur and Ddur.
| - | Cdur | Desdur | Ddur | Esdur | Edur | Fdur | Fisdur | Gdur | Asdur | Adur | Bdur | Hdur | Cmoll | Cismoll | Dmoll | Dismoll | Emoll | Fmoll | Fismoll | Gmoll | Gismoll | Amoll | Bmoll | Hmoll |
| Cdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Desdur | Cdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Ddur | Ddur | Ddur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Esdur | Cdur | Esdur | Ddur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Edur | Cdur | Edur | Ddur | Edur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Fdur | Fdur | Fdur | Ddur | Fdur | Fdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Fisdur | Cdur | Desdur | Ddur | Esdur | Edur | Fdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Gdur | Gdur | Gdur | Ddur | Gdur | Gdur | Fdur | Gdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Asdur | Cdur | Asdur | Ddur | Esdur | Edur | Fdur | Asdur | Gdur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Adur | Adur | Adur | Ddur | Adur | Adur | Adur | Adur | Adur | Adur | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Bdur | Cdur | Bdur | Ddur | Bdur | Edur | Fdur | Bdur | Gdur | Bdur | Adur | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Hdur | Cdur | Hdur | Ddur | Esdur | Edur | Fdur | Hdur | Gdur | Asdur | Adur | Bdur | - | - | - | - | - | - | - | - | - | - | - | - | - |
| Cmoll | Cdur | Cmoll | Ddur | Cmoll | Cmoll | Fdur | Cmoll | Gdur | Cmoll | Adur | Bdur | Cmoll | - | - | - | - | - | - | - | - | - | - | - | - |
| Cismoll | Cdur | Cismoll | Ddur | Esdur | Edur | Fdur | Cismoll | Gdur | Cismoll | Adur | Bdur | Cismoll | Cmoll | - | - | - | - | - | - | - | - | - | - | - |
| Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | Dmoll | - | - | - | - | - | - | - | - | - | - |
| Dismoll | Cdur | Dismoll | Ddur | Esdur | Edur | Fdur | Dismoll | Gdur | Dismoll | Adur | Bdur | Dismoll | Cmoll | Cismoll | Dmoll | - | - | - | - | - | - | - | - | - |
| Emoll | Cdur | Emoll | Ddur | Emoll | Emoll | Fdur | Emoll | Gdur | Emoll | Adur | Emoll | Emoll | Cmoll | Emoll | Dmoll | Emoll | - | - | - | - | - | - | - | - |
| Fmoll | Cdur | Fmoll | Ddur | Fmoll | Edur | Fdur | Fmoll | Gdur | Fmoll | Adur | Bdur | Fmoll | Cmoll | Fmoll | Dmoll | Fmoll | Emoll | - | - | - | - | - | - | - |
| Fismoll | Cdur | Fismoll | Ddur | Esdur | Edur | Fdur | Fismoll | Gdur | Fismoll | Adur | Bdur | Fismoll | Cmoll | Fismoll | Dmoll | Fismoll | Fismoll | Fismoll | - | - | - | - | - | - |
| Gmoll | Cdur | Gmoll | Ddur | Gmoll | Gmoll | Fdur | Gmoll | Gdur | Gmoll | Adur | Gmoll | Gmoll | Gmoll | Gmoll | Dmoll | Gmoll | Gmoll | Gmoll | Gmoll | - | - | - | - | - |
| Gismoll | Cdur | Gismoll | Ddur | Esdur | Edur | Fdur | Gismoll | Gdur | Asdur | Adur | Bdur | Hdur | Cmoll | Cismoll | Dmoll | Dismoll | Emoll | Fmoll | Fismoll | Gmoll | - | - | - | - |
| Amoll | Cdur | Amoll | Ddur | Amoll | Edur | Amoll | Amoll | Gdur | Amoll | Adur | Amoll | Amoll | Amoll | Amoll | Dmoll | Amoll | Amoll | Amoll | Amoll | Amoll | Amoll | - | - | - |
| Bmoll | Cdur | Bmoll | Ddur | Esdur | Edur | Fdur | Bmoll | Gdur | Bmoll | Adur | Bdur | Bmoll | Cmoll | Cismoll | Dmoll | Bmoll | Emoll | Fmoll | Fismoll | Gmoll | Bmoll | Amoll | - | - |
| Hmoll | Cdur | Hmoll | Ddur | Hmoll | Hmoll | Fdur | Hmoll | Gdur | Hmoll | Adur | Hmoll | Hmoll | Hmoll | Hmoll | Dmoll | Hmoll | Hmoll | Hmoll | Hmoll | Hmoll | Hmoll | Amoll | Hmoll | - |
