video2midi
youtube synthesia video to midi, just for fun )
ΠΠΎΡΡΠΎΠ·Π΄Π°Π½ΠΈΠ΅ midi Ρ Π²ΠΈΠ΄Π΅ΠΎ ΡΠΎΠ»ΠΈΠΊΠΎΠ² synthesia ΠΈ Π΅ΠΉ ΠΏΠΎΠ΄ΠΎΠ±Π½ΡΠΌ..
Video example on YouTube, creating midi from hybrid synthesia video & real piano video2midi 0.4.5.2
Video example on YouTube, creating 2 channels midi, version video2midi 0.3.9.1
Video example on YouTube, creating 6 channels midi, version video2midi 0.3.9.1
Video example on YouTube, creating 2 channels midi, version video2midi 0.3.0
Manual "How to" Or "How to use video2midi after installing"
GH already added new discuss functionality so we can talk without opening an issue... if you have any questions or comments, you can open discuss or you can write to whatsapp / telegram , i'll try to answer / help ;)
EN:
dependency
- python-opengl
- python-opencv
- python-pygame
- python-midiutil
install dependency
Debian based Linux:
sudo apt install python-opencv python-pygame python-midiutil python-opengl
Arch Linux:
sudo pacman -S hdf5 python-opencv python-pygame python-opengl
sudo pip install midiutil
Or thanks to C0rn3j you can install video2midi just from AUR now using your favorite AUR helper:
yay -S video2midi-git
Windows + Anaconda2 (python 2.7)/Anaconda3 (python 3.7):
usage
Download the video with your favorite melody (I recommend 720p), launch the program, adjust the keys in it, press Q and after the completion of the work a midi file will be created.
GNU/Linux:
./v2m.py ./synthesia_video.mkv
Windows+Anaconda2 (python 2.7)/Anaconda3 (python 3.7):
in start menu search and open Anaconda command prompt:
cd path to v2m.py
python v2m.py synthesia_video.mkv
Control:
- h - show/hide this help
- q - begin to recreate midi
- s - set start frame, (mods : shift, set processing start frame to the beginning)
- e - set end frame, (mods : shift, set processing end frame to the ending), on some formats, it does not work correctly, it seems like an OpenCV bug
- p - if the key is specified, it will forcibly divide the output into 2 channels regardless of the settings. Splitting will be carried out depending on the position of the key relative to the specified key. Used on video with one key color.
- o - enable or disable the ability to recognize the overlap of keys with each other. In this case, at the moment of overlapping, the end of one note and the beginning of another will be created.
- i - enable or disable ignore/lengthening of notes with minimal duration ( if enabled the notes whose duration will be less than that specified will be ignored. If disabled the notes whose duration will be less than that specified will be automatically equated to the minimum duration.)
- r - enable or disable resize function
- Mouse wheel - keys adjustment
- Left mouse button - dragging the selected key / select color from the color map
- CTRL + Left mouse button - update selected color in the color map
- CTRL + 0 - disable selected color in the color map
- Right mouse button - dragging all keys, if the key is selected, the transfer is carried out relative to it.
- Arrows - keys adjustment (mods : shift)
- PageUp/PageDown - scrolling video (mods : shift)
- Home/End - go to the beginning or end of the video
- F2/F3 - save / load settings.
- [ / ] - change base octave
- Escape - quit
- Space - abort re-creation and save midi file to disk
how it works
Frame by frame we scan the virtual keyboard and write the keys to the midi file...
Additional features
All settings are moved to an ini file which can be used as common for all directories if it is located in the home directory (~/.v2m.ini) or as a separate for local directory (./v2m.ini).
You can also customize the separation into channels depending on the color of the key. To do this, in the v2m.ini file, you need to modify the color matching to the midi channel of the track. By default, each color key will be recorded in its own channel, so if you want to combine the channels, simply specify the same numbers for different colors.
color_channel_accordance = 0,0, 1,1, 2,2, 3,3, 4,4, 5,5
You can customize the channel mapping to a MIDI instrument. To do this, in the v2m.ini file, you need to modify the correspondence of the midi channel to the instrument. The default all channels is 0 midi instrument.
channel_prog_accordance = 0,0, 0,0, 0,0, 0,0, 0,0, 0,0
RU:
Π·Π°Π²ΠΈΡΠΈΠΌΠΎΡΡΠΈ
- python-opengl
- python-opencv
- python-pygame
- python-midiutil
ΡΡΡΠ°Π½ΠΎΠ²ΠΊΠ° Π·Π°Π²ΠΈΡΠΈΠΌΠΎΡΡΠ΅ΠΉ.
Debian based Linux:
sudo apt install python-opencv python-pygame python-midiutil python-opengl
Arch Linux:
sudo pacman -S hdf5 python-opencv python-pygame python-opengl
sudo pip install midiutil
Windows + Anaconda2 (python 2.7)/Anaconda3 (python 3.7):
ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅
Π‘ΠΊΠ°ΡΠΈΠ²Π°Π΅ΠΌ Π²ΠΈΠ΄Π΅ΠΎ Ρ ΠΏΠΎΠ½ΡΠ°Π²ΠΈΠ²ΡΠ΅ΠΉΡΡ ΠΌΠ΅Π»ΠΎΠ΄ΠΈΠ΅ΠΉ ( ΡΠ΅ΠΊΠΎΠΌΠ΅Π½Π΄ΡΡ 720p ) , Π·Π°ΠΏΡΡΠΊΠ°Π΅ΠΌ ΠΏΡΠΎΠ³ΡΠ°ΠΌΠΌΡ, Π² Π½Π΅ΠΉ ΡΠ΅Π³ΡΠ»ΠΈΡΡΠ΅ΠΌ ΠΊΠ»Π°Π²ΠΈΡΠΈ ΠΆΠΌΡΠΌ Q ΠΈ ΠΏΠΎΡΠ»Π΅ Π·Π°Π²Π΅ΡΡΠ΅Π½ΠΈΡ ΡΠ°Π±ΠΎΡΡ Π±ΡΠ΄Π΅Ρ ΡΠΎΠ·Π΄Π°Π½ midi ΡΠ°ΠΉΠ».
GNU/Linux:
./v2m.py ./synthesia_video.mkv
Windows+Anaconda2 (python 2.7)/Anaconda3 (python 3.7):
in start menu search and open Anaconda command prompt:
cd path to v2m.py
python v2m.py synthesia_video.mkv
Π£ΠΏΡΠ°Π²Π»Π΅Π½ΠΈΠ΅:
- h - ΠΏΠΎΠΊΠ°Π·Π°ΡΡ/ΡΠΏΡΡΡΠ°ΡΡ ΠΏΠΎΠΌΠΎΡΡ
- q - ΠΏΡΠΈΡΡΡΠΏΠΈΡΡ ΠΊ Π²ΠΎΡΡΠΎΠ·Π΄Π°Π½ΠΈΡ midi
- s - Π£ΡΡΠ°Π½ΠΎΠ²ΠΈΡΡ Π½Π°ΡΠ°Π»ΡΠ½ΡΠΉ ΠΊΠ°Π΄Ρ ΠΎΠ±ΡΠ°Π±ΠΎΡΠΊΠΈ (ΠΌΠΎΠ΄ΠΈΡΠΈΠΊΠ°ΡΠΎΡ: shift, ΡΠ±ΡΠΎΡ Π½Π° Π½Π°ΡΠ°Π»ΡΠ½ΡΠΉ ΠΊΠ°Π΄Ρ Π²ΠΈΠ΄Π΅ΠΎ)
- e - Π£ΡΡΠ°Π½ΠΎΠ²ΠΈΡΡ ΠΊΠΎΠ½Π΅ΡΠ½ΡΠΉ ΠΊΠ°Π΄Ρ ΠΎΠ±ΡΠ°Π±ΠΎΡΠΊΠΈ (ΠΌΠΎΠ΄ΠΈΡΠΈΠΊΠ°ΡΠΎΡ: shift, ΡΠ±ΡΠΎΡ Π½Π° ΠΊΠΎΠ½Π΅ΡΠ½ΡΠΉ ΠΊΠ°Π΄Ρ Π²ΠΈΠ΄Π΅ΠΎ), Π½Π° Π½Π΅ΠΊΠΎΡΠΎΡΡΡ ΡΠΎΡΠΌΠ°ΡΠ°Ρ , Π½Π΅ ΡΠ°Π±ΠΎΡΠ°Π΅Ρ ΠΊΠΎΡΡΠ΅ΠΊΡΠ½ΠΎ, ΠΏΠΎΡ ΠΎΠΆΠ΅ Π½Π° Π±Π°Π³ OpenCV
- p - ΠΡΠ»ΠΈ ΠΊΠ»Π°Π²ΠΈΡΠ° ΡΠΊΠ°Π·Π°Π½Π°, ΠΏΡΠΈΠ½ΡΠ΄ΠΈΡΠ΅Π»ΡΠ½ΠΎ ΡΠ°Π·Π΄Π΅Π»ΠΈΡ Π²ΡΠ²ΠΎΠ΄ Π½Π° 2 ΠΊΠ°Π½Π°Π»Π° Π½Π΅ Π·Π°Π²ΠΈΡΠΈΠΌΠΎ ΠΎΡ Π½Π°ΡΡΡΠΎΠ΅ΠΊ. Π Π°Π·Π΄ΠΈΠ»Π΅Π½ΠΈΠ΅ Π±ΡΠ΄Π΅Ρ ΠΏΡΠΎΠ²Π΅Π΄Π΅Π½ΠΎ Π² Π·Π°Π²ΠΈΡΠΈΠΌΠΎΡΡΠΈ ΠΎΡ ΠΏΠΎΠ»ΠΎΠΆΠ΅Π½ΠΈΡ ΠΊΠ»Π°Π²ΠΈΡΠΈ ΠΎΡΠ½ΠΎΡΠΈΡΠ΅Π»ΡΠ½ΠΎ ΡΠΊΠ°Π·Π°Π½ΠΎΠΉ ΠΊΠ»Π°Π²ΠΈΡΠΈ. ΠΡΠΏΠΎΠ»ΡΠ·ΡΠ΅ΡΡΡ Π½Π° Π²ΠΈΠ΄Π΅ΠΎ Ρ ΠΎΠ΄Π½ΠΈΠΌ ΡΠ²Π΅ΡΠΎΠΌ ΠΊΠ»Π°Π²ΠΈΡ.
- o - ΠΠΊΠ»ΡΡΠΈΡΡ/Π²ΡΠΊΠ»ΡΡΠΈΡΡ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡΡ ΡΠ°ΡΠΏΠΎΠ·Π½Π°Π²Π°ΡΡ ΠΏΠ΅ΡΠ΅ΠΊΡΡΡΠΈΠ΅ ΠΊΠ»Π°Π²ΠΈΡ Π΄ΡΡΠ³ Π΄ΡΡΠ³ΠΎΠΌ. Π Π΄Π°Π½Π½ΠΎΠΌ ΡΠ»ΡΡΠ°Π΅ Π² ΠΌΠΎΠΌΠ΅Π½Ρ ΠΏΠ΅ΡΠ΅ΠΊΡΡΡΠΈΡ Π±ΡΠ΄Π΅Ρ ΡΠΎΠ·Π΄Π°Π½ΠΎ ΠΎΠΊΠΎΠ½ΡΠ°Π½ΠΈΠ΅ ΠΎΠ΄Π½ΠΎΠΉ Π½ΠΎΡΡ ΠΈ Π½Π°ΡΠ°Π»ΠΎ Π΄ΡΡΠ³ΠΎΠΉ.
- i - ΠΠΊΠ»ΡΡΠΈΡΡ/Π²ΡΠΊΠ»ΡΡΠΈΡΡ ΠΈΠ³Π½ΠΎΡΠΈΡΠΎΠ²Π°Π½ΠΈΠ΅/ΡΠ΄Π»ΠΈΠ½Π΅Π½ΠΈΠ΅ Π½ΠΎΡ ΠΌΠ΅Π½ΡΡΠ΅ ΠΌΠΈΠ½ΠΈΠΌΠ°Π»ΡΠ½ΠΎΠΉ Π΄Π»ΠΈΡΠ΅Π»ΡΠ½ΠΎΡΡΠΈ Π½ΠΎΡΡ ( ΠΡΠ»ΠΈ Π²ΠΊΠ»ΡΡΠ΅Π½ΠΎ Π΄Π°Π½Π½ΡΠ΅ Π½ΠΎΡΡ Π½Π΅ Π±ΡΠ΄ΡΡ Π·Π°ΠΏΠΈΡΠ°Π½Ρ Π² midi. ΠΡΠ»ΠΈ Π²ΡΠΊΠ»ΡΡΠ΅Π½ΠΎ Π½ΠΎΡΡ Π΄Π»ΠΈΡΠ΅Π»ΡΠ½ΠΎΡΡΡ ΠΊΠΎΡΠΎΡΡΡ Π±ΡΠ΄Π΅Ρ ΠΌΠ΅Π½ΡΡΠ΅ ΡΠΊΠ°Π·Π°Π½Π½ΠΎΠΉ Π±ΡΠ΄ΡΡ Π°Π²ΡΠΎΠΌΠ°ΡΠΈΡΠ΅ΡΠΊΠΈ ΠΏΡΠΈΡΠ°Π²Π½Π΅Π½Ρ ΠΌΠΈΠ½ΠΈΠΌΠ°Π»ΡΠ½ΠΎΠΉ Π΄Π»ΠΈΡΠ΅Π»ΡΠ½ΠΎΡΡΠΈ. )
- r - ΠΠΊΠ»ΡΡΠΈΡΡ/Π²ΡΠΊΠ»ΡΡΠΈΡΡ ΡΡΠ½ΠΊΡΠΈΡ ΠΌΠ°ΡΡΡΠ°Π±ΠΈΡΠΎΠ²Π°Π½ΠΈΡ
- Mouse wheel - ΠΏΠΎΠ΄ΡΡΡΠΎΠΉΠΊΠ° ΠΊΠ»Π°Π²ΠΈΡ
- ΠΠ΅Π²Π°Ρ ΠΊΠ½ΠΎΠΏΠΊΠ° ΠΌΡΡΠΈ - ΠΏΠ΅ΡΠ΅ΡΠ°ΡΠΊΠΈΠ²Π°Π½ΠΈΠ΅ Π²ΡΠ±ΡΠ°Π½Π½ΠΎΠΉ ΠΊΠ»Π°Π²ΠΈΡΠΈ / Π²ΡΠ±ΠΎΡ ΡΠ²Π΅ΡΠ° ΠΈΠ· ΠΊΠ°ΡΡΡ ΡΠ²Π΅ΡΠΎΠ².
- CTRL + ΠΠ΅Π²Π°Ρ ΠΊΠ½ΠΎΠΏΠΊΠ° ΠΌΡΡΠΈ - ΠΎΠ±Π½ΠΎΠ²ΠΈΡΡ Π²ΡΠ±ΡΠ°Π½Π½ΡΠΉ ΡΠ²Π΅Ρ ΠΊ ΠΊΠ°ΡΡΠ΅ ΡΠ²Π΅ΡΠΎΠ².
- ΠΡΠ°Π²Π°Ρ ΠΊΠ½ΠΎΠΏΠΊΠ° ΠΌΡΡΠΈ - ΠΏΠ΅ΡΠ΅ΡΠ°ΡΠΊΠΈΠ²Π°Π½ΠΈΠ΅ Π²ΡΠ΅Ρ ΠΊΠ»Π°Π²ΠΈΡ, Π΅ΡΠ»ΠΈ ΠΊΠ»Π°Π²ΠΈΡΠ° Π²ΡΠ±ΡΠ°Π½Π°, ΠΏΠ΅ΡΠ΅Π½ΠΎΡ ΠΎΡΡΡΠ΅ΡΡΠ²Π»ΡΠ΅ΡΡΡ ΠΎΡΠ½ΠΎΡΠΈΡΠ΅Π»ΡΠ½ΠΎ Π½Π΅Ρ
- CTRL + 0 - ΠΡΠΊΠ»ΡΡΠΈΡΡ Π²ΡΠ±ΡΠ°Π½Π½ΡΠΉ ΡΠ²Π΅Ρ.
- Π‘ΡΡΠ΅Π»ΠΊΠΈ - ΠΏΠΎΠ΄ΡΡΡΠΎΠΉΠΊΠ° ΠΊΠ»Π°Π²ΠΈΡ (ΠΌΠΎΠ΄ΠΈΡΠΈΠΊΠ°ΡΠΎΡ: shift)
- PageUp/PageDown - ΠΏΡΠΎΠΊΡΡΡΠΊΠ° Π²ΠΈΠ΄Π΅ΠΎ (ΠΌΠΎΠ΄ΠΈΡΠΈΠΊΠ°ΡΠΎΡ: shift, ΡΠ°Π³ ΠΏΠΎ ΠΊΠ°Π΄ΡΡ)
- Home/End - ΠΏΠ΅ΡΠ΅Ρ ΠΎΠ΄ Π² Π½Π°ΡΠ°Π»ΠΎ ΠΈΠ»ΠΈ ΠΊΠΎΠ½Π΅Ρ Π²ΠΈΠ΄Π΅ΠΎ
- [ / ] - ΠΈΠ·ΠΌΠ΅Π½ΠΈΡΡ Π±Π°Π·ΠΎΠ²ΡΡ ΠΎΠΊΡΠ°Π²Ρ
- F2/F3 - Π·Π°ΠΏΠΈΡΠ°ΡΡ / Π·Π°Π³ΡΡΠ·ΠΈΡΡ Π½Π°ΡΡΡΠΎΠΉΠΊΠΈ.
- ESCAPE - Π²ΡΡ ΠΎΠ΄ / quit
- SPACE - ΠΏΡΠ΅ΡΠ²Π°ΡΡ Π²ΠΎΡΡΠΎΠ·Π΄Π°Π½ΠΈΠ΅ ΠΈ Π·Π°ΠΏΠΈΡΠ°ΡΡ midi ΡΠ°ΠΉΠ» Π½Π° Π΄ΠΈΡΠΊ / abort re-creation and save midi file to disk
ΠΊΠ°ΠΊ ΡΡΠΎ ΡΠ°Π±ΠΎΡΠ°Π΅Ρ
ΠΠ°Π΄Ρ Π·Π° ΠΊΠ°Π΄ΡΠΎΠΌ ΡΠΊΠ°Π½ΠΈΡΡΠ΅ΡΡΡ Π²ΠΈΠ΄Π΅ΠΎ ΠΏΠΎΡΠΎΠΊ ΠΎΡΡΠ»Π΅ΠΆΠΈΠ²Π°Ρ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΡ Π²ΠΈΡΡΡΠ°Π»ΡΠ½ΠΎΠΉ ΠΊΠ»Π°Π²ΠΈΠ°ΡΡΡΡ ΠΏΠΎΡΠ»Π΅ Π²ΡΡ ΡΡΠΎ Π·Π°ΡΠΈΠΊΡΠΈΡΠΎΠ²Π°Π½ΠΎ Π΄Π°ΠΌΠΏΠΈΠΌ Π½Π° Π²ΠΈΠ½Ρ.
ΠΠΎΠΏΠΎΠ»Π½ΠΈΡΠ΅Π»ΡΠ½ΡΠ΅ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡΠΈ
ΠΡΠ΅ Π½Π°ΡΡΡΠΎΠΉΠΊΠΈ Π²ΡΠ½Π΅ΡΠ΅Π½Ρ Π² ΡΠ°ΠΉΠ» ini ΡΠ°ΠΉΠ» ΠΊΠΎΡΠΎΡΡΠΉ ΠΌΠΎΠΆΠ΅Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°ΡΡΡΡ ΠΊΠ°ΠΊ ΠΎΠ±ΡΠΈΠΉ Π΄Π»Ρ Π²ΡΠ΅Ρ ΠΊΠ°ΡΠ°Π»ΠΎΠ³ΠΎΠ² Π΅ΡΠ»ΠΈ Π½Π°Ρ ΠΎΠ΄ΠΈΡΡΡ Π² Π΄ΠΎΠΌΠ°ΡΠ½Π΅ΠΌ ΠΊΠ°ΡΠ°Π»ΠΎΠ³Π΅ ~/.v2m.ini Π»ΠΈΠ±ΠΎ ΠΎΡΠ΄Π΅Π»ΡΠ½ΡΠΉ Π»ΠΎΠΊΠ°Π»ΡΠ½ΡΠΉ Π΄Π»Ρ ΠΊΠ°ΡΠ°Π»ΠΎΠ³Π° ./v2m.ini.
ΠΡ ΠΌΠΎΠΆΠ΅ΡΠ΅ Π½Π°ΡΡΡΠΎΠΈΡΡ ΡΠ°Π·Π±ΠΈΠ΅Π½ΠΈΠ΅ Π½Π° ΠΊΠ°Π½Π°Π»Ρ Π² Π·Π°Π²ΠΈΡΠΈΠΌΠΎΡΡΠΈ ΠΎΡ ΡΠ²Π΅ΡΠ° ΠΊΠ»Π°Π²ΠΈΡΠΈ. ΠΠ»Ρ ΡΡΠΎΠ³ΠΎ Π² ΡΠ°ΠΉΠ»Π΅ v2m.ini Π½ΡΠΆΠ½ΠΎ ΠΏΠΎΠΏΡΠ°Π²ΠΈΡΡ ΡΠΎΠΎΡΠ²Π΅ΡΡΡΠ²ΠΈΠ΅ ΡΠ²Π΅ΡΠ° ΠΊΠ°Π½Π°Π»Ρ midi ΡΡΠ΅ΠΊΠ°. ΠΠΎ ΡΠΌΠΎΠ»ΡΠ°Π½ΠΈΡ ΠΊΠ°ΠΆΠ΄ΡΠΉ ΡΠ²Π΅Ρ Π°ΠΊΡΠΈΠ²ΠΈΡΡΡΡΠΈΠΉ ΠΊΠ»Π°Π²ΠΈΡΡ Π±ΡΠ΄Π΅Ρ Π·Π°ΠΏΠΈΡΠ°Π½ Π² ΡΠΎΠ±ΡΡΠ²Π΅Π½Π½ΡΠΉ ΠΊΠ°Π½Π°Π», ΡΠ°ΠΊΠΈΠΌ ΠΎΠ±ΡΠ°Π·ΠΎΠΌ Π΅ΡΠ»ΠΈ Ρ ΠΎΡΠΈΡΠ΅ ΠΎΠ±ΡΠ΅Π΄ΠΈΠ½ΠΈΡΡ ΠΊΠ°Π½Π°Π»Ρ ΠΏΡΠΎΡΡΠΎ ΡΠΊΠ°ΠΆΠΈΡΠ΅ Π΄Π»Ρ ΡΠ°Π·Π½ΡΡ ΡΠ²Π΅ΡΠΎΠ² ΠΎΠ΄ΠΈΠ½Π°ΠΊΠΎΠ²ΡΠ΅ Π½ΠΎΠΌΠ΅ΡΠ°.
color_channel_accordance = 0,0, 1,1, 2,2, 3,3, 4,4, 5,5
ΠΡ ΠΌΠΎΠΆΠ΅ΡΠ΅ Π½Π°ΡΡΡΠΎΠΈΡΡ ΡΠΎΠΎΡΠ½Π΅ΡΠ΅Π½ΠΈΠ΅ ΠΊΠ°Π½Π°Π»Π° ΠΊ MIDI ΠΈΠ½ΡΡΡΡΠΌΠ΅Π½ΡΡ. ΠΠ»Ρ ΡΡΠΎΠ³ΠΎ Π² ΡΠ°ΠΉΠ»Π΅ v2m.ini Π½ΡΠΆΠ½ΠΎ ΠΏΠΎΠΏΡΠ°Π²ΠΈΡΡ ΡΠΎΠΎΡΠ²Π΅ΡΡΡΠ²ΠΈΠ΅ ΠΊΠ°Π½Π°Π»Π° midi ΠΈΠ½ΡΡΡΡΠΌΠ΅Π½ΡΡ. ΠΠΎ ΡΠΌΠΎΠ»ΡΠ°Π½ΠΈΡ ΠΊΠ°Π½Π°Π» ΡΠ°Π²Π΅Π½ 0 midi ΠΈΠ½ΡΡΡΡΠΌΠ΅Π½ΡΡ
channel_prog_accordance = 0,0, 0,0, 0,0, 0,0, 0,0, 0,0