I had 6 red stickers on the wall to reconstruct camera position relative to the scene, warped the original effects by SeeMusic and pasted into the 3D scene.
python (opencv mostly) + shotcut. The six red stickers on the wall are used to resolve the camera position/angle, calculate the corners of the 1) lighting on the keyboard, 2) virtual screen with falling notes and 3) shadows on the wall. Then warp and blend the seemusic video into the scene.
python (opencv mostly) + shotcut. The six red stickers on the wall are used to resolve the camera position/angle, calculate the corners of the 1) lighting on the keyboard, 2) virtual screen with falling notes and 3) shadows on the wall. Then warp and blend the seemusic video into the scene.
This is really interesting! I am wondering what kind of program did you use it for?