Jump to content
Search Community

How to do a motion tracking editor in the browser?

Gilbert test
Moderator Tag

Recommended Posts

Hey Gilbert. It looks like Chris built out this custom editor himself. It looks like he created some drag and drop controls for certain properties, fine tuned those, then also made a process to step frame by frame through a video and sync each frame with each set of properties. 

 

Maybe he can tell you more about it himself. @chrisgannon?

  • Like 1
Link to comment
Share on other sites

44 minutes ago, Gilbert said:

he didn't say how he built it. So, I wanted to get some ideas

 

From his old blog you can find two pages of articles where he mentions using Adobe Edge Animate CC 2014 & Greensock’s Draggable along with some vague other hints. Edge stopped being developed in November of 2015 though. https://chrisgannon.wordpress.com/?s=motion

  • Like 1
Link to comment
Share on other sites

 

What you see there was some software I was developing that was a progression from a more convoluted process involving motion tracking. I was never very happy with the results and ended up using my initial process which I'll describe.

 

So this initial process involved several steps using different software. I never posted the steps because it was really complicated and had a lot of quirks.

 

If I remember correctly, I generated tracking data using Mocha inside After Effects. I then wrote a parser to convert it to a corner pin data format (basically an array of X and Y vector points).

 

Then I used a library called PerspectiveTransform.js to apply the corner pin data to an element (this was crucial) — this could be anything from a video to an SVG or JPG. This library skewed and scaled elements based on their X and Y data.

 

Then I converted the video I wanted to composite the element onto to an image sequence (massive faff!). A lot of people thought I was using video but at the time, video events and keyframes were really unreliable, the browser specs kept changing and it couldn't keep up with the data stream half the time anyway.

 

I then created a player that accepted the image sequence, the element(s) to composite (these could be interactive like a DIV of HTML elements) and the corner pin data and married the whole lot up on a draggable timeline (that could also be played back).

 

My overall aim with this project was to have interactive elements composited onto video and tracked in 3D space which I managed to do (I even managed to track an image sequence in 3D space composited on top of another video) — bizarrely all of the demos were done in Edge Animate because at the time I was transitioning from Flash to not Flash and I didn't know pure JS, CSS or anything.

 

Chrome kept changing its video specs so every week or so I had to update my libraries. These changes were so frequent (and often broke all my work) that I eventually gave up because it was just far too much work to keep it maintained, plus the whole process of getting an interactive element tracked in 3D and composited onto video was a sprawling mess! 

 

Having said that, the results were pretty good so it wasn't all wasted time :)

 

Hope that helps!

  • Like 5
Link to comment
Share on other sites

2 hours ago, chrisgannon said:

Chrome kept changing its video specs so every week or so I had to update my libraries. These changes were so frequent (and often broke all my work) that I eventually gave up because it was just far too much work to keep it maintained, plus the whole process of getting an interactive element tracked in 3D and composited onto video was a sprawling mess!


I feel ya.. but you gotta love Chrome, the new IE :)
 

  • Like 1
Link to comment
Share on other sites

@chrisgannon, thanks for the detailed explanation.

I also came from Flash (Flex) and I was doing mainly visualization stuff. After Flash I learned D3 and I was doing lots of experiments with Edge Animate.

 

Now, I am exploring what I can do to combine video with animation overlays with GSAP. (on my free time)

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...