Usefulness of FPS meters

[numblock]

Hi gsap,

 

First of all thanks for all your brilliant work, i've been a big fan since AS2 days!

 

I have read articles/posts in the past where you mentioned that FPS meters were not useful or unreliable.

Here: https://css-tricks.com/myth-busting-css-animations-vs-javascript/

and here:

http://greensock.com/forums/topic/11244-fps-examples/

 

Could you provide some more details about your reasons for not wanting to rely on FPS meters for measuring animation performance? Like what were the inconsistencies you were seeing in browser numbers and their causes?

And maybe some alternative methods for measuring performance?

 

Thanks!

 

[GreenSock]

 Here are a few of the reasons that I can recall:

1. I was trying to compare some JS-based things with CSS animations/transitions. The only way to do an FPS meter is with JavaScript. The browsers were not reporting things accurately for non-JS things. For example, if I used a requestAnimationFrame loop to measure how many times the screen actually got repainted, it might report as 12fps even though visually it was blatantly obvious that it was more often than that. This is at least partially due to the fact that JS always runs on the main thread but *sometimes* transforms or opacity animations are spun off to a different thread. 
2. I found that Dev Tools wasn't reliably reporting things, particularly for CSS-based animations. See http://greensock.com/css-performance/ It made them *appear* to be much less expensive than they actually were. 
3. I was told by extremely well-respected members of the Chrome team that they knew FPS counters were inaccurate and there was no way to solve that (at least at that point). 
4. Some animations entailed a hybrid of JS and CSS, like Zepto would use JS to fire off animations by applying CSS transitions (via JS). Thus when the CSS stuff was doing its thing, it actually slowed the main thread way down (again, because even if things are theoretically spun off to another thread, there is a bunch of overhead expense, as shown in my video link above). So you'd see this funky delay between when animations would start (they'd get clumped together) and then during the actual animation it'd appear smoother. Since the FPS counter was in the JS level, it was inaccurate.
5. Some FPS counters used requestAnimationFrame while others used setTimeout or setInterval. Some of the core engines (like jQuery) used setTimeout() for animation ticks/iterations, but if you try to measure things with a requestAnimationFrame meter, it's again inaccurate because it's not synced with those setTimout calls. In other words, if jQuery is set to try to run at 100fps, that means 1 tick every 10 milliseconds whereas requestAnimationFrame is capped at 60fps (1 every 16.7ms). You could have some occasions where jQuery runs 2 updates inbetween requestAnimationFrame ticks (two movements, one screen refresh). You can't really use setTimeout() to drive an FPS counter because those aren't synced with screen refreshes (which are typically what people associate with a "frame").

So basically, in order to have a somewhat accurate FPS meter, you must make sure that **everything** you're measuring is also perfectly in sync with requestAnimationFrame ticks. The browser doesn't always cooperate. Oh, and some older browsers don't even offer requestAnimationFrame support. So it just gets really tricky to build something that you can measure accurately across a bunch of devices.

 

In my opinion, it's very easy to miss the point of FPS meters and get side-tracked by the tech. Ultimately, why do we even care about FPS meters? Typically it's because we want to benchmark things so that we know which one will perform better. For animation, "perform better" almost always means "looks smoother". So I tend to put a lot more emphasis on that. Use your eyes. Push things to their limits and see where they break down, and measure the different techniques at that point rather than getting hyper-focused on FPS numbers. I've seen cases where an animation reports a higher frame rate but looks visibly worse than something with a lower FPS number. I have also seen things that made it seem like CSS was better on Chrome on a fast Mac, but then I tried it on some mobile devices and the JS version blew the doors off the CSS one. So it wasn't as easy as just cracking open Chrome and saying "oh, look, the FPS meter says this way is better...cool, we'll go with that."

 

There are other factors too, like the way animation delays are calculated, avoiding layout thrashing, and synchronization across every part of an engine. I explain some of that here along with some comparisons: http://greensock.com/gsap-1-12-0. Again, if you myopically focus on an FPS counter, you may miss some of these other factors that make a big difference in how things will actually look. 

 

I hope that's at least a little bit helpful rather than just being utterly confusing.  

[numblock]

Thanks so much for such a detailed answer, Jack! This is exactly what I was looking for!