Comments on: FPStress I think I might have found the FPS culprit... it's a great marketting tool.. Looking at GPU hardware sites they always compare GPU speeds by playing games and comparing the FPS.. 1. It is the actual final measurement that a player notices... Any timing that doesn't increase or decrease the FPS is irrelevant to a player. 2. It conveniently curves the results so as to show extra deviation amongst the high end values... so 69fps, 70fps, 71fps. Generally the cards of interest are all the fastest and usually closest to the same speed... so having the most accuracy at the high end is useful for making them look noticeably faster (when in reality they are not). I think I might have found the FPS culprit… it’s a great marketting tool..
Looking at GPU hardware sites they always compare GPU speeds by playing games and comparing the FPS..

1. It is the actual final measurement that a player notices… Any timing that doesn’t increase or decrease the FPS is irrelevant to a player.

2. It conveniently curves the results so as to show extra deviation amongst the high end values… so 69fps, 70fps, 71fps. Generally the cards of interest are all the fastest and usually closest to the same speed… so having the most accuracy at the high end is useful for making them look noticeably faster (when in reality they are not).

]]>
By: Marshall Robin/2011/02/28/fpstress/#comment-1172 Marshall Robin Thu, 03 Mar 2011 03:03:44 +0000 I think over other concerns you shouldn't use FPSs because it's really hard to sum : "I want a 30 FPS simulation, I have a physic engine running at 200 FPS, a simulated sea running at 50 FPS and a navigation engine at 40 FPS, will it miss the mark?" I think over other concerns you shouldn’t use FPSs because it’s really hard to sum :
“I want a 30 FPS simulation, I have a physic engine running at 200 FPS, a simulated sea running at 50 FPS and a navigation engine at 40 FPS, will it miss the mark?”

]]>
By: Aurélien Pocheville/2011/02/28/fpstress/#comment-1130 Aurélien Pocheville Tue, 01 Mar 2011 11:34:57 +0000 Funny... I sent a document to our team about half a year ago that is very similar to your post. It was called '5 reasons why the framerate counter is not a profiler'. It was intended to artists/designers but also to some programmers. Things in my doc that I did not see in your list were (fairly obvious ones perhaps): 1) Framerate only shows the performance of the weakest link, referring to multiprocessor architectures and whether you are CPU/GPU bound. Too often artists, designer or perhaps even programmers use the framerate counter to see if a modification to the game changed anything to the framerate of the game. If you are heavily GPU bound in the section you are testing, you might have introduced a major CPU performance problem without noticing. 2) If you are measuring framerate through a custom framerate counter (and not through PIX, for instance), then chances are that you have a mechanism to average the counter over several frames to make the counter better readable. This will surely hide spikes, so it is easy to miss those. Also, I actually have worked on projects where the framerate counter had a severe impact on performance because of costly font rendering :D. Funny… I sent a document to our team about half a year ago that is very similar to your post. It was called ’5 reasons why the framerate counter is not a profiler’. It was intended to artists/designers but also to some programmers.

Things in my doc that I did not see in your list were (fairly obvious ones perhaps):
1) Framerate only shows the performance of the weakest link, referring to multiprocessor architectures and whether you are CPU/GPU bound. Too often artists, designer or perhaps even programmers use the framerate counter to see if a modification to the game changed anything to the framerate of the game. If you are heavily GPU bound in the section you are testing, you might have introduced a major CPU performance problem without noticing.

2) If you are measuring framerate through a custom framerate counter (and not through PIX, for instance), then chances are that you have a mechanism to average the counter over several frames to make the counter better readable. This will surely hide spikes, so it is easy to miss those.

Also, I actually have worked on projects where the framerate counter had a severe impact on performance because of costly font rendering :D.

]]>
By: James Podesta/2011/02/28/fpstress/#comment-1115 James Podesta Tue, 01 Mar 2011 01:03:58 +0000 ...and that in turn caused lots of arguments between NTSC, PAL50 and PAL60 folks about what a raster line actually is. Ah, happy memories :-) …and that in turn caused lots of arguments between NTSC, PAL50 and PAL60 folks about what a raster line actually is. Ah, happy memories :-)

]]>
By: James Podesta/2011/02/28/fpstress/#comment-1110 James Podesta Mon, 28 Feb 2011 22:30:30 +0000 Any time you need to do math on numbers, you need to use time, e.g. "this blur effect costs 2ms and this particle system costs 3ms, we're already running at 12ms per frame so if we turn them both on we'll blow out the frame budget of 16ms". In ideal cases, it is possible to get identical intuition by using either FPS or mSPF. But so often I see people get the wrong intuition from using FPS and end up saying inadvertently silly things. You only do that a few times before everyone agrees to just ban use of FPS. Measuring and reasoning about graphics performance is already really difficult without adding a reciprocal in at every step. The one exception is when talking to pointy-haired-bosses and small children. Any time you need to do math on numbers, you need to use time, e.g. “this blur effect costs 2ms and this particle system costs 3ms, we’re already running at 12ms per frame so if we turn them both on we’ll blow out the frame budget of 16ms”.

In ideal cases, it is possible to get identical intuition by using either FPS or mSPF. But so often I see people get the wrong intuition from using FPS and end up saying inadvertently silly things. You only do that a few times before everyone agrees to just ban use of FPS. Measuring and reasoning about graphics performance is already really difficult without adding a reciprocal in at every step.

The one exception is when talking to pointy-haired-bosses and small children.

]]>
By: snake5/2011/02/28/fpstress/#comment-1106 snake5 Mon, 28 Feb 2011 19:57:14 +0000 A classic, but that bears repeating. How often do I have people make this mistake. And they start to complain on why they've "lost" some many FPS. @snake5: Maybe the argument is every time the same, but the fact is that the human brain is <b>much</b> better at linear computation that at inverses. -what if I need to compare numbers relatively to see if it’s an increase or a decrease in speed/time? It makes no difference what I use. => so better to use the intuitive one -what if everyone knows how to do basic number inversion without any calculators? One simple rule to remember: bigger FPS=smaller changes. => true, but I have yet to find theses "everyone" -what if I try to implement something slow and I just want to see if the speed is above or below the requirements? => spec your requirements in seconds. The beauty is that the minimum FPS can be expressed in seconds, so you have to do the inversion only once. A classic, but that bears repeating. How often do I have people make this mistake. And they start to complain on why they’ve “lost” some many FPS.
@snake5:
Maybe the argument is every time the same, but the fact is that the human brain is much better at linear computation that at inverses.
-what if I need to compare numbers relatively to see if it’s an increase or a decrease in speed/time? It makes no difference what I use.
=> so better to use the intuitive one
-what if everyone knows how to do basic number inversion without any calculators? One simple rule to remember: bigger FPS=smaller changes.
=> true, but I have yet to find theses “everyone”
-what if I try to implement something slow and I just want to see if the speed is above or below the requirements?
=> spec your requirements in seconds. The beauty is that the minimum FPS can be expressed in seconds, so you have to do the inversion only once.

]]>
By: snake5/2011/02/28/fpstress/#comment-1103 snake5 Mon, 28 Feb 2011 18:40:22 +0000 Amen brother. Amen brother.

]]>