By Kirk Spragg

The retail cost of video hardware is not a reliable guide to how well Leapfrog’s 3D visualisation functionality performs on that hardware. The more expensive workstation grade hardware solutions such as NVIDIA’s Quadro range of desktop cards are designed to accelerate operations that Leapfrog does not use. As a result, the 3D performance in Leapfrog is often no better than less expensive gaming and home grade video hardware.

In this post Applications Specialist Kirk Spragg compares five home and gaming grade video cards with a workstation grade Quadro 4000 by benchmarking the cards to determine their relative performance. The most expensive card is the Quadro 4000, the least expensive the Radeon HD 6450. The performance of each card is measured by the amount of detail Leapfrog can display with each card while maintaining smooth visualisation. To do so we will take a Leapfrog model, vary the model resolution and then measure the smoothness of the visualisation. The results and the details describing how they were obtained are given later in this post. We have also constructed a scene file for you to download. You can use this scene file to compare your video hardware with cards we examined in this post.

Hardware tested

We have a wide range of video cards here at our R&D centre in New Zealand which we use to test our software for driver and compatibility issues. We have six cards in our collection that are representative of what is currently available on the market. These are the cards whose performance we evaluated. The retail value of these cards ranges from $31.99 US to $719.99 US (see Note 1). The model names of the video cards and their retail price relative to the most expensive card is shown in Table 1 below:

Video card Relative price to Quadro 4000
NVIDIA Quadro 4000 100%
NVIDIA GTX 670 45%
AMD Radeon HD 7850 2GB 22%
NVIDIA 650 Ti 2GB 20%
AMD Radeon HD 7790 1GB 18%
NVIDIA GT 440 1 GB 9%
AMD Radeon HD 6450 1GB 5%

Table 1 – A list of the benchmarked video cards with their prices relative to the Quadro 4000

Model used for comparison

To stress test the visualisation capabilities of each graphics card, we needed a large regional model with sufficient complexity. This model also had to be in the public domain, so that we could show our results publicly. The Canterbury plains groundwater model (see Figure 1 below) fits both of these criteria. You may recognise this model from our case study on the Leapfrog website. By varying the model resolution we were able to generate a range of models, each of which contained more or less detail depending on the chosen model resolution.

A screenshot of the model used for benchmarking the video cards

Figure 1 – A screenshot of the model used for benchmarking the video cards in this blog post

Benchmarking procedure

To measure the performance of each video card, we took the Canterbury plains model and changed the model resolution until we could rotate a scene with all the model volumes displayed at a frame rate of 24 FPS (Frames Per Second) using full acceleration (see Note 2). Most people perceive 24 FPS as continuous motion. This is why movies in cinemas are displayed using this frame rate. At 24 FPS, the human eye cannot easily distinguish each frame as it is displayed. The result is the illusion of continuous motion. This illusion is what we experience when we watch a movie. Below 24 FPS, the individual frames are more easily recognisable to the eye. The result is a jerky sequence of images that no longer blends into a continuous motion.

We displayed each scene at full screen on a 1080p monitor (1920×1080 resolution) and rotated it using the left arrow key for 30 seconds measuring the frame rate. To measure the frame rate reliably, we had to rotate the scene for more than several seconds. The FPS was highly variable at first, but would stabilise after 20 seconds. So we added another 10 seconds to be safe.

Each card was measured in this way and we created a scene with the model that would display on that card at 24 FPS. We also estimated the level of detail in each model by calculating the number of triangles in that model. This is a reasonable measure of the detail in a model as all Leapfrog models are made from meshes, which are constructed out of triangles. A more detailed model has a large number of small triangles, which capture more detail. A less detailed model will have fewer triangles that are larger in size and capture less detail. The results of our testing are shown in the table below:

Video hardware Number of triangles in model
NVIDIA GTX 670 2,070,234
NVIDIA GTX 650 Ti 1,158,134
NVIDIA Quadro 4000 (see Note 3) 544,506
AMD Radeon HD 7850 2GB 359,050
NVIDIA Geforce GT 440 (see Note 3) 225,330
AMD Radeon HD 7790 1GB 200,784
AMD Radeon 6450 3,958

Table 2 – The results of our benchmarking procedure. The most powerful card, the NVIDIA GTX 670, is at the top, the least powerful at the bottom.

We created a Leapfrog Viewer file with the scenes of each model we constructed to benchmark the graphic cards which can be downloaded here (37MB). You will need the Leapfrog Viewer to view this file. So if you haven’t downloaded the free Leapfrog Viewer software yet, please visit the Leapfrog website.

Relative performance of graphics video cards

Figure 2 – A bar graph of both performance and cost of each of the video cards. Performance is shown by bar height, with more triangles equaling better performance. The cost is shown by the width of the bars. Bar width is directly proportional to the video card’s cost.

The results in Table 2 are shown graphically in Figure 2  above. The width of the bars is scaled by the price relative to the Quadro 4000. The height is the number of triangles a card could display at 24 FPS. The widest bar belongs to the most expensive card, the Quadro 4000. Unfortunately, this price does not translate into performance with Leapfrog. You can see that both, the NVIDIA GTX 670 and the GTX 650 Ti, offer better performance and both are less than half the cost of the Quadro 4000. While the GTX 670 offers the best performance, the 650 Ti has the best price/performance ratio. The 650Ti is half the cost of the GTX 670 and offers slightly more than half the performance.

How to compare your graphics hardware

The scenes in the scene file are ordered by the model resolution. The least detailed model is listed first, the most last. To see where your video hardware falls with respect to the cards we reviewed, display each scene in full screen mode with full acceleration turned on if possible (see Note 2). Rotate the scene by holding down one of the arrow keys for 30 seconds. Then look at the frame rate at the bottom right hand corner of the Leapfrog Viewer window as shown in Figure 3 below. You may need to exit full screen mode by pressing F11 to do this.

Leapfrog Viewer frame rate

Figure 3 – A screenshot of the Leapfrog Viewer. The frame rate is shown at the bottom of the window.

If the FPS value is greater than 24, your video hardware is performing better than the card used to create that scene. If the FPS is lower, your video hardware is performing worse than the video hardware used to make that particular scene. When you find two scenes, one of which rotates faster than 24 FPS and one that is slower, your video hardware is in between both in terms of the amount of detail Leapfrog can display while maintaining smooth visualisation.

Conclusion

It is clear from the graph of our benchmark results in Figure 2, that price does not necessarily equate to performance. Expensive workstation grade solutions such as NVIDIA’s Quadro range provide additional functionality that Leapfrog does not use. The functionality that Leapfrog does use is accelerated best by medium to high end home and gaming graphics hardware. This is why the NVIDIA GTX 670 and GTX650 Ti performed so well. If you are looking to improve the performance of Leapfrog’s visualisation capabilities, these cards offer the best performance.

Notes:

  1. Retail prices were taken from www.newegg.com on 2013/06/27 of the lowest cost video card with the same GPU and memory as those we reviewed in this post.
  2. We used full acceleration for all of our benchmarks as it improves the visual quality, particularly the rendering of transparency. Full acceleration also uses your video card’s memory more efficiently and allows larger models to be displayed. Older or less powerful video hardware may perform better using partial acceleration. However, this comes at the cost of visual quality and increased video memory usage, which may prevent these cards from displaying the more detailed models.
  3. There is an issue with Leapfrog and older NVIDIA cards that degrades performance when running in full screen mode. You can work around this by not running Leapfrog in full screen. This benchmark was measured with the scene displayed in a large window, NOT full screen to avoid this known issue.