SVG benchmarks
 | Outdated
This page largely represents the 2006 situation. |
For rasterizing SVG images we've used librsvg for some time. Another contender which we didn't choose originally is Batik. There are several respects in which librsvg is unsatisfactory and Batik might be preferable.
librsvg: In favour: fast; totally free/libre. Against: Lots of GNOME-ish dependencies, which makes tracking updates difficult; We have to maintain a security patch to disable the loading of external URL resources, since librsvg maintainers weren't interested in it; Lots of rendering bugs (but it's improved over time); Unclear fonts/text rendering situation.
Batik: In favour: more mature; higher quality rendering; built-in security mode. Against: slow!; doesn't (or at least didn't) run on free/libre Java (GCJ etc).
Benchmark
[edit]Brion did a quick test of SVG rendering (19 Dec 2006), with 50 images pulled off Commons. Test run on a MacBook, 2 GHz Intel Core 2 Duo, in Ubuntu running under Parallels (using one virtual processor). rsvg is Ubuntu's 2.16.0; batik is 1.6, running on Sun's J2SE 6 runtime.
- rsvg: 18.07 seconds; 361.46ms per image
- batik: 108.44 seconds; 2168.84ms per image
Batik is probably slowed down by having to start up the VM and JIT on every call. It likely can be sped up by using a daemon process and handing off render requests to that.
Batik also refused to render one file that rsvg did render.
ivar notes that to avoid JDK startup time you can use nailgun.
Additional notes
[edit]- Batik command-line renderer appears to scale image height to be proportional with the given width. This could produce width/height inconsistencies between specified heights from MW and results.
- I'm pretty sure Batik has a mode for increased safety (rejecting external links), which we have to maintain as a patch hack for librsvg. Not having to maintain a patch is attractive.
- We frequently get requests about font support. It'd be nice to know how to answer such questions.
- rsvg's resource usage can be limited with ulimit etc easily; what about a batik daemon? How to stop giiaaannttttt files from eating up RAM without taking down the server?
- You'd have to look at jvm args... -- chris
Further testing
[edit]2008 (57 images)
[edit]David Gerard ran a test of ImageMagick 6.2.4, Inkscape 0.45.1, Batik 1.7.0 (on Sun Java 1.6.0_03-b05) and rsvg 2.18.2 on 57 Commons images, including some in the megabytes. Those which reliably gave an error from the renderers were left out. Test box was a Dell Latitude D610, Pentium M 1.6 GHz, 512MB memory, running Kubuntu 7.10. All test applications except Batik were Ubuntu repository versions.
All four programs were run through the 57 images 115 times and timed with /usr/bin/time. X running in Failsafe (konsole as only app), nothing else running.
| median real | mean real | median CPU | mean CPU | |
|---|---|---|---|---|
| Batik | 139.71 | 139.95 | 137.02 | 137.04 |
| ImageMagick | 112.43 | 112.67 | 111.7 | 111.73 |
| Inkscape | 51.43 | 51.52 | 51.18 | 51.2 |
| rsvg | 25.97 | 26.03 | 25.8 | 25.8 |
Worrying:
- Batik's JVM crapped itself several times for no apparent reason - 39 times in the run, i.e. 39 in 4485 attempted renderings. Not on any particular SVG either.
- Inkscape thrashed the disk. (Single physical disk in laptop.)
(Possible explanation: that rsvg is a lib whereas Inkscape is a full app - could the overhead of starting and stopping the app be significant?)
What this suggests: use rsvg if possible (anything Unix), use Inkscape if rsvg's dependencies are too troublesome (e.g. on Windows, where Inkscape is available as a standalone app) and you have a lot of memory.
See /test 20080112 for full data.
2009 (56 files)
[edit]Hk kng re-ran the tests in 2009 using a http server solution for Batik: batikd. Full test results are at /test 20090816. WikiMedia Commons presents a gallery of the test images.
A comparison of CPU times is not as easily done, since the real image conversion for Batik is done in the separate server process. The wallclock times look like this:
| Program | Median time |
|---|---|
| Batik | 57.76 s |
| ImageMagick | 81.86 s |
| Inkscape | 70.95 s |
| rsvg | 41.48 s |
2021 (337+1338+141 files, including correctness ratio)
[edit]JoKalliauer ran several test suites in 2021 and wrote detailed explanations about rendering issues; see c:User:JoKalliauer/SVG_test_suites for details.
| librsvg 2.50 | resvg 0.14.0 | Inkscape 1.0 | Batik 1.13; 1.14[clarification needed] | |
|---|---|---|---|---|
| Correctness according to the W3C test suite | 0,662 | 0,831 | 0,745 | 0,801 |
| cpu-time taken in the W3C test suite (512 px) | 1m 46,776s | 1m 04,490s | 7m 57,465s | 6m 51,560s |
| Correctness according to resvg test suite | 0.754 | 0.956 | 0.729 | 0.703 |
| Time measured by resvg test suite | 4min 05sek | 2min 30sek | 46min 22sek | 61min 29sek |
| Correctness with Wikimedia featured pictures | 0.92 | 1.00 | ||
| Time with Wikimedia featured pictures | 5m 17,701s | 4m 46,639s | 15m 28,202s | 11m 30,768s |