Is Safari Really That Fast?

Late last week, Apple held their annual iPad and Mac event where they unveiled next generation iPads, iMacs and Mac Minis. If you're interested, you can watch the whole stream. And while new Apple hardware is always pretty exciting, the software announcements they make are often overlooked. During the event, they announced that the new version of OSX, called Yosemite, is now available. Having been testing it for a few months through the public beta program, this announcement didn't excite me all that much - it wasn't new to me. But during the announcement they discussed the new version of Safari, and they included a baffling claim.

Browser makers love benchmarks, most of which are pretty irrelevant to real-world use, but make great "I'm faster than you" tools. Apple's claim with Safari is that it's 6.6 times faster than Firefox (and Chrome is 1.1 times faster than Firefox) at "javascript on a typical website". The statement alone is pretty worthless because what on Earth qualifies as a "typical website"? The implication here seems to be that Safari is 6 times faster in real world use. If that's true, then it's bafflingly fast and most every website should feel way faster than in other browsers. The only problem is, I have no idea how they tested it.

If you're unsure what javascript is, let me clarify. Javascript is a programming language used on the web to interact with web documents (pages). Almost all interaction on a web page is powered, at least in part if not entirely, by javascript. Some of the most popular websites have a ton of javascript - Facebook, for example. If your browser's javascript engine is faster, the website will feel faster while you interact with it. Javascript benchmarks do not indicate how quickly a web page will load - that's a totally different issue.

It's entirely possible that Apple found one website that met their mysterious "typical website" criteria and it happened to be an edge case where Safari is actually way faster than the other browsers, but the claim is presented in a way that suggests a thorough test. They also could have tested it against older versions of other browsers and not the newest ones, making this a really unfair comparison. I'm going to spend a little time benchmarking the new Safari using the most widely used benchmarks to see if Safari even competes with them, let alone is significantly faster. Here's what I'll be using:

• SunSpider 1.0.2
  SunSpider is a benchmark that tests the performance of the javascript language, not interactions with pages or other browser APIs.
• Kraken 1.1
  Kraken is a benchmark created by Mozilla that's based on SunSpider, but includes some test cases pulled from "real-world" applications.
• Octane 2
  Octane is a benchmark created by Google that focuses on javascript execution and "jankiness" that occurs when javascript interacts with documents. It aims to be a modern benchmark that simulates "real world use".
• Peacekeeper
  Made by Futuremark, Peacekeeper is a modern benchmarks that tests a wide range of javascript functionality, including newer APIs introduced by the HTML5 specification.
• Minesweper
  The Minesweeper benchmark is just a game of Minesweeper developed by Microsoft that includes an option to have your browser solve the puzzle.

As far as benchmarks go, it's the standard set. Each of them tends to favor their creator's browser, Chrome typically scores highest in the Octane test, Firefox in the Kraken test and so on. Together, though, they show a somewhat comprehensive view of javascript performance.

My expectation is that Safari is basically as fast as every other browser. Javascript is more or less as fast as it's going to get, and performance issues plaguing the web these days more often come from poor network connections and unoptimized websites (unoptimized for network conditions). Javascript is hardly the determining factor in web performance on a modern browser.

So, in benchmarking Safari against its competition, I'm not setting out to declare "the best browser". The best browser is the newest version of which ever browser works for you. They're all about as fast as each other, but Apple wants us to think differently. I'm simply testing their claim to see if Safari is at all faster than the others. I'll report back with my findings.