Web Browsers Malware Benchmark 2011

By | July 18, 2011 | 37 Comments

Web Browsers Malware Benchmark 2011Internet Explorer 9 scores 99-100%.

NSS Labs, an independent network testing facility, has tested 5 most popular web browsers against the socially engineered malware, resulting in more controversy and flame wars all over the Internet.

Although some claim that results are sponsored by Microsoft, Rick Moy, the president of NSS Labs, said that while Microsoft did hire them to benchmark various web browsers few years ago, it was only to improve Internet Explorer’s security. However, after seeing some positive numbers, it was only then sent to the marketing department to do their thing.

Tested web browsers

Google Chrome 10.0.648.204
Windows Internet Explorer 8 (build 8.0.7600.16385)
Windows Internet Explorer 9 (build 9.0.8112.16421)
Mozilla Firefox 4.0
Opera 11.01 Build 1190
Safari 5.0.5(7533.21.1)

Web Browsers Malware Benchmark 2011

Web Browsers Malware Benchmark 2011

IE9 achieved a unique URL blocking score of 89% and over-time protection rating of 92%. Enabling Application Reputation on top of SmartScreen increased the unique URL block rate of Internet Explorer 9 by 11% (to 100%) at zero hour as well as the over-time protection by 8% (to 100%). Internet Explorer 9 was by far the best at protecting against socially-engineered malware, even before App Rep’s protection is layered on top of SmartScreen.

Firefox 4 achieved a 13% protection rating, on par with protection offered by Chrome and Safari — 86% less protection than Internet Explorer 9 and 77% less than Internet Explorer 8. Firefox exhibited deterioration in protection compared with our Q3 2010 global test that can be attributed to either the implementation of Safe Browsing API v2 or new tactics being used by cybercriminals that Safe Browsing has not yet adapted to. There was a slight 1% improvement between zero-hour protection (16%) and eventual protection at the 19 day mark (17%).

Safari 5 achieved a 13% protection rating on par with Firefox and Chrome converging at a roughly 17% block rate after 19 days. However, Safari presented a notable lag in protection vs. Firefox with zero-hour protection of 11% (vs. Firefox’s 16%).

With a protection rating of 13%, Chrome 10 offered nearly identical protection to Safari and Firefox.

Opera 10’s overall blocking rate of 5% was consistently the lowest in the group. However, this was an improvement over the 0% scores in our previous global tests, and is likely attributable to the company’s partnership with antivirus firm AVG.

Anyhow, we would like to hear your thoughts about the results. Sponsored and untrustworthy or legit and worth a mentioning?

[Thanks, Ichann, Rafael Luik]


About (Author Profile)


Vygantas is a former web designer whose projects are used by companies such as AMD, NVIDIA and departed Westood Studios. Being passionate about software, Vygantas began his journalism carrier back in 2007 when he founded FavBrowser.com. Having said that, he is also an adrenaline junkie who enjoys good books, fitness activities and Forex trading.

  • Paulius Vaičiulis

    Well done. IE9 is awesome in many ways.

    • Anonymous

      You would have to be a total retard to believe NSS labs, they are cherry-picking things to test.

      • Paulius Vaičiulis

        I am not sitting in the internet all day and don’t have time for conspiracy
        theories

        gadgets.util.runOnLoadHandlers();.

        • Guest

          But you do have time to blindl accept Microsoft’s flawed tests? The problems with this test are well known and documented. No need for any conspiracy theories, especially when we know that Microsoft pays for fake tests all the time.

          • Paulius Vaičiulis

            And I KNOW that they do not. lol

        • http://twitter.com/TiagoMorbusSa Tiago Sá

          The fact that you think it’s a conspiracy theory strips your posts of any relevance whatsoever.

  • http://twitter.com/DanielHendrycks Daniel Hendrycks

    Seems like the Fox News of benchmarks.

    • http://www.favbrowser.com Vygantas Lipskas

      Why? Do you base your opinion by comparing NSS Labs results with other web browser security companies results (that tested anti social-malware features) and show completely different picture or simply saying that because you are a big fan of Opera? :)

      • everybody

        You can easily realize how unreliable NSS Labs is by simply reading that report. Here is the most ridiculous part:
        “Ultimately, 650 URLs passed our post-validation process and are included in the results, providing a margin of error of 3.84% with a confidence interval of 95%.”

        And:
        “Care was taken not to overweight any one domain to represent more than 10% of the test set.”

        It means that in worst case they might have used only 10 sites for that “test”!

        • http://www.favbrowser.com Vygantas Lipskas

          You are correct, guess it is shady after all.

      • http://twitter.com/DanielHendrycks Daniel Hendrycks

        With IE8, they, NSS, said it was the best, malware blocking wise. All else said differently, at the time. IE9 now is out and has better security, making the results more trust-worthy, but not enough to consider them straight. These results are still very questionable.
        They should share the URL list, the advantage would be it’d be good for filter’s to know and blacklist the sites, and there’d be a benefit to NSS’s reputation as to what they used and whether it is confirmable.

  • Anonymous

    Of course this test was done by NSS Labs (funded by MS every year for this). NSS Labs every year publish a similar bullshit table claiming IE v XYZ blocks the malwares most. :)

    Don’t get me wrong, IE9 is a great product and is a step in the right direction but still it lags behind other browsers in terms of functionality and customisability.

    People shouldn’t believe in these Bullshit tests (like what NSS Labs does) and use their own mind while browsing. No browser can save you if you’re devoid of common-sense.exe.

    • http://www.favbrowser.com Vygantas Lipskas

      They tested anti-malware features and not overall functionality or customization abilities.

      Therefore, if browser underperforms in one area, it does not mean that IE does a bad job in all the other areas as well :)

      • Guest

        The latest report from NSS Labs is as crappy as the rest of them. An extremely small sample size, funded by Microsoft, refusing to share the URL list, no verification, contradictions, and all sorts of smaller and bigger problems. The NSS Labs reports have no credibility at all.

  • http://my.opera.com/rafaelluik Rafael

    Vygantas you know that makes no sense IE9 to be that effective while others that are aways improving security and closing security bugs are in that positions…
    At most maybe they have Security Essentials or something installed that integrates to IE?

    COMMON Opera has AVG. AVG! (just as an example, not that I’m defending Opera, I’d use this argument if any other had this kind of security planned).

    • Nobody

      avg is anti virus company primarily – this test is for social malware (ie. mind trick’y stuff, that is not easy to catch using the same set of skills)

      it is entirely possible that AVG in this respect that is tested simply sux

      and as well it is entirely possible that this test sux – but i’d like to see some more reason to say like that – nss had shaky reputation and MS as a previous sponsor, but on the other hand IE9 is a reaaaaaly good piece of software, that put emphasis on this type of malware from the get go – it is as well that they’ve managed to be really good at it.

      what made me laugh – no matter what the test, no matter how the results are ‘unprofessional’ – opera as always is trailing the pack. it will be less than a day before certain Blog Comment Manager writes some witty and not at all misleading rant about yet another ‘bad bad test’. maybe one in a while opera should learn from their shortcomings instead of covering them with cheap rhetorics of their prestigious censors?

      • WOFall

        One of the primary protections offered by a good anti-virus is url-blocking, which aims to prevent the malware reaching the pc in the first place. When an antivirus program detects a virus, the originating url of the file is sent back to the vendor and it’ll be blacklisted. This makes sense, and AVG’s sizable audience would lead you to expect a reasonable score in this test.
        So from where does Microsoft get its list? They would have to scan the entire Internet to find the bad downloads, and while Microsoft has a lot of resources, I don’t see this happening. Google are in the business of scanning the Internet though, so it’s perfectly reasonable for them to have a good list for use in Chrome, Firefox, and Safari.
        They could also be examing gray areas where it’s difficult to define whether or not the program is malware (would the Zwinky family count?). I should not have to take this sort of swipe against the test, but as they methodology and list of testcases isn’t released, it becomes impossible to trust.

        Microsoft: “We’ve just tested IE9 against malware. It tests 90% better than all other browsers with a score of 100!”
        Users: “Cool, can we see the test?”
        Microsoft: “Nope! :)”
        Users: “Riiiiight.”

        NSS Labs: “We’ve just tested browsers against malware. IE9 tests 90% better than all other browsers with a score of 100!”
        Users: “Cool! Seeing as you’re not directly affiliated with Microsoft we trust you! Go IE9!!!”

        • Nobody

          remind me please, what operating system is used on over 95% machines around the world? what system most likely malware creators use? who made this system? and who made all its security measures?

          microsoft is the company that had to tackle with malware, spyware, crapware, shitware and animated binders for decades – and boy, they are good at it now. also, they have bing, not entirely small search engine.

          that makes this result entirely possible.

          win7 is targeted by various small penis owners day by day and it still is relatively safe system – why ie9 shouldnt?

          i dont want to say, that i believe this test (nss has bad history, really) – but im closer to believing it, than to dismissing it ‘because ms is crap’. MS can do fantastic and briliant things – kinnect, visual studio to name the best – why not IE? people are already having difficulties feeding their prejudices against this browser.. it is really good now.

          • http://my.opera.com/rafaelluik Rafael

            Mr nobody… How can IE9 surpass a browser with malware protection from AVG, Netcraft AND Haute Secure?
            I have nothing against IE, but I have all against NSS Labs tests.

          • Nobody

            ignoring the advantage of having most popular operating system in the world and second largest search engine for a moment

            ms security essentials – these 3 words should help you understand how ms can collect information about all the shit that floats around the web. before you say that MSE is crap, do some research

            there are probably other reasons, not limited to avg, netcraft and hs being really second tier companies, even in their own field (that does not mean that ms buys not effect of their work)

          • http://my.opera.com/rafaelluik Rafael

            Ok I don’t need to prove anything…

            And see the comment below if you want… It doesn’t prove anything too…
            http://www.favbrowser.com/web-browsers-malware-benchmark-2011/#comment-256343083

            Well, NSS Labs doesn’t prove anything too! So we’re tied (or not)! :)

          • Nobody

            well, for once you should at least provide a theory why you don’t believe these results. ‘i don’t like them’ is not enough.

            ms being artsy at deceptive marketing or not – people need to adapt to a world when IE is a good browser and in some aspects actually better than long standing kings. it took them ages but they’ve finally managed it

            and it is a good time to mention, that opera as always closed the pack :) this is the reason why you dont believe this test? so what test do you believe?

          • Guest

            Lots of comments here explain why the test can’t be trusted (extremely low sample size, impossible to verify, no url list, the fact that IE block nearly everything and other browser nearly nothing being extremely suspect, etc.).

            I don’t know why people are discussing whether IE is a good browser or not when this is about the NSS Labs test. IE8 and 9 (9 in particular) have had some major problems with site compatibility, but that’s probably just down to those not being the #1 browser right away unlike other IE versions.

            Bottom line, don’t trust poorly conducted tests funded by Microsoft. Microsoft will be happy to lie.

      • Guest
  • Mikah

     
     I use Opera & the free Comodo Internet Security 
    but If I was surfing the Internet without a firewall active anti virus & defense programs I would use IE9 .

  • mav

    “Don’t get me wrong, IE9 is a great product and is a step in the right direction but still it lags behind other browsers in terms of functionality and customisability”  by the metrix

    @metrix -  Agree with the cutomizability, but not the functionality.

    • http://my.opera.com/rafaelluik Rafael

      Disagree with both, what cutomizability? :o

  • Tito

    From the report:
    “For clarity, the following definition is used for a socially-engineered malware URL: a web page link that directly leads to a download that delivers a malicious payload whose content type would lead to execution, or more generally a website known to host malware links. These downloads appear to be safe, like those for a screen saver application, video codec upgrade, etc., and are designed to fool the user into taking action. Security professionals also refer to these threats as “consensual” or “dangerous” downloads.”

    I think it’s safe to say that unless you’re an idiot who runs random .exe files these results are irrelevant.

    Of course, I’m well aware that a big number of people (most?) are in fact idiots and do run random .exe files.

  • Guest

    LOL, NSS Labs again. Paid and bought by Microsoft. Does anyone really think IE catches nearly everything, but every single other browser catches nearly nothing? Nothing has changed. M$ lying again..

    http://www.thetechherald.com/article.php/200912/3268/Can-you-trust-the-NSS-Labs-report-touting-the-benefits-of-IE8

    Remember when they claimed that Opera had updated itself automatically during the test, even though they were using a version that didn’t even support automatic updates? LOL.

    And as someone mentioned, they could be using as little as 10 sites for this test! It’s pseudo-science at its worst. No wonder NSS Labs was thrown out of the security testing standards organization.

  • Arthur Ţiţeică

    When downloading Foxit reader version 5 (quite recently released) IE9 marks it as unsafe and one has to click a lot in order to actually run the installer. I wouldn’t call that security – it just shows that they have no f**cking clue about what’s OK or not.

    Besides that people who can’t protect themselves shouldn’t really do *administrative* tasks on computers. It’s like saying I treat myself of hepatitis by reading blogs on the internet.

    • http://pulse.yahoo.com/_5F7DFC3FHUM4SYSW5J4XSQ2KSA I

      So if much of the ‘malware’ IE8/9 detects is just perhaps the non-default alternatives to common programs like adobe reader etc etc…. a load of false positives?
      something has to explain the massive difference

  • Mikah

    Microsoft have a long history of making misleading claims
    http://www.sitepoint.com/really-ie9-100-compatible-with-html5/#comment-949000

  • greG 8-)

    LOL Look at all these people crying there browser sucks at Malware …I bet if your browser won you wouldn’t be saying nothing ..you would nod your head like ”yeah thats true” ..and no one would fight about anything cause you already have in your mind IE sucks

    Lesson learn dont be a fan boy give credit where credit is due

    • Guest

      If the browser was won by a browser whose parent company did not pay for the test, the test would have been much more trustworthy.

      But even with that, the test is just pseudoscientific, unverifiable nonsense. There are major problems with the methodology (the sample size could be as small as 10 sites, etc.), and there’s no way for independent parties not paid by Microsoft to verify the data.

      My guess is that no other browser vendor would dare to be caught red-handed publishing a terrible test like this.

  • greG 8-)

    Guest….you comment like 6 times already on here you must be mad cause they didnt use your lil extensions or addons for your browser to block malware in this test ..maybe when they implement them in the browser someone will do the test again soon..untill then have fun with extensions/addons  (especially firefox) that slow down your performance and be buggy :)

    • Guest

      Extension to block malware? What are you talking about? Can’t you just read what people are writing instead of being an ass?

  • Banananjam

    Maybe but i have added McAfee Site Advisor to Chrome and it works like a charm.
    http://www.siteadvisor.com/ 

«
»