Monday, May 26, 2003
Tail between My Legs: MSFT vs. Linux
OK. I have made several mistakes and the general thrust of much of what Thom says is correct. That said, I will not cede the point that I believe at some level MSFT must have the ability to be or become more secure than Linux. In my earlier argument, I made several errors about the relative security of MSFT web servers versus Linux servers. I have recently completed an updated analysis that compares an estimate of the proportion of MSFT vs. Linux web servers and the proportion of MSFT vs. Linux web defacements. By comparing these figures, I was hoping to discover empirically whether MSFT or Linux was relatively more secure. As part of the analysis, I developed a simple quantification of Linux and MSFT's relative lack of security. I defined security weakness factor (SWF) as SWF = SWD / SS, where SWD = share of total web defacements and SS = share of total web servers. The higher the SWF score, the worse the security. I retrieved data on web defacements from Attrition and data on web servers from The Internet Operating System Counter. I have copied below the spreadsheet I used to perform the calculations.
Attrition Share of Total Web Defacements by OS, selected months in 2000 and 2001
OK. I have made several mistakes and the general thrust of much of what Thom says is correct. That said, I will not cede the point that I believe at some level MSFT must have the ability to be or become more secure than Linux. In my earlier argument, I made several errors about the relative security of MSFT web servers versus Linux servers. I have recently completed an updated analysis that compares an estimate of the proportion of MSFT vs. Linux web servers and the proportion of MSFT vs. Linux web defacements. By comparing these figures, I was hoping to discover empirically whether MSFT or Linux was relatively more secure. As part of the analysis, I developed a simple quantification of Linux and MSFT's relative lack of security. I defined security weakness factor (SWF) as SWF = SWD / SS, where SWD = share of total web defacements and SS = share of total web servers. The higher the SWF score, the worse the security. I retrieved data on web defacements from Attrition and data on web servers from The Internet Operating System Counter. I have copied below the spreadsheet I used to perform the calculations.
Attrition Share of Total Web Defacements by OS, selected months in 2000 and 2001
|
Jun-00 |
Jul-00 |
Aug-00 |
Sep-00 |
Oct-00 |
Nov-00 |
Dec-00 |
Jan-01 |
Feb-01 |
Mar-01 |
Apr-01 |
May-01 |
Totals |
Windows (NT + 2000) |
245 |
263 |
219 |
181 |
321 |
472 |
465 |
537 |
440 |
515 |
1028 |
930 |
5616 |
Linux (all varieties) |
53 |
95 |
219 |
189 |
152 |
147 |
89 |
310 |
335 |
250 |
274 |
102 |
2215 |
Others |
81 |
41 |
104 |
106 |
100 |
103 |
129 |
50 |
73 |
122 |
244 |
105 |
1258 |
Totals |
379 |
399 |
542 |
476 |
573 |
722 |
683 |
897 |
848 |
887 |
1546 |
1137 |
9089 |
|
|||||||||||||
|
Jun-00 |
Jul-00 |
Aug-00 |
Sep-00 |
Oct-00 |
Nov-00 |
Dec-00 |
Jan-01 |
Feb-01 |
Mar-01 |
Apr-01 |
May-01 |
Totals |
Windows (NT + 2000) |
65% |
66% |
40% |
38% |
56% |
65% |
68% |
60% |
52% |
58% |
66% |
82% |
62% |
Linux (all varieties) |
14% |
24% |
40% |
40% |
27% |
20% |
13% |
35% |
40% |
28% |
18% |
9% |
24% |
Others |
21% |
10% |
19% |
22% |
17% |
14% |
19% |
6% |
9% |
14% |
16% |
9% |
14% |
Totals |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
100% |
|
Internet Operating System Counter, April of 1999
Windows 24.40%
Linux 28.50%
Relative Insecurity Levels of Windows and Linux
Windows 2.53
Linux 0.86
As you can see, Linux appears to be more secure by a factor of roughly 3x. However, I should caveat that the operating system counter statistics are from April of 1999 and the web defacement statistics are from mid-2000 to mid-2001. That said, assuming that Linux has not achieved a 3x advantage in share of web servers, it is likely still more secure.
Now that I have conceded this point, I would like to address a few of Thom's arguments and then conclude. Thom makes an interesting but ultimately flawed "good eyes" versus "bad eyes" argument. Thom admits that a key variable for his model is the number of crackers and percentage of users who actually read the code. However, these variables, while important, are not in my view the most important. I would wholeheartedly agree with Thom that there are likely more "good eyes" as he claims than "bad eyes"--(see Thom I am optimistic!). However, I think that a debate about these variables fundamentally misses the problem with open source. I find it hard to believe that "bad eyes" looking at open source code are equally effective in creating security breaches in open source as the "bad eyes" looking at closed source are in creating security breaches in closed source code. The crux of my argument is that "bad eyes" get a lot of good and useful information for their cracking endeavors by looking at the source code of an OS. Stated differently, my point is that "bad eyes" attacking Linux are likely more effective than "bad eyes" attacking MSFT. Now, I could be wrong since it appears from my argument above that Linux is currently more secure than MSFT. Thom does briefly address this point by saying that such tools as hex editors and reverse compilers might level the playing field between crackers attacking closed source and crackers attacking open source. However, I think that most everybody would agree, Thom included, that using reverse compilers and hex editors is much more painstaking, complicated, and error-ridden process than is simply looking at the source code.
I grant Thom his argument on repair. I think it is fundamentally correct that repairs likely occur more quickly in Linux than they do in MSFT. I also operated under an incorrect assumption in my previous argument that Linux was not the OS of choice for large scale corporations on key servers. I am now increasingly aware of the fact that my assumption is likely false. So I agree with Thom. As my statistics above reveal, more web servers as of late 1999 ran Linux than MSFT. Though I am not totally sure what is going on at the largest corporations (where presumably security efforts receive more budget funding), I would assume that Linux has a larger share than MSFT. On the ethical question Thom raised, I will have to address that later.
Now, how could I believe that MSFT might have the potential to become more secure when the data reveal that it is currently quite a bit less secure? There are several reasons that this might be the case. It could be the case that crackers are much more more likely to attack MSFT servers than Linux servers. For example, perhaps crackers feel some kind of loyalty toward open source for their own cultural reasons or perhaps they simply despise MSFT for their own cultural reasons. It could also be the case that the types of sites that crackers want to crack happen to run MSFT more frequently than Linux. This might be the case if sites that have valuable data or information typically run MSFT. Finally, it could be the case that there are security relationships between the server market (where MSFT and Linux have roughly equal market share) and the client market (where MSFT clearly dominates) that make crackers want to attack MSFT servers more. For example, it may be possible to crack an MSFT server and then exploit this weakness to distribute an e-mail virus to a larger group of MSFT client computers.
However, I will grant Thom that it might simply be the case that there are inherent net security benefits to open source over closed source. I still doubt it but if the analysis I performed above continues to show MSFT at such a security disadvantage in say 2-3 years, I may have to cede the point entirely.
On a side note, Gartner released a survey in 2001 that said that 45% of new server shipments were MSFT and only 25% were Linux. The survey also concluded that Linux remains an option for those companies seeking a "lower cost alternatives" to MSFT. Gartner also said that it "does not predict any massive groundswell of support for Linux that would have any serious impact on Windows server market share." Statements like the latter require some justification. However, this data is pregnant with hermeneutic possibilities. Despite our quibble about security, is Linux ever more than a "low cost" alternative to MSFT? Will it every have a serious impact on MSFT? Perhaps it already has in that Windows has publicly addressed it as a threat. But is this simply lip service? Questions for another day.