Package manager vulnerability study flawed ?

Submitted by dag on Sat, 2008/07/12 - 15:43

A study from the University of Arizona (recently posted on slashdot) looked at weaknesses in package managers (and mirror setup). By becoming an official mirror and delaying or stalling a mirror's updates they tried to lower the security of servers using that mirror and increasing the window of opportunity for a successful attack.

In itself it is very useful to make people aware of weaknesses in technology or abuse of trust, but in this case (and certainly for CentOS) I think they overstated the impact or at least ignored mechanisms used to prevent possible security risks.

By default CentOS uses yum with mirrorlist enabled. This means that instead of using a single mirror all the time, you are not using one, but different mirrors. This reduces the risk of a single mirror being out-of-date somewhat. But next to that CentOS has several tiers of mirrors depending on the update-frequency of each mirror (and the form of control the CentOS project has of those mirrors).

And the mirrorlist that CentOS users actually use is being created based on the correctness of the individual mirrors, we are continuously verifying mirror content, metadata, filesizes and signatures on checksums. This means that CentOS users are only working with an up-to-date mirrorlist and mirrors that stall or delay packages are left out of this mirrorlist. You can see how it works on the CentOS mirror status page.

Unfortunately this mechanism was not mentioned in the study.

The conclusion seems to be that any theoretical risk is very minimal and indirect, however some of the recommendations for improving the package manager's robustness should definitely be taken seriously by their developers.

Update: CentOS developer Johnny Hughes blogged about the same topic.

Hmm

Should that "Before publishing" part be in this post?

Human error

You actually catched my article when it was being reviewed by others before it was ready for publication.

I guess I need to look into a better way of offering a new article for review to others without making it public and available from a feed :-/

Maybe there is a Drupal module that allows this functionality ?

Inaccuracy?

Hello, I'm one of the authors of the study. I wanted to first of all thank you for commenting on our research. One of the major benefits we hoped would come from making this public is that the Linux community would become more aware and interested in fixing the problems we point out.

I also wanted to respond to several of the issues you brought up in your blog post. First, I appreciate you pointing out that many distributions check to see if their mirrors are current and try to remove mirrors that are not. We under-emphasized this in the webpage and other documents because we did not view this as a mechanism used to detect a malicious party (we thought the intent was to detect negligent administrators and broken scripts). As I'm sure you and the savvy reader are aware, it is possible for a web server to serve different content to different users. We examined our web request logs from our CentOS mirror and I believe we can identify the "checking bot" IP addresses. If we were malicious, we could serve "good" content to your checking bot and "malicious" content to other users. I would be happy to provide what I believe to be the IPs used to check if a mirror is current to you offline for verification / rebuttal. However, since you view this information as important to the security of your users, I will not list the information here.

Additionally, I wanted to mention that we found significant security problems with Fedora's MirrorManager (our FAQ talks about how it can be used to target attacks). However, other redirectors we looked at (like Download Redirector for OpenSUSE) do improve security in a similar manner to what you describe. I was wondering if we could talk more offline about how your mirror list redirection works so we can discuss the potential for abuse?

I also wondered if you might want to look in detail at the other attacks page of the web site and the technical report which mentions detailed information about flaws in YUM. We would be happy to discuss the feasibility of attacks that target these issues with you. However, I will point out one attack that is extremely simple that I hope illustrates there is a real danger to your users. If I control a mirror and you attempt to retrieve a file from my mirror, I can return an endless stream of data which (on YUM) will fill the disk and crash the client system (stopping logging, corrupt databases, etc.). This is obviously a real threat to all of your users regardless of any mirror redirection strategies you perform.

Anyways, we thank you for taking a look at our research and hope to hear more rebuttal / confirmation in the future.

Thanks,
Justin Cappos

Right

Justin,

You are correct when you say that there are potential problems with the depsolvers (yum and others) and I think my article ends to point out that we should not minimize those risks and developers definitely should look into them.

What I simply wanted to make clear is that we do more than just accept mirror-requests and that a mirror is not just offered whatever the accuracy of the mirror is. We do verify for forgery or malicious content.

But here also, yes, you have a good point about providing different content to different clients and unfortunately we trust other mechanisms (from the depsolver) to make sure that the user is not affected.

Other security mechanisms?

Can you explain more about your checking for forgery / malicious content (privately via email is fine)? I understand you have a checker bot that looks to see if the repomd.xml file is up to date. Can you talk more about other security checks you have that we may have overlooked?

I want to make sure that we don't provide any incorrect information about CentOS' security mechanisms. :)

I definitely appreciate the comments and suggestions on our research. Our goal isn't to point fingers at distributions. We're trying to provide accurate information about the real risks involved. We thank you for helping to point out any items we missed in our broad examination of package manager and distribution security.

Justin

Why the silence?

Did the discussion between dag and justin go offline? I would be very interested in reading the rest of this thread (being a CentOS admin).