Kernel Traffic
Latest | Archives | People | Topics
Wine
Latest | Archives | People | Topics
GNUe
Latest | Archives | People | Topics
Czech
Home | News | RSS Feeds | Mailing Lists | Authors Info | Mirrors | Stalled Traffic
 

Kernel Traffic #300 For 29 Mar 2005

By Zack Brown

Table Of Contents

Mailing List Stats For This Week

We looked at 1649 posts in 9MB. See the Full Statistics.

There were 649 different contributors. 234 posted more than once. The average length of each message was 78 lines.

The top posters of the week were: The top subjects of the week were:
51 posts in 311KB by Adrian Bunk
48 posts in 233KB by Jeff Garzik
39 posts in 152KB by Ingo Molnar
34 posts in 212KB by Ray Bryant
32 posts in 141KB by Linus Torvalds
136 posts in 556KB for "[BK] upgrade will be needed"
47 posts in 250KB for "[RFC 2.6.11-rc2-mm2 0/7] mm: manual page migration -- overview"
46 posts in 178KB for "2.6: drivers/input/power.c is never built"
46 posts in 172KB for "[ANNOUNCE] hotplug-ng 001 release"
45 posts in 267KB for "2.6.11-rc3-mm2"

These stats generated by mboxstats version 2.2

1. Linux 2.6.11-rc3-mm2 Released; Some Debate Over Security

10 Feb 2005 - 18 Feb 2005 (51 posts) Archive Link: "2.6.11-rc3-mm2"

Topics: Capabilities, FS: accessfs, Kernel Release Announcement, Real-Time

People: Andrew MortonChristoph HellwigOlaf DietscheJack O'QuinChris WrightIngo MolnarMatt Mackall

Andrew Morton announced Linux 2.6.11-rc3-mm2, saying:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.11-rc3/2.6.11-rc3-mm2/

Regarding the security module patches, Christoph Hellwig said, "Even if we accept a module that grants capabilities to groups this isn't fine yet because it only supports two specific capabilities (and even those two in different ways!) instead of adding generic support to bind capabilities to groups." Olaf Dietsche remarked:

Unless I misunderstood the code, this one is available for quite some time: <http://www.olafdietsche.de/linux/accessfs/> or a newer, self-contained version <http://lkml.org/lkml/2005/1/11/221>

Or you could use a real solution - filesystem capabilities: <http://www.olafdietsche.de/linux/capability/> and if you don't like this one :-), there's also an alternative existing here: <http://www.stanford.edu/~luto/linux-fscap/>

Andrew also said to Christoph, "I'm sure that got discussed somewhere in the 1000 emails which flew past last time. Jack?" And Jack O'Quin replied:

Most people felt that a more general capabilities module would be nice to have. But, no one offered any code, or volunteered to work on it.

I have no objection to that approach, but am not willing or able to do it myself. My opinion is that expanding the scope of the LSM would significantly increase its security risk. That job needs to be done very carefully, by someone with a deep understanding of the kernel's internal use of capabilities.

Perhaps, Christoph's suggestion could become part of a more general module, which might replace the RT-LSM in the 2.8 timeframe. Our LSM is a modest solution aimed at solving the immediate needs of audio developers and users with minimal impact on kernel security or correctness.

Matt Mackall asked what happened to the RT rlimit code from Chris Wright, and Chris replied, "I still have it, but I had the impression Ingo didn't like it as a long term solution/hack (albeit small) to the scheduler. Whereas the rt-lsm patch is wholly self-contained."

It turned out there were many opinions about the various implementations. Chris was right that his rlimit patch was considered too invasive. Actually, the main reason Andrew chose the RT LSM solution was because none of the alternatives were self-contained enough. And although, as Matt pointed out, Chris' patches were more straightforward than the other althernatives considered; there was still the problem that, as Ingo Molnar pointed out, "it didnt solve the problem (unprivileged user can lock up the system) in any way. So after it became visible that all the existing 'dont allow users to lock up' solutions are too invasive, we went to recommend the solution that introduces the least architectural problems: RT-LSM." Elsewhere in the thread, Ingo made another point about Chris' rlimit patch. He said, "if it turns out to be a mistake then it's already codified into the ABI, while RT-LSM is much less 'persistent' and could be replaced much easier." Christoph and Matt, however, were very dubious about the possibility of removing RT-LSM later; they said once RT-LSM was in the kernel, people would begin to rely on it, and removing it would not be just a simple technical issue. Ingo replied:

i somewhat share that view. (despite all the promises from the audio folks - if they are just half as agressive resisting removal as they were pushing integration then it will never be removed ;-)

but i'm not sure how rlimits will contain the whole problem - can rlimits be restricted to a single app (jackd)? The most canonical use of rlimits is per-user (per-group), so the rlimit could end up _widening_ the effects of the hack ...

Matt confirmed that rlimits could indeed be restricted to a single application. Ingo asked how this could be accomplished, and Matt replied that the application had to be invoked by a setuid launcher; it was not possible to use rlimits to elevate the rlimits of a running process. Ingo argued that if there was already a requirement for a setuid application launcher, there was no need for anything else; the launcher itself could give the needed privileges. Matt argued that the rlimits patch would do a lot more than just this; but the discussion ended abrubtly, with no resolution.

2. New Hotplug Project; Reducing Boot-Time

10 Feb 2005 - 16 Feb 2005 (76 posts) Archive Link: "[ANNOUNCE] hotplug-ng 001 release"

Topics: FS: devfs, FS: initramfs, FS: ramfs, Hot-Plugging, Klibc, PCI, Version Control, Virtual Memory

People: Greg KHPatrick McFarlandVojtech Pavlik

Greg KH said:

I'd like to announce, yet-another-hotplug based userspace project: hotplug-ng. This collection of code replaces the existing linux-hotplug package with very tiny, compiled executable programs, instead of the existing bash scripts.

It currently provides the following:

But why redo this all in .c code? What's wrong with shell scripts? Nothing is wrong with shell scripts, unless you don't want to have an interpreter in your initramfs/initrd and you want to provide /sbin/hotplug and autoload module functionality. Or if you have a huge box that spawns a zillion hotplug events all at once, and you need to be able to handle all of that with the minimum amount of processing time and memory.

So, how small are these programs? Take a look:

   text    data     bss     dec     hex filename
   4669      32     124    4825    12d9 hotplug
   5077       8     348    5433    1539 module_pci
   4925       8     412    5345    14e1 module_scsi
   5349       8     348    5705    1649 module_usb

Those are all static binaries, linked with klibc (which is included in the hotplug-ng package, just like udev.)

This compares to the following bash scripts:

-rwxr-xr-x  1 root root  4412 Feb 10 15:28 /sbin/hotplug
-rw-r--r--  1 root root   702 Sep 24 08:04 /etc/hotplug/blacklist
-rw-r--r--  1 root root  5293 Sep 24 08:04 /etc/hotplug/hotplug.functions
-rwxr-xr-x  1 root root  3739 Sep 24 08:04 /etc/hotplug/pci.agent
-rwxr-xr-x  1 root root  1459 Sep 24 08:04 /etc/hotplug/scsi.agent
-rwxr-xr-x  1 root root 13466 Sep 24 08:04 /etc/hotplug/usb.agent
-rw-r--r--  1 root root 39306 Sep 24 08:04 /etc/hotplug/usb.distmap
-rw-r--r--  1 root root  4364 Sep 24 08:04 /etc/hotplug/usb.handmap
-rw-r--r--  1 root root   189 Sep 24 08:04 /etc/hotplug/usb.usermap

All of which are loaded into memory for each hotplug event (for specific hotplug events, only that bus type of file is loaded.)

But what about speed? With a completely unscientific measurement on my old, slow laptop, it takes about 2 seconds from the time I plug a usb device into the machine, for the proper module to be loaded. With the hotplug-ng program, it takes less than a second.

And for those of you who might remember the old dietHotplug program that also did the same thing in a tiny amount of space, this project obsoletes that one. dietHotplug had to be rebuilt for every kernel that was used on the system, hotplug-ng uses the ability for modprobe to determine the module that needs to be loaded based on the module aliases (modprobe as it currently works stops loading modules when it finds an alias that matches. This does not work for drivers that claim to support "all devices" and then later on fail on devices that they really don't support. For that, all matching drivers need to be loaded for the system to work properly. The linux-hotplug scripts handle this correctly, so if you rely on this functionality, please stick with that package for now. I'll be modifying modprobe to add this feature in the near future.).

The code can be found at:
kernel.org/pub/linux/utils/kernel/hotplug/hotplug-ng-001.tar.gz
for those who wish to poke around in it.

I still have a few more programs to write to get it up to the same functionality as the existing hotplug scripts (firmware, ieee1392, etc.) but those will be done soon. I'd like to get people's comments on the idea, and welcome suggestions and even patches :)

hotplug-ng development is done in a BitKeeper repository located at:
bk://linuxusb.bkbits.net/hotplug-ng

Patrick McFarland replied:

Wow, thats pretty awesome. Just the other day I said, "Why is hotplug written in sh? Isn't that horribly inefficient way of handling something that needs to be done quickly using the least amount of resources possible?" It seems you were reading my mind.

Please, continue this project and encourage distros to switch to it (when it exceeds hotplug in functionality and stability). Ubuntu currently is trying to reduce boot time, and I bet something like this would factor in (even a few seconds helps).

Greg was grateful for the praise, and he did confirm that he'd been talking to the Ubuntu folks; but he didn't think his code would really have an impact on boot-time. Vojtech Pavlik, however, said, "Hotplug scripts were identified as one of the major culprits of slow boot when we did the analysis for SuSE 9.2. They took 40+ seconds from the total boot time."

At this point the debate veered off, into which distributions put in how much effort to reduce boot time.

3. BitKeeper Licensing; Kernel Developers Unhappy

13 Feb 2005 - 23 Feb 2005 (159 posts) Subject: "[BK] upgrade will be needed"

Topics: Version Control

People: Larry McVoyAlan Cox

Larry McVoy said:

This is a heads up that the BK tree for the kernel is currently at 59,000 changesets give or take a few. The BK that you are using uses unsigned shorts for the internal names of each delta which means you folks are about 100 days away from things no longer working.

The good news is that the openlogging tree for the kernel has 135,000 changesets so we've obviously long since fixed this problem.

The bad news is that you will need to upgrade your BK binary in order to pass over the 64K changeset boundary. The data is stored on disk in ascii so it doesn't matter if you upgrade until you hit the problem but sooner or later you will need to upgrade.

We'll get the fix into bk-3.2.4 which should be out by the end of the month. When we release that we'll send out notice and it would be good if people gave it a try and let us know if they hit issues because in a couple of months everyone is going to have to upgrade.

It's possible that we'll be changing the BK license to conform more with our commercial license but we won't do that without running it by Linus & Co to make sure that it's acceptable. One change we'd like to make there is to clarify the non-compete stuff. We've had some people who have indicated that they believed that if they used BK they were agreeing that they would never work on another SCM system. We can see how it is possible that people would interpret the license that way but that wasn't our intent. What we would like to do is change the language to say that if you use BK you are agreeing that you won't work on another SCM for 1 year after you stop using BK. But after that you would be able to hack on anything that you wanted. That was more of what we had in mind in the first place but we didn't make it clear. If you all thought that using BK meant you had no right to hack on SCM ever again, that's just not fair. We need to protect our IP but you should have the right to choose to go hack SCM if that's what you (our first choice is that you came and worked here, we really like kernel hackers, but if you don't want to that's cool too).

Various folks protested that it was absurd to have a license that prohibited working on a competing project for a year; and some folks said it was unenforceable, or that it had no standing in many parts of the world. A big discussion started up, in which some folks accused Larry of misrepresenting his objectives; while others pleaded with him to use a more sensible license. At one point Alan Cox remarked, "The real fix is to replace BK with something free and better, but thats *not* a trivial task."

4. An Attempt At /proc/cpumem

16 Feb 2005 - 17 Feb 2005 (11 posts) Archive Link: "[PATCH] /proc/cpumem"

People: Itsuro OdaEric W. Biederman

Itsuro Oda said:

Attached is an implementation of /proc/cpumem. /proc/cpumem shows the valid physical memory ranges.

example: amd64 8GB Mem

# cat /proc/cpumem
0000000000000000 000000000009b800
0000000000100000 00000000fbe70000
0000000100000000 0000000100000000
#

start address and size. hex digit.

Eric W. Biederman replied, "Interesting. My imagination when I proposed this was something based on struct resource that works like /proc/iomem on x86 but can be meaningfully be used on systems where ram lives in a separate address space from io device memory." He still felt that adding a type field to /proc/cpumem would be a more appropriate solution, though he didn't feel it would be necessary for x86 machines, as the information was available elsewhere.

5. New yaird Replacement For mkinitrd

17 Feb 2005 - 21 Feb 2005 (4 posts) Archive Link: "[ANNOUNCE] yaird, a mkinitrd based on hotplug concepts"

Topics: Backward Compatibility, Device Mapper, Disk Arrays: EVMS, Disk Arrays: LVM, Disks: IDE, Disks: SCSI, FS: NFS, FS: devfs, FS: initramfs, FS: ramfs, FS: sysfs, Hot-Plugging, Klibc, Serial ATA, Software Suspend, USB

People: Erik van KonijnenburgJeff Garzik

Erik van Konijnenburg said:

OK, time to stop polishing and start publishing.

This is to announce yaird, Yet Another mkInitRD, a rewrite of mkinitrd based on hotplug algorithms.

MOTIVATION

Why a rewrite? The versions of mkinitrd that I studied, Debian sid and Fedora FC3, have some problems: they capture a lot of knowledge about the boot process, but don't always understand when a new kernel uses a different module name than the old kernel, may rely on modules compiled into the kernel, and don't always catch errors at the earliest opportunity.

Assumption: there are three issues that cause most of these problems.

Yaird is intended to find out whether that assumption is correct: if so, a program to build initrd images will be more reliable if it's written in perl, if it uses sysfs to determine what hardware needs to be supported, and if it closely follows the methods hotplug uses find the modules needed to support some hardware.

STATUS

There is working code: yaird booted the machine this note is written on. Code is available online, there also is a paper that discusses the workings of the application in detail.

http://www.xs4all.nl/~ekonijn/yaird/

So far, the program works correctly on every machine it's been tested on, but with only two test boxes that does not mean much.

Basic creature comforts are in place: "configure; make" but no RPM or deb files, a README and help option but no manual page.

Features:

There are rough edges in practically every feature: this is suitable for testing, but not for production use.

TODO

  1. Testing so far is 100% successful, but with just two boxes to play with, that's not saying much. If you can find space to test the code on your system, your results are highly appreciated. At this point, hardware testing is most valuable: I already know that dm-crypt is unsupported for now, but whether this stuff can boot a powerbook, sparc, or just about anything else is an open question.
  2. Feedback. This may be an interesting idea, but how does it relate to other new stuff floating about? In particular, what about the work that's going on putting udev on initramfs: are these approaches complementary or alternatives? Different perspectives on where this stuff fits in would help.
  3. Support more configurations. dm-crypt is unsupported for now, and so are multipath, swsusp, EVMS, NFS and loopback mounts. Implementing this stuff becomes interesting once there are some tests results that the basic ideas work in practice.

Jeff Garzik replied:

Having a mkinitrd that's not a shell script is a godsend. I would endorse yaird on that fact alone :)

I've long wanted a "mkinitfoo" that would create .cpio.gz for initramfs by default. So, good job there, too.

Eventually your script will need pull in, into the initramfs, klibc and programs linked against klibc.

6. Extended Attribute Support For JFFS3

22 Feb 2005 (3 posts) Archive Link: "JFFS2 Extended attributes support & SELinux in handhelds"

Topics: Extended Attributes, FS: ReiserFS

People: Lorenzo Hernández García-HierroJames Morris

Lorenzo Hernández García-Hierro said:

I've been working in implementing extended attributes support in the JFFS2 filesystem.

During the (short) time I worked on it, I just decided to try to bring back the thread on SELinux in hand-held and embedded devices and see if there's someone interested in contributing to it and collaborating to make something out of it.

The current work is just a draft, I've started with the standard Vanilla kernel sources plus mtd JFFS2 sources, used to patch the vanilla ones for latest code (I'm confused on which one has the most updated tree or if there are special differences between mtd's trees and vanilla's), and implemented the skeleton using the reiserfs xattr code base.

Then Russell Coker commented to me that I should use the handhelds.org kernel, so, I have the doubt on which one use, even if porting the changes made to JFFS2 code base to vanilla sources wouldn't be difficult, at least I suppose.

I've opened a few wiki pages for discussion and documenting until it gets further developed:

http://wiki.tuxedo-es.org/tuxedo/JFFS2xattr
http://wiki.tuxedo-es.org/tuxedo/SELinuxEmbedded

In addition, having someone experienced with xattr API could be great, as development documentation seems inexistent, among James Morris' merged xattr consolidation code.

Josh Boyer suggested talking to the JFFS2 developers; he said xattr support would probably be added to JFFS3. After an IRC discussion between the two of them, Lorenzo migrated his code to JFFS3, and posted a new patch.

7. Linux 2.4.30-pre2 Released

23 Feb 2005 (1 post) Archive Link: "Linux 2.4.30-pre2"

Topics: FS: JFS, USB

People: Marcelo Tosatti

Marcelo Tosatti announced Linux 2.4.30-pre2, saying:

Here goes the second pre of v2.4.30.

It contains a bunch of important networking fixes, most noticeably the brlocks rework.

Plus USB fixes, megaraid2 driver update, JFS update, amongst others.

8. yaird 0.0.4 Released

23 Feb 2005 (1 post) Archive Link: "[ANNOUNCE] yaird 0.0.4, a mkinitrd based on hotplug concepts"

Topics: Disks: SCSI, FS: initramfs, FS: ramfs, FS: sysfs, Hot-Plugging, Klibc

People: Erik van Konijnenburg

Erik van Konijnenburg said:

Version 0.0.4 of yaird is now available at:

http://www.xs4all.nl/~ekonijn/yaird/yaird-0.0.4.tar.gz

Yaird is a perl rewrite of mkinitrd. It aims to reliably identify the necessary modules by using the same algorithms as hotplug, and comes with a template system to to tune the tool for different distributions and experiment with different image layouts. It requires a 2.6 kernel with hotplug. There is a paper discussing it at:

http://www.xs4all.nl/~ekonijn/yaird/yaird.html

There are rough edges in practically every feature, and numerous features still need to be added: this is suitable for testing, but not for production use.

Thanks to all who gave feedback, sent patches and were brave enough to test this stuff. Below are highlights from the change log and todo list.

Summary of user visible changes for version 0.0.4:

On top of the todo list are now:

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.