Kernel Traffic #309 For 4 Jun 2005

By Zack Brown

Table Of Contents

Mailing List Stats For This Week

We looked at 1166 posts in 7MB. See the Full Statistics.

There were 442 different contributors. 169 posted more than once. The average length of each message was 87 lines.

The top posters of the week were: The top subjects of the week were:
48 posts in 269KB by andrew morton
34 posts in 144KB by pavel machek
32 posts in 214KB by dmitry torokhov
31 posts in 208KB by jesper juhl
25 posts in 102KB by greg kh
71 posts in 327KB for "linux 2.6.12-rc3"
65 posts in 393KB for "2.6.12-rc2-mm3"
59 posts in 269KB for "git-commits mailing list feed."
54 posts in 303KB for "[patch x86_64] live patching function on 2.6.11.7"
47 posts in 201KB for "[patch] pci: add pci shutdown ability"

These stats generated by mboxstats version 2.8

1. Linux 2.6.12-rc2-mm3 Released

11 Apr 2005 - 27 Apr 2005 (75 posts) Archive Link: "2.6.12-rc2-mm3"

Topics: Disks: SCSI, FS: sysfs, Kernel Release Announcement

People: Andrew MortonNick Piggin

Andrew Morton announced Linux 2.6.12-rc2-mm3, saying:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.12-rc2/2.6.12-rc2-mm3/

Later that day, he said the anticipatory I/O scheduler patch had been broken, and that he was working on it; he and Nick Piggin discussed the problem, but no immediate solution came out of it.

2. Isolating Sets Of CPUs For Targeted Processes

18 Apr 2005 - 25 Apr 2005 (35 posts) Archive Link: "[RFC PATCH] Dynamic sched domains aka Isolated cpusets"

Topics: Hot-Plugging, Hyperthreading

People: Dinakar GuniguntalaNick PigginPaul Jackson

Dinakar Guniguntala said:

Here's an attempt at dynamic sched domains aka isolated cpusets

There are some things that may/will change

Nick Piggin was thrilled to see someone working on this, though he had problems with Dinakar's implementation. Paul Jackson also liked the project but had issues with the implementation. Nick, Paul, and several other folks had a lively technical discussion, resulting in at least one revised version of the patch by Dinakar.

3. Linux 2.6.12-rc3 Released; First Release Using git

20 Apr 2005 - 26 Apr 2005 (82 posts) Subject: "Linux 2.6.12-rc3"

Topics: Kernel Release Announcement, Version Control

People: Linus TorvaldsGreg KHPavel MachekAndrew MortonPetr BaudisPierre Ossman

Linus Torvalds announced Linux 2.6.12-rc3, saying:

you know what the subject line means by now, but this release is a bit different from the usual ones, for obvious reasons. It's the first in a _long_ time that I've done without using BK, and it's the first one ever that has been built up completely with "git".

It's available both as a patch (against 2.6.11) and as a tar-ball, and for non-BK users the biggest difference is probably that the ChangeLog format has changed a bit. And it will probably continue to evolve, since I don't have my "release-script" tools set up for the new setup, so this release was done largely manually with some ad-hoc scripting to get the ChangeLog information etc out of git.

For BK users, I hope we can get a BK tree that tracks this set up soon, and it should hopefully not be too disruptive either.

And for the crazy people, the git archive on kernel.org is up and running under /pub/scm/linux/kernel/git/torvalds/linux-2.6.git. For the adventurous of you, the name of the 2.6.12-rc3 release is a very nice and readable:

a2755a80f40e5794ddc20e00f781af9d6320fafb

and eventually I'll try to make sure that I actually accompany all releases with the SHA1 git name of the release signed with a digital signature.

One of the tools I don't have set up yet is the old "shortlog" script, so I did this really hacky conversion. You don't want to know, but let's say that I'm re-aquainting myself with 'sed' after a long time ;). But if some lines look like they got hacked up in the middle, rest assured that that's exactly what happened, and the long log should have the rest ...

Various folks tried using git, asked usage questions, and requested features. Petr Baudis, the author of the Cogito version control layer above the git filesystem, pointed out that the UI would be changing drastically in the near future (it has since done so). At one point, Linus remarked:

A word of warning: in many ways it's easier to work with patches. In particular, if you want to have me merge from your tree, I require a certain amount of cleanliness in the trees I'm pulling from. All of the people who used to use BK to sync are already used to that, but for people who didn't historically use BK this is going to be a learning experience.

The reason patches are easier is that you can start out from a messy tree, and then whittle down the patch to just the part you want to send me, so it doesn't actually matter how messy your original tree is, you can always make the end result look nice.

One of the things a distributed SCM brings with it is that you can't edit history after the fact, which means that if you use git and you've got a messy tree, you can't just "clean it up". You either have to keep your tree clean all the time, or you have to generate a new clean tree (usually by exporting patches from your messy one) and throw the messy ones away periodically.

("throw-away" git trees are actually very very useful).

git is actually even _more_ strict than BK in this respect, since the git model means that everything is based on SHA1 hashes, and you can't edit _anything_. With BK, some people were used to edit the checkin comments after the fact, and you could do that kind of limited cleanup before you asked me to merge. With git, that's all hashed cryptographically and is part of the "name" of the result, so if you want to change the checkin comments, you literally have to throw the old one (and every later checkin that has it as its parents) away, and re-generate the whole chain.

This is very much by design. This is how git (and I) can trust the end result. It is how git can know that if we have a common parent, all the history before that common parent is guaranteed to be the same for both you and me, and git can thus ignore it. But as mentioned, it does mean that git history is set in stone, and the only way to "fix" things is literally to re-create it all.

Pierre Ossman asked what the main parts of the 'learning experience' consisted of; Greg KH replied:

The main issue is if you want to use git for development and accepting patches from others, you need to be used to not using that git tree to send patches to Linus. To send patches to him, do something like the following:

Because of all of this, I've found that it is easier to use quilt for day-to-day development and acceptance of patches. Then use git to build up trees for Linus to pull from.

But you might find your workflow is different :)

Pavel Machek asked, "How does Andrew fit into this picture, btw? I thought all patches ought to go through him... Is Andrew willing to pull from git trees? Or is it "create one version for akpm, and when he ACKs it, create another for Linus"?" Greg replied:

Yeah, getting Andrew into the picture is a bit different. Previously, with bk, I could just have him pull from my trees, and generate a patch from that. And actually, with git that would work just as well, so if you make your git working trees public, he can pull from them and you're fine.

But with quilt it's different. That's why I make up a big patch which is the sum of my individual patches and put them on a public site. Right now you can see this at:
kernel.org/pub/linux/kernel/people/gregkh-2.6/
The patches in that directory are the "rolled up" ones. The script there is what I use to build these patches, if you want to do something like it.

In the patches/ subdir below that one, is a mirror of my quilt patches directory, series file and all. That way people can still see the individual patches if they want to.

Does this help some? It's all still under flux as to how this all works, try something and go from there :)

Andrew Morton said:

Andrew has some work to do before he can regain momentum:

I don't know how all this will pan out. I guess the next -mm won't have many subsystem trees and I'll gradually add them as things get sorted out.

He added:

Of course, whatever gets done, I'd selfishly prefer that most (or even all) subsystem maintainers work the same way and adopt the same work practices.

I guess it's too early to think about that, but if one maintainer (hint) were to develop and document a good methodology and toolset, others might quickly follow.

4. Git Commits Mailing List Feed; Some Discussion Of Tagging

20 Apr 2005 - 25 Apr 2005 (59 posts) Archive Link: "Git-commits mailing list feed."

Topics: Version Control

People: Jan DittmerDavid WoodhouseLinus TorvaldsThomas GlanzmannSean EstabrooksGreg KH

David Woodhouse announced that the [email protected] mailing list would begin carrying commits from Linus Torvalds's git repository. Jan Dittmer asked if daily snapshots were in the offing, and David said he might do it at some point. Greg KH pointed out that the scripts to so it were floating around somewhere, and could be ported to work for git repositories. He added that he also would love to see someone do this. Jan had trouble finding the script, so he wrote a new one himself, and said, "The produced patch will create a file git-commit-id in the top-level directory with the commit id used to create the patch."

Jan's script had some odd dependencies (it relied on a modified version of ketchup (http://www.selenic.com/ketchup) ), which seemed a bit excessive to David. David posted his own simpler version. Jan pointed out that David's method of determining the most recent versioned release would not work, as it relied on version tagging in the repository, which Linus did not yet do. That was why Jan had resorted to using ketchup. But David said, "Nah, asking Linus to tag his releases is the most comfortable way." Linus replied:

The reason I've not done tags yet is that I haven't decided how to do them.

The git-pasky "just remember the tag name" approach certainly works, but I was literally thinking o fsetting up some signing system, so that a tag doesn't just say "commit 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 is v2.6.12-rc2", but it would actually give stronger guarantees, ie it would say "Linus says that commit 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 is his 2.6.12-rc2 release".

That's something fundamentally more powerful, and it's also something that I actually can integrate better into git.

In other words, I actually want to create "tag objects", the same way we have "commit objects". A tag object points to a commit object, but in addition it contains the tag name _and_ the digital signature of whoever created the tag.

Then you just distribute these tag objects along with all the other objects, and fsck-cache can pick them up even without any other knowledge, but normally you'd actually point to them some other way too, ie you could have the ".git/tags/xxx" files have the pointers, but now they are _validated_ pointers.

That was my plan, at least. But I haven't set up any signature generation thing, and this really isn't my area of expertise any more. But my _plan_ literally was to have the tag object look a lot like a commit object, but instead of pointing to the tree and the commit parents, it would point to the commit you are tagging. Somehting like

        commit a2755a80f40e5794ddc20e00f781af9d6320fafb
        tag v2.6.12-rc3
        signer Linus Torvalds

        This is my official original 2.6.12-rc2 release

        -----BEGIN PGP SIGNATURE-----
        ....
        -----END PGP SIGNATURE-----

with a few fixed headers and then a place for free-form commentary, everything signed by the key (and then it ends up being encapsulated as an object with the object type "tag", and SHA1-csummed and compressed, ie it ends up being just another object as far as git is concerned, but now it's an object that tells you about _trust_)

(The "signer" field is just a way to easily figure out which public key to check the signature against, so that you don't have to try them all. Or something. My point being that I know what I want, but because I normally don't actually ever _use_ PGP etc, I don't know the scripts to create these, so I've been punting on it all).

If somebody writes a script to generate the above kind of thing (and tells me how to validate it), I'll do the rest, and start tagging things properly. Oh, and make sure the above sounds sane (ie if somebody has a better idea for how to more easily identify how to find the public key to check against, please speak up).

He replied to himself a few minutes later:

Btw, in case it wasn't clear, one of the advantages of this is that these objects are really _not_ versioned themselves, and that they are totally independent of the objects that they actually tag.

They spread together with all the other objects, so they fit very well into the whole git infrastructure, but the real commit objects don't have any linkages to the tag and the tag objects themselves don't have any history amongst themselves, so you can create a tag at any (later) time, and it doesn't actually change the commit in any way or affect other tags in any way.

In particular, many different people can tag the same commit, and they don't even need to tage their _own_ commit - you can use this tag objects to show that you trust somebody elses commit. You can also throw the tag objects away, since nothing else depends on them and they have nothing linking to them - so you can make a "one-time" tag object that you can pass off to somebody else, and then delete it, and now it's just a "temporary tag" that tells the recipient _something_ about the commit you tagged, but that doesn't stay around in the archive.

That's important, because I actually want to have the ability for people who want me to pull from their archive to send me a message that says "pull from this archive, and btw, here's the tag that not only tells you which head to merge, but also proves that it was me who created it".

Will we use this? Maybe not. Quite frankly, I think human trust is much more important than automated trust through some technical means, but I think it's good to have the _support_ for this kind of trust mechanism built into the system. And I think it's a good way for distributors etc to say: "this is the source code we used to build the kernel that we released, and we tagged it 'v2.6.11-mm6-crazy-fixes-3.96'".

And if my key gets stolen, I can re-generate all the tags (from my archive of tags that I trust), and sign them with a new key, and revoke the trust of my old key. This is why it's important that tags don't have interdependencies, they are just a one-way "this key trusts that release and calls it xyzzy".

In the course of discussion, Jan pointed out that with Linus's solution, the only way to truly clone a repository would be to use rsync. The tag objects would not be directly connected to the history of the repository, and so any more sophisticated mechanism of synchronizing repositories by grabbing any missing patches would not work here. The repository itself would not be aware of the tags, and so the synchronization tools would not see the tags in order to pull them into the target repository. Linus replied:

But this is a _feature_.

Other people normally shouldn't be interested in your tags. I think it's a mistake to make everybody care.

So you normally would fetch only tags you _know_ about. For example, one of the reasons we've been _avoiding_ personal tags in teh BK trees is that it just gets really ugly really quickly because they get percolated up to everybody else. That means that in a BK tree, you can't sanely use tags for "private" stuff, like telling somebody else "please sync with this tag".

So having the tag in the object database means that fsck etc will notice these things, and can build up a list of tags you know about. It also means that you can have tag-aware synchronization tools, ie exactly the kind of tools that only grab missing commits can also then be used to select missing tags according to some _private_ understanding of what tags you might want to find..

Elsewhere, in the course of discussion, the location of the signature came up. It would be easier to put the PGP header around the full entry, instead of just at the end as Linus had proposed. Thomas Glanzmann said:

# This creates the signature.
gpg --clearsign < sign_this > signature

# And this verifies it.
gpg --verify < signature && echo valid

But Linus replied:

This really doesn't work for me - I do not want to have the gpg header above it, only the signature below. Since I want git to actually understand the tags, but do _not_ want git to have to know about whatever signing method was used, I really want the resulting file to look like

commit ....
tag ...

here goes comment
here goes signature

and no headers.

Whether that can be faked by always forcing SHA1 as the hash, and then just removing the top lines, and re-inserting them when verifying, or whether there is some mode to make gpg not do the header crud at all, I don't know. Which is exactly why I never even got started.

Sean Estabrooks replied, "A script that knows how to validate signed tags, can easly strip off all the signing overhead for display. Users of scripts that don't understand will see the cruft, but at least it will still be usable." Linus said:

NO.

Guys, I will say this once more: git will not look at the signature.

That means that we don't "strip them off", because dammit, they DO NOT EXIST as far as git is concerned. This is why a tag-file will _always_ start with

commit <commit-sha1>
tag <tag-name>

because that way we can use fsck and validate reachability and have things that want trees (or commits) take tag-files instead, and git will automatically look up the associated tree/commit. And it will do so _without_ having to understand about signing, since signing is for trust between _people_ not for git.

And that is why I from the very beginning tried to make ti very clear that the signature goes at the end. Not at the beginning, not in the middle, and not in a different file. IT GOES AT THE END.

Sean replied, "Okay now you're just being difficult <g> You're acting like it's impossible for git to grab the SHA1 out of the clear text message if there is signing overhead above the tag reference. That is nonesense. You simply state that tag must include a SHA1 object reference preceded by "REF:" in the comment. Git can surely use this regardless of what signing overhead is above, below or beside it. The suggestion for stripping out the signing overhead was for _human_ readability; git won't care a gnit." But Linus said:

No. It's not "impossible" for git to parse crap. But git won't.

There are two ways you can write programs:

and I do the first one. That means that a program I write does something _repeatable_. It does the same thing, regardless of whether a human happened to write "REF:" in the comment section, or anything else.

The thing is, great programs come not out of great coding, but out of great data structures. The whole git philosophy bases itself on getting the data structure right.

And what you are asking for is doing it _wrong_. So in git I don't just parse random free-form text and guess that a line that starts with REF: is a reference to a commit. It has very rigid and well-specified data structures, and that's how you make reliable programs.

I don't care what anybody else does on top of git, but dammit, I'll make sure that the core infrastructure is designed the right way.

And that means that we don't guess, and that we don't parse random ASCII blobs. It means that we have very very fixed formats so that programs can either do the right thing or unambiguously say "that's crap".

I've said it before, and I'll say it again: we have enough crap that calls itself SCM's out there already. I want git to be reliable and _simple_, not a collection of crap that just happens to work.

5. Increasing The Limit Of e820 Entries From 32 To 128

22 Apr 2005 - 26 Apr 2005 (6 posts) Archive Link: "[PATCH] Increase number of e820 entries hard limit from 32 to 128"

People: Venkatesh PallipadiLinus TorvaldsAndi Kleen

Venkatesh Pallipadi said, "The specifications that talk about E820 map doesn't have an upper limit on the number of E820 entries. But, today's kernel has a hard limit of 32. With increase in memory size, we are seeing the number of E820 entries reaching close to 32. Patch below bumps the number upto 128" for i386 and x86-64. Linus Torvalds remarked, "Hmm. Anything that changes setup.S tends to have bootloader dependencies. I worry whether this one does too.." Venkatesh replied:

The setup.S change in this patch should be OK. As it is adding to the existing zero-page and keeping it within one page. I tested it on systems with grub, adding some dummy E820 entries and it worked fine.

However, there is another place that needs to be changed. Boot loaders also calls E820 while booting directly with vmlinux (instead of usual bzImage - which is handled by this patch) and that needs to change to incorporate more E820 entries. But, there we may need more changes, to the boot protocol version and the like. On a side note, looking at the grub source, it seems to have a limit of 50 entries today, which doesn't agree with current 32 entry limit in the kernel. Not sure why grub has this different limit though.

Andi Kleen remarked, "The last time I tried to extend the zero page (with a longer command line) it broke lilo on systems with EDID support and CONFIG_EDID enabled. Make sure you test that case." Venkatesh replied:

Tested this patch with some more configuration and I did not see any breakage.

And in the current zero-page, EDID info is at a lower address (before E820MAP). So, there should not be any issues with EDID info. Only field (other than E820) that is changing in zero page is EDDBUF (that comes after E820MAP). The patch changes the reference to EDDBUF inside kernel to new position in zero page. And I don't see EDDBUF being used by boot loader anywhere. So, we should be OK with that change.

6. Linux 2.4.31-pre1 Released; 2.4 Tree To Migrate To git

25 Apr 2005 (1 post) Archive Link: "Linux 2.4.31-pre1"

People: Marcelo Tosatti

Marcelo Tosatti announced Linux 2.4.31-pre1, saying, "Here goes the first pre of v2.4.31. It contains a very small number of changes, mostly an x86_64 update. Side note: I'm planning on moving the v2.4 repository along with the full history information to a git repository soon."

7. git-pasky Renamed To Cogito; Version 0.8 Released

25 Apr 2005 - 26 Apr 2005 (10 posts) Archive Link: "[ANNOUNCE] Cogito-0.8 (former git-pasky, big changes!)"

People: Petr Baudis

Petr Baudis announced:

here goes Cogito-0.8, my SCMish layer over Linus Torvald's git tree history tracker. This package was formerly called git-pasky, however this release brings big changes. The usage is significantly different, as well as some basic concepts; the history changed again (hopefully the last time?) because of fixing dates of some old commits. The .git/ directory layout changed too.

Upgrading through pull is possible, but rather difficult and requires some intimacy with both git, git-pasky and Cogito. So probably the best way to go is to just get cogito-0.8 tarball at

http://www.kernel.org/pub/software/scm/cogito/

or

ftp://ftp.kernel.org/pub/software/scm/cogito/

build and install it, and do

cg-clone rsync://rsync.kernel.org/pub/scm/cogito/cogito.git

Yes, this is a huge change. No, I don't expect any further changes of similar scale. I think the new interface is significantly simpler _and_ cleaner than the old one.

First for the concept changes. There is no concept of tracking anymore; you just do either cg-pull to just fetch the changes, or cg-update to fetch them as well as merge them to your working tree. Even more significant change is that Cogito does not directly support local branches anymore - git fork is gone, you just go to new directory and do

cg-init ~/path/to/your/original/repository

(or cg-clone, which will try to create a new subdirectory for itself). This now acts as a separate repository, except that it is hardlinked with the original one; therefore you get no additional disk usage. To get new changes to it from the original repository, you have to cg-update origin. If you decide you want to merge back, go to the original repository, add your new one as a branch and pull/update from it.

As for the interface changes, you will probably find out on your own; cg-help should be of some help. All the scripts now start with 'cg-', and you should ignore the 'cg-X*' ones. The non-trivial mapping is:

git addremote -> cg-branch-add
git lsremote -> cg-branch-ls
git patch -> cg-mkpatch
git apply -> cg-patch
git lsobj -> cg-admin-lsobj

Commands that are gone:

git fork
git track

New commands:

cg-clone
cg-update

Of course other changes include various bugfixes, and latest Linus' stuff (although we do not make use of Linus' tags yet).

Note that I don't know how many time will I have for hacking Cogito until the next Sunday/Monday. I hope I will get some time to at least apply bugfixes etc, but I don't know how much more will I be able to do. You would make me a happy man if you could please port your pending patches from git-pasky to Cogito; I promise to apply them and I hope there isn't going to be another so big change in the foreseeable future, which would cause major conflicts for your patches etc.

8. OpenSSI 1.9.0 Released

28 Apr 2005 (1 post) Archive Link: "[ANNOUNCE] OpenSSI 1.9.0 released"

People: Aneesh Kumar

Aneesh Kumar said:

OpenSSI 1.9.0 is a development release. For a stable release, download OpenSSI 1.2.2 instead. This is the first release based on a 2.6 kernel, 2.6.10 to be precise.

To know more about OpenSSI visit www.openssi.org (http://www.openssi.org) .

To download and install

http://www.openssi.org/cgi-bin/view?page=docs2/1.9/debian/INSTALL.html

9. Umbrella 0.7 Released And Feature Complete

28 Apr 2005 (1 post) Archive Link: "[RELEASE] Umbrella is now feature complete! (v0.7 released)"

People: Kristian Sørensen

Kristian Sørensen said:

Finally, we now present you with a feature complete version of Umbrella, namely version 0.7.

Followin is the main major changes:

For instructions on how to try out the Process-Based Access Control and Digitally Signed Binaries in Umbrella, please download the complete 0.7 tarball from SourceForge: http://prdownloads.sourceforge.net/umbrella/umbrella-0.7.tar.bz

Please refer to the README file in the tarball for further instructions.

As always we appreciate any comments, suggestions etc. you may have :-)

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.