From samuel.verstraete at gmail.com Mon Oct 5 14:13:12 2009 From: samuel.verstraete at gmail.com (samuel) Date: Mon, 5 Oct 2009 14:13:12 +0200 Subject: my merge went wrong, sorry Message-ID: Hi, I just fucked up moonbase... I have reverted all of it so we should be fine once again... I did need to use git reset --hard + a git push --force so if any of you updated his local moonbase in the time between that reset and my merge you will need to clone moonbase again. Sorry for the noise. Samuel From zmcgrew at gmail.com Tue Oct 6 02:11:28 2009 From: zmcgrew at gmail.com (Zachary McGrew) Date: Mon, 5 Oct 2009 17:11:28 -0700 Subject: Bash 4 patches Message-ID: <4b1a641e0910051711g741c97fbs84569bb1554fa5b5@mail.gmail.com> Attached are my patches to make Bash 4 work for us. Bash 4 (+ official patches) is in my zmcgrew branch on doppio, commit id 902543aaddccdf0ed85a14b93d6e9c8ea25d9610. Here's a link for the lazy: http://foo-projects.org/git/?p=zmcgrew/moonbase.git;a=commit;h=902543aaddccdf0ed85a14b93d6e9c8ea25d9610 edit.lunar is a patch for /var/lib/lunar/functions/edit.lunar init-functions is a patch for /lib/lsb/init-functions -- Zachary McGrew Web: http://zmcgrew.no-ip.com -------------- next part -------------- A non-text attachment was scrubbed... Name: edit.lunar.patch Type: application/octet-stream Size: 212 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: init-functions.patch Type: application/octet-stream Size: 1767 bytes Desc: not available URL: From zbiggy at o2.pl Tue Oct 6 22:27:21 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Tue, 6 Oct 2009 22:27:21 +0200 Subject: [PATCH] add lzma and xz to unpack plugin Message-ID: <200910062227.21659.zbiggy@o2.pl> Hello, many sites use lzma and xz for keeping files small to save on bandwidth cost. This is important for open source, freeware sites to keep costs low so please patch lunar tarball with this patch. For us this will be faster downloads. For me applying this patch will make developer work easier: I will not have to backport module updates anymore. The compression from the best to the worst: lzma xz bz2 gz Z have a nice day, Zbigniew Luszpinski -------------- next part -------------- A non-text attachment was scrubbed... Name: unpack-generic.plugin.patch Type: text/x-patch Size: 1064 bytes Desc: not available URL: From auke at foo-projects.org Wed Oct 7 04:48:37 2009 From: auke at foo-projects.org (Auke Kok) Date: Tue, 06 Oct 2009 19:48:37 -0700 Subject: Bash 4 patches In-Reply-To: <4b1a641e0910051711g741c97fbs84569bb1554fa5b5@mail.gmail.com> References: <4b1a641e0910051711g741c97fbs84569bb1554fa5b5@mail.gmail.com> Message-ID: <4ACC0185.4060401@foo-projects.org> Zachary McGrew wrote: > Attached are my patches to make Bash 4 work for us. > > Bash 4 (+ official patches) is in my zmcgrew branch on doppio, commit > id 902543aaddccdf0ed85a14b93d6e9c8ea25d9610. > Here's a link for the lazy: > http://foo-projects.org/git/?p=zmcgrew/moonbase.git;a=commit;h=902543aaddccdf0ed85a14b93d6e9c8ea25d9610 > > edit.lunar is a patch for /var/lib/lunar/functions/edit.lunar > init-functions is a patch for /lib/lsb/init-functions ACK, looks OK to me. good to know we can't use negative values again... strange change though. Auke From auke at foo-projects.org Wed Oct 7 04:50:06 2009 From: auke at foo-projects.org (Auke Kok) Date: Tue, 06 Oct 2009 19:50:06 -0700 Subject: [PATCH] add lzma and xz to unpack plugin In-Reply-To: <200910062227.21659.zbiggy@o2.pl> References: <200910062227.21659.zbiggy@o2.pl> Message-ID: <4ACC01DE.4050006@foo-projects.org> Zbigniew Luszpinski wrote: > Hello, > > many sites use lzma and xz for keeping files small to save on bandwidth cost. > This is important for open source, freeware sites to keep costs low so please > patch lunar tarball with this patch. For us this will be faster downloads. > For me applying this patch will make developer work easier: I will not have > to backport module updates anymore. why not put these as plugins in the lzma and xz modules? Auke From zbiggy at o2.pl Wed Oct 7 19:01:51 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Wed, 7 Oct 2009 19:01:51 +0200 Subject: [PATCH] add lzma and xz to unpack plugin In-Reply-To: <4ACC01DE.4050006@foo-projects.org> References: <200910062227.21659.zbiggy@o2.pl> <4ACC01DE.4050006@foo-projects.org> Message-ID: <200910071901.51431.zbiggy@o2.pl> > Zbigniew Luszpinski wrote: > > Hello, > > > > many sites use lzma and xz for keeping files small to save on bandwidth cost. > > This is important for open source, freeware sites to keep costs low so please > > patch lunar tarball with this patch. For us this will be faster downloads. > > For me applying this patch will make developer work easier: I will not have > > to backport module updates anymore. > > why not put these as plugins in the lzma and xz modules? > > Auke Why not: xz is replacement for gz for source packages. xz/lzma has better compression than bz2 and gzip and decompression time much better than bz2. It looks it will gain more acceptance than bz2. Take a look at gnu.org ftp for example. They skipped bz2 and provide for new releases xz and gz as backward compatibility. Still downloading gz from their site is bandwidth waste. Because the most key modules comes from gnu.org it is better for us to use xz. Because in the future xz will have the same importance or greater than gz it is better to have it in core plugin and on ISO. As you may see in the patch xz is integrated into tar as 3rd unpacker (after gz and bz2). That is why the easiest way was to extend generic-unpack plugin instead of adding plugin. If you find generic unpack plugin to be decentralized I can write unpacker plugin for bzip2 and gzip modules too to replace the generic one. I found generic plugin supports all compressors supported by tar. Because tar supports xz/lzma as 3rd compressor it is still obvious to me to add support for it to generic plugin. lzma and xz are my prefered source packages. Next is bzip2 and gzip. All future module updates from me will use lzma and xz if possible. Let me know your decision. I see sense of writing xz/lzma unpack plugin only when bzip2 and gzip also will be split to separate plugins. have a nice day, Zbigniew Luszpinski From sofar at foo-projects.org Fri Oct 9 01:57:52 2009 From: sofar at foo-projects.org (Auke Kok) Date: Thu, 08 Oct 2009 16:57:52 -0700 Subject: [PATCH] add lzma and xz to unpack plugin In-Reply-To: <200910071901.51431.zbiggy@o2.pl> References: <200910062227.21659.zbiggy@o2.pl> <4ACC01DE.4050006@foo-projects.org> <200910071901.51431.zbiggy@o2.pl> Message-ID: <4ACE7C80.6000902@foo-projects.org> Zbigniew Luszpinski wrote: >> Zbigniew Luszpinski wrote: >>> Hello, >>> >>> many sites use lzma and xz for keeping files small to save on bandwidth cost. >>> This is important for open source, freeware sites to keep costs low so please >>> patch lunar tarball with this patch. For us this will be faster downloads. >>> For me applying this patch will make developer work easier: I will not have >>> to backport module updates anymore. >> why not put these as plugins in the lzma and xz modules? >> >> Auke > > Why not: xz is replacement for gz for source packages. > xz/lzma has better compression than bz2 and gzip and decompression time much better than bz2. > It looks it will gain more acceptance than bz2. > Take a look at gnu.org ftp for example. > They skipped bz2 and provide for new releases xz and gz as backward compatibility. > Still downloading gz from their site is bandwidth waste. > Because the most key modules comes from gnu.org it is better for us to use xz. > Because in the future xz will have the same importance or greater than gz > it is better to have it in core plugin and on ISO. > As you may see in the patch xz is integrated into tar as 3rd unpacker (after gz and bz2). > That is why the easiest way was to extend generic-unpack plugin instead of adding plugin. > If you find generic unpack plugin to be decentralized I can write unpacker plugin > for bzip2 and gzip modules too to replace the generic one. > I found generic plugin supports all compressors supported by tar. Because tar supports > xz/lzma as 3rd compressor it is still obvious to me to add support for it to generic plugin. > lzma and xz are my prefered source packages. Next is bzip2 and gzip. > All future module updates from me will use lzma and xz if possible. > > Let me know your decision. > I see sense of writing xz/lzma unpack plugin only when bzip2 and gzip also will be split to separate plugins. that is the logical succession of my point, however, I really thing that this move should START by making a plugin in the xz/lzma modules, not the other way around. stop bickering, write a plugin, it's easy, and any dev can commit it to moonbase. :P Auke From zbiggy at o2.pl Fri Oct 9 12:34:54 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Fri, 9 Oct 2009 12:34:54 +0200 Subject: [PATCH] add lzma and xz to unpack plugin In-Reply-To: <4ACE7C80.6000902@foo-projects.org> References: <200910062227.21659.zbiggy@o2.pl> <200910071901.51431.zbiggy@o2.pl> <4ACE7C80.6000902@foo-projects.org> Message-ID: <200910091234.54590.zbiggy@o2.pl> > that is the logical succession of my point, however, I really thing that > this move should START by making a plugin in the xz/lzma modules, not > the other way around. > > stop bickering, write a plugin, it's easy, and any dev can commit it to > moonbase. :P > > Auke OK. I wrote and commited xz plugin. bzip2, gzip and tar also should be converted to plugins? Zbigniew From sofar at foo-projects.org Fri Oct 9 21:19:48 2009 From: sofar at foo-projects.org (Auke Kok) Date: Fri, 09 Oct 2009 12:19:48 -0700 Subject: [PATCH] add lzma and xz to unpack plugin In-Reply-To: <200910091234.54590.zbiggy@o2.pl> References: <200910062227.21659.zbiggy@o2.pl> <200910071901.51431.zbiggy@o2.pl> <4ACE7C80.6000902@foo-projects.org> <200910091234.54590.zbiggy@o2.pl> Message-ID: <4ACF8CD4.1070400@foo-projects.org> Zbigniew Luszpinski wrote: >> that is the logical succession of my point, however, I really thing that >> this move should START by making a plugin in the xz/lzma modules, not >> the other way around. >> >> stop bickering, write a plugin, it's easy, and any dev can commit it to >> moonbase. :P >> >> Auke > > OK. I wrote and commited xz plugin. > bzip2, gzip and tar also should be converted to plugins? if there are more xz/lzma files in moonbase than bz2/gz, I'll gladly consider that. From zbiggy at o2.pl Sat Oct 10 02:26:42 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Sat, 10 Oct 2009 02:26:42 +0200 Subject: Do not build openoffice-src when dpkg module is installed - install will fail Message-ID: <200910100226.42528.zbiggy@o2.pl> Hello, do not build openoffice-src when dpkg module is installed - install will fail. When epm - the install tool for openoffice-src will find dpkg tool (which is installed by dpkg module) instead of making file tree for Lunar installer it will auto pack everything to deb packages and tar.gz files. Lunar installer will not find file tree to install and will fail. If you lrm dpkg before lining openoffice-src all will go fine, without error. You can lin dpkg after openoffice-src installation to keep using dpkg. If I find plenty of time for building OO again I will patch epm installer so it will not find dpkg anymore to fix build problem. If you would like fix it - feel free to do it. The install fail begins at line 103 of module's BUILD script file: cd instsetoo_native/unxlngi6.pro/OpenOffice/native/install/en-US/linux-2.6-intel/buildroot/opt && because when deb files are built the directory structure is different and above directory path can not be entered into. have a nice day, Zbigniew Luszpinski From zbiggy at o2.pl Wed Oct 14 05:56:06 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Wed, 14 Oct 2009 05:56:06 +0200 Subject: What to do with xz? Message-ID: <200910140556.06750.zbiggy@o2.pl> Hello, Terry Chan wrote in commit 7ca5e78a4e8347e4b6b995b76a63966e22c53d94: "gzip: KEEP the source as tar.gz NOT tar.xz This is really stupid to require the xz module to be able to unpack the GZIP source code. Everyone has gzip on their box from the Lunar ISO. Do NOT change this module over to tar.xz. This means you!" I would like to know if we are for or against xz tarballs in moonbase. What are rules of using xz in moonbase. There are 5 choices: a) Boycott xz and keep using gzip,bzip and compress for everything. There is no problem for me to convert modules from xz to gz before commit. Script keeping this done is trivial to write and use for me. b) Prepare a list of blacklisted modules which should always be legacy compressed (this is important because gnu.org only uses gz and xz) c) make DETAILS conditional so those who have xz can use smaller and faster downloads. Other will use legacy gz and older. d) ugly hack: add xz to DEPENDS of every xz/lzma source packed module e) Return to my plan of comfortable migration which was: 1) lunar depends xz + patched generic unpack plugin (working patch already present on ML) 2) lunar update will force patched and updated lunar module to be installed as first and xz dependency will be also installed so nobody is hurt even if he/she has fresh install from ISO. 3) Now we can start pushing xz packed tarballs to moonbase because everyone who do lunar update will have support for xz. Because 1st point was not successful done to the end all plan collapsed. But we can always try it again if there is more acceptance. I prefer e) solution. Because xz is used by new gnu.org releases I vote for putting xz on new ISO because soon the most new core modules can be in xz or gz tarball. xz depacks almost equally fast as gzip and tarballs are ~40% smaller. For me it is smart to keep gzip packed with other compressor because if I loose gzip I can still unpack it, build it and install it without the need of having gzip. Believe or not but some people do not have iso after installation. For example they burn it on CD-RW, install and erase disc. Or they are lazy to not search for it. For me this was obvious: bzip2 is packed with gzip, xz is packed with bzip2 so gzip should be packed with xz to close the security circle. Gnu organisation only supports gzip and xz for new releases on their servers - that is why bzip2 was not chosen by me. Because I had to rewrite xz to external unpack plugin I realized that someone who has not yet xz will receive on screen message that xz is needed to unpack archive. Maybe this is not elegant as e) solution but works. The xz message is only displayed once if someone will install xz module after this message. Not a big issue. have a nice day, Zbigniew Luszpinski From sofar at foo-projects.org Thu Oct 15 17:35:27 2009 From: sofar at foo-projects.org (Auke Kok) Date: Thu, 15 Oct 2009 08:35:27 -0700 Subject: What to do with xz? In-Reply-To: <200910140556.06750.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> Message-ID: <4AD7413F.2060107@foo-projects.org> Zbigniew Luszpinski wrote: > Hello, > > Terry Chan wrote in commit 7ca5e78a4e8347e4b6b995b76a63966e22c53d94: > "gzip: KEEP the source as tar.gz NOT tar.xz This is really stupid to > require the xz module to be able to unpack the GZIP source code. > Everyone has gzip on their box from the Lunar ISO. Do NOT change this > module over to tar.xz. This means you!" > > I would like to know if we are for or against xz tarballs in > moonbase. What are rules of using xz in moonbase. There are 5 > choices: a) Boycott xz and keep using gzip,bzip and compress for > everything. There is no problem for me to convert modules from xz to > gz before commit. Script keeping this done is trivial to write and > use for me. b) Prepare a list of blacklisted modules which should > always be legacy compressed (this is important because gnu.org only > uses gz and xz) c) make DETAILS conditional so those who have xz can > use smaller and faster downloads. Other will use legacy gz and older. > d) ugly hack: add xz to DEPENDS of every xz/lzma source packed > module e) Return to my plan of comfortable migration which was: 1) > lunar depends xz + patched generic unpack plugin (working patch > already present on ML) 2) lunar update will force patched and updated > lunar module to be installed as first and xz dependency will be also > installed so nobody is hurt even if he/she has fresh install from > ISO. 3) Now we can start pushing xz packed tarballs to moonbase > because everyone who do lunar update will have support for xz. > Because 1st point was not successful done to the end all plan > collapsed. But we can always try it again if there is more > acceptance. > > I prefer e) solution. Because xz is used by new gnu.org releases I > vote for putting xz on new ISO because soon the most new core modules > can be in xz or gz tarball. xz depacks almost equally fast as gzip > and tarballs are ~40% smaller. > > For me it is smart to keep gzip packed with other compressor because > if I loose gzip I can still unpack it, build it and install it > without the need of having gzip. Believe or not but some people do > not have iso after installation. For example they burn it on CD-RW, > install and erase disc. Or they are lazy to not search for it. For me > this was obvious: bzip2 is packed with gzip, xz is packed with bzip2 > so gzip should be packed with xz to close the security circle. wait what? what is wrong with the current gzip module? we've always assumed a baseline of gzip+bzip2 compression in the lunar ISO's. let's not change that. I want you to work on adding xz/lzma support. I do not want you to work on messing everything up by changing EVERY moonbase module to go from gz/bzip2 compression to xz/lzma. Doing so is completely counterproductive and will just frustrate other developers. All you are contributing is changes for the sake of changes. Leave gziped GNU modules that already exist alone, please. Auke From zbiggy at o2.pl Sun Oct 18 03:32:20 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Sun, 18 Oct 2009 03:32:20 +0200 Subject: What to do with xz? In-Reply-To: <4AD7413F.2060107@foo-projects.org> References: <200910140556.06750.zbiggy@o2.pl> <4AD7413F.2060107@foo-projects.org> Message-ID: <200910180332.20273.zbiggy@o2.pl> > what is wrong with the current gzip module? OMG O_O Keeping gzip unpacker packed inside gz file is insane. When my girlfriend sent me rar unpacker inside rar archive it was funny. If Lunar developer says it is right to keep gzip unpacker packed inside gz file it is not. If you think that all of us have Lunar CD in drive ready to restore gunzip you are terribly wrong. I feel really embarrassed explaining this _again_ on Linux distro _developer_ list. If you loose gunzip you will not be able to install it again because lin will be unable to unpack gzip-*.tar.gz file to build gunzip. If xz unpacks gzip and gzip unpacks bzip2 and bzip2 unpacks xz then if you loose one of these you can fix it immediately with just a lin of missing unpacker. Now to fix missing gunzip you have to have Lunar CD or resurrect it (if you keep compiled sources in bzips - waste of space). > we've always assumed a baseline of gzip+bzip2 compression in the lunar > ISO's. let's not change that. OK. Now I understand. This is politics not technology. My technical explanation or discussion is pointless. > I want you to work on adding xz/lzma support. Already done. Commited 8 days ago. Works perfect for me. If someone encounters xz/lzma packed source he/she is asked to install xz if it is not installed. Then xz modules are unpacked the same way as bzips or gzips. Nobody is left broken. However this means nothing now because there is no xz sources in moonbase. Writing this plugin was waste of time and resources. > I do not want you to work > on messing everything up by changing EVERY moonbase module to go from > gz/bzip2 compression to xz/lzma. Doing so is completely > counterproductive and will just frustrate other developers. All you are > contributing is changes for the sake of changes. I followed gnu.org path: With new release xz/lzma tarball appears together with legacy, fatty gz next to it. If you browse ftp.gnu.org you will find that old releases are gz only and new are both xz and gz releases. (compare sizes of autoconf, coreutils, gzip, gawk, libtool (lzma), m4, parted, texinfo (lzma), wget(lzma)) By updating module with new app release new version tarball is downloaded. When new gzip release appeared I bumped version 1.3.12->13 in moonbase and chosen xz tarball instead of gz. No mess. Just ordinary version update. The same way some time ago I asked people to use bz2 instead of gz or Z and ftp instead of http. Just to make downloads faster. And packing gzip with xz is safer to me than packing gzip with gzip but others find this otherwise. I think the most frustrating and counterproductive thing are white spaces added by some people to make diffs in git bigger. This is awful to read someone's path via web git interface to realize that 80% of changes are spaces added here or there. BTW: Please add --ignore-all-space (or -w to be short) to web git-diff config to cut these empty changes out. > > Leave gziped GNU modules that already exist alone, please. > OK. 2 people against xz : 1 for, the rest is silent. No more xz on public. I will use xz personally in zlocal. For future updates I will preserve packing method present in moonbase so you do not have to afraid my updates. EOT on my side. I think all is clear. Zbigniew Luszpinski From ratler at lunar-linux.org Sun Oct 18 08:54:11 2009 From: ratler at lunar-linux.org (Stefan Wold) Date: Sun, 18 Oct 2009 08:54:11 +0200 Subject: What to do with xz? In-Reply-To: <200910180332.20273.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> <4AD7413F.2060107@foo-projects.org> <200910180332.20273.zbiggy@o2.pl> Message-ID: <1255848851.4536.53.camel@localhost> On Sun, 2009-10-18 at 03:32 +0200, Zbigniew Luszpinski wrote: > > what is wrong with the current gzip module? > > OMG O_O > Keeping gzip unpacker packed inside gz file is insane. > When my girlfriend sent me rar unpacker inside rar archive it was funny. > If Lunar developer says it is right to keep gzip unpacker packed inside gz file it is not. > If you think that all of us have Lunar CD in drive ready to restore gunzip you are terribly wrong. > I feel really embarrassed explaining this _again_ on Linux distro _developer_ list. > If you loose gunzip you will not be able to install it again because lin will be unable to unpack gzip-*.tar.gz file to build gunzip. > If xz unpacks gzip and gzip unpacks bzip2 and bzip2 unpacks xz then if you loose one of these you can fix it immediately with just a lin of missing unpacker. Now to fix missing gunzip you have to have Lunar CD or resurrect it (if you keep compiled sources in bzips - waste of space). > I understand your concern, but seriously how many times have your gzip gone broken the last hmm 6-7 years that lunar have existed? I can't remember a single time. Not saying it can't happen, but statisticly it's safe. There are other ways to restore gzip in case you don't have a Lunar CD at hand, but no point diving into that discussion here. > > we've always assumed a baseline of gzip+bzip2 compression in the lunar > > ISO's. let's not change that. > > OK. Now I understand. This is politics not technology. My technical explanation or discussion is pointless. Not really, let us wait a moment with this change. If we were to jump each technology that arrive that would be a full time job. The day gnu stop serving .gz or when more then 50% of the moonbase tarballs have an .xz equivalent, then it's certainly time to do the switch, right now I don't see the benefit. > > > I want you to work on adding xz/lzma support. > > Already done. Commited 8 days ago. Works perfect for me. > If someone encounters xz/lzma packed source he/she is asked to install xz if it is not installed. > Then xz modules are unpacked the same way as bzips or gzips. Nobody is left broken. > However this means nothing now because there is no xz sources in moonbase. > Writing this plugin was waste of time and resources. What you're saying are just proving the point on how few maintainers that are using .xz. I don't see a problem adding modules with .xz or changing modules to .xz that are _not_ considered CORE modules. > > > I do not want you to work > > on messing everything up by changing EVERY moonbase module to go from > > gz/bzip2 compression to xz/lzma. Doing so is completely > > counterproductive and will just frustrate other developers. All you are > > contributing is changes for the sake of changes. > > I think the most frustrating and counterproductive thing are white spaces added by some people to make diffs in git bigger. > This is awful to read someone's path via web git interface to realize that 80% of changes are spaces added here or there. > BTW: Please add --ignore-all-space (or -w to be short) to web git-diff config to cut these empty changes out. I probably missed a context switch somewhere, weren't we talking about xz? I very much people add spaces on purpose. Some people have broken editors that doesn't remove trailing spaces etc etc. That can be fixed by teaching people to configure their editors. But that's another discussion. > > > > Leave gziped GNU modules that already exist alone, please. > > > > OK. 2 people against xz : 1 for, the rest is silent. No more xz on public. I will use xz personally in zlocal. > For future updates I will preserve packing method present in moonbase so you do not have to afraid my updates. I'm ok with xz for unimportant modules, ie as I earlier stated, non-core modules. And it works perfectly fine as a plugin. -- Sincerely Stefan Wold Lunar Linux developer - PGP public key 6E810F05 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From dagbrown at lart.ca Sun Oct 18 08:44:18 2009 From: dagbrown at lart.ca (Dave Brown) Date: Sun, 18 Oct 2009 15:44:18 +0900 Subject: What to do with xz? In-Reply-To: <200910180332.20273.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> <4AD7413F.2060107@foo-projects.org> <200910180332.20273.zbiggy@o2.pl> Message-ID: <20091018064418.GA27854@phb> On Sun, Oct 18, 2009 at 03:32:20AM +0200, Zbigniew Luszpinski wrote: > > what is wrong with the current gzip module? > > If Lunar developer says it is right to keep gzip unpacker packed > inside gz file it is not. The GNU folks distribute gzip as an uncompressed tar file. For people who don't have gzip handy. ftp://ftp.gnu.org/pub/gnu/gzip/gzip-1.3.12.tar Perhaps that should be added as a possible alternative URL for gzip. --Dave From sofar at foo-projects.org Mon Oct 19 09:04:51 2009 From: sofar at foo-projects.org (Auke Kok) Date: Mon, 19 Oct 2009 00:04:51 -0700 Subject: What to do with xz? In-Reply-To: <200910180332.20273.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> <4AD7413F.2060107@foo-projects.org> <200910180332.20273.zbiggy@o2.pl> Message-ID: <4ADC0F93.7030900@foo-projects.org> Zbigniew Luszpinski wrote: >> what is wrong with the current gzip module? > > OMG O_O > Keeping gzip unpacker packed inside gz file is insane. > When my girlfriend sent me rar unpacker inside rar archive it was funny. > If Lunar developer says it is right to keep gzip unpacker packed inside gz file it is not. > If you think that all of us have Lunar CD in drive ready to restore gunzip you are terribly wrong. > I feel really embarrassed explaining this _again_ on Linux distro _developer_ list. > If you loose gunzip you will not be able to install it again because lin will be unable to unpack gzip-*.tar.gz file to build gunzip. > If xz unpacks gzip and gzip unpacks bzip2 and bzip2 unpacks xz then if you loose one of these you can fix it immediately with just a lin of missing unpacker. Now to fix missing gunzip you have to have Lunar CD or resurrect it (if you keep compiled sources in bzips - waste of space). > >> we've always assumed a baseline of gzip+bzip2 compression in the lunar >> ISO's. let's not change that. > > OK. Now I understand. This is politics not technology. My technical explanation or discussion is pointless. > >> I want you to work on adding xz/lzma support. > > Already done. Commited 8 days ago. Works perfect for me. > If someone encounters xz/lzma packed source he/she is asked to install xz if it is not installed. > Then xz modules are unpacked the same way as bzips or gzips. Nobody is left broken. > However this means nothing now because there is no xz sources in moonbase. > Writing this plugin was waste of time and resources. > >> I do not want you to work >> on messing everything up by changing EVERY moonbase module to go from >> gz/bzip2 compression to xz/lzma. Doing so is completely >> counterproductive and will just frustrate other developers. All you are >> contributing is changes for the sake of changes. > > I followed gnu.org path: > With new release xz/lzma tarball appears together with legacy, fatty gz next to it. > If you browse ftp.gnu.org you will find that old releases are gz only and new are both xz and gz releases. > (compare sizes of autoconf, coreutils, gzip, gawk, libtool (lzma), m4, parted, texinfo (lzma), wget(lzma)) > By updating module with new app release new version tarball is downloaded. > When new gzip release appeared I bumped version 1.3.12->13 in moonbase and chosen xz tarball instead of gz. > No mess. Just ordinary version update. The same way some time ago I asked people to use bz2 > instead of gz or Z and ftp instead of http. Just to make downloads faster. > And packing gzip with xz is safer to me than packing gzip with gzip but others find this otherwise. > > I think the most frustrating and counterproductive thing are white spaces added by some people to make diffs in git bigger. > This is awful to read someone's path via web git interface to realize that 80% of changes are spaces added here or there. > BTW: Please add --ignore-all-space (or -w to be short) to web git-diff config to cut these empty changes out. > >> Leave gziped GNU modules that already exist alone, please. >> > > OK. 2 people against xz : 1 for, the rest is silent. No more xz on public. I will use xz personally in zlocal. > For future updates I will preserve packing method present in moonbase so you do not have to afraid my updates. you write way too much text for something as simple as adding support for xz files. you try to make way too large of a change touching the very core of lunar's way of packing. And all at once. Take a step back, and introduce your changes in small and non-invasive changes, and over time. Time, as in months, not days. For some of the changes we previously did in lunar we took 6+ months to merge features. Cheers, Auke From zbiggy at o2.pl Wed Oct 21 01:50:45 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Wed, 21 Oct 2009 01:50:45 +0200 Subject: What to do with xz? In-Reply-To: <1255848851.4536.53.camel@localhost> References: <200910140556.06750.zbiggy@o2.pl> <200910180332.20273.zbiggy@o2.pl> <1255848851.4536.53.camel@localhost> Message-ID: <200910210150.45898.zbiggy@o2.pl> > > > we've always assumed a baseline of gzip+bzip2 compression in the lunar > > > ISO's. let's not change that. > > > > OK. Now I understand. This is politics not technology. My technical explanation or discussion is pointless. > > Not really, let us wait a moment with this change. If we were to jump > each technology that arrive that would be a full time job. The day gnu > stop serving .gz or when more then 50% of the moonbase tarballs have > an .xz equivalent, then it's certainly time to do the switch, right now > I don't see the benefit. I would like to see Lunar as the most fresh and innovative distro. That is why linux-2.6, sse3, VDPAU appeared, now I tried with xz. In IT the one who stays always looses. Nobody will drop gzip just because xz appeared. Like bzip2 did not dropped gzip. This is just fourth, better compressor than older compress, gzip, bzip2. If we will not start using xz in moonbase in even single module it will never reach 50%. The benefit is obvious like with bzip2: smaller files. > > However this means nothing now because there is no xz sources in moonbase. > > Writing this plugin was waste of time and resources. > > What you're saying are just proving the point on how few maintainers > that are using .xz. I don't see a problem adding modules with .xz or > changing modules to .xz that are _not_ considered CORE modules. Nobody uses xz in Lunar because: 1. There was no unpack plugin. 2. People usually increase version and count checksum in DETAILS. They do not check what lies next to $SOURCE on http/ftp/sourceforge. I do this. That is why I see xz/lzma and the rest is not. 3. Response on devel ML is showing xz is unwelcome in moonbase because there is common agreement we can use only gzip+bzip2. Nice to see you would like to see xz for non core modules. How about making common agreement that all non sustained modules can be converted to xz? (Of course we do change to xz only when bumping version to new one). Is it OK or too wide? If you have other idea share it on ML please. > I'm ok with xz for unimportant modules, ie as I earlier stated, non-core > modules. And it works perfectly fine as a plugin. Thanks for trying. I hope your voice will help xz not dying in Lunar. Zbigniew From zbiggy at o2.pl Wed Oct 21 02:08:20 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Wed, 21 Oct 2009 02:08:20 +0200 Subject: What to do with xz? In-Reply-To: <4ADC0F93.7030900@foo-projects.org> References: <200910140556.06750.zbiggy@o2.pl> <200910180332.20273.zbiggy@o2.pl> <4ADC0F93.7030900@foo-projects.org> Message-ID: <200910210208.20782.zbiggy@o2.pl> > you write way too much text for something as simple as adding support > for xz files. I hoped I will convince people to xz. But failed. This was the reason for so much text. Lunar could have the best innovative packaging but no way only preserving history matters. > you try to make way too large of a change touching the very core of > lunar's way of packing. And all at once. Argh. This was just a one module - new version of gzip. Not massive action. xine-lib-vdpau was packed with lzma by me so no other packaging available. I hoped it will go the same way as bzip2 replaced gzip over time with every next module version update. > Take a step back, and introduce your changes in small and non-invasive > changes, and over time. Time, as in months, not days. For some of the > changes we previously did in lunar we took 6+ months to merge features. If you would like to see xz packages in moonbase tell me the road map (which modules can be xz which not and when). The working xz plugin is in moonbase. Now it is just a matter of using xz in DETAILS or not. That's all. lzma-sdk is in moonbase since 20080129. lzma-utils appeared a little bit later. Then lzma-utils were replaced by xz on 20090224. xz is upgrade to lzma-utils. Same author, same code, same site. I use xz/lzma since long time without any issue. That is why I decided to start pushing it slowly to moonbase. Before commit to moonbase I have to download legacy gz or bz2 to recalculate checksum. After commit rollback to xz. No problem but I prefer to be in sync with moonbase instead of driving overloaded zlocal. Zbigniew From zbiggy at o2.pl Wed Oct 21 03:26:06 2009 From: zbiggy at o2.pl (Zbigniew Luszpinski) Date: Wed, 21 Oct 2009 03:26:06 +0200 Subject: What to do with xz? In-Reply-To: <200910140556.06750.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> Message-ID: <200910210326.06431.zbiggy@o2.pl> Hello, I solved the problem of unwelcome xz in the following way: I added to DETAILS XZSOURCE and XZSOURCE_VFY - you can see this in recent aria2 commit. Soon I will post here an optional patch which will make use of these new variables. The Revolutionists will have a chance to start using xz: +enjoying smaller files, +enjoying faster downloads, +proving that xz is present in more places than Conservatists think, +being proud Lunar is innovative and on cutting edge of IT development. The Conservatists will be happy: +knowing they are safe from xz enemy. +knowing they do not have to do anything to keep Lunar as it was before. +the xz/lzma tarball will appear only together with module version update. (I hope the other xz fans will behave the same). +the quietness will come back to ML :) The Liberalists will be happy: +there is another optional choice for them with rollback option. I will be happy: +my zlocal will be smaller == easier to maintain == less time wasted == more time for real Lunar development. The Extremists will not benefit from this because they always build from repos. :) If you are another group I forgot to make happy let me know - I will find a yet better solution :) further mails will be shorter, I promise :) Zbigniew From ratler at lunar-linux.org Wed Oct 21 06:48:27 2009 From: ratler at lunar-linux.org (Stefan Wold) Date: Wed, 21 Oct 2009 06:48:27 +0200 Subject: What to do with xz? In-Reply-To: <200910210326.06431.zbiggy@o2.pl> References: <200910140556.06750.zbiggy@o2.pl> <200910210326.06431.zbiggy@o2.pl> Message-ID: <1256100507.4536.69.camel@localhost> On Wed, 2009-10-21 at 03:26 +0200, Zbigniew Luszpinski wrote: > Hello, > > I solved the problem of unwelcome xz in the following way: > I added to DETAILS XZSOURCE and XZSOURCE_VFY - you can see this in recent aria2 commit. > Soon I will post here an optional patch which will make use of these new variables. No! Don't do that, don't extend DETAILS with support for new variables that obviously do exactly what SOURCE, SOURCE_VFY already do. Please revert that commit. If you want to try xz on that module do it the proper way by adding .xz by using SOURCE and SOURCE_VFY, then let the xz plugin handle it. Now you're just trying to make stuff even more complex then they have to be. > > The Revolutionists will have a chance to start using xz: > +enjoying smaller files, > +enjoying faster downloads, > +proving that xz is present in more places than Conservatists think, > +being proud Lunar is innovative and on cutting edge of IT development. > > The Conservatists will be happy: > +knowing they are safe from xz enemy. > +knowing they do not have to do anything to keep Lunar as it was before. > +the xz/lzma tarball will appear only together with module version update. (I hope the other xz fans will behave the same). > +the quietness will come back to ML :) > > The Liberalists will be happy: > +there is another optional choice for them with rollback option. > > I will be happy: > +my zlocal will be smaller == easier to maintain == less time wasted == more time for real Lunar development. > > The Extremists will not benefit from this because they always build from repos. :) > If you are another group I forgot to make happy let me know - I will find a yet better solution :) Please there is no point trying to start a flamewar on our ML. So lets end this discussion right here. There is input in this thread already on how to move forward with an adoption of .xz, start with the minor modules and then we evaluate the outcome. -- Sincerely Stefan Wold Lunar Linux developer - PGP public key 6E810F05 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: