VectorLinux

Cooking up the Treats => Distro development => Topic started by: Joe1962 on June 22, 2007, 01:52:24 pm

Title: Discussion of "Patches" repository guidelines
Post by: Joe1962 on June 22, 2007, 01:52:24 pm
Originating from Caitlyn's post:
(http://www.vectorlinux.com/forum2/index.php?topic=3385.msg21645#msg21645
and due to my reticence to split the topic, as it was a logical continuation of that thread, I decide to quote and reply to her here, in the hopes of starting a good discussion of the subject at hand:

The bad news:  That gtk+ upgrade placed in patches to allow for the latest and greatest GNOME.  That's an upgrade no other distro would have dared do -- a major upgrade of gtk without spinning a new release of the distro.
I agree with this.

No.  headacher raised a valid point.  Either gtk+2 2.10.11 should have immediately become our new standard (i.e.: everything built against it) or it should never have gone into patches.   I argued against the gtk+ upgrade, if you remember.  Now you know why.  What has been done is contrary to the practices of every major distro for a reason.  You can't have two standards at once in the same version of the distro.
I argued agaisnt this too. I also wasn't happy with all the upgrading that took place between 5.8 Standard and SOHO, as it makes common packages and repo all the more complicated. I proposed a common base during the pre-development phase of Dynamite 5.9, which was to become 6.0, but actually became 5.8. I hope this common base can really be achieved in 6.0, as it did not work out for 5.8.

This thread really should be moved somewhere where the major movers and shakers of VL can discuss it.
[snip]
Someone care to move this thread?
Hope you don't mind, but since your post fitted well in that thread, I didn't want to split it off, so as you can see, I took another route.

My solution, which I'm fairly sure will be rejected, would be to take the current state of VL 5.8, gtk+ 2.10.11 and all, since it does all work and is now well tested, and respin as Vector Linux Standard 5.8.1.  Kind of like Mandriva 2007.01 (Spring), it isn't a major new release but it does set a new base to work from with lots of cool upgrades.
FWIW, I support this proposal.
Title: Re: Discussion of "Patches" repository guidelines
Post by: easuter on June 22, 2007, 03:10:12 pm
Quote from: caitlyn
The bad news:  That gtk+ upgrade placed in patches to allow for the latest and greatest GNOME.  That's an upgrade no other distro would have dared do -- a major upgrade of gtk without spinning a new release of the distro.

[...]

Either gtk+2 2.10.11 should have immediately become our new standard (i.e.: everything built against it) or it should never have gone into patches.   I argued against the gtk+ upgrade, if you remember.  Now you know why.  What has been done is contrary to the practices of every major distro for a reason.  You can't have two standards at once in the same version of the distro.

Its a bit of a double edged blade...

Both are very valid points though. There were no plans at all for a "home-brew" GNOME when VL 5.8 Standard was being developed so no "rule" was set for what would happen to the package upgrades needed by GNOME.

However, when GNOME and its deps were added to the repo, SOHO hadn't even been released yet (it was still RC1 or RC2), and neither had the LiveCDs IIRC. Still a lot of work was being done on those projects.
So to be able to use GNOME at all, we would have had to wait until these were finally finished to be able to make (and test) a VL 5.8.1 Standard that included the GTK and all the other upgrades...?

It would have made perfect sense to do that, but unfortunately manpower is not one on VL's strong points...

Hopefully a clear (and publicly accessible) roadmap can be drawn up for VL6, which addresses issues like as to how far packagers can go before a new VL release is required, with all the "do's and don't-do's". However, Slackware 12 will also be a major "leap into the present", with lots up-do-date packages in the default installation, so this will reduce (maybe almost eliminate) the need for such prolific package upgrades that we've seen for VL 5.8.

EDIT: better direct communication between packagers/devs/etc would be nice too...
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 23, 2007, 03:58:02 pm
joe1962:  Your solution to getting this issue discussed is perfectly fine with me if the core developers and decision makers are here.

FWIW, I agree with everything said by joe1962 and eauster so far.  I'd like to bring a couple of additional points from the previous thread which I made to this one to hopefully focus on why this discussion is important:

Quote
A lot of places (particularly businesses and government) as well as security paranoid individuals like me want to keep any known vulnerabilities closed.  I do this stuff for a living, though not with Vector Linux.  I'd love VL to get to the point where businesses and government think it's worth considering.  You may have no idea how close, quality wise, this distro is.

You've just raised another point.  gtk+ 2.8.x wasn't a security vulnerability and it wasn't broken.  There wasn't a compelling reason other than GNOME 2.18.0 to put gtk+ 2.10.11 in patches.  It, with GNOME, probably needed to be put in a separate (new) repository so that everyone who does a

slapt-get --update
slapt-get --upgrade

wouldn't get it.  It's too late for that now.

The point I'm making is that we need to have a sensible upgrade policy and a build policy that tracks it.  This isn't about individual choices but rather what our user community, especially people migrating from things like the very broken recent release of Xubuntu Feisty Fawn, is likely to expect from us.

When Daniel Baranger of Red Hat suggested that VL lacked karma (he couldn't find another way to say why he didn't like it) I kind of agreed.  I think this is an example of why.  It isn't that there is anything wrong with VL.  It's actually pretty darned impressive at this point.  It's more suffering growing pains and needs some more rigorous (professional?) standards that we all follow.  Oh, and yeah, that includes me even if I don't like them -- hence my feeling that I need to build against a clean box since that is the current standard -- at least until this issue is addressed.

In every other major distro (I'm assuming VL is moving into the ranks of majors here) once something is in patches or the distro equivalent it is the de facto standard.  Every other major distro recommends keeping up with patches.  Up until VL 5.8 there was no workable mechanism to do that.  With VL 5.1 the docs advised against a slack-get --upgrade for good reasons: 

1) slack-get wasn't terribly mature, and
2) the repository management wasn't there yet 

If you tried to do a mass upgrade on 5.1 you broke things.  On 5.8 that is no longer true.  It works like a charm, even with the rather daring gtk+ 2.10.11 upgrade.  The sole broken piece after the gtk+ upgrade is wifi-radar and the package which fixes that is in testing.  Doing a mass upgrade is what someone migrating from another distro with sane package management (read: NOT Slackware) is probably used to and it's what they likely expect.

The nice thing about respinning the current iso is that you basically replace the stuff on the current iso with the stuff from patches (plus wifi-radar 1.9.8 from testing) and go with what you have.  It's pretty much all tested already except for running the installer with the newer packages.  Considering the huge amount of stuff already upgraded (xfce, seamonkey, firefox, gtk+, glib2, vl-hot, cairo, pango, etc...) I don't think it would be much a stretch to "release" 5.8.1 as it has as many or more changes than what typically happens between two releases of Ubuntu or Fedora or whatever.  You wouldn't need new repositories either... just continue with the existing ones.  You also get the bonus of new community press on the new "release".

Re: SOHO.  My forthcoming review for O'Reilly makes the point that it should have been called 5.8.1.  It has so many changes and upgrades from Standard that they can't really be considered the same version IMHO.  BTW, you're going to like my review of SOHO.  I haven't seen KDE move that fast on my aging hardware in ages.

-Cait


Title: Re: Discussion of "Patches" repository guidelines
Post by: The Headacher on June 24, 2007, 01:28:02 am
So the question becomes, if somebody builds a new version of a package that comes with VL and is likely to give compatibility issues (not a bugfix or security update ), where DO we put it (we already established it shouldn't go into patches)???? Unstable?
Title: Re: Discussion of "Patches" repository guidelines
Post by: easuter on June 24, 2007, 02:26:58 am
Quote
if somebody builds a new version of a package that comes with VL and is likely to give compatibility issues (not a bugfix or security update ), where DO we put it (we already established it shouldn't go into patches)Huh? Unstable?

I think there also has to be a destiction in future of whether the package is a "major" upgrade. If its something trivial its probably not  a problem to be added to the patches repo...or maybe there should be an "updates/" directory to destinguish the non-critical package upgrades from the critical ones?...

Title: Re: Discussion of "Patches" repository guidelines
Post by: bigpaws on June 24, 2007, 05:35:17 am
I am going to jump in here.

First:
Quote
A lot of places (particularly businesses and government) as well as security paranoid individuals like me want to keep any known vulnerabilities closed.  I do this stuff for a living, though not with Vector Linux.


There are businesses that are looking at Vector. As far as security and stability, Vector is decent. There are too many services offered on a stock install. The firewall decision is good but has been mind boggling to many who are trying the get networking running. Then they find out through several hours of testing that a firewall is in place. A note on the install would be nice.

Quote
Up until VL 5.8 there was no workable mechanism to do that.  With VL 5.1 the docs advised against a slack-get --upgrade for good reasons:

1) slack-get wasn't terribly mature, and
2) the repository management wasn't there yet 

The repository is not there still IMHO. In order to handle that type of request a sane base needs to be developed and adhered to. Then scripts to evaluate all packages and then upgrade. Mass upgrades are happening in other distributions. However other than reading the whole changelog how does a person know what happened to there system? Both of the above require manpower that may not be
available at present.

The GTK issue should have not become an issue. In order to preserve sanity only form of GTK should
have been used. As noted too many chances of things going wrong.

Why is it that everytime that a new update (not a patch) comes down the road that it has to be included? IMHO sticking to a base for standard and SOHO keeps stability. A patch repository is
a good idea.

Slackware is a stable platform without any suprises. Vanilla everything, no tweaking. Slackware current is moving toward a desktop vs a server. The less tweaking the better the ability of users to
keep up with there system if they so choose.

The GUI vs Text installer why is that? Most others do it so we have to? Lets stick to what  Vector is
straightforward and clean. During an install how about all on board hardware runs that is possible, in the end users will love you for it.

Bigpaws
Title: Re: Discussion of "Patches" repository guidelines
Post by: saulgoode on June 24, 2007, 04:38:32 pm
... another distro with sane package management (read: NOT Slackware) ...

Now, now. Considering the nature of this thread, not to mention the problems being experienced by Red Hat, Ubuntu, et alia WRT improper handling of dependencies and maintenance of repositories, are you sure this is not a case of the lunatics running the asylum?
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 24, 2007, 06:25:29 pm
I haven't experienced any serious problems with Ubuntu.  Fedora Core has had serious problems, not Red Hat Enterprise Linux.  In any case, all the majors generally do have sane package management.  Lately so does Vector Linux.  Slackware still doesn't handle dependencies so it doesn't.

-Cait
Title: Re: Discussion of "Patches" repository guidelines
Post by: saulgoode on June 25, 2007, 12:53:52 am
I am not sure what your definition of "serious problems" is, but for me it means a package manager demands more of my time working around other people's screwups than I do handling my own.

Try performing an 'apt-build world --recompile' with Ubuntu and tell me the packages are being managed properly; even Ubuntu's developers recommend against performing version upgrades.

RPM still can't handle the order in which packages are required to meet dependencies. Maybe the RHEL repositories have this sussed, but for a $300 a year subscription, I should expect some accountability. FWIW, I have seen recent discussions about PostGreSQL dependency problems with RH and, as you stated, FC seems to be something of a mess.

There is nothing "insane" about trusting one's own abilities to maintain his system over those claimed by someone else, especially when those claiming to be able to handle the task have a ten-year track record of failure. One of the reasons Slackware is so reliable is because its developers are able to focus on fixing the problems in the packages, not the ones in the package manager.
Title: Re: Discussion of "Patches" repository guidelines
Post by: Darin on June 25, 2007, 05:00:18 am

RPM still can't handle the order in which packages are required to meet dependencies.


that is not entirely true...it is depending on the spec file format they use...and how they put in the dependancies...you can force program to be installed in a certain order correctly
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 25, 2007, 08:50:02 am
Quote
RPM still can't handle the order in which packages are required to meet dependencies.

This hasn't been true for years, as in since rpm 4.x.

10 year track record of failure?  SuSe, Mandriva, Fedora up to FC4, Red Hat, Ubuntu, and Debian all have done a good job -- to the point where most users will never notice a problem.  Slackware is *known* for dependency hell because it doesn't even try to handle dependencies at all.  It's why I wrote off Slackware as  a geek toy years ago.

The reason VL is getting so much attention is because it has successfully matched Slackware stability, reliability, and speed with the ease of use, most especially including package management, that the major distros are known for.

Now, let's get back to solving the very real issues rather than attacking other distros which are far more popular than either Slack or VL -- and for good reason.

-Cait
Title: Re: Discussion of "Patches" repository guidelines
Post by: Joe1962 on June 25, 2007, 08:58:51 am
Now, let's get back to solving the very real issues rather than attacking other distros which are far more popular than either Slack or VL -- and for good reason.
Indeed... Let's get back to discussing how to make our repository better. You can always find someone worse than yourself and use this as justification, but I find the best way is to constantly strive for perfection, only then will you get as close as humanly possible to it... 8)
Title: Re: Discussion of "Patches" repository guidelines
Post by: saulgoode on June 25, 2007, 10:38:40 am
This hasn't been true for years, as in since rpm 4.x.
My mistake. I guess I misinterpreted the problem (http://www.linuxquestions.org/questions/showthread.php?t=549496&highlight=rpm+hell). Apparently, RPM didn't recognize that some installed packages conflicted with each other.

10 year track record of failure?  SuSe, Mandriva, Fedora up to FC4, Red Hat, Ubuntu, and Debian all have done a good job -- to the point where most users will never notice a problem.  Slackware is *known* for dependency hell because it doesn't even try to handle dependencies at all.  It's why I wrote off Slackware as  a geek toy years ago.

Slackware is in no way "known" for dependency hell (try googling "slackware dependency hell" and see what comes up). The reason Slackware doesn't have a dependency hell is because they don't split every program up into a half dozen different packages. If you have the packages from the first two installation disks, 99% of your system needs are met. Ever hear of the Pareto Principle? If you actually examine your entire process of system maintenance, you will find that the complexity of the RPM and APT systems creates more problems than it solves. Vector developers would be well-served to keep the big picture in view while improving their own system of package management.
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 25, 2007, 10:52:34 am
Perhaps I'll continue the discussion (argument?) with Saul in another thread.  It's gotten us way off topic. No disrespect to Saul intended, but...

Right now the issues are:

1-  Is doing a major upgrade of core libraries with a release cycle (i.e.: gtk+2, glib2) was a good idea or not?
2-  Should such upgrades should go into patches so that anyone who does a slapt-get --upgrade gets them or rather should they be in a separate repository with patches reserved for security patches and bugfixes and
3-  Since so many manjor upgrades to 5.8 have been offered should VL respin the Standard ISO and call it VL 5.8.1. or something, including all the patches to date since they are tested

There seems to be support for a separate repository other than patches (I agree with this, BTW) and for a respin, which I originally proposed.

I'd like to hear from the movers and shakers, the people who actually make such decisions.

-Cait
Title: Re: Discussion of "Patches" repository guidelines
Post by: uelsk8s on June 25, 2007, 08:01:38 pm
1. It was a bad idea, and IIRC was warned against by a few different people.
2. these packages should have been put into the Gnome folder in the 5.8 repo, and that is where I understood they were going to be put.
3. IMO Yes, but ultimately that will be vecs decision.

Uelsk8s
Title: Re: Discussion of "Patches" repository guidelines
Post by: The Headacher on June 26, 2007, 02:55:27 am
Quote from: Caitlyn
1- Is doing a major upgrade of core libraries with a release cycle (i.e.: gtk+2, glib2) was a good idea or not?
I'd rather stay with the latest stable Slack versions of those libraries if possible, or Slack current if we really have to upgrade.

Quote
2-  Should such upgrades should go into patches so that anyone who does a slapt-get --upgrade gets them or rather should they be in a separate repository with patches reserved for security patches and bugfixes and
Well, at least they shouldn't go into patches, that much seems obvious to me now.

Quote
3-  Since so many manjor upgrades to 5.8 have been offered should VL respin the Standard ISO and call it VL 5.8.1. or something, including all the patches to date since they are tested
I'm not sure what would be the benefit of this. If we wait a few weeks/months we can start playing with slack 12 and apply what we have learned with VL 5.8 on VL 6.

Quote from: Uelsk8s
2. these packages should have been put into the Gnome folder in the 5.8 repo, and that is where I understood they were going to be put.
Let's please watch out for creating a lot of different repo's, that's very Debianesque and juist plain confusing (Last time I tried some Debian based distro I even needed to add some weird repo for the Nvidia driver, the binary didn't compile on it).. I mean, just for Gnome is fine of course, we'd better watch out for fragmenting the repo's too much though.
Title: Re: Discussion of "Patches" repository guidelines
Post by: Darin on June 26, 2007, 06:21:45 am
why not have 2 branches...one is testing and the other is normal....now you can move the testing-updates over to normal pretty easily...as for the foundation of the branch...why not use the same as a the package was made for..lets come up with a revised skeleton for kde...or even use the repo branch to be the same...makes where to put packages alot easier as you can have gnome or any other window manager in a window manager section....simplicity is the answer especially with people contributing packages..just my 2 cents on how to clear some confusion in things
Title: Re: Discussion of "Patches" repository guidelines
Post by: Joe1962 on June 26, 2007, 11:07:32 am
I agree too many repos is complicated, but too few is a problem, so I guess we need to find a balance. Maybe we need to set up an "experimental" repo for stuff like that gtk upgrade? Then we can keep "patches" for bug or security fixes and "extra" for non-iso packages built on the vanilla release (or built on the patches at most).
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 26, 2007, 01:36:14 pm
I like Joe1962's suggestion.  It seems like a good, balanced approach to the repos for future interesting upgrades that we might not want to be universal.

The advantages of a respin are two fold:

1.  For packagers and developers it gives us a new base from which to build.  Right now if I build on a "clean machine:" per VL standards and build against gtk+2 2.8.x there is no guarantee my package will work on a machine where patches have been applied.  If I build against gtk+2 2.10.11 my package will force an upgrade if installed with gslapt or slapt-get and may not work on a machine where patches have been applied.  By spinning a new release gtk+2 2.10.11 as well as the upgraded versions of cairo, glib2, pango, atk, etc... all become the standard that everyone works from.

2.  For users they get a tested and fully upgraded, GNOME-ready VL Standard built on the same 2.6.20 kernel as SOHO.  It saves hundreds of megabytes (or possibly over a gig now if you include a kernel upgrade) or downloads of patches if you choose to keep your machine up to date for security and bugfixes.

If it meant a lot of work for the developers I'd agree with waiting for VL 6.  The point is that other than testing the installer with new package bundles everything has already been developed and thoroughly tested.  This would only be an incremental release, not a new development cycle.
Title: Re: Discussion of "Patches" repository guidelines
Post by: The Headacher on June 26, 2007, 03:00:02 pm
Quote
For packagers and developers it gives us a new base from which to build.  Right now if I build on a "clean machine:" per VL standards and build against gtk+2 2.8.x there is no guarantee my package will work on a machine where patches have been applied.  If I build against gtk+2 2.10.11 my package will force an upgrade if installed with gslapt or slapt-get and may not work on a machine where patches have been applied.  By spinning a new release gtk+2 2.10.11 as well as the upgraded versions of cairo, glib2, pango, atk, etc... all become the standard that everyone works from.

And break Slack 11 compatibility... the gtk package that comes with vl 5.8 is the default one from Slack 11. So if gtk 2.2.8 packages don't work properly on 2.2.10 it would be a release for which linuxpackages.net and slacky.eu are no longer useful package resources (at least for apps that use gtk).

I for one don't care much for a new version of VL 5.8 just because some package went to the wrong repo and now we're stuck with some weird packages we're not sure what to do with. The differences between the 2 versions aren't that huge IMO. If we're going to release a new standard with updated gtk etc. we'll also need to release a new SOHO, or just accept the fact that packages on "5.8.1 standard" will not work on "5.8 SOHO" without massive forced upgrading of packages. Or package on 5.8 standard after all so it will work on both most of the times.

Everything I've built on a clean 5.8 standard install has worked just dandy on SOHO (at least for me) . I honestly don't see much of a point in starting completely over with very little packages in the repo. We'll have to do that when VL 6 is released, which is one of the reasons I'm trying to learn SlackBuilds ATM. 

How's about nuking the offending packages (or at least moving them somewhere that isn't a repo that's enabled by default) to start with?
Title: Re: Discussion of "Patches" repository guidelines
Post by: easuter on June 26, 2007, 03:28:23 pm
Quote
And break Slack 11 compatibility... the gtk package that comes with vl 5.8 is the default one from Slack 11. So if gtk 2.2.8 packages don't work properly on 2.2.10 it would be a release for which linuxpackages.net and slacky.eu are no longer useful package resources (at least for apps that use gtk).

But the GTK upgrade doesn't break GTK 2.8-based packages, Xfce and the rest continue to work after the upgrade (except for wifi-radar).
Anyway, I wouldn't have uploaded the packages if they were "destructive", its one of the reasons it took me so long to get GTK and GNOME finished: test packages, fix problems, test again, fix other problems, test again, fix, fix, fix, test, test, test, and so forth untill it worked...  :-X

But yes, I do agree with sticking to "major" Slackware packages for official releases of VL.

So we have a few options, based on the discussion in the thread (but not limited to):

-nuke the packages
-create an "updates" repo
-create a gnome repo, where everything that gnome relies on, upgrade-wise, must be moved to along with the gnome core packages
-create a new release of VL with these packages as a base

Now its really up to the VL elders to make a decision regarding this ;)
Title: Re: Discussion of "Patches" repository guidelines
Post by: Joe1962 on June 26, 2007, 04:27:01 pm
If we go the new repo route, I'd rather it isn't called "upgrades" or anything like that, because it sounds confusingly similar to "patches". I would prefer "experimental", or something in that vein. Even "contrib" if necessary... ;D
Title: Re: Discussion of "Patches" repository guidelines
Post by: caitlyn on June 26, 2007, 05:58:19 pm
To The Headacher:  It's not just Standard.  A slapt-get --upgrade will install gtk+2 2.10.11 into SOHO as well.  It's in both places.  The good news:  It's still only wifi-radar that gets broken.  Anyone going to move wifi-radar 1.9.8 out of testing so that a slapt-get --upgrade doesn't break anything?
Title: Re: Discussion of "Patches" repository guidelines
Post by: joec on June 27, 2007, 05:21:47 am
If we go the new repo route, I'd rather it isn't called "upgrades" or anything like that, because it sounds confusingly similar to "patches". I would prefer "experimental", or something in that vein. Even "contrib" if necessary... ;D

From one of those new to linux, new to VL (not so good with Windows either) users -- confusion is bad. :) The two major issues i have faced are understanding partitioning and figuring out the repositories. VL seems to install with a large list of repositories, mirrors and potential repositories.
It is not always clear where to look for what. A repository for "patches" which only fixes security/bug problems without doing anything else is a good thing.