VectorLinux

Cooking up the Treats => General Development => Topic started by: kidd on December 04, 2008, 07:22:37 am

Title: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: kidd on December 04, 2008, 07:22:37 am
Sbbuilder (SlackBuild Builder) is a small perl script for automating the
generation of the directory structure and the SlackBuild necessary for
making packages compliant with the VectorLinux repository.
It was made with three goals. The first one is to give facilities for
making a standard repository. The script would reproduce the same
structure for every package making more easy the maintenance and the
realization of automated tasks. The second is to give a tool for packagers
in order to make your life a bit easier and improve your productivity.
Developers and enthusiastics that build many packages will find it (we hope)
useful. Finally, we hope it will be part of a bigger app developed for porting
an existing VL repository to different architectures.
First the installation. We are using Template-Toolkit, you need to install it
in your system. It is very easy:

Code: [Select]
cpan Template
If you install it as root the resource will be available for all users, if you
are planning to use sbbuilder as your regular user you can install TT with your
regular account. You will be prompted with a few questions, defaults are OK.
If you never used cpan before, it's the perl modules repository. It ask for
your location the first time in order to use the nearest mirror. We are going
to include TT in the default installation in the future.

Now we are ready to take a look into Sbbuilder.  Install it with
Code: [Select]
slapt-get --install sbbuilder
Its usage is pretty straight forward. Two parameters are required:

Code: [Select]
ssbuilder --package=<name_of_the_package> --version=<version>
This will build the directories and place a SlackBuild with the package name
and version information.
The directory tree is like this:

Quote
AppName
|
`-- Version Number
        |
        `-- src
             |
             `-- AppName.SlackBuild

The SlackBuild will be filed with AppName and Version where is pertinent.
A number of optional arguments can be used in a way that it allows to
generate the whole SlackBuild without need of extra editing.

--user Your packager ID. If not used it will look for the
VL_PACKAGER environment variable (you can define it in your shell
startup script, tipically ~/.bashrc). If that var is not set the
script will use the current username returned by `whoami`.

--build Define a build number (as in 1vl60). If not defined the
script will default to 1.

--link This is a new variable in the SlackBuild template. You can
define a URL to the tar.gz or tar.bz2. The SlackBuild will download
the sources using wget if no sources are found in the working dir.
This variable will be useful if a repackaging is needed.

--type Different kinds of sources are available in the OpenSource
world :). We need to configure the sources properly.It defaults to usual configure
The possible values of this argument are:

* perl Use this to build perl modules and general perl
applications. Perl packages need a special configuration for
working and this option provides it.

* python Obviously for building python apps :). This option
writes the classic python setup.py lines in the proper place.

* noarch This will replace the ARCH (tipically i586)value with
"noarch". In this way users know that they can use your package
in any architecture.

* cmake For modern and cool developers that are using cmake
instead the old lady GNU Autoconf.

--configure_options Pass extra options for the configure script (or cmake)
with this option.

--ldflags Define extra ldflags for the actual compiling process.

--help You wont believe it, it shows a short help message.

--manual Well, shows the manual.

As you note, if all the options are used you get a fully working SlackBuild with a
oneliner (I admit it, a long one), and if you place a slack-desk file in the same
dir of the SlackBuild, you may not need extra editing. Of course this is for
applications in general, many will need extra tweaking of the SlackBuild.
The new template written by uelsk8s provides a special place for the application of
patches, and you have the classic "extra tweakings" section after the configure
lines. It's very important to use the indicated places for doing this stuff, so the
script will be easy to understand for other packagers and also automated tasks can
be made (grep is powerful! ;)).

Enjoy and as always, suggestions, patches, fixes and insults are welcome.

uelsk8,rbistolfi,kidd
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: uelsk8s on December 04, 2008, 08:13:14 am
I just wanted to say, that I cant take the credit for writing the Template.
IT was written by lots of people in the packagers forum and bits found throughout the internet, I merely composed it
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: M0E-lnx on December 16, 2008, 09:10:35 pm
Not having any luck with the CPAN modules here.

so I can't test
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: toothandnail on December 17, 2008, 12:35:10 am
I've never used cpan before ( :-[ and didn't read the instructions until it failed the first time, naturally). However, once I did read the directions, was able to get cpan configured and (presumably) got the templates module loaded.

Only tried it a couple of times for pretty standard builds, but it has worked for both of them (the libpcap and wireshark builds were both configured using sbbuilder). Looks good. I will be trying some Python based stuff next, since that is something I always have difficulty building conventional SlackBuilds for.

Very nice. Thanks, a very useful tool.

paul.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: MikeCindi on December 17, 2008, 05:32:04 am
I couldn't find the sbbuilder package but cpan setup okay as far as I could tell. Can't go any further for now...
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: kidd on December 17, 2008, 06:10:24 am
MikeCindi,

Thanks for testing :)   sbbuilder is in the testing repo of 5.9 and 6.0 aswell.

Make sure you have it enabled.

Keep questions/suggestions coming!


Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: MikeCindi on December 17, 2008, 06:19:03 am
Okay, I'll try again later as I still can't find it (perhaps the repo mirror needs to "refresh").

It's later...this is what I get:

Code: [Select]
root:# slapt-get --install sbbuilder
Reading Package Lists... Done
The following NEW packages will be installed:
  sbbuilder
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 4.0kB of archives.
After unpacking 30.0kB of additional disk space will be used.
1/1 Get http://vectorlinux.osuosl.org/veclinux-6.0/packages/ sbbuilder 0.4.8-noarch-1vl60 [4.0kB]...Failed to download: HTTP response code said error
I'm getting this with a lot of packages from the 6.0 repo the last two days.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: toothandnail on December 18, 2008, 04:01:34 am
I got the same error when trying to reinstall (after I reinstalled beta 2.2 in place of beta 2.4). Fortunately, I had the older version in the local cache, so I was able to manually install it.

:) Waiting for the update...

paul.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: MikeCindi on December 23, 2008, 09:07:47 pm
Okay, now with rc1 the sbbuild seems to be working as it should. Now the question is am I working as I should on building packages. I've been using checkinstall for long enough that using makepkg is not coming back to me as I would want it to. Thus after getting into the source dir my process should be:
Code: [Select]
sbbuilder --package=<name_of_the_package> --version=<version> ...
./configure && make
su
make install DESTDIR=/myworkarea
cd /myworkarea
makepkg <name_of_the_package>-<version>-<build>-1vl60.tlz
Perhaps I will grow to like this over time but
Code: [Select]
./configure && make
su
checkinstall -L
seems to be much simpler.  :'(
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: uelsk8s on December 23, 2008, 09:34:09 pm
actually for the simple apps it would be just
Code: [Select]
ssbuilder --package=<name_of_the_package> --version=<version> ...
sh <name_of_the_package>.SlackBuild

Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: MikeCindi on December 23, 2008, 10:39:58 pm
Thank you; sounds easy enough. I've obviously gotten lazy with making packages and not kept up...
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: MikeCindi on December 24, 2008, 10:32:59 am
actually for the simple apps it would be just

This is TOO easy. I guess I've mostly just made "simple" packages as only a few have required me to pass args to configure. As I read through the SlackBuild script it does everything that I would do (and much more) and should make it easy for anyone to create standardized packages of the "simple apps". With a little reading and tweaking of the script the more complex apps (to build) should also be easy enough.

I realize what I've just written is the intent of kidd's original post but somehow it didn't register with me. Thanks again for making VL so user-friendly.

Mike
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: kidd on December 24, 2008, 11:18:34 am
It's great you liked it.

It's a pretty simple script, in fact, the perl part has nearly no logic at all, but with a wise use of Template Toolkit we've managed to write a fairly user-friendly tool.

Now there's no excuse to not give packages to community and repos ;)

Cheers,

Kidd
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: toothandnail on January 29, 2009, 03:53:26 am
I've hit a bit of a problem with sbbuilder's handling of info files. Also have a suggestion for an addition...

I've just built a package which by default places info files in /usr/share/info. While the routine in the sbbuilder-generated script looks as though it should work, it leaves the directory in place, so the info files don't get moved and don't get gzipped. I'm still trying to work out why the routine doesn't work....

I also have a suggestion for an improvement in handling info files. A lot of packages that include info files also include a 'dir' file. As things stand at the moment, that file will be placed in /usr/info and then gzipped along with the info file.  It will overwrite a normal /usr/info/dir file. Looking at the package list from 6.0 RC4, there are two packages that come with /usr/info/dir.gz files (aspell and texi2html).

I would like to suggest that the handling of info files be expanded to search for and delete any dir files during the build, and if info files are processed during the build, a doinst.sh be generated to update /usr/info/dir during the install. I normally remove and rebuild the completely, since I don't know what state the existing /usr/info/dir file may be in. For instance, in the texinfo package I built, I create this doinst.sh:

Code: [Select]
doinst() {
# Install ddrescue info file.
    cd usr/info
if [ -f dir ]; then
    rm dir
    fi
if [ -f dir.gz ]; then
    rm dir.gz
    fi
for i in *.info.gz
        do
        install-info $i dir
    done
}

Adding something like that would make sure that any info files in a package would be applied, and would also ensure that /usr/info/dir was correctly updated.

paul.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: rbistolfi on January 29, 2009, 04:25:52 pm
Thank you very much for your input Paul. Uelsk8s addressed the issue and a new version (0.4.11.3) was uploaded.
Kidd implemented a new feature. Now, if no --package and --version options are used, sbbuilder will attempt to get them from the url passed to --link. This will work only with source tarballs named with the classic scheme (name)-(version).tar* though. The error is not handled yet, I forgot about it. If sbbuilder cant guess version and name, it will spit some perl garbage, we will implement a nice message and a clean exit for the next one :)
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: toothandnail on February 03, 2009, 01:12:41 pm
Another request/suggestion. Requiredbuilder sometimes misses deps (and sometimes adds deps that are build-time, not runtime). It would be useful to have a mechanism to specify extra deps, or, alternately, specify deps that shouldn't be put in slack-required.

Not sure how easy (or even if its possible) that would be to implement, but it would add some useful extra functionality to sbbuilder.

paul.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: no2thesame on March 21, 2009, 09:42:07 pm
What am I doing wrong?

sbbuilder --package=unetbootin-source-319.tar.gz --version=319
produces
File exists. Failed creating top dir.
Not a directory. Failed creating version folder.
Not a directory. Failed creating src folder.
Not a directory at /usr/bin/sbbuilder line 192.


I thought that sbbuilder might have been corrupted because it worked a couple of weeks ago and so I Uninstalled v 4.12 and reinstalled (could only choose v.4.11... Why is that?)

Previously ssbuilder .4.12 with the same command line produced:
File exists. Failed creating top dir.
Not a directory. Failed creating version folder.
Not a directory. Failed creating src folder.
Not a directory at /usr/bin/sbbuilder line 198.


Thanks
no2
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: kidd on March 22, 2009, 01:33:42 am

sbbuilder --package=unetbootin-source-319.tar.gz --version=319

should be

sbbuilder --package=unetbootin --version=319


then you'll have to move the source package to the correct dir.

Keep in mind this tarball name isn't the 'standard' for most tarballs, and sbbuilder will not see it (it'll look for unetbootin-319.(tgz|tar.gz|tar.bz2)  . you can try renaming the tarball, but I suspect when unpacking it, it'll create a dir called unetbootin-source-319 , not unetbootin-319 .

Well, I think you'll have to edit the slackBuild to make everything run.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: no2thesame on March 22, 2009, 03:03:03 am

sbbuilder --package=unetbootin-source-319.tar.gz --version=319

should be

sbbuilder --package=unetbootin --version=319

Well, I think you'll have to edit the slackBuild to make everything run.

Thanks for picking that up, Kidd. At least now sbbuilder works but...
running the SlackBuild ends with this error:

./unetbootin.SlackBuild: line 113: cd: /home/no2thesame/Unetbootin/unetbootin/319/tmp/unetbootin-319: No such file or directory
Setting permissions...
Configuring source...
./unetbootin.SlackBuild: line 156: ./configure: No such file or directory


What do I need to do now?
no2
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: kidd on March 22, 2009, 03:31:15 am
you can look the error lines: 

line 113 is 

Code: [Select]
cd $TMP/$NAME-$VERSION
you should first unpack your tar.gz manually, and see what's the directory name it creates.  Then adapt some lines to its format.  By default, tar.gz files are name-version.tar.gz and when untarred, they create a name-version dir . as I said previously, you'll have to check the generated slackbuild manually to make everything work.

HTH
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: no2thesame on March 22, 2009, 06:44:17 am
.. as I said previously, you'll have to check the generated slackbuild manually to make everything work.

This is way over my head... The two previous attempts at packaging "just worked". I suspect this is not a "default" tar.gz!

After I changed the name of the package to unetbootin-319.tar.gz, sbbuilder did its job and produced unetbootin/319/src/unetbootin.SlackBuild

However once I ran the SlackBuild, there was no unetbootin/319/tmp/unetbootin-319, which is what the SlackBuild line 115 is looking for. There are also no .configure files, in the tmp directory which  is what line 156 is looking for.

When I manually unpack the source it gives me language files and some .ts and .cpp The build-nostatic file which I take to be equivalent to our SlackBuild file make lots of reference to Debian things and according to the ReadMe I will need qmake and qt4.

All I want to do is make a bootable USB! So I'm giving up on unetbootin and will ask for help in a more appropriate area of the forum.

Thanks a lot though, Kidd. Appreciated it.
Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: kidd on March 22, 2009, 08:23:48 am
no2thesame, sbbuilder nor a proper SlackBuild are not required at all to make a package for your own use.

If you just want to install unetbootin, just follow the instructions you will (or should) find in a file called INSTALL or README , or search for  unetbootin installation.  you'll have to manually compile it, and install to your box.


Title: Re: Packaging mini-tool for 6.0 (and maybe 5.9)
Post by: no2thesame on March 22, 2009, 02:35:25 pm
Thanks Kidd,
no2thesame, sbbuilder nor a proper SlackBuild are not required at all to make a package for your own use.
But I would rather have something I could share. Anyway the dependancies look messy (to my untrained eye) and there seems to be a simpler way to do the job.

Learned more about packaging though, so all is not lost.
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: Joe1962 on December 06, 2010, 08:36:27 am
Correct me if I am wrong, but I could not find a development thread for sbbuilder besides this oldish one, so I renamed it and madde it sticky...

That said, I have a suggestion to change the documentation files that are copied to the package. This is based on a longish time of using sbbuilder to set up my build scripts and then tweaking them. The changes go in /usr/share/sbb/template/tweaks.

replace:
Code: [Select]
cp -a AUTHORS COPYING ChangeLog Credits MANIFEST README TODO \with:
Code: [Select]
cp -a ANNOUNCE* AUTHORS* BUGS* COPYING* ChangeLog* CHANGELOG* Credits* DEVELOPERS LICENSE* MANIFEST PROVENANCE README* TODO* Exception_to_GPL_for_Qt.txt \
I also changed the package format from tlz to txz, but that would be SOHO specific. This could be made into a runtime parameter, if it isn't one already.
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: uelsk8s on February 06, 2011, 11:20:56 am
I found an error in sbbuilder
as it is now it creates this
Code: [Select]
cd usr/info
if [ -f dir ]; then
    rm dir
fi
if [ -f dir.gz ]; then
    rm dir.gz
fi
for i in *.info.gz
        do
        install-info  dir
    done
in any pkg that has a usr/info dir

the install-info line is missing $i
not a big deal the info just doesnt get installed, I doubt anyone reads info anyways :)
the bad thing this does though is it changes to the usr/info dir and does not allow any symlinks added to the pkg to be created

this section should generate
Code: [Select]
CWD=$(pwd)
cd usr/info
if [ -f dir ]; then
    rm dir
fi
if [ -f dir.gz ]; then
    rm dir.gz
fi
for i in *.info.gz
        do
        install-info $i dir
done
cd $CWD

I have updated sbbuilder in the VL7.0 ISO's and will get with rbistolfi to get the changes added
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: Joe1962 on February 07, 2011, 03:09:49 am
I have seen this and my solution was to comment the lot out... ;D  I am not sure it is even necessary, since when installing packages with info files (after removing those lines, of course) I see a line in the console saying something to the effect that the info files are being regenerated (or similar). That is with installpkg and upgradepkg, I cannot say if this works with gslapt/slapt-get too.

BTW, I have made quite a few changes to the sbbuilder templates, so if anyone wants to try them out, I will attach them here. They go in /usr/share/sbb/template. I compressed them all, since they are very small, but I have not modified all of them.

One thing I added was handling for cases when the source tarball or the top-level directory within it have non-standard names, different from what the package name should be.

I also modified requiredbuilder to fix the dependency mess created by deps like:
Code: [Select]
pixman >= 0.19.2-i586-1vl60when it should only be:
Code: [Select]
pixman >= 0.19.2This way, deps are met as long as you have the same or higher version number installed, regardless of whether it is a VL or Slackware package.
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: budulay on February 22, 2011, 09:33:52 pm
As it is right now, to add another file to /usr/doc, I have to write it in between the standard ones, making it hard to see:

"cp -a ANNOUNCE* <<file I want to add>> AUTHORS* BUGS* COPYING* ChangeLog* CHANGELOG* Credits* DEVELOPERS \
LICENSE* MANIFEST PROVENANCE README* TODO* Exception_to_GPL_for_Qt.txt \
$PKG/usr/doc/$NAME-$VERSION"

So to improve readability I put the names of all files I want to add into a variable, and then simply plug the variable name in the above list:

#Add all the files you want copied to docs, space separated
ADDITIONAL_DOCS=""
........
## Add common docs files:
cp -a \
$ADDITIONAL_DOCS \
ANNOUNCE* \
AUTHORS* \......

@Joe1962:
I love that tarname and dirname idea. This is way better than just changing the line manualy:
tar xvf $CWD/FileZilla_$VERSION"_src".tar.* || exit 1
which is what I was doing so far...


On a side note, I can see configure.cmake and configure.python files in the templates directory. Does that mean that sbbuilder can create SlackBuilds that configure packages with those? If so, how can I do that?
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: uelsk8s on February 22, 2011, 10:01:13 pm
sbbuilder --type=python --link=

       --type=string
               Type of SlackBuild.  There are 5 types of SlackBuilds :

               normal . Default configure; make and make install flags.

               perl . Slackbuild for perl packages.

               python . Slackbuild for python packages.

               noarch . noarch package? This is for you.

               cmake . Use cmake instead GNU autoconf.


sbbuilder --manual
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: budulay on February 22, 2011, 11:03:46 pm
Thanks Uel, I didn't know sbbuilder had a manual, just the help page(sbbuilder --help).
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: budulay on February 23, 2011, 07:57:15 pm
@Joe1962
While using your templates, I noticed that although you changed NAME to PKGNAME and VERSION to PKGVERSION(by the way, what was the reason for that?), when "sbbuilder --link" is used, the parsed LINK variable still contains NAME and VERSION, which pretty much breaks automatic source downloading:
Code: [Select]
LINK=${LINK:-"http://sourceforge.net/projects/alleg/files/$NAME/$VERSION/$NAME-$VERSION.tar.gz/download"}  #Enter URL for package here!
I looked through the templates, intending to fix this, but I can't find where the --link input is being parsed.
Title: Re: sbbuilder development. (was: Packaging mini-tool for 6.0 (and maybe 5.9))
Post by: Joe1962 on February 26, 2011, 10:25:55 am
Yes, sorry, since I never used direct source downloading, I didn't fix that part. I will look into it.