VectorLinux
September 14, 2014, 07:43:03 pm *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Visit our home page for VL info. To search the old message board go to http://vectorlinux.com/forum1. The first VL forum is temporarily offline until we can find a host for it. Thanks for your patience.
 
Now powered by KnowledgeDex.
   Home   Help Search Login Register  
Please support VectorLinux!
Pages: [1]
  Print  
Author Topic: Help me with scripting this...  (Read 6045 times)
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« on: November 04, 2007, 01:39:58 pm »

Hey!

I'm planning on really working on the multimedia disc this next week. I'm going to have some days off. I've got all my scripting stuff done but I want to tie it all together. Here's the problem: I need to be able to (re)compile all of my programs with one script. But first, I need to know the order that they should be compiled. That's where I need help. Let's say that I've already compiled every program individually and have the slack-required file there. What I think needs to happen is this:

1. Search for all the slack-required files (They will be named like the name of the program i.e. ardour-slack-required)

2. Go through the dependencies of those files one by one and check and see if I have a $NAME.SlackBuild available. If there is one, then put that the name of the dependency above the name of the program who's slack-required file we are searching.

3. Output it all to a file

Basically I want to be able to run this script and it would search through all the slack-required files, ordering the programs that I have by dependency and outputs it to a file.

Here's an example:
Say I have just sox and libsndfile and libmad and libA in my compiling folder. The script would read the sox slack-required and see that libsndfile and libmad are both in there. So, it would put them before sox. The resulting "order" file would look like this:

Quote
libsndfile
libmad
sox

Now say that it reads the libmad slack-required file and finds that it depends on libA. The script searches my compileing folder and finds that the libA.SlackBuild is there. It would then put libA before libmad:

Quote
libsndfile
libA
libmad
sox

Does this make sense? Could I get some help with the technical details of writing this script? Is there a better way of resolving all these dependencies and compiling them all? Thanks in advance!
Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #1 on: December 23, 2007, 08:19:34 pm »

1. Search for all the slack-required files (They will be named like the name of the program i.e. ardour-slack-required)

Presume that all slack-required files are somewhere under the current directory, or take a directory on the cli?

2. Go through the dependencies of those files one by one and check and see if I have a $NAME.SlackBuild available. If there is one, then put that the name of the dependency above the name of the program who's slack-required file we are searching.

Where would I search for the SlackBuild files? Is case sensitivity important? (I've seen .slackbuild also.)

What happens if I don't find a SlackBuild?


Does this make sense? Could I get some help with the technical details of writing this script? Is there a better way of resolving all these dependencies and compiling them all? Thanks in advance!

My idea would be to have an array of dependency tiers and a tabela associativa (rbistolfi would know Smiley) indicating the tier level for each dependency. Assume that there is only one tier to begin with. Every program added without any dependencies is on tier 0 and recorded as such. Then dependencies come along for programs in tier 0 and are added at tier 1 below. If it so happens that something on tier 0 is required by something, a new tier 0 is formed and the old one gets shifted down with entries in the associative array updated accordingly; if something on tier n has dependencies, they are recorded on tier n+1, etc...

A crappy algorithm indeed, but maybe I'll be able to think of something better when I complete the next semester.

(I assume it wouldn't be necessary to detect cyclic dependencies because your packages aren't RPM after all. Smiley)
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
rbistolfi
Packager
Vectorian
****
Posts: 2282


« Reply #2 on: December 23, 2007, 08:37:32 pm »

My idea would be to have an array of dependency tiers and a tabela associativa (rbistolfi would know Smiley)

http://www.lua.org/portugues.html

Smiley
Logged

"There is a concept which corrupts and upsets all others. I refer not to Evil, whose limited realm is that of ethics; I refer to the infinite."
Jorge Luis Borges, Avatars of the Tortoise.

--
Jumalauta!!
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« Reply #3 on: December 23, 2007, 08:50:00 pm »

Hanumizzle,

Take a look at my vectorcontrib/Dweeberkitty/BonusDisc/. You will see what I'm talking about. I have an organized set of folders with a standard naming scheme across all of them. Also, the slackbuilds are all named "NAME.SlackBuild" there aren't any that are lower case. Here is what folders and files I have:

NAME/
NAME/VERSION
NAME/VERSION/slack-desc
NAME/VERSION/slack-required
NAME/VERSION/NAME-VERSION-ARCH-BUILD.tlz
NAME/VERSION/src/NAME.SlackBuild

Every single package that I have built follows this standard and it is uniform enough that the algorithm used does not have to be written in a broad manner to account for several different scenarios but can be specific for this standard.

If you look in my BonusDisc/Configurations/ORDER file, you'll see why I want to be able to automate this. It is quite daunting to keep track of all dependencies to ensure that a complete rebuild of everything on a clean machine will be successful later down the road.

EDIT: Here's my ORDER file
Code:
libmad
soundtouch
scons
cvs
imaging
dvd-slideshow
fftw
fltk
libgsasl
gst-python
wxGTK
scrollkeeper
libavc1394
libiec61883
liblo
ilmbase
openexr
libsndfile
libsamplerate
twolame
ladspa
VCO-plugins
REV-plugins
MCP-plugins
FIL-plugins
AMB-plugins
caps-plugins
swh-plugins
tap-plugins
jack
dssi
portaudio
jackbeat
bitmeter
libquicktime
mjpegtools
transcode
sox
libgnomecanvas
ffmpeg2theora
raptor
liblrdf
dvgrab
dvdauthor
denemo
lash
gtick
kino
ardour
mandvd
terminatorX
audacity
muse
mixxx
jackEQ
jdelay
jamin
jack-rack
qjackctl
vkeybd
hydrogen
libgig
linuxsampler
liblscp
qsampler
manslide
avidemux
blop
tovid
omins
libpano13
libpano12
glew
libxmi
freeglut
enblend
cinepaint
sweep
mhwaveedit
gc
pigment
python-setuptools
qtractor
lives
fluidsynth
swami
csound
qsynth
TiMidity++
guile
fontforge
autotrace
potrace
t1utils
mftrace
tetex
qdvdauthor
XML-Twig
rosegarden
libatomic_ops
pulseaudio
tapiir
libsigc++
yafray
blender
tse3
ttf2pt1
lilypond
fluidsynth-dssi
lmms
gavl
gmerlin
openmovieeditor
speex
hugin
k9copy
oggconvert

This is what the final product should look like. Only the process of getting to that should be automated. Anyway, take a look in my contrib and see if that helps clear things up. Oh, also take a look at my CompileAll.sh script. That might give you some ideas.
« Last Edit: December 23, 2007, 09:02:35 pm by Dweeberkitty » Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #4 on: December 24, 2007, 10:00:58 pm »

Would it be necessary to detect dependencies of dependencies? If not, then my task is greatly simplified.
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« Reply #5 on: December 24, 2007, 10:45:20 pm »

In a way, yes. The first thing that the script will have to do is search for all the .SlackBuilds that exist. After that is done we will have a list of all packages that then needs to be ordered. Whatever the process that is used to generate the order, at the end of the day, it must be able to work much the same way as Gslapt and make sure that each dependency is met. However, unlike Gslapt where the dependency of the dependency order doesn't matter because it is all recompiled, we are compiling the dependencies which in turn possibly require other programs to be present which should have been compiled before them. So to answer your question, yes, it does need to detect dependencies of dependencies. Sorry!  Undecided
Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #6 on: December 24, 2007, 11:02:42 pm »

There's a topological sort module for Python, that might help me. (Yikes!)
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #7 on: December 25, 2007, 07:38:55 am »

Merry Christmas, Dweeberkitty

http://www.sendspace.com/file/nzu496

You unpack this guy somewhere, cd gen-order, then run python gen-order.py with the root sources directory as the only argument. The default source directory is the current directory, but that would make no sense for this distribution. At the moment, I don't have it looking for SlackBuild files, only printing out the order. Should I raise an exception or something if none is found?? Not sure.
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« Reply #8 on: December 28, 2007, 11:27:11 am »

Thanks hanumizzle! I'll give this a try real soon. Merry Christmas to you too!
Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #9 on: December 28, 2007, 12:05:28 pm »

Thanks hanumizzle! I'll give this a try real soon. Merry Christmas to you too!

Err, it might be in reverse order. In which case, pipe the output through perl -we 'print sort { $b cmp $a } <>'
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« Reply #10 on: January 06, 2008, 01:52:08 pm »

Hanumizzle,

I was just going to use your script but I get
Quote
Sorry, the file you requested is not available.
from the link. Could you either reupload it or send it to the email address in my profile. Thanks!
Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Witek Mozga
Vectorite
***
Posts: 113



WWW
« Reply #11 on: January 07, 2008, 01:56:49 am »

Hanumizzle,

I was just going to use your script but I get
Quote
Sorry, the file you requested is not available.
from the link. Could you either reupload it or send it to the email address in my profile. Thanks!

If hanumizzle made a script for you already that`s OK. But if not then I could try. I don`t promise it will be fast as I constantly lack time but I`ll try to do my best. I don`t know where to look for vectorcontrib/Dweeberkitty/BonusDisc/ so if you could specify where to exactly find those example files, that would be nice. Technical note: I will use Tcl (interpreter for this language is present in Vector, it`s tclsh)
Logged

Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #12 on: January 07, 2008, 08:08:27 am »

Hanumizzle,

I was just going to use your script but I get
Quote
Sorry, the file you requested is not available.
from the link. Could you either reupload it or send it to the email address in my profile. Thanks!

Second attempt:

http://www.sendspace.com/file/gdnpeq
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Dweeberkitty
Packager
Vectorian
****
Posts: 836



WWW
« Reply #13 on: January 07, 2008, 02:37:34 pm »

Cool. It works! However, I find that it produces dependencies that the packager doesn't care about because they are not compiling it. In other words, the script needs to search to see if the .SlackBuild for the dep exists before adding it to the list. Could you do this? Thanks!

~Dweeberkitty
Logged

Registered Linux User #443399
Desktop: Intel Pentium D 3.33Ghz, 320GB hard drive, 2 gigs DDR2 533mhz RAM, NVIDIA Geforce 7800 GS, X2GEN 22" widescreen monitor;
Laptop: Dell Mini 9, Intel Atom 1.6Ghz, 1GB ram
Multimedia Bonus Disc website: http://www.vectorlinuxsolutions.com/
Triarius Fidelis
Vecteloper
Vectorian
****
Posts: 2399


Domine, exaudi vocem meam


WWW
« Reply #14 on: January 07, 2008, 03:29:01 pm »

http://pastebin.ca/845915

Try replacing gen-order.py with the file above.
Logged

"Leatherface, you BITCH! Ho Chi Minh, hah hah hah!"

Formerly known as "Epic Fail Guy" and "Döden" in recent months
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!