Please support VectorLinux!

Author Topic: [UPDATE: v0.4 !] Introducing 'lsneeds' - a package dependency tool.  (Read 4897 times)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Ever wondered what existing package on your system, is wanting to pull in a bunch of new dependencies when you upgrade?

Or found a disk-hog package, and want to make sure nothing needs it?

Hopefully this little shell script that I wrote up can help with those situations. Just bunzip the attachment and drop it into /usr/local/bin. (Now also avail at pastebin.)

CHANGELOG from 0.3 to 0.4:
  • --package_data-dir and --library option added to allow greater flexibility.
  • Separated -h and --help. -h illustrates options in standard POSIX help format
    and --help includes usage examples.
  • Speed of --not increased by several tweaks. Including reduction of reads of /var/log/packages. Time on my personal system has dropped from 21s to 14s, and is even faster if using ash instead of bash.
  • --category option removed.

Code: [Select]
vluser:$ lsneeds -h
lsneeds version 0.4

Usage: lsneeds [-q] [-d <path>] <package-name>
Usage: lsneeds [-n] [-l] [-d <path>]
Usage: lsneeds [-h] [--help]

Provides a list of items that have a dependency
on <package-name>. a.k.a. "what needs <package-name>".

Options:
-q, --quiet
Don't output messages to stderr.
-d, --package_data-dir <path>
Specify alternative path to package_data file.
-l, --library
Start in shell library mode. Do not run any functions.
Allow functions and variables to be sourced into an
interactive POSIX environment.
-n, --not
Invert the operating mode. Display packages that are
not a dependency of anything else.
-h
Output this help message.
--help
Output this help message, and usage examples.

Note:
Execution speed can be increased, by launching lsneeds in a non-bash
POSIX subshell such as "ash" or "dash". ex.
vluser:$ ash lsneeds --not;
Changelog and source link for prior version archived at http://pastebin.com/0QzjiCBz
« Last Edit: February 23, 2011, 03:37:33 pm by pierce.jason »
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #1 on: January 13, 2011, 11:16:20 pm »
I'm kind of surprised there were not any replies to my original release thread for this tool. Are there not as many power users on Vector as I thought? This seems to be one of the larger capabilities missing from Vector/slapt-get package management.

Comments?
Quote
I personally would sure like to get the --not mode to run faster!
On my 7200rpm seagate PATA (ata100, mode5) drive, with 748 packages... it takes just under 21 seconds.

Suggestions?

Requests?
Quote
I originally implemented this, with hopes of creating a system that could show me dependencies that have been orphaned. Now I need to find a way to "mark" packages that have been explicitly installed (not pulled as deps), to compare to the output of --not.

pierce.jason
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

davidlondonuk

  • Member
  • *
  • Posts: 55
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #2 on: January 14, 2011, 03:07:33 am »
Thanks mate, will give the script a whirl.

stretchedthin

  • Administrator
  • Vectorian
  • *****
  • Posts: 3780
    • Opensourcebistro.com
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #3 on: January 14, 2011, 07:19:45 am »
I'll give it a try as well.
Vectorlinux screencasts and  tutorials can be found at....
http://www.opensourcebistro.com/blog1
http://www.youtube.com/user/vid4ken?feature=mhee

nightflier

  • Administrator
  • Vectorian
  • *****
  • Posts: 4074
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #4 on: February 01, 2011, 11:44:31 am »
Using it for VL7 development now. Thanks!

budulay

  • Packager
  • Vectorian
  • ****
  • Posts: 568
  • NewComer
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #5 on: February 02, 2011, 08:45:23 am »
How could I've missed this? Will give it a shot, thanks ;)
VL6.0-Light@MSI MS-1652(AMD TurionX2 ZM-84 2300 MHZ, 4 GB RAM, 250 GB HDD, Nvidia GForce 9600M GT, GIGABYTE AirCruiser N300)
Registered Linux user #473602
SlackBuild Repository for VL7.Not much but it's a start. Join in!

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #6 on: February 04, 2011, 12:19:03 am »
Sweet. Glad to see some usage here guys!

Any of you have a request on how I could make it more usefull or easier to work with?

(pssssttt....
keep it on the "hush-hush" but,
I'm working on a modification that should hopefully,
greatly boost the speed for --not,
as well as provide possibility for integration with python scripts!)
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #7 on: February 11, 2011, 02:12:13 am »
Are any of you running hardware with odd or sub-par disk IO?

I've been in contact with a fellow who reports 2-3sec per package in --not mode, and similar results for revdep'n a single package. Seems it is because his hard drive controller chipset is integrated with CPU.

If anyone else experiences delays such as this, I'd love to hear.

For lsneeds python, I get about 0.047sec, with 19 packages depending on python.

I personally get about 21sec for a --not operation, with about 800 packages installed. That averages out to 0.03sec per package if my math doesn't fail me.

I'm considering implementing a pre-loader option for --not, that copies the ~3mb package_data file to a ramdisk. Seems vector implements ramdisks in 15mb chunks, but I don't know if this is "growing" or static size. This would of course require that lsneeds be ran as root, or that a sudo command be setup for ramdisk instantiation.
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

roarde

  • Vectorian
  • ****
  • Posts: 584
  • move the needle
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #8 on: February 12, 2011, 04:02:08 am »
It's not disk IO. I set up lsneeds to use /dev/shm/tmp and the problem persists. It is using /dev/shm/tmp, and not using a physical drive for the fifo.

Also, since a temporary file is used, you might want to trap cleanup, where cleanup () at least removes the tempfiles.
Robert
VL Light RC 1.2

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #9 on: February 12, 2011, 04:28:41 pm »
Thanks for the tips there roarde!

I wasn't familiar with sh traps, but this should make emergency-bailouts a bit nicer.

The majority of IO won't be related to /tmp or fifo, but rather read access to /var/log/packages and the package_data under /home/ftp/.

The ~2MB package_data is what I would load to a ramdisk if I implemented something like a --io switch. package_data is accessed once for every package that is installed, when running --not, or once when revdep'n a single package.

More thanks to roarde, I just spotted where I can cut down read access to the /var/log/packages directory for --not! Looks like I can easily reorganize some data access and reduce reads of /var/log/packages by about 800% (ie. reduce to one read, instead of one per package).
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

roarde

  • Vectorian
  • ****
  • Posts: 584
  • move the needle
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #10 on: February 13, 2011, 09:13:29 am »
Mentioned this to you elsewhere pierce.jason, but to say it for everyone else, my speed troubles are local. Has to do with the disconnect between VIA and open source; mainly related to processor (non-)usage with IO a distant second.

At least chrome 9 (video) info has finally been released to a public location, at Xorg.
Robert
VL Light RC 1.2

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.4 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #11 on: February 23, 2011, 03:40:32 pm »
Just uploaded new version. 0.4

Pretty fair speedup. Few new options. Deleted some stuff that didn't seem useful.
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)