VectorLinux

Please login or register.

Login with username, password and session length
Advanced search  

News:

Visit our home page for VL info. For support and documentation, visit the Vector Linux Knowledge Center or search the Knowledge Center and this Forum using the search box above.

Author Topic: [UPDATE: v0.4 !] Introducing 'lsneeds' - a package dependency tool.  (Read 4940 times)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250

Ever wondered what existing package on your system, is wanting to pull in a bunch of new dependencies when you upgrade?

Or found a disk-hog package, and want to make sure nothing needs it?

Hopefully this little shell script that I wrote up can help with those situations. Just bunzip the attachment and drop it into /usr/local/bin. (Now also avail at pastebin.)

CHANGELOG from 0.3 to 0.4:
  • --package_data-dir and --library option added to allow greater flexibility.
  • Separated -h and --help. -h illustrates options in standard POSIX help format
    and --help includes usage examples.
  • Speed of --not increased by several tweaks. Including reduction of reads of /var/log/packages. Time on my personal system has dropped from 21s to 14s, and is even faster if using ash instead of bash.
  • --category option removed.

Code: [Select]
vluser:$ lsneeds -h
lsneeds version 0.4

Usage: lsneeds [-q] [-d <path>] <package-name>
Usage: lsneeds [-n] [-l] [-d <path>]
Usage: lsneeds [-h] [--help]

Provides a list of items that have a dependency
on <package-name>. a.k.a. "what needs <package-name>".

Options:
-q, --quiet
Don't output messages to stderr.
-d, --package_data-dir <path>
Specify alternative path to package_data file.
-l, --library
Start in shell library mode. Do not run any functions.
Allow functions and variables to be sourced into an
interactive POSIX environment.
-n, --not
Invert the operating mode. Display packages that are
not a dependency of anything else.
-h
Output this help message.
--help
Output this help message, and usage examples.

Note:
Execution speed can be increased, by launching lsneeds in a non-bash
POSIX subshell such as "ash" or "dash". ex.
vluser:$ ash lsneeds --not;
Changelog and source link for prior version archived at http://pastebin.com/0QzjiCBz
« Last Edit: February 23, 2011, 03:37:33 pm by pierce.jason »
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #1 on: January 13, 2011, 11:16:20 pm »

I'm kind of surprised there were not any replies to my original release thread for this tool. Are there not as many power users on Vector as I thought? This seems to be one of the larger capabilities missing from Vector/slapt-get package management.

Comments?
Quote
I personally would sure like to get the --not mode to run faster!
On my 7200rpm seagate PATA (ata100, mode5) drive, with 748 packages... it takes just under 21 seconds.

Suggestions?

Requests?
Quote
I originally implemented this, with hopes of creating a system that could show me dependencies that have been orphaned. Now I need to find a way to "mark" packages that have been explicitly installed (not pulled as deps), to compare to the output of --not.

pierce.jason
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

davidlondonuk

  • Member
  • *
  • Posts: 55
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #2 on: January 14, 2011, 03:07:33 am »

Thanks mate, will give the script a whirl.
Logged

stretchedthin

  • Administrator
  • Vectorian
  • *****
  • Posts: 3780
    • Opensourcebistro.com
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #3 on: January 14, 2011, 07:19:45 am »

I'll give it a try as well.
Logged
Vectorlinux screencasts and  tutorials can be found at....
http://www.opensourcebistro.com/blog1
http://www.youtube.com/user/vid4ken?feature=mhee

nightflier

  • Administrator
  • Vectorian
  • *****
  • Posts: 4086
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #4 on: February 01, 2011, 11:44:31 am »

Using it for VL7 development now. Thanks!
Logged

budulay

  • Packager
  • Vectorian
  • ****
  • Posts: 568
  • NewComer
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #5 on: February 02, 2011, 08:45:23 am »

How could I've missed this? Will give it a shot, thanks ;)
Logged
VL6.0-Light@MSI MS-1652(AMD TurionX2 ZM-84 2300 MHZ, 4 GB RAM, 250 GB HDD, Nvidia GForce 9600M GT, GIGABYTE AirCruiser N300)
Registered Linux user #473602
SlackBuild Repository for VL7.Not much but it's a start. Join in!

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #6 on: February 04, 2011, 12:19:03 am »

Sweet. Glad to see some usage here guys!

Any of you have a request on how I could make it more usefull or easier to work with?

(pssssttt....
keep it on the "hush-hush" but,
I'm working on a modification that should hopefully,
greatly boost the speed for --not,
as well as provide possibility for integration with python scripts!)
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #7 on: February 11, 2011, 02:12:13 am »

Are any of you running hardware with odd or sub-par disk IO?

I've been in contact with a fellow who reports 2-3sec per package in --not mode, and similar results for revdep'n a single package. Seems it is because his hard drive controller chipset is integrated with CPU.

If anyone else experiences delays such as this, I'd love to hear.

For lsneeds python, I get about 0.047sec, with 19 packages depending on python.

I personally get about 21sec for a --not operation, with about 800 packages installed. That averages out to 0.03sec per package if my math doesn't fail me.

I'm considering implementing a pre-loader option for --not, that copies the ~3mb package_data file to a ramdisk. Seems vector implements ramdisks in 15mb chunks, but I don't know if this is "growing" or static size. This would of course require that lsneeds be ran as root, or that a sudo command be setup for ramdisk instantiation.
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

roarde

  • Vectorian
  • ****
  • Posts: 590
  • move the needle
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #8 on: February 12, 2011, 04:02:08 am »

It's not disk IO. I set up lsneeds to use /dev/shm/tmp and the problem persists. It is using /dev/shm/tmp, and not using a physical drive for the fifo.

Also, since a temporary file is used, you might want to trap cleanup, where cleanup () at least removes the tempfiles.
Logged
Robert
VL Light RC 2

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #9 on: February 12, 2011, 04:28:41 pm »

Thanks for the tips there roarde!

I wasn't familiar with sh traps, but this should make emergency-bailouts a bit nicer.

The majority of IO won't be related to /tmp or fifo, but rather read access to /var/log/packages and the package_data under /home/ftp/.

The ~2MB package_data is what I would load to a ramdisk if I implemented something like a --io switch. package_data is accessed once for every package that is installed, when running --not, or once when revdep'n a single package.

More thanks to roarde, I just spotted where I can cut down read access to the /var/log/packages directory for --not! Looks like I can easily reorganize some data access and reduce reads of /var/log/packages by about 800% (ie. reduce to one read, instead of one per package).
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)

roarde

  • Vectorian
  • ****
  • Posts: 590
  • move the needle
Re: [UPDATE: v0.3 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #10 on: February 13, 2011, 09:13:29 am »

Mentioned this to you elsewhere pierce.jason, but to say it for everyone else, my speed troubles are local. Has to do with the disconnect between VIA and open source; mainly related to processor (non-)usage with IO a distant second.

At least chrome 9 (video) info has finally been released to a public location, at Xorg.
Logged
Robert
VL Light RC 2

pierce.jason

  • Packager
  • Vectorite
  • ****
  • Posts: 250
Re: [UPDATE: v0.4 !] Introducing 'lsneeds' - a package dependency tool.
« Reply #11 on: February 23, 2011, 03:40:32 pm »

Just uploaded new version. 0.4

Pretty fair speedup. Few new options. Deleted some stuff that didn't seem useful.
Logged
pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)