VectorLinux
December 21, 2014, 05:02:17 pm *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Visit our home page for VL info. To search the old message board go to http://vectorlinux.com/forum1. The first VL forum is temporarily offline until we can find a host for it. Thanks for your patience.
 
Now powered by KnowledgeDex.
   Home   Help Search Login Register  
Please support VectorLinux!
Pages: [1]
  Print  
Author Topic: [UPDATE: v0.4 !] Introducing 'lsneeds' - a package dependency tool.  (Read 4692 times)
pierce.jason
Packager
Vectorite
****
Posts: 250



« on: December 29, 2010, 07:11:51 pm »

Ever wondered what existing package on your system, is wanting to pull in a bunch of new dependencies when you upgrade?

Or found a disk-hog package, and want to make sure nothing needs it?

Hopefully this little shell script that I wrote up can help with those situations. Just bunzip the attachment and drop it into /usr/local/bin. (Now also avail at pastebin.)

CHANGELOG from 0.3 to 0.4:
  • --package_data-dir and --library option added to allow greater flexibility.
  • Separated -h and --help. -h illustrates options in standard POSIX help format
    and --help includes usage examples.
  • Speed of --not increased by several tweaks. Including reduction of reads of /var/log/packages. Time on my personal system has dropped from 21s to 14s, and is even faster if using ash instead of bash.
  • --category option removed.

Code:
vluser:$ lsneeds -h
lsneeds version 0.4

Usage: lsneeds [-q] [-d <path>] <package-name>
Usage: lsneeds [-n] [-l] [-d <path>]
Usage: lsneeds [-h] [--help]

Provides a list of items that have a dependency
on <package-name>. a.k.a. "what needs <package-name>".

Options:
-q, --quiet
Don't output messages to stderr.
-d, --package_data-dir <path>
Specify alternative path to package_data file.
-l, --library
Start in shell library mode. Do not run any functions.
Allow functions and variables to be sourced into an
interactive POSIX environment.
-n, --not
Invert the operating mode. Display packages that are
not a dependency of anything else.
-h
Output this help message.
--help
Output this help message, and usage examples.

Note:
Execution speed can be increased, by launching lsneeds in a non-bash
POSIX subshell such as "ash" or "dash". ex.
vluser:$ ash lsneeds --not;
Changelog and source link for prior version archived at http://pastebin.com/0QzjiCBz
« Last Edit: February 23, 2011, 03:37:33 pm by pierce.jason » Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
pierce.jason
Packager
Vectorite
****
Posts: 250



« Reply #1 on: January 13, 2011, 11:16:20 pm »

I'm kind of surprised there were not any replies to my original release thread for this tool. Are there not as many power users on Vector as I thought? This seems to be one of the larger capabilities missing from Vector/slapt-get package management.

Comments?
Quote
I personally would sure like to get the --not mode to run faster!
On my 7200rpm seagate PATA (ata100, mode5) drive, with 748 packages... it takes just under 21 seconds.

Suggestions?

Requests?
Quote
I originally implemented this, with hopes of creating a system that could show me dependencies that have been orphaned. Now I need to find a way to "mark" packages that have been explicitly installed (not pulled as deps), to compare to the output of --not.

pierce.jason
Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
davidlondonuk
Member
*
Posts: 55


« Reply #2 on: January 14, 2011, 03:07:33 am »

Thanks mate, will give the script a whirl.
Logged
stretchedthin
Administrator
Vectorian
*****
Posts: 3780


WWW
« Reply #3 on: January 14, 2011, 07:19:45 am »

I'll give it a try as well.
Logged

Vectorlinux screencasts and  tutorials can be found at....
http://www.opensourcebistro.com/blog1
http://www.youtube.com/user/vid4ken?feature=mhee
nightflier
Administrator
Vectorian
*****
Posts: 4039



« Reply #4 on: February 01, 2011, 11:44:31 am »

Using it for VL7 development now. Thanks!
Logged
budulay
Packager
Vectorian
****
Posts: 568


NewComer


« Reply #5 on: February 02, 2011, 08:45:23 am »

How could I've missed this? Will give it a shot, thanks Wink
Logged

VL6.0-Light@MSI MS-1652(AMD TurionX2 ZM-84 2300 MHZ, 4 GB RAM, 250 GB HDD, Nvidia GForce 9600M GT, GIGABYTE AirCruiser N300)
Registered Linux user #473602
SlackBuild Repository for VL7.Not much but it's a start. Join in!
pierce.jason
Packager
Vectorite
****
Posts: 250



« Reply #6 on: February 04, 2011, 12:19:03 am »

Sweet. Glad to see some usage here guys!

Any of you have a request on how I could make it more usefull or easier to work with?

(pssssttt....
keep it on the "hush-hush" but,
I'm working on a modification that should hopefully,
greatly boost the speed for --not,
as well as provide possibility for integration with python scripts!)
Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
pierce.jason
Packager
Vectorite
****
Posts: 250



« Reply #7 on: February 11, 2011, 02:12:13 am »

Are any of you running hardware with odd or sub-par disk IO?

I've been in contact with a fellow who reports 2-3sec per package in --not mode, and similar results for revdep'n a single package. Seems it is because his hard drive controller chipset is integrated with CPU.

If anyone else experiences delays such as this, I'd love to hear.

For lsneeds python, I get about 0.047sec, with 19 packages depending on python.

I personally get about 21sec for a --not operation, with about 800 packages installed. That averages out to 0.03sec per package if my math doesn't fail me.

I'm considering implementing a pre-loader option for --not, that copies the ~3mb package_data file to a ramdisk. Seems vector implements ramdisks in 15mb chunks, but I don't know if this is "growing" or static size. This would of course require that lsneeds be ran as root, or that a sudo command be setup for ramdisk instantiation.
Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
roarde
Vectorian
****
Posts: 555


move the needle


« Reply #8 on: February 12, 2011, 04:02:08 am »

It's not disk IO. I set up lsneeds to use /dev/shm/tmp and the problem persists. It is using /dev/shm/tmp, and not using a physical drive for the fifo.

Also, since a temporary file is used, you might want to trap cleanup, where cleanup () at least removes the tempfiles.
Logged

Robert
VL STD 7.1 RC3.1, icewmvmods
pierce.jason
Packager
Vectorite
****
Posts: 250



« Reply #9 on: February 12, 2011, 04:28:41 pm »

Thanks for the tips there roarde!

I wasn't familiar with sh traps, but this should make emergency-bailouts a bit nicer.

The majority of IO won't be related to /tmp or fifo, but rather read access to /var/log/packages and the package_data under /home/ftp/.

The ~2MB package_data is what I would load to a ramdisk if I implemented something like a --io switch. package_data is accessed once for every package that is installed, when running --not, or once when revdep'n a single package.

More thanks to roarde, I just spotted where I can cut down read access to the /var/log/packages directory for --not! Looks like I can easily reorganize some data access and reduce reads of /var/log/packages by about 800% (ie. reduce to one read, instead of one per package).
Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
roarde
Vectorian
****
Posts: 555


move the needle


« Reply #10 on: February 13, 2011, 09:13:29 am »

Mentioned this to you elsewhere pierce.jason, but to say it for everyone else, my speed troubles are local. Has to do with the disconnect between VIA and open source; mainly related to processor (non-)usage with IO a distant second.

At least chrome 9 (video) info has finally been released to a public location, at Xorg.
Logged

Robert
VL STD 7.1 RC3.1, icewmvmods
pierce.jason
Packager
Vectorite
****
Posts: 250



« Reply #11 on: February 23, 2011, 03:40:32 pm »

Just uploaded new version. 0.4

Pretty fair speedup. Few new options. Deleted some stuff that didn't seem useful.
Logged

pierce.jason
Email: $(echo -e "moc\x2eliamg\x40nosaj.ecreip" | rev)
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!