Let’s say we’ve got a bunch of directories and we’d like to get counts of how many directories are how many levels deep.
Our directory structure:
$ find . -type d . ./test1 ./test1/test1.1 ./test2 ./test2/test2.1 ./test2/test2.1/test2.2 ./test3 ./test3/test3.1 ./test3/test3.1/test3.2 ./test3/test3.1/test3.2/test3.3
There are a number of ways to do this, but I found awk to be most elegant as I wanted to be able to substract an arbitrary number from the result and awk can count and do arithmetic all by itself:
$ find . -type d | while read f; do awk -F'/' '{print NF -1}' | sort | uniq -c | sort -nr; done 3 2 3 1 2 3 1 4
Got thousands of tar archives being transferred from various machines to one repository. Found that some of the archives are bad, either because the network transfer didn’t complete or the archive process was interrupted on the machine that created the archive (due to insufficient disk space, etc). Whatever the cause, needed a method to verify the archive integrity.
Tar provides options for verify the archive integrity by comparing it with the file system. In this case, I needed the ability to verify the archive without access to the original file system.
The BASH one liner below will find all archives below current directory, loop through each one attempting to list its contents, discard the archive content list, capture and log the tar command’s exit status along with archive name and redirect both to a log. Exit status of 0 means the archive is good, 2 means it’s bad (haven’t seen a 1). Redirecting output to /dev/null while capturing exit status turned out trickier than I thought, but this seems to work well:
1 | for f in $(find . -name "*tar.bz2"); do tar tfj $f &> /dev/null; err="$?"; echo $err $f >> tar-check.list; done |
The split command is one gem of a Giant Dork tool, but it expects to be given the number of lines to split a file along. It’s easy enough to fire up a calculator to do this, but sometimes a programmatic method is desirable. Here’s a bashism that’ll do it:
1 | split -l $(echo $(( $(cat sourcefile | wc -l) / 3))) sourcefile sourcefile. |
I liked Jason’s solution of passing a file list to du via xargs to produce results sorted by size in human readable format, but wanted ability to limit file list by name and age. Combining the find command with du, worked out nicely:
find . -name "*tar.bz2" -mmin -120 -ls | sort -k7rn | awk '{print$NF}' | xargs du -sh
122M ./sls-monitor/830144.tar.bz2 98M ./sls-monitor/830156.tar.bz2 67M ./sls-off1/905895.tar.bz2 50M ./sls-off1/893748.tar.bz2 16M ./sls-off1/893759.tar.bz2 7.3M ./sls-off1/905897.tar.bz2 5.1M ./sls-off1/854850.tar.bz2 4.4M ./sls-monitor/804331.tar.bz2 3.8M ./sls-monitor/804333.tar.bz2 612K ./sls-off1/893755.tar.bz2 512K ./sls-off1/905898.tar.bz2
A project I’m working required a comma delimited file containing date ranges, in one week increments, from July 1987 to the present. That’s 1166 week ranges. What sounds like a potentially time consuming ordeal, takes about 10 sec to produce with the following shell one liner:
for n in $(seq 0 1166); \ do echo -e \ $(date +%m/%d/%Y --date="1987-07-01 +$n week")","\ $(date +%m/%d/%Y --date="1987-07-01 +$n week +6 day") \ >> date.ranges; done
Most of the stuff I do is via the shell, but once in a while need to attach to an existing X session on my work desktop. My requirements are that the method:
- Uses SSH for authentication
- Tunnels VNC traffic over SSH for security and so no additional ports need to be opened
x11vnc is what I’ve been using. Here’s a little shell script I wrote that will take user@hostname as a variable, enumerate an existing GDM session on the remote computer, create an SSH tunnel and attach to it via VNC. It uses aggressive compression, which looks a bit ugly but still works over a low bandwidth connection. The auth file enumeration bit works with Gnome, but could probably be adapted for use with other window managers.
1 2 3 4 5 | sudo -s if [ -f /tmp/nmap.* ]; then rm /tmp/nmap.*; fi nmap -sP -PA 192.168.0.0/24 >> /tmp/nmap.sweep for h in $(grep Host /tmp/nmap.sweep | awk '{print$2}'); do $c -PN -A $h >> /tmp/nmap.scan; done less /tmp/nmap.scan |
With WordPress MU, all blogs share the same database. A number is added to standard WordPress table names for each blog. Main blog is usually wp_1_xxx, second blog that was created is wp_2_xxx, etc.
To modify a WPMU blog directly in the DB, one would usually want to identify which tables are used for which blog, to ensure the correct blog is being modified. Here’s a little shell one liner to do just that:
1 2 3 4 | $ CMD="mysql giantdorks --skip-column-names -Be"; TBS=$($CMD "show tables like 'wp%options'"); for TB in $TBS; do URL=$($CMD "select option_value from $TB where option_name='siteurl'") && echo $TB is for $URL; done wp_1_options is for http://giantdorks.org/ wp_2_options is for http://giantdorks.org/alain/ wp_3_options is for http://giantdorks.org/jason/ |
Pad filenames starting with a single digit year (e.g. 1, 2, etc) with a zero while leaving filesnames starting with two digit years (e.g. 11, 12) untouched.
To trim a video clip, mencoder and ffmpeg expect a start time in hh:mm:ss, but instead of end time, want duration:
$ mencoder -ss 00:02:55 -endpos 00:07:15 -oac copy -ovc copy in.avi -o out.avi $ ffmpeg -ss 00:02:55 -t 00:07:15 -vcodec copy -acodec copy -i in.avi out.avi
Video players make it easy to identify start/stop times, but difficult to figure out duration, so I wrote a little shell script that will calculate duration from start/stop times.
Usage example:
$ calc-duration 00:02:55 00:10:10 Start: 00 hrs 02 min 55 sec Stop: 00 hrs 10 min 10 sec +----------------------+ | duration is 00:07:15 | +----------------------+