24
Sep

Linux Motherboard upgrade

I’ve always been pretty amazed at how easy it is to change hardware under a Linux system and for it mostly to just work – especially with most modern distros. Unlike those that are stuck with Windows systems, it is possible to do a motherboard upgrade on a linux system with the minimum of fuss, and without doing a system install!

Indeed, mostly it’s just a matter of remembering to make the system forget about the MAC address of the on-board ethernet. Otherwise after the system is booted, the system thinks that the eth0 is missing and instead configures the hardware as eth1.

Maybe networkmanager, or whatever, copes, but I like to have command-line control over my network setup, and on my Debian based systems it’s eth0 that is configured properly in /etc/network/interfaces.

So before I shut down my old motherboard system for the last time, I delete the eth0 line in the file /etc/udev/rules.d/70-persistent-net.rules so that when the system reboots with new hardware, it allocates eth0 to the new onboard ethernet interface.

However, I recently recycled an Intel D945GSEJT motherboard as a replacement for a VIA EPIA MoBo in my home server. The Intel board had been my guts of my desktop before I upgraded to a dual core ATOM based board – I like my hardware to use less Watts. My server runs Ubuntu server 8.04 LTS, and, despite the forced amnesia of the VIA’s ethernet MAC address, I still didn’t have an ethernet connection after the Motherboard transplant. A quick google discovered a load of issues with Realtek r8169 driver in older kernels – and Ubuntu Server 8.04 LTS is hardly state of the art.

Not having time to do a real fix, I reached for an old PCI Ethernet card I sometimes used when I needed a machine with 2 ethernet ports. Remembering to hack /etc/udev/rules.d/70-persistent-net.rules, disabling the onboard ethernet device in the BIOS and rebooting immediately got me a working system.

I’ll leave it like it is until after I get up the courage to do the upgrade to Ubuntu server 10.04LTS. Then I’ll rescue the PCI ethernet card and add it to my box of useful bits and pieces that might just get me out of a fix. That box has all sorts of cables and adapters, as well as an old PCI graphics card, a PCI SATA adapter, a PCI IDE adapter and the PCI Ethernet card. Anyone got recommendations for other useful bits and pieces to add to the box?

cheers
Jim Jackson

18
Sep

Self Documenting Scripts

As a “good” programmer I like to put comments at the top of my scripts to say what the script does, and how it is used. I also like the script to output a useful help message when a user gets options/arguments wrong, or when they use the option ‘-h’. I found it was a pain keeping the 2 in step, and developed this simple scheme to only have one place for this info.

I’ll use bash here but I’m sure people can adapt to other scripting languages.

#!/bin/sh
#
## 
## Usage: helpdemo 
##
## This demos a self documenting scheme for scripts.
##

prog="$0"
me=`basename "$prog"`

dohelp () {
  grep '^##' "$prog" | sed -e 's/^##//' -e "s/_PROG_/$me/" 1>&2
}

echo "Program name is: $me"
echo "Program file is: $prog"
echo

dohelp
exit

Prefix any lines you want to be output as “help” by ‘##’ at the beginning of the line. All such lines are printed out to stderr by the dohelp function. ‘sed’ in this function also strips off the leading ‘##’ from the lines and substitutes the filename of the invoked script for ‘_PROG_’, so that if you change the name of the script, it still magically refers to the new name.

It’s a simple scheme, and can obviously be extended, e.g. one could change the dohelp function thus…

dohelp () {
  pfx="$1"
  if [ "$pfx" = "" ]; then pfx='##' ; fi
  grep "^$pfx" "$prog" | sed -e "s/^$pfx//" -e "s/_PROG_/$me/" 1>&2
}

dohelp can now be called to select out lines with a different prefix, but its default behaviour when given no prefix, is as before.

This has formed part of my standard shell script template for many years. Hope others find it useful.

Jim Jackson

10
Sep

Roll your own ‘lsof’ – sort of

Ever read the ‘lsof’ utility man page? It’s a shame, because the idea of the utility is wonderful – list open files and who is using them.

I had need to see what files were in use by imapd on my server. “lsof | grep imapd” gave me far more than I needed, along with long lines of info I didn’t really need all of, and all the libraries in use, etc, etc. That’s when I thought, RTFM[1] … so I did, and eventually gave up. Maybe I was having a bad day, but the manual seemed impenetrable, so I turned to
rolling my own.

I’d explored around the /proc file system a bit before, and was sure everything I need was there, and it turned out a lot easier than I’d thought. So here goes…

Under /proc, every process has a directory, with an entry called “cmdline” which gives access to that process’s command line arguments. There is a sub-directory called “fd” that contains links to each file the process has open. With this info I knocked together this little script, I called it ‘pof’:

#!/bin/sh
#
cd /proc
while [ "$1" != "" ]; do
  for n in [1-9]* ; do
    if [ "$n" != "self" -a -f $n/cmdline ] then
      if grep "$1" $n/cmdline > /dev/null ; then
        cmd=`tr '\0' ' ' < $n/cmdline`
        printf "%-8d %s\n" $n "$cmd"
        for m in $n/fd/* ; do
          printf " %-14s %s %s\n" `stat --printf="%N" $m`
        done
      fi
    fi
  done
  shift
done

It is called with an string or regular expression that you want to search each command line for. If it is found, the open file links are displayed for that process. So now ‘pof imapd’ gives me what I want.

The ‘cmdline’ file contains each argument of the command line as a null (‘\0’) terminated string, so I need to use ‘tr’ to convert the nulls to spaces before printing out.

On my system the imapd daemon is launched from inetd, so it’s first 3 files, stdin/stdout/stderr, are network sockets – so I need to dig a bit deeper and find out how to get the remote IP address of the socket.

Anyway I’m sure someone will tell me how easy ‘lsof’ is to use to get any info you want, but delving into /proc is fun!

Jim Jackson

[1] //en.wikipedia.org/wiki/RTFM

7
Sep

Script to Watch for Changes to Disk and Run Script Immediately

By utilizing inotifywait from the inotify-tools package, we can monitor changes to disk and immediately run a script or command on that file. No need for a cron job here…

To install inotify-tools, we simply use apt-get (or yum, depending on your distro):

sudo apt-get install inotify-tools

Here is one example script which utilizes inotifywait:

#!/bin/sh

watch_dir="/media/md0/"
inotifywait -mr -e create -e delete -e modify --timefmt '%Y-%m-%dT%H%M%S' --format '%w;%f;%e;%T' "$watch_dir" | while IFS=';' read -r DIR FILE EVENT TIME; do

  echo "${EVENT} ${TIME}: ${DIR}${FILE} >> /home/user/watched

  case "$EVENT" in
    "CREATE"|"MODIFY")
      echo "Run create/modify script on file: ${DIR}${FILE}" >> /home/user/watched
      ;;
    "DELETE")
      echo "Run delete script on file: ${DIR}${FILE}" >> /home/user/watched
      ;;
  esac
done

Give your script a name, executable rights, and run it in the background. For example:
chmod +x myscript.sh && ./myscript.sh &

Obviously, you’ll need to modify the script to fit your needs, but this should give you a jump start on your project.

Note: There’s an infinite loop bug in the script above. Remove the ‘modify’ options and this will clear things up. However, if you do that, then you’re really only looking for file creation or deletion. You could add more conditionals to the script above to prevent the loop. If anybody has a suggestion, I’d be happy to update the script.

Take this a step further by following my guide on Automatically Start a Script at Linux Bootup

5
Sep

The watch Command

The watch command allows you to periodically run a command (default, every 2 seconds but can be specified using -n or --interval option). It’s like an alias for a while 1; sleep loop and is really simple to use.

watch [options] command command_options

When I had used it, I was wanting to see the progress of the ‘dd’ command. Follow this link if you’re interested as well: http://www.commandlinefu.com/commands/view/1868/watch-the-progress-of-dd
http://linuxcommando.blogspot.com/2008/06/show-progress-during-dd-copy.html

4
Sep

Utility to Clean C Like Language Syntax

Check out uncrustify next time you need to stylize your c like application. Their website gives a short, concise summary of what it does. Check it out here: uncrustify on sourceforge.

It’s also highly configurable via the uncrustify.cfg config file.

Example usage:

uncrustify -c uncrustify.cfg -f source_file.c

You should be able to install it using apt-get or yum, depending on your Linux distribution.

3
Sep

What Connections Have I Got

It is easy to use the utility netstat to list the active connections you have to your machine

netstat -t

provides the information. But often it is good to know which process/program the connection is to. Again netstat obliges

netstat -tp

However, if like me you use 80 column xterms (or even one of the linux VTs in 80 column mode), then the long lines make the output less than easy reading. So using the “cut” utility we can cut down the verbiage to what is important

netstat -tp | cut -c21-63,80-

I often like to use the “--numeric-hosts” option to netstat too.

Another use of netstat is to show those programs that are listening for connections. This can help check if services have crashed, or whether you have unwelcome “services” running on your machine.

netstat -tpl

Again “cut” can be used to trim the fat and keep the output manageable

netstat -tpl | cut -c21-36,56-63,80-

It is worth giving the man page a netstat a check, it is a very useful utility.

28
Aug

Creating Busybox Links

This tip came in from a subscriber/regular contributor. It’s a script for quickly creating busybox links based on the commands included in your build. I know it will be helpful to somebody out there.

Hi Derek,

Dunno if you are still doing your “DailyLinux” thing, but here’s
a bit of script I knocked together today to create busybox links.
Yes I know there is an “–install” option on a lot of busybox builds,
but it doesn’t seem to play well with installing in a chroot area.

So copy the busybox executable into the “bin” directory…

cp -p /bin/busybox /data/chroot/bin

and cd into that directory

cd /data/chroot/bin

Then run this shell snippet…

  ./busybox  --help | \
  sed -e '1,/^Currently defined functions:/d' \
      -e 's/[ \t]//g' -e 's/,$//' -e 's/,/\n/g' | \
  while read app ; do
    if [ "$app" != "" ]; then
      printf "linking %-12s ...\n" "$app"
      ln -sf "./busybox" "$app"
      ls -ld "$app"
    fi
  done

and all the links for the apps built-in to the busybox version you have
get built for you.

It takes advantage of the fact that “busybox –help” gives a list of the
built apps at the end of its output, after the line “Currently defined
functions:”. The sed script, removes all line upto and including this line,
and then removes spaces, tabs and trailing commas, then it converts all
commas to newlines. This gives a list of apps, one app to a line. The while
loop reads each line and does the linking.

Dunno if this would interest anyone else.

all the best
Jim

14
Aug

Python Script to Recursively Search For and Generate Thumbnails for Video Files

This script will recursively search for files based on listed extension names and then use the ‘ffmpegthumbnailer’ program to generate a thumbnail for each file it finds.

If you’re on a Ubuntu system, you should be able to install ffmpegthumbnailer using apt-get install ffmpegthumbnailer.

#!/usr/bin/python

import os

for root, dirnames, filenames in os.walk('/var/www/media/'):
  for filename in filenames:
    if filename.lower().endswith(('.m4v', '.mov', '.mpeg', 'mp4')): 
      ifile = os.path.join(root, filename)
      ofile = os.path.splitext(ifile)[0] + ".jpg"
      try:
        with open(ofile) as f: pass
      except IOError as e:
        print "Generating thumbnail for: " + ifile

        fftoptions = "-s0 -f"
        command = "ffmpegthumbnailer -i %s -o %s %s" % (ifile, ofile, fftoptions)

        p = os.popen(command,"r")
        while 1:
          line = p.readline()
          if not line: break
          print line

I had a specific purpose for this script, but you could easily modify it to fit your needs. When you save the file, don’t forget to mark it as executable with chmod +x scriptname.py.

21
Mar

Convert PDF to SWF with pdf2swf

I stumbled upon pdf2swf when looking for the best way to embed a PDF into a webpage. It works really well and was easy to compile on my shared hosting account (bluehost.com). I installed using the following steps:

Download – Download version 0.9.1 (visit http://www.swftools.org/download.html for latest)

wget http://www.swftools.org/swftools-0.9.1.tar.gz

Extract – Extract the newly downloaded files

tar xzvf swftools-0.9.1.tar.gz

Configure – Make sure the environment is setup

cd swftools-0.9.1; ./configure

Compile – Getting source to binaries

make config; make

Test Run – You should get a help menu

./src/pdf2swf

Install – Copy the binaries to proper locations on the system
(You won’t be able to run this step if you’re installing on a shared host. If that’s the case, copy the binaries of interest to ~/bin and add ~/bin to PATH variable)

make install

Usage – Run the program (Run man pdf2swf for more info)

pdf2swf myfile.pdf myfile.swf

That’s it! Open the .swf using your browser to see pdf2swf’s handywork.

As a side note, I ended up ditching the .swf method for embedding my document on my website and going with Google’s PDF viewer which is done with the following line of HTML:

<iframe style="width: 700px; height: 900px; border: 0px none;" src="http://docs.google.com/gview?url=http://mydomain.com/path/to/myfile.pdf&amp;embedded=true" width="320" height="240"></iframe>
« Previous PageNext Page »