Mark Sobell Blog

Use := to Assign a Default Value to an Unset Variable

Posted on May 3, 2012 by mark

You can change the value of a null or unset variable to its default in a script using the := modifier:

${name:=default}

When the shell expands the expression ${name:=default}, it sets the value of name to the expanded value of default. If a script contains a line such as the following and LIT is unset or null at the time this line is executed, the shell assigns LIT the value /home/max/literature:

$ ls ${LIT:=/home/max/literature}

Shell scripts frequently start with the : (null) builtin followed on the same line by the := expansion modifier to set any variables that might be null or unset. The : builtin evaluates each token in the remainder of the command line but does not execute any commands.
Use the following syntax to set a default for a null or unset variable in a shell script (a SPACE follows the first colon). Without the leading colon (:), the shell would evaluate and attempt to execute the “command” that results from the evaluation.

: ${name:=default}

When a script needs a directory for temporary files and uses the value of TEMPDIR for the name of this directory, the following line assigns to TEMPDIR the value /tmp if TEMPDIR is null:

: ${TEMPDIR:=/tmp}

Posted in bash, Shell scripts | Comments Off

Performing a Join from the Command Line

Posted on February 23, 2012 by mark

The examples in this section use the following files:

$ cat one
9999 first line file one.
aaaa second line file one.
cccc third line file one.

$ cat two
aaaa FIRST line file two.
bbbb SECOND line file two.
cccc THIRD line file two.

The first example shows the simplest use of join. The files named one and two are joined based, by default, on the first field in each line of both files. Both files are in sorted order based on the join field. The join fields on two pairs of lines match and join displays those lines.

$ join one two
aaaa second line file one. FIRST line file two.
cccc third line file one. THIRD line file two.

You can use the – –check-order option to see if both files are properly sorted. Following, sort with the –r option sorts one in reverse alphabetical order and sends the output through a pipe to join. The shell replaces the – argument to join with the standard input to join which comes from the pipe; join displays an error message.

$ sort -r one | join --check-order - two
join: file 1 is not in sorted order

Next, the –a option with an argument of 1 causes join to display, in addition to its normal output, lines from the first file (one) that do not have a matching join field.

$ join -a 1 one two
9999 first line file one.
aaaa second line file one. FIRST line file two.
cccc third line file one. THIRD line file two.

Use –v in place of –a to inhibit the display of lines that join normally displays (those that have a matching join field).

$ join -v 1 one two
9999 first line file one.

The final example uses onea as the first file and specifies the third field of the first file (–1 3) as the match field. The second file (two) uses the default (first) field for matching.

$ cat onea
first line aaaa file one.
second line 1111 file one.
third line cccc file one.
$ join -1 3 onea two
aaaa first line file one. FIRST line file two.
cccc third line file one. THIRD line file two.

Posted in bash, Sysadmin | Comments Off

What does dereference mean?

Posted on January 30, 2012 by mark

A symbolic link is a file that refers to another file (a target file) without pointing directly to the target file: It is a reference to the target file. To dereference a symbolic link means to follow the link to the target file rather than work with the link itself. When you dereference a symbolic link, you end up with a pointer to the file (the filename of the target file). The term no-dereference is a double negative: It means reference. To no-dereference a symbolic link means to work with the link itself (do not dereference the symbolic link).

Many Linux utilities have dereference and no-dereference options, usually invoked by the –L (– –dereference) and the –P (– –no-dereference) options, respectively. Some utilities, such as cp and ls, also have a partial dereference option that is usually invoked by –H. With a –H option, a utility dereferences files listed on the command line only, not files found by traversing the directory hierarchy of a file listed on the command line. This section uses examples to explain each of these options.

Most utilities assume no-dereference as the default. The GNU ls utility, which is used in most Linux distributions does not have a –P (– –no-dereference) option, although the BSD ls utility does. Following, ls with the –l option displays information about the files in the working directory and does not dereference the memoA symbolic link; it displays the symbolic link including the pathname of the file the link points to (the target file). The first character of the memoA line is an l, indicating the line is describing a link; Max created the symbolic link and owns it.

$ ls -l
lrwxrwxrwx. 1 max max    19 01-27 16:19 memoA -> /home/max/sam/memoA
-rw-r--r--. 1 max max 39016 01-27 15:10 memoD
-rw-r--r--. 1 max max 25291 01-27 15:09 memoE

The next command specifies on the command line the file the symbolic link points to (the target file) and displays information about that file. The file type, permissions, owner, and time for the file are different from that of the link. Sam created the file and owns it.

$ ls -l /home/max/sam/memoA
-rw-r--r--. 1 sam sam 38889 01-27 15:07 /home/max/sam/memoA

Next, the –L (– –dereference) option to ls displays information about the files in the working directory and dereferences the memoA symbolic link; it displays the file the link points to (the target file). The first character of the memoA line is a –, indicating the line is describing a regular file; the command displays the same information about memoA as the preceding command.

$ ls -lL
-rw-r--r--. 1 sam sam 38889 01-27 15:07 memoA
-rw-r--r--. 1 max max 39016 01-27 15:10 memoD
-rw-r--r--. 1 max max 25291 01-27 15:09 memoE

When you do not specify a symbolic link as an argument to ls, the –H (partial dereference; this short option has no long version) option displays the same information as the –l option.

$ ls -H
lrwxrwxrwx. 1 max max    19 01-27 16:19 memoA -> /home/max/sam/memoA
-rw-r--r--. 1 max max 39016 01-27 15:10 memoD
-rw-r--r--. 1 max max 25291 01-27 15:09 memoE

When you specify a symbolic link as an argument to ls, the –H option causes ls to dereference the symbolic link; it displays information about the file the link points to (the target file; memoA in the example).

$ ls -H memoA
-rw-r--r--. 1 sam sam 38889 01-27 15:07 memoA

In the following example, the shell expands the * to a list of the names of the files in the working directory and passes that list to ls. Specifying an ambiguous file reference that expands to a symbolic link produces the same results as explicitly specifying the symbolic link (because ls does not know it was called with an ambiguous file reference, it just sees the list of files).

$ ls -H *
-rw-r--r--. 1 sam sam 38889 01-27 15:07 memoA
-rw-r--r--. 1 max max 39016 01-27 15:10 memoD
-rw-r--r--. 1 max max 25291 01-27 15:09 memoE

Posted in bash, Sysadmin, Uncategorized | Comments Off

The local Builtin

Posted on January 17, 2012 by mark

You can use the local builtin only within a function. This builtin causes its arguments to be local to the function it is called from and its children. Without local, variables declared in a function are available to the shell that called the function (functions are run in the shell they are called from). The following function demonstrates the use of local.

$ demo ()
> {
> x=4
> local y=8
> echo "demo: $x $y"
> }
$ demo
demo: 4 8
$ echo $x
4
$ echo $y

$

The demo function, which is entered from the keyboard, declares two variables: x and y and displays their values. The variable x is declared with a normal assignment statement while y is declared using local. After running the function, the shell that called the function has access to x but knows nothing of y.

Posted in bash, Sysadmin | Comments Off

Burn a CD/DVD ISO image file from the command line using wodim

Posted on November 8, 2011 by mark

Install wodim; its name is derived from Write data to Optical DIsk Media. This utility has many options; in a simple setup it figures out what you want to do and does it.

To burn an ISO image file, first determine the device name of the CD/DVD:

$ wodim --devices
wodim: Overview of accessible drives (1 found) :
-------------------------------------------------------------------------
 0  dev='/dev/scd0'     rwrw-- : '_NEC' 'DVD_RW ND-2510A'
-------------------------------------------------------------------------

Then, burn ISO image file:

$ wodim dev=/dev/scd0 -v -data name.of.file.iso

Replace -data with -audio to burn .wav files. That’s it!

Posted in Sysadmin | Comments Off

Fun with hdparm!

Posted on September 25, 2010 by mark

The hdparm utility reports on and set device parameters. This blog explains a few of the hdparm options—see the man page for more options and details.

CAUTION: You can wreak havoc on a disk with some hdparm commands. You can wipe a disk with a hdparm Secure Erase command (see https://ata.wiki.kernel.org/index.php/ATA_Secure_Erase). Use this utility with caution.

If you want more information than this blog hosts, see the hdparm man page, http://freshmeat.net/projects/hdparm/, and http://sourceforge.net/projects/hdparm/.

In the following examples, /dev/sdf is a Hitachi 1TB drive model HDS721010CLA332.

Display General Information

The –I option displays information about a drive. Add – –verbose for even more information.

$ sudo hdparm -I /dev/sdf | head -20
/dev/sdf:
ATA device, with non-removable media
        Model Number:       Hitachi HDS721010CLA332
        Serial Number:      JP2921HQ049J1A
        Firmware Revision:  JP4OA25C
        Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6; Revision: ATA8-AST T13 Project D1697 Revision 0b
Standards:
        Used: unknown (minor revision code 0x0029)
        Supported: 8 7 6 5
        Likely used: 8
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455

With the –H option, hdparm displays the temperature of the drive you specify as an argument. It also tells you whether the drive temperature is in a safe range, which may not be accurate. This option works for Hitachi drives and may work for some others.

$ sudo hdparm -H /dev/sdf
/dev/sdf:
  drive temperature (celsius) is:  29
  drive temperature in range:  yes

Control Power Settings

The –C option displays the power setting of the drive:

mark@tea:~$ sudo hdparm -C /dev/sd[ef]
/dev/sde:
 drive state is:  active/idle
/dev/sdf:
 drive state is:  standby

Although the man page refers to a sleep and a standby mode, I could get my drive to report only standby. The –Y option is supposed to put a drive into the lowest power mode (sleep), which means it shuts down completely. The –y option is supposed to put a drive into the low power mode (standby), which usually means it spins down.

$mark@tea:~$ sudo hdparm -C /dev/sdf
/dev/sdf:
 drive state is:  active/idle
mark@tea:~$ sudo hdparm -Y /dev/sdf
/dev/sdf:
 issuing sleep command
mark@tea:~$ sudo hdparm -C /dev/sdf
/dev/sdf:
 drive state is:  standby

Before testing another option, I spin up the hard drive by listing a directory on the hard drive. Five-ten seconds elapse before the listing appears, confirming the drive was spun-down. If you do not see this pause, the drive is not spun-down or you are reading from cache.

mark@tea:~$ ls -l /backup/bu1/*/p01.26
...
mark@tea:~$ sudo hdparm -C /dev/sdf
/dev/sdf:
 drive state is:  active/idle
>>> Mon 17:05
mark@tea:~$ sudo hdparm -y /dev/sdf
/dev/sdf:
 issuing standby command
>>> Mon 17:05
mark@tea:~$ sudo hdparm -C /dev/sdf
/dev/sdf:
 drive state is:  standby

With an argument of 0 (zero), the –S option prevents the drive from entering low-power mode and spinning down. The numeric arguments cause the drive to spin down after a specified period of inactivity. An argument in the range of 1-240 causes the drive to spin down after nx5 seconds (120 causes the drive to spin down after 10 minutes of inactivity). An argument in the range 241-251 cause the drive to spin down after (n-240)x30 minutes. These values may vary between drive manufacturers.

Performance Testing

The –T and –t options yield benchmarks for the drive. The uppercase option displays cache read benchmarks while the lowercase option displays device read benchmarks.

mark@tea:~$ sudo hdparm -Tt /dev/sdf
/dev/sdf:
 Timing cached reads:   15200 MB in  2.00 seconds = 7609.46 MB/sec
 Timing buffered disk reads:  408 MB in  3.01 seconds = 135.52 MB/sec

Posted in Hard disk, Sysadmin | Comments Off