Rainer
Raab
Originally
Found at: http://www.samag.com/documents/s=1153/sam0103f/0103f.htm
Scripting and
UNIX systems administration go hand in hand. Writing scripts to automate
repetitive tasks is necessary if you want to find the time to work on other
things, such as tightening security on the systems you manage, migrating
applications to promote server utilization, capacity planning, and system
tuning. These types of tasks often fall at the bottom of the food chain if you
are in a typical shop, constantly putting out the fires started by buggy
applications, compounded by poor vendor support, and unreliable hardware. They
also lose priority if you are supporting a development team with a time to
market of 3-6 months for their application and the developers expect you to
manage the test, staging, and production environments, along with application
deployment and installation scripts. Fortunately, most UNIX variants provide an
ideal tool for developing and deploying scripts -- the Korn shell.
The Korn
shell is not just a command interpreter; it provides a powerful and versatile
programming language for the rapid development of automation scripts and the
capability to write globally accessible functions. In its basic form, a Korn
shell script is a series of shell commands, strung together in an executable
file. In its most advanced form, a Korn shell script may utilize built-in shell
commands that provide support for arrays, integer arithmetic and arithmetic
conditions, advanced UNIX I/O redirectors, and functions. It is the support for
functions that allows the rapid development of scripts that provide solutions
to the tasks of routine UNIX systems administration.
Defining
Global Functions
Korn shell
functions, known as procedures or subroutines in other programming languages,
allow for the organization of related shell commands by name. By organizing
shell commands into functions, long shell scripts are easier to read and
follow, and thus maintain. More importantly, functions provide a mechanism
allowing shell scripts to be quickly developed, because Korn shell functions
can be made available for use by other Korn shell scripts.
Korn shell
functions are normally defined within the script from which they are called.
However, in order for them to be made available to other Korn shell scripts
(i.e., made global), they must be defined separately and saved in their own
files, with only one function per file. To define a global function in Korn
shell, use the same syntax for defining all functions. Either one of the
following forms will work:
function function_name { shell commands...}
Or the Bourne
shell form:
function_name () { shell commands...}
I always use
the first form since it is the one that I am more familiar with and seems to be
the predominantly used one.
A global
function is one that returns the system date and time in a format that is
suitable for timestamps in log files:
function get_time { TIME=$(date '+%m/%d/%y-%H:%M:%S') printf "$TIME\n"}
To make the get_time function global and
invokable from Korn shell scripts, create a special directory (e.g., ksh_functions) in a system-wide
accessible directory (e.g., /usr/local). Save the get_time function in that
directory, using the function name as the filename. I prefer to use the /usr/local/ksh_functions directory as the
global Korn shell function directory:
[rainer@fostercity]/usr/local/ksh_functions>ls -ltotal 2-rw-r--r-- 1 rainer gurus 93 Nov 28 10:00 get_time
Add the Korn
shell built-in variable FPATH=/usr/local/ksh_functions and autoload get_time to your .kshrc file, or to the file
you have defined as the environmental file for your login shell. If you have
not previously done so, adding export ENV=~/.kshrc to your .profile initialize file will set
your environmental file to .kshrc:
[oksana@fostercity]/export/home/oksana>more .kshrcexport FPATH=/usr/local/ksh_functionsautoload get_time
The use of
the built-in variable FPATH is similar to that of the PATH environment variable.
FPATH contains the list of directories that have function definitions,
and is used by the autoload function_name command to locate
functions. The autoload command not only instructs the shell to load the function
only when invoked, but also enables us to store functions in locations other
than in our local .profile or .kshrc files, providing the
entire framework for creation of global functions. Actually, the autoload command is an alias
for the typeset -fu command and can be used instead. I prefer to
use the autoload command because it is easier to remember.
To use get_time from a script, simply
invoke the function name from a shell script:
[oksana@fostercity]/export/home/oksana/scripts>more test_time1#!/bin/kshPATH=/usr/bin:/usr/ucbget_time
You can even
assign a variable to hold the results from get_time:
[oksana@fostercity]/export/home/oksana/scripts>more test_time2#!/bin/kshPATH=/usr/bin:/usr/ucbSYSTIME=$(get_time)
Securing Global Functions
You may have
noticed that in the get_time function I did not specify the path for the
system commands date and printf. This is because the PATH environment variable is set
explicitly, from the shell scripts test_time1 and test_time2, to include the
standard system command directories /usr/bin and /usr/ucb. This is my preferred
method for writing functions and scripts. If you view this as a security
concern, simply include the full path for system commands in all functions and
set the PATH to null (PATH="") in your shell
scripts.
For added
security, create a special group, such as kshteam, and add
supplementary group membership to users who want to create and access global
functions in the ksh_functions directory:
[rainer@fostercity]/usr/local/ksh_functions>groups rainer gurus kshteam
Then, assign
the user bin as the owner, with group name kshteam, to the ksh_functions directory with
permissions of read, write, and execute for the owner/group and no permissions
for world. Next, set the sticky bit on the ksh_functions directory to only
allow the owner of a global function to delete, rename, or modify the function:
[rainer@fostercity]/usr/local/ksh_functions>ls -la total 6 drwxrwx--T 2 bin kshteam 512 Nov 28 12:41 . drwxr-xr-x 40 root root 1024 Oct 22 14:50 .. -rw-r--r-- 1 rainer gurus 93 Nov 28 14:50 get_time
This scheme
will prevent unauthorized access to global functions, as well as prevent
authorized users from modifying functions that they do not own. However, be
sure to inform authorized users that their functions must be world readable to
allow other authorized users access (unless the other authorized users all
belong to the same primary group and the appropriate permissions are set to
allow group access to the functions). Setting the global functions to world
readable is not a security concern in this scheme because the permissions on
the ksh_functions directory are set to none for world, making
the directory inaccessible by anyone other than root or members of the kshteam group.
Managing
Global Functions
There are
several useful commands for managing functions. To verify which functions are
available to your shell, use the functions command. The functions command displays a
list of all defined functions, listed in alphabetical order by function name,
defined in your login session:
[oksana@fostercity]/export/home/oksana/scripts>functionsfunction get_time
The functions command, like autoload, is yet another Korn
shell alias. The typeset -f command can be used instead. There are a
number of predefined Korn shell aliases that might be of interest. Use the alias command to display a
list of all aliases defined for your login session:
[oksana@fostercity]/export/home/oksana/scripts>aliasautoload='typeset -fu'command='command 'functions='typeset -f'history='fc -l'integer='typeset -i'local=typesetnohup='nohup 'r='fc -e -'stop='kill -STOP'suspend='kill -STOP $$'
The functions command does not just
list functions available to your current login session, but all of your shells,
because the FPATH=/usr/local/ksh_functions and the autoload get_time statements have been
added to your .kshrc environment file. The get_time function, as well as
other functions defined by the autoload command in your .kshrc file, are available
to all subshells and Korn shell scripts run under your login.
whence is another useful
built-in command. It identifies the source of a command. When used with the -v option (verbose), whence produces useful
information as to what type of command it is (e.g., is it a shell function, a
built-in command, a reserved shell keyword, an alias, etc.):
[oksana@fostercity]/export/home/oksana/scripts>whence -v get_timeget_time is a function[oksana@fostercity]/export/home/oksana/scripts>whence -v whencewhence is a shell builtin[oksana@fostercity]/export/home/oksana/scripts>whence -v functionfunction is a reserved shell keyword[oksana@fostercity]/export/home/oksana/scripts>whence -v functionsfunctions is an exported alias for typeset -f
Use whence for clarification
when there is some uncertainty as the output of a command. Strange results are
often not so strange when you realize that the command you have been running is
not the command you thought you were running. A good example of this is the pwd command, which is
also a built-in shell command, and takes precedence over /usr/bin/pwd when typed without the
full path. This conundrum becomes apparent when you cd to a directory via a
symbolic link, and find yourself wondering where you really are:
[oksana@fostercity]/export/home/oksana/scripts>cd /usr/pub \ [oksana@fostercity]/usr/pub>pwd/usr/pub[oksana@fostercity]/usr/pub>/usr/bin/pwd /usr/share/lib/pub
This occurs
because pwd is a built-in shell command that displays the contents of the
present working directory set by the cd command and stored in the PWD shell
environment variable. PWD stores the relative path, while /usr/bin/pwd displays the absolute
path name of the current working directory.
Practical
Global Functions
The get_time function was a simple
example of a function, made global, that may be useful for incorporation into
other Korn shell scripts. However, it is hardly worth the effort to make get_time global just to save
3-4 lines of future typing. A more useful global function would be one to send
out email alerts (see Listing 1):
The first
thing you might notice that is different with this function, than with the
previous get_time function, is that it contains a documentation header.
When writing functions for system-wide usage, it is helpful to include a small
description of the function that describes (at a minimum) what the function
does and its usage. Develop a standard documentation header and stick to it. As
the system-wide function library grows, you will be glad to see the
documentation header there when you want to use someone else's function or need
to use one of your older functions.
To make the send_email function globally
available and invokable from Korn shell scripts, save the send_email function to our system-wide
accessible /usr/local/ksh_functions directory and add the autoload send_email command to your .kshrc file:
[rainer@fostercity]/usr/local/ksh_functions>ls -ltotal 4-rw-r--r-- 1 rainer gurus 93 Nov 28 10:00 get_time-rw-r--r-- 1 rainer gurus 290 Nov 28 10:05 send_email [oksana@fostercity]/export/home/oksana>more .kshrcexport FPATH=/usr/local/ksh_functionsautoload get_timeautoload send_email
To use the send_email function from a
script, simply assign values to the variables EMAIL_LIST, EMAIL_SUBJECT, and EMAIL_MESSAGE, then invoke the
function name from a script:
[oksana@fostercity]/export/home/oksana/scripts>more test_email#!/bin/kshPATH=/usr/bin:/usr/ucbEMAIL_LIST="raabrr@yahoo.com 4151234567@alphapage.airtouch.com"EMAIL_SUBJECT="Testing"EMAIL_MESSAGE="This was only a test.."send_email
Notice that
you can pass the value of variables to functions. In the test_email script, we are
passing a space-separated list of the users to whom we want to send email using
the EMAIL_LIST variable, and the subject of the email message with the EMAIL_SUBJECT variable, and the
email message itself with the EMAIL_MESSAGE variable. This little
bit of magic works because functions do not run in separate subshells, as
scripts normally do (the exception is if you invoke a script with the
"." command). Utilizing functions in this manner allows you to use
functions within your scripts as if they were present locally!
Another
useful function is one to read a configuration file (see Listing 2. When developing complex scripts, I find it
useful to be able to change the values of variables from a configuration file,
rather than hardcoding the values within the script itself. This not only eases
script maintenance, but also allows for user-adjustable settings. Since the
configuration file is a text file, you can make appropriate comments to assist
the individuals who may be responsible for managing the process that your
script performs.
Listing 3 contains a configuration file that could be used
with the read_config function. To make the read_config function globally
available and invokable from Korn shell scripts, save it to our system-wide
accessible /usr/local/ksh_functions directory and add autoload read_config to your .kshrc file. Then to use it
from a shell script, simply invoke the function name from a script:
[oksana@fostercity]/export/home/oksana/scripts>more test_read_cfg#!/bin/kshPATH=/usr/bin:/usr/ucbCONFIG_FILE=/export/home/oksana/scripts/test_read_cfg.confread_configprintf "Email list: $EMAIL_LIST\n"printf "Log file: $LOG_FILE\n"printf "Debugging is: $DEBUG\n"
Notice how
the values of the variables DEBUG, LOG_FILE, and EMAIL_LIST are magically
available to the test_read_cfg script. This is accomplished by exporting the
variables found in the configuration file to the current shell for use. I use
the read_config function for all of my shell scripts that require regular
updates or are maintained by someone else. For example, to add someone to the
email list, just edit the test_read_cfg.conf file with your
favorite text editor and then append an additional email address to the EMAIL_LIST variable. Someone
with no Korn shell scripting experience can do this, since the configuration
file is easily readable and in plain text.
Conclusion
The Korn
shell provides a powerful and versatile programming language and is my
scripting language of choice for automating routine UNIX systems administration
tasks. The Korn shell function facility not only provides the mechanism for
allowing shell scripts to be quickly developed, but it promotes the sharing of
code through the use of global functions. There is no point in reinventing the
wheel every time a new script has to be developed. With some proper management,
the use of global Korn shell functions can become an invaluable tool for
everyone on a system, not just the systems administrator.
References
Learning
the Korn Shell, Bill Rosenblatt, O'Reilly and Associates, Inc., 1993
Rainer
Raab is a senior UNIX Systems Administrator, consulting for Wells Fargo Bank in
San Francisco, California. When not scripting, he spends as much time as
possible with his lovely wife, Oksana. He can be contacted at: raabrr@yahoo.com.