When we first set up a Network Installation Management server for AIX in our own test lab, the unclear and somewhat self-contradictory NIM manual for AIX 4.3.3 made it a painful experience. Our goal is to make your path to a working NIM server/client environment shorter and more interesting. This article is intended as a practical, step-by-step guide for setting up a NIM server for AIX, and we provide it as a complement to the official manual.
|
We will not go into great detail of NIM basics. We still encourage you to read the "Network Installation Management Guide and Reference" for additional information, but we hope the manual will be easier to understand after you read this article.
A Few Assumptions
NIM client name: aix7
NIM master name: march
DNS domain name: testlab.com
Network where the machines are located: 192.168.1.0/24
The master should run the highest version of AIX that you are going to install via NIM. You cannot install AIX 5.2 from a master that runs on AIX 4.3.3. We also encourage you to install the latest firmware available for your machines. You can get it from the IBM Support Web site:
http://www-1.ibm.com/servers/eserver/support/pseries/index.html
NIM Server Installation
Before you start, we advise you to decide where you want the NIM lpp_source, SPOT, and other resources allocated. We found that having it on a separate logical volume makes sense for future optimization. We created separate logical volumes for lpp_source and SPOT resources. I cannot tell you if it's really a great idea to have it separate; you may find it unnecessary. In our situation, however, we were tight for hard-disk space, so saving a few gigabytes in this fashion made sense.
This article assumes you have created two filesystems: /export/nim/lpp_source and /export/nim/spot. The following two commands will create these filesystems. Note that both are created within nimvg. You might want to create a separate volume group (as we did) for NIM resources to improve disk I/O performance even further:
crfs -v jfs -g nimvg -a size=$((2000*500)) -m \
/export/nim/lpp_source \
-Ayes -prw -tno -a frag=4096 -a nbpi=4096 -a compress=no
crfs -v jfs -g nimvg -a size=$((2000*300)) -m /export/spot \
-Ayes -prw -tno -a frag=4096 -a nbpi=4096 -a compress=no
Check that the NIM server fileset is installed (bos.sysmgt.nim):
lslpp -l | grep bos.sysmgt.nim
If it's not installed, put the first AIX CD into the server's CD drive and run:
installp -aXgd /dev/cd0 bos.sysmgt.nim
Once you have the disk space allocated for resources and the NIM fileset installed, you can carry on with the configuration.
To begin, let's define the network; run:
nimconfig -a netname=testlab -a pif_name=en0 -a platform=chrp \
-a cable_type1='N/A'
Our master server was of chrp architecture. You can find out about yours by running:
lscfg | grep Arch
Now we can define lpp_source and SPOT:
nim -o define -t lpp_source -a source=/dev/cd0 -a server=master \
-a location='/export/nim/lpp_source/aix53_cd_lpp' aix53_cd_lpp
nim -o define -t spot -a source=aix53_cd_lpp -a server=master \
-a location='/export/nim/aix53_cd_spot' aix53_cd_spot
In our lab, we defined a name convention and named directories containing a resource's files the same as the resource name. So, in the command above, the resource name is aix53_cd_lpp, and the location option defines where the files will be kept (i.e., in the directory called /export/nim/lpp_source/aix53_cd_lpp). The source option can be a directory that contains files from the AIX CD or any set of files that you want to define as a resource (see "Custom Software Installation with NIM" below).
We will need two services to be running on the master to facilitate network boot of clients: TFT and BOOTP. Put the following lines into the master's /etc/inetd.conf file:
tftp dgram udp6 SRC nobody /usr/sbin/tftpd tftpd
bootps dgram udp wait root /usr/sbin/bootpd bootpd /etc/bootptab
and make inetd re-read the file by running:
refresh -s inetd
You may already have these lines in place if you didn't change the defaults. For the security-conscious (as we all should be), we will later show how to wrap this up with the TCP Wrapper program.
NIM Client Installation
Now we have defined the server resources, and we can set up our first NIM client. You will need to install the NIM client fileset (bos.sysmgt.nim.client) on the client machine. Put the first AIX CD into the CD drive and run:
installp -aXgd /dev/cd0 bos.sysmgt.nim.client
We need remote shell service to be running on the client side to make NIM operations possible. Put the following line into your /etc/inetd.conf file on client machine:
shell stream tcp6 nowait root /usr/sbin/rshd rsh
and make inetd re-read the file by running:
refresh -s inetd
Again, this line might already be in your /etc/inetd.conf if you didn't change the defaults. We will later show how to wrap it up with TCP Wrapper. For the moment, however, let's concentrate on getting the system working without the security features for easier troubleshooting.
We do need to put the master's root account name into the client's root $HOME/.rhosts file. The file should look like this:
march root
march.testlab.com root
It's good to have both short and fully qualified names of the master server in the file. This can save you the hassle of wondering why the rshell is not working.
At this point, we can use either of the following approaches:
1. We can define an NIM client by running the nim command on the master:
nim -o define -t standalone -a platform='rspc' \
-a netboot_kernel='mp' -a if1='find_net aix7 000629F71EB' \
-a cable_type1='N/A' -a net_definition='ent 255.255.255.0 \
192.168.1.1 192.168.1.1' aix7
Remember that the client machine name is aix7. The machine is an older model 43P (rspc architecture) with single CPU (see parameter netboot_kernel; 'up' -- single CPU, 'mp' -- for SMP system). Note the long number after the machine name (000629F71EB) is the MAC address of the client Ethernet adapter. The two identical IP addresses later in the command (192.168.1.1) are the default gateway and the NIM master server default gateway, respectively. There might be a case when you have your master server on a different NIM client network.
2. We can use the niminit command on the client machine to do the same:
niminit -a name='aix7' -a master=march -a pif_name='en0' \
-a cable_type1='N/A' -a platform=rspc -a netboot_kernel='mp'
As you can see, there are only a few differences between the nim and niminit parameters. Both commands do the same operation resulting in a record created for the machine (aix7) on the master NIM server and file /etc/niminfo on the client.
NIM Operations
Now we have everything necessary to try our first Base Operating System (BOS) installation. The process involves installing and configuring the minimum amount of software needed to bring a machine to the running state. All NIM operations can be initiated from either a server (push installation) or client (pull installation).
Let's look at a push installation:
nim -o bos_inst -a source=rte -a lpp_source=aix53_cd_lpp \
-a spot=aix53_cd_spot -a boot_client=no aix7
In this example, we asked the NIM server to begin the installation of the OS from lpp_source and SPOT resources without a client reboot. Client reboot is the main thing here. You can push-install the OS and make the client machine reboot immediately (boot_client=yes) or you can prevent the immediate reboot (as we did using boot_client=no) and only allocate the resources for the client. The actual installation will begin when you reboot the client and force it to boot over the network.
For both methods, you will need to use SMS (System Management Services) to have the correct server, client, and gateway IP addresses configured for the network boot.
Access the SMS Menus
The SMS main menu looks similar to the following examples. Although it's slightly different on different pSeries models, it's easy to find your way through the menu.
To get to SMS menu, type F1 (or Esc+1 if you working through the serial port) when you see this line on the screen during initial boot of the machine:
memory keyboard network scsi speaker
Here is the menu you will see on pSeries 6C1:
pSeries Firmware
Version xxxxxxxx
(c) Copyright IBM Corp. 2000, 2002 All rights reserved.
------------------------------------------------------------------
Main Menu
1 Select Language
2 Change Password Options
3 View Error Log
4 Setup Remote IPL (Initial Program Load)
5 Change SCSI Settings
6 Select Console
7 Select Boot Options
8 View System Configuration Components
9 Update System/Service Processor Firmware
------------------------------------------------------------------
Navigation Keys:
X = eXit System Management Services
------------------------------------------------------------------
Type the number of the menu item and press Enter or Select a Navigation Key:
And this is menu you will see on 44P-170:
RS/6000 Firmware
Version xxxxxxxx
------------------------------------------------------------------
System Management Services
1 Display Configuration
2 Multiboot
3 Utilities
4 Select Language
.------.
|X=Exit|
'------'
You have to change the IP addresses of Server, Client, and Gateway for the machine to boot successfully over the network:
RS/6000 Firmware
Version xxxxxxxx)
------------------------------------------------------------------
IP Parameters
1. Client IP Address [192.168.1.3]
2. Server IP Address [192.168.1.2]
3. Gateway IP Address [192.168.1.1]
4. Subnet Mask [255.255.255.0]
When you do boot_client=no, you must alter boot sequence of the client machine to get it to boot over the network first. With boot_client=yes, the sequence will be altered for you by NIM system and will return to "hard drive first" when the installation is over.
If the nim operation failed, use the following command to reset the client state:
nim -Fo reset aix7
You can investigate the problem using the NIM log facility:
nim -o showlog aix7
You may encounter difficulty if you forgot to add the remote shell to the client's /etc/inetd.conf file, if you forgot to mention the master's root account in client's root $HOME/.rhost file, or you have name resolution problems (check DNS settings). If everything was configured properly, you will see the usual AIX installation screen on the client after network boot.
If you go one step further and create a custom bosinst.data file, you can get a non-prompted network installation. We will talk later about using mksysb images in your NIM environment. This will let you clone your systems over the network using the power of NIM.
Custom Software Installation with NIM
Here we will cover installation of extra software, APARs, or maintenance-level packages. Let's say we have our client installed and we have maintenance level 9 on it (because that's the level of AIX on the CDs that we used for creation of the lpp_source and spot resources). Later, we need to upgrade the client to maintenance level 11. Let's do that using our freshly installed NIM system.
First, we must create an lpp_source resource consisting of the maintenance-level files. The files are in /mnt/patches/aix/433/ml0911. So, we run the following command on the master to define the lpp_source:
nim -o define -t lpp_source -a server=master \
-a source=/mnt/patches/aix/433/ml0911 \
-a location=/export/nim/lpp_source/aix53_ml0911_lpp \
-a comments="4.3.3 maintenance level upgrade 09->11" aix53_ml0911_lpp
Second, we can use the "cust" operation to install from the lpp_source:
nim -o cust -a lpp_source=aix53_ml0911_lpp -a filesets='all' aix7
Note that for AIX 5.x lpp_source definition, the command would be different. We noticed that in 4.3.3, even if the source directory didn't have all the filesets for the lpp_source resource to have the "simages" attribute (meaning you can run the BOS installation using the lpp_source), the command above would finish successfully with a warning that you cannot use the lpp_source for bos_inst operation.
To define an lpp_source with not enough filesets (see NIM manual; lpp_source resource description) for the resource to have the "simages" attribute, you must use the "packages" attribute in the command. The attribute will list all the filesets you want to have in the resource directory. That is a lot of filesets! But, thanks to Unix, we have a solution. Here is the command:
nim -o define -t lpp_source -a server=master \
-a source=/mnt/patches/aix/520/ml4fixes \
-a location=/export/nim/lpp_source/aix520_ml4fixes_lpp \
-a packages="`installp -L -d /mnt/patches/aix/520/ml4fixes \
| awk -F: '{print $1}'`" aix520_3apars
We use a combination of the installp command and awk to list the filesets available in the source directory.
One thing that is not mentioned in the NIM manual is the installation of rpm packaged software that comes on the "Linux for AIX toolbox" CD.
Let's say you want to add the vnc rpm package to NIM lpp_source and install it using the NIM system. How would you make NIM use rpm instead of installp to do this? Simple. To begin, add the package to the lpp_source. To do this, you must copy the rpm package from the CD into the RPMS/ppc directory of the lpp_source.
For example, our aix53_cd_lpp resource directory is /export/nim/lpp_source/aix53_cd_lpp, so we copy the file into the /export/nim/lpp_source/aix53_cd_lpp/RPMS/ppc directory. Then we tell NIM about the change. Run the following command:
nim -o check aix53_cd_lpp
Now we can run the installation of vnc to the client:
nim -o cust -a lpp_source=aix53_cd_lpp -a filesets='R:vnc-3.3.3r1-2'
That's it. Thanks to the guys from the AIX newsgroup for the tip!
The same approach is valid when you want to add extra software into your lpp_source, which wasn't installed there by NIM during the define operation. Just copy the filesets into the installp/ppc directory under the lpp_source directory.
For example, our aix53_cd_lpp resource directory is /export/nim/lpp_source/aix53_cd_lpp, so we copy the filesets into the /export/nim/lpp_source/aix53_cd_lpp/installp/ppc directory. Then we can use the "check" operation shown previously for the vnc rpm file.
Mksysb and NIM
Another handy resource is mksysb. This tool allows you to clone your machines even faster than before. Say you already had a mksysb image of your client machine and now you'd like to use NIM to install it. All you need to do is to tell NIM where the mksysb image file is when defining a mksysb resource:
nim -o define -t mksysb -a server=master \
-a location=/export/nim/mksysb/aix7.433.mksysb aix7_433_mksysb
Here we already copied mksysb image file aix7.433.mksysb into the /export/nim/mksysb directory, and we want to define it as a resource.
If you want to take the mksysb image from the client, then do:
nim -o define -t mksysb -a server=master \
-a location=/export/nim/mksysb/aix7.433.mksysb \
-a source=aix7 -a mk_image='yes' aix7_433_mksysb
Note that "aix7" as a source is not a machine name but is the NIM resource called aix7, the one the defines the machine aix7.
Now you can run the client installation using the image:
nim -o bos_inst -a source=mksysb -a mksysb=aix7.433.mksysb \
-a spot=aix53_cd_spot -a boot_client=yes aix7
Note that the SPOT resource must be defined for the same version of AIX as your mksysb image. The maintenance level of the SPOT and the mksysb image must be the same as well.
Although it's handy to have an image of every machine in your lab, that takes a lot of disk space on your NIM master server. We found it more flexible to have a neutral mksysb image with all the necessary software but without any machine-name or IP-specific settings. This allowed us to clone many machines using a single mksysb image and apply final scripts to set the IP and other machine-specific parameters at the end of the BOS installation. This setup saves loads of disk space and makes maintenance of NIM easier.
Security
It's always good to have firewalls to secure your company networks, but really paranoid sys admins (as we all should be) would go one step further and secure each server. An extremely useful tool for this is the TCP Wrapper program, written by Wietse Venema. We will not describe the tool here. For those who want to know more, we recommend Practical UNIX & Internet Security by Garfinkel, Spafford, and Schwartz.
You can download TCP Wrapper already built for your version of AIX from:
http://www.bullfreeware.com/
Or, you can build it yourself with sources available from:
ftp://coast.cs.purdue.edu/pub/tools/tcp_wrappers
For our NIM installation, we want to protect both the server and the client. Let's do the server first. Change the lines in /etc/inetd.conf file (as mentioned at the beginning of the article) to the following:
tftp dgram udp6 SRC nobody /usr/local/bin/tcpd /usr/sbin/tftpd
bootps dgram udp wait root /usr/local/bin/tcpd /usr/sbin/bootpd \
/etc/bootptab
and make inetd re-read the file by running:
refresh -s inetd
Create file /etc/hosts.allow and put the following line in it:
tftpd,bootpd: 192.168.1.
Next, create file /etc/hosts.deny and put the following line in it:
ALL: ALL
We will protect the client in a similar way. The line in /etc/inetd.conf transforms into:
shell stream tcp6 nowait root /usr/local/bin/tcpd /usr/sbin/rshd
Then we make inetd re-read the file by running:
refresh -s inetd
Next, add march into /etc/hosts.allow:
rshd: march.testlab.com
The /etc/hosts.deny file is the same as on the server.
Run /usr/local/bin/tcpdchk -v to verify that your settings are correct. That's the simplest way to protect our NIM operations.