xCAT Blade Center NOTES (WIP)

This document is for xCAT 1.2.0 or later.

This HOWTO is for xCAT experts.  Please be very familiar with the xCAT 1.1.0 Redbook and the xCAT HOWTO.

1. I recommend a non-blade for a management node.
    
2. Blade Center Management Module (BCMM) firmware version 1.02 tested.
    
3. SSL not supported.  Please disable SSL support.
    
4. Create an xcat user ID and password that is identical per BCMM and enter in $XCATROOT/etc/passwd.tab.  Read the managementprocessor-HOWTO for information.  I recommend that it be the last user ID and password on the list.
    
5. Each Blade Center Management Module (BCMM) should be defined as a host and a node.  Read the managementprocessor-HOWTO for information on testing and troubleshooting all nodes management processors.  Review etc/samples/*.tab for examples of blades.
   
   NOTE:  blades in mp.tab use the bay number, not the internal name.
   
   NOTE:  BCMMs in mpa.tab use type "bc".
    
6. Assign each BCMM with an IP address.  Use static or DHCP, NOT "Try DHCP, if fails use static IP".  If using DHCP use xCAT static DHCP assignment.  Internal and external BCMM adapters IP addresses MUST be unique.  I recommend only using one or the other leaning toward external only for security.
    
7. Use mpasetup to complete the setup of each BCMM.
    
8. Check with mpacheck, e.g.:
   
   # mpacheck -l bc1
   bc1: Host IP Address: 192.168.1.45
   bc1: Subnet mask: 255.255.255.0
   bc1: Gateway IP Address: 192.168.1.1
   bc1: Data Rate: Auto
   bc1: Duplex Mode: Auto
   bc1: SNMP Traps: Enabled
   bc1: SNMP Agent: Enabled
   bc1: SNMP Community Name: public
   bc1: SNMP IP Address 1: 192.168.1.100
    
9. Scan for blades with mpascan, e.g.:
   
   # mpascan bc1
   bc1: blade1 1,2
   bc1: blade2 3,4
   bc1: blade3 5
   bc1: blade4 7
   bc1: Empty 6
   bc1: Empty 8
   bc1: Empty 9
   bc1: Empty 10
   bc1: Empty 11
   bc1: Empty 12
   bc1: Empty 13
   bc1: Empty 14
   
   In this example BC bc1 has (2) double wide blades in bays 1,2 and 3,4, and (2) single wide blades in bays 5 and 7.  The remaining bays are empty.
    
10. Change each nodes boot order with rbootseq, e.g.
    
    [root@foo xcat]# rbootseq blade1-blade4 c,f,n,hd0
    blade1: cdrom floppy network harddrive0
    blade2: cdrom floppy network harddrive0
    blade3: cdrom floppy network harddrive0
    blade4: cdrom floppy network harddrive0
    
    In this example blades 1-4 have their boot order changed to boot network before hard drive.  Use the 'list' option to display existing setup.
     
11. Use getmacs to collect MAC addresses from the BC internal switch.  The nodehm.tab getmacs method should be mpcli2.
    
    E.g.:
    
    # makedhcp --new
    .
    .
    .
    # rpower blade1-blade4 boot
    blade1: on reset
    blade2: on reset
    blade3: on reset
    blade4: on reset
    
    # sleep 60
    # getmacs blade1-blade4
    
    Please reset nodes: blade1 blade2 blade3 blade4
    
    Press [Enter] when ready...
    
    Saving output to mac.lst in current directory /opt/xcat.
    
    blade1 00:09:6b:00:63:e5
    blade2 00:09:6b:00:4a:7e
    blade3 00:09:6b:00:3b:2a
    blade4 00:09:6b:00:55:18
    
    Auto merge mac.lst with /opt/xcat/etc/mac.tab(y/n)?
    
    If you get "NO_MAC" you may have not completely setup xCAT TFTP/DHCP for stage2.  Read the xCAT Redbooks and HOWTOs.  Or, stage2 has not completed boot.  NOTE to troubleshoot stage2 issues set serialmac in site.tab to NA before you run ./mkstage.  stage2 pings the management server to keep the MAC alive in the switch for collection.
    
    NOTE:  The user ID and password for all the BC switches must be defined in passwd.tab as bcnetuser and bcnetpass.
    
    NOTE:  The bottom BC switch connects to all blade eth0.
     
12. Disable eth1 PXE boot per blade unless that is your intent.  It will save time.
     
13. Remote console.
    
    rvid works with BC.  Unlike the rvid with ASMA, BC rvid uses VNC.  High speed text and graphics works well.  However only one blade per BC can have the focus.  If you have an rvid session open and start another on the same BC, the existing session will switch.  No new window will open.
    
    Some BCs have no serial port solution.  Do NOT enable the serial BIOS redirection in the BIOS, it will just slow you down.
    
    Press F8 to popup a window with options.  E.g.  ctrl-alt-del.
     
14. Remote install console.  To monitor multiple blades during install and runtime use the following instructions.
    
    RH9 and RHEL3.0 support remote install console using telnet.  To use add the following to $XCATROOT/etc/conserver.cf for each node:
    
    nodename:|conserver.telnet nodename::&:
    
    E.g.:
    
    node4:|conserver.telnet node4::&:
    
    Use winstall/wcons as normal, when the install starts you'll see it. If you actually want a telnet login for runtime you will need to create a post install script to enable telnet.
    
    Not recommended if you have a serial port and are using it for serial console.
     
15. There is not need to boot stage3.  Use mpname to rename the Hawk management processor on each blade.  Validate with mpascan.
    
    NOTE:  mpncheck is not useful since all BC hardware control is based on physical position not internal management processor name.
    
    # mpname blade1-blade4
    blade1: blade bc1,1 renamed from junk to blade1
    blade2: blade bc1,3 renamed from junk to blade2
    blade3: blade bc1,5 renamed from junk to blade3
    blade4: blade bc1,7 renamed from junk to blade4
     
16. The rest of xCAT functions as expected with Blade Center.
     
17. SNMP hardware alerts support under development.
     

Support

http://xcat.org

Egan Ford
egan@us.ibm.com
October  2003