|
xCAT Force10 Networks E1200 Setup HOWTO If you are unfamiliar with Force10 and the E1200 you can visit their website at http://www.force10networks.com. This document does not attempt to describe the architecture of the E1200 nor does it give details about running it in an enterprise environment. It does detail the steps involved in running the E1200 as a high throughput gigabit switch in a Linux HPC environment. Helpful Hints · The CLI for the switch supports command completion via the <tab> key. · Command history can be accessed via the <up arrow>. · Much of the CLI matches or closely resembles that of Cisco’s IOS. · Use “?” to bring up context sensitive help. Contents
The switch comes with a blue 8-wire flat cable and a RJ45 to DB9 serial adapter. This cable needs to be connected between the console port on the primary RPM module and the PC or terminal. The primary RPM is located in slot RP0 and is labeled as a LC-EC-RPM. The console port is labeled as such and is the top RJ45 connector on the RPM. The terminal configuration should be set to 9600, 8, N, 1 with no flow control. Testing was done using Windows HyperTerminal. After connecting the cable and bringing up the terminal session you should login to a priviledged session. Type <enter> to get a prompt and type enable. Press RETURN to get
started. Force10>enable Force10# User Configuration /
Maintenance By default there isn’t a password for the privileged session nor is there a user and password for remotely logging in. From the Force10# prompt type in the following to set a privileged password of “foobar” and add a user “foo” with a password of “foobar”. Force10#configure Force10(conf)#enable
password 0 foobar Force10(conf)#username foo password 0 foobar Force10(conf)#end Force10# Management Network
Configuration In order to access the switch remotely an ip address needs to be assigned to the 10/100 Ethernet port on the RPM. This port will usually then be connected to an external customer network to allow remote access to the switch even in the event that the management node cannot be accessed. The following will configure the management interface on the primary RPM and assign it an address on the users network. Force10#configure Force10(conf)# interface managementEthernet 0/0 Force10(conf-if-mgmt)#description
Force10(conf-if-mgmt)#no
shutdown Force10(conf-if-mgmt)#ip address 192.168.1.1 Force10(conf-if)#end Force10# Gigabit Linecard Configuration By default all of the ports on all of the gigabit linecards are in the down state. You need to bring them all up manually. This only has has to be done one time but each port must be done one at a time using the following sequence. Force10#configure Force10(conf)# interface gigabitEthernet 2/0 Force10(conf-if)#switchport Force10(conf-if)#no
shutdown Force10(conf-if)#end Force10# The above sequence will configure the first gigabit port of the linecard located in chassis slot 2 to be a layer 2 switchport and make it active. Repeat those steps for every port in the switch. To make it somewhat easier you can use the <up arrow> key to recall each of the commands and backspace to change the slot/port numbers. Example: Force10#configure Force10(conf)# interface gigabitEthernet 2/0 Force10(conf-if)#switchport Force10(conf-if)#no
shutdown Force10(conf-if)#<up
arrow><up arrow><up
arrow> Force10(conf)# interface gigabitEthernet 2/1 Force10(conf-if)#<up
arrow><up arrow><up
arrow> Force10(conf-if)#switchport Force10(conf-if)#<up
arrow><up arrow><up
arrow> Force10(conf-if)#no
shutdown Force10(conf-if)#end Force10# Continue on for the remaining ports. VLAN
Configuration A typical configuration requires that two VLANs be configured on the switch. The first one is the main cluster traffic and all nodes will connect into ports assigned to that VLAN. The second VLAN is for the management subnet and will consist mainly of ports that the management node and 10/100 switches will connect into. The tested environment had six gigabit linecards assigned to the cluster VLAN and one linecard assigned to the management VLAN. The management VLAN is also assigned an ip address to allow remote access from the management node via an internal network. Management VLAN config: Force10#configure Force10(conf)#interface
vlan 20 Force10(conf-if-vlan)#description Management
VLAN Force10(conf-if-vlan)#ip address
172.20.100.1/16 Force10(conf-if-vlan)#no shutdown Force10(conf-if-vlan)#untagged gigabitEthernet 13/0-23 Force10(conf-if-vlan)#end Force10# Cluster VLAN config: Force10#configure Force10(conf)#interface
vlan 21 Force10(conf-if-vlan)#description Cluster VLAN Force10(conf-if-vlan)#no shutdown Force10(conf-if-vlan)#untagged gigabitEthernet 2/0-23 Force10(conf-if-vlan)#untagged gigabitEthernet 4/0-23 Force10(conf-if-vlan)#untagged gigabitEthernet 6/0-23 Force10(conf-if-vlan)#untagged gigabitEthernet 8/0-23 Force10(conf-if-vlan)#untagged gigabitEthernet 10/0-23 Force10(conf-if-vlan)#untagged gigabitEthernet 12/0-23 Force10(conf-if-vlan)#end Force10# Verifying and Saving
Configuration You will need to verify the changes made to the switch and save them to nvram. During the configuration the changes are made to the running configuration and take place immediately. The changes will not however survive a reboot unless you save them down to the startup configuration. Verify Config: Force10#show linecard all -- Line cards -- Slot Status
NxtBoot ReqTyp
CurTyp Version
Ports -------------------------------------------------------------------------- 0 online
online EX2YB EX2YB 3.1.4.2
2 1 not
present 2 online online E24SB E24SB 3.1.4.2
24 3 not
present 4 online
online E24SB E24SB 3.1.4.2
24 5 not
present 6 online
online E24SB E24SB 3.1.4.2
24 7 not
present 8 online
online E24SB E24SB 3.1.4.2
24 9 not
present 10 online
online E24SB E24SB 3.1.4.2
24 11 not
present 12 online
online E24SB E24SB 3.1.4.2
24 13 online
online E24SB E24SB 3.1.4.2
24 Force10# Use the above output as a reference to
ensure you configured all of the linecards and
ports. Force10#show vlan Codes: * - Default VLAN, G -
GVRP VLANs NUM Status Q
Ports * 1
Inactive
20 Active U Gi 13/0-23 21 Active U Gi 2/0-23
U Gi 4/0-23
U Gi 6/0-23
U Gi 8/0-23
U Gi 10/0-23
U Gi 12/0-23 Force10# Make sure all the ports are in the correct
VLAN from the output similar to above. Force10#show ip interface brief linecard
2
ß leave off the “linecard 2” if you want to list all linecards Interface
IP-Address OK? Method
Status
Protocol GigabitEthernet 2/0 unassigned YES Manual
up
up GigabitEthernet 2/1 unassigned YES Manual
up
up GigabitEthernet 2/2 unassigned YES Manual
up
up GigabitEthernet 2/3 unassigned YES Manual
up
up GigabitEthernet 2/4 unassigned YES Manual
up
up GigabitEthernet 2/5 unassigned YES Manual
up
up GigabitEthernet 2/6 unassigned YES Manual
up up GigabitEthernet 2/7 unassigned YES Manual
up
up GigabitEthernet 2/8 unassigned YES Manual
up
up GigabitEthernet 2/9 unassigned YES Manual
up
up GigabitEthernet 2/10 unassigned NO Manual up
down GigabitEthernet 2/11 unassigned NO Manual up
down GigabitEthernet 2/12 unassigned NO Manual up
down GigabitEthernet 2/13 unassigned NO Manual up
down GigabitEthernet 2/14 unassigned YES Manual
up
up GigabitEthernet 2/15 unassigned YES Manual
up
up GigabitEthernet 2/16 unassigned NO Manual up
down GigabitEthernet 2/17 unassigned NO Manual up
down GigabitEthernet 2/18 unassigned NO Manual up
down GigabitEthernet 2/19 unassigned NO Manual up
down GigabitEthernet 2/20 unassigned NO Manual up
down GigabitEthernet 2/21 unassigned NO Manual up
down GigabitEthernet 2/22 unassigned NO Manual up
down GigabitEthernet 2/22 unassigned NO Manual up
down Force10# Make sure the output similar to above shows
all interface to have a status of up. Save Config: Force10#copy running-config startup-config Resetting Switch to Factory
Defaults Resetting the switch to the factory defaults can be done by deleting (or renaming) the startup-config from the flash: filesystem. List the files in flash: Force10#dir Directory of
flash: 1 drwx
512
2 -rwx 11001 3 -rwx 12108 4 -rwx
0
5 drwx
512
flash: 32096256 bytes total
(22284288 bytes free) Force10# Delete the startup-config: Force10#delete startup-config Directory of
flash: 1 drwx
512
2 -rwx 12108 3 -rwx
0
4 drwx
512
flash: 32096256 bytes total
(22211289 bytes free) Force10#reload The “reload” above will restart the switch and with the startup-config missing a new one will be created with factory defaults. Upgrading the E1200
software Upgrading the software does require reloading the switch so downtime will need to be scheduled for the upgrade. Download the new release from http://www.force10networks.com/support and put in the temp directory of the management node. Alternatively, you can put it on any machine running sshd that can be reached from the E1200. Login to the E1200 and run through the following steps: List the current version to make sure you are not running the latest: Force10#show
version Force10 Networks Real Time
Operating System Software Force10 Operating System
Version: 1.0 Force10 Application Software
Version: 3.1.4.2 Copyright (c) 1999-2002 by
Force10 Networks, Inc. Build Time: Fri Oct 18
Build Path:
/local/sw/huiyuan/2b1_0/depot/main/Dev/Cyclone/Release/3.1.3b1/SW/SRC Force10 uptime is 2 days, 5
hours, 6 minutes System image file is
"flash://E1200_405-3.1.4.2.bin" Chassis Type: E1200
Control Processor: IBM
PowerPC 405GP (rev D) with 268435456 bytes of
memory. Route Processor 1: IBM
PowerPC 405GP (rev D) with 536870912 bytes of
memory. Route Processor 2: IBM
PowerPC 405GP (rev D) with 536870912 bytes of
memory. 128K bytes of non-volatile
configuration memory. 2 Route Processor
Module 9 Switch Fabric
Module 1 2-port 10GE LAN PHY line card
with 10Km optics (EB) 7 24-port GE line card with SX
optics (EB) 2 FastEthernet/IEEE 802.3
interface(s) 2 Ten GigabitEthernet/IEEE 802.3
interface(s) 168 GigabitEthernet/IEEE 802.3
interface(s) Force10# Copy the software from the management server to the E1200. There are several methods available but the simplest seems to be to use the scp support. Force10# Force10#copy scp: flash: Address or name of remote
host []: 172.20.1.1 Source file name []:
/tmp/e1200-4.1.1.0.bin User name to login remote
host: root Password to login remote
host: foobar Destination file
name[e1200-4.1.1.0.bin]: <enter> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!! 10029048 bytes successfully
copied Force10# Configure the E1200 to use the new software on boot. Force10#
configure Force10(conf)# boot system rpm0 primary
flash://e1200-4.1.1.0.bin Force10(conf)# boot system
rpm0 default flash://e1200-4.1.1.0.bin Force10(conf)#
end Force10# Reload switch. Force10#reload Compatibility with Cisco Catalyst
Switches There have been observed problems connecting a Cisco Catalyst with the Force10 E1200. Turning off flow control on the Cisco ports that connect directly into the E1200 will fix the problem. For further information on setting up the Cisco refer to the Cisco Howto. Wes
Kinard |