Document last changed by: Jim Sibley on 7/23/2001

OSA cards for Linux - Practical Notes

I've attempted to pull together the practical knowledge I've acquired in implementing these cards based on information from Boeblingen, Germany (linux390@de.ibm.com) and trial and error. This is by no means complete, but at least I have gotten these parameters to work on S/390 Linux.

• Linux IBM Object Code Only OSA Drivers
• Types of OSA cards
• Relating Interface with OSA card and TP parameters (SuSE)
• HCD and IOCDS considerations
• Native IPL
• Operational Notes
• References

Linux IBM Object Code Only OSA Drivers

The are two sets of Linux IBM OCO drivers to support OSA cards

• LAN Channel Station (LCS) Ethernet and Token Ring (lcs.o). Normal ESCON protocols.
• Gigabit Ethernet (GBE). QDIO is a new S/390 architecture that allows an OSA-Express Gigabit Ethernet feature to communicate directly with system memory through the use of queues and a never-ending channel program.
  • Queued Direct I/O (QDIO) protocol driver (qdio.o)
  • Gigabit Ethernet driver (qeth.o)

back to top

Types of OSA cards

Currently, there are two flavors of OSA cards: OSA2 and OSA-Express. The OSA-Express cards can only be installed in S/390 G5 or above. The zSeries OSA-Express (Hydra 1.5 cards) are functionally the same as two (2) G5+ cards (Hydra 1).

• OSA2-ENTR - OSA2 Ethernet/Token Ring
• OSA2-FE - OSA2 Fast Ethernet
• OSA-FEX - OSA Fast Ethernet Express
• OSA-GBX - OSA Gigabit Express

• Mbps - Megabits per second (approximate).
• Connection: The card may support other protocols, but Linux only supports Token Ring or Ethernet.
• CPU: Minimum CPU required
• HCD CHP: HCD/IOCDS CHP type
• # CHP: Number of CHP addresses per card
• Port/CHP: number of ports per CHP
• Multiple IP: You must use OSA/SF to allow multiple IP on an OSA2 card. The OSAX card automatically assigns the correct value to the correct LPAR.
• MAX Addresses/port: OSA2 cards allow 16 IP addresses per port. For OSAX, there are 240 addresses per port. Since an LCS card takes 2 S/390 addresses per IP addresses, the limit would be 120 IP addresses per chpid for LCS and since a QDIO card takes 4 addresses (LPAR) mode, the limit would be 60 IP addresses.
• Access IP's other LPARS: IP addresses of other LPARS are directly addressable within the card
• Linux Module: the modules are in "/lib/modules/$RELNO/net". The release number may change
• Sample /etc/modules.conf
  • LCS cards
    • The addresses are in even-odd pairs. You only code the even addresses.
    • Only one LCS options card should be present. Multiple cards would be coded on the same options card. Linux detects whether the card is T/R or Ethernet and it assigns the interface name sequentially, as appropriate to the card (tr0,tr1,eth0,eth1, etc.)
    • It assigns the interface name sequentially, as appropriate to the card (tr0,tr1,eth0,eth1, etc.).

      1 chp, 1 port/chp	options LCS noauto=1 devno_portno_pairs=0x2000,0
      1 chp, 2 ports/chp	options LCS noauto=1 devno_portno_pairs=0x2000,0,0x2000,1
      2 chp, 1 ports/chp	options LCS noauto=1 devno_portno_pairs=0x2000,0,0x3000,0

  • QETH cards
    • The addresses are in triplets. However, under native, the developers currently recommend quadruplets on EVEN address boundaries; this may be relaxed in the future. VM without autosense may be able to use the triplets.
    • Only one QETH options line should be present. Multiple cards would be coded on the same options line.
    • If no interface name is present, it assumes eth0. It is probably safer to code the explicit interface name
    • The QETH modules has a dependency on the QDIO module in /lib/modules/$RELNO/modules.dep, so both modlues are loaded when QETH is loaded

      1 chp	options qeth qeth_options=noauto,0x4000,0x4001,0x4002,eth0
      2 chp	options qeth qeth_options=noauto,0x4000,0x4001,0x4002,eth0,0x5000,0x5001,0x5002,eth1

back to top

Relating Interface with OSA card and TP parameters (SuSE)

SuSE initializes the network with the "/sbin/init.d/network" script.

It needs to tie together four things:

1) The module name
2) The module parameters
3) The device addresses
4) The TCP/IP parameters

The interface name from the IFCONFIG command ("eth0", "tr0", etc) is used as the common thread (as in "train of thought").

• /etc/modules.conf
  • "alias eth0 lcs" ties the interface name with the module name
  • "options lcs" ties the device address and card options with the module name
• /etc/rc.config
  • NETCONFIG="_0 _1" defines the subscripts for TCP/IP devices (in this case 2 "_0" and "_1")
  • IPADDR specifies the IP address to use for each interface
    • IPADDR_0="9.112.50.115"
    • IPADDR_1="10.32.56.119"
  • NETDEV assigns the interface (or net device) names
    • NETDEV_0="tr0"
    • NETDEV_1="eth0"
  • IFCONFIG then specifies the parameters for the "ifconfig" command
    • IFCONFIG_0="9.112.50.115 broadcast 9.112.50.255 netmask 255.255.255.0 up"
    • IFCONFIG_1="10.32.56.119 broadcast 10.32.56.255 netmask 255.255.255.0 up"

  back to top

HCD and IOCDS considerations

Generally, I've coded 16 addresses for each OSA and the last entry at xFE must be genned for OSA/SF as "OSAD" to read any of the cards or update the OSA2 cards. This number of addresses may vary according to the number of IP addresses you need for an LPAR - 2 addresses/IP if LCS, 4 addresses/IP if QDIO (Linux developers recommendation - this may be relaxed to 3 addresses in the future). VM may allow 3 addresses/IP if you do not use autosense.   

CARD	IOCDS
OSA2-ENTR OSA2-FE	CHPID PATH=(00),SHARED,                                        *                PARTITION=((LPAR1,LPAR2)),TYPE=OSA           CNTLUNIT CUNUMBR=2000,PATH=00),UNIT=OSA           IODEVICE ADDRESS=(2000,016),CUNUMBR=(2000),UNIT=OSA           IODEVICE ADDRESS=(20FE,001),CUNUMBR=(2000),UNIT=OSAD
OSA-FEX LCS	CHPID PATH=(FD),SHARED,                                        *                PARTITION=((LPAR1,LPAR2)),TYPE=OSE           CNTLUNIT CUNUMBR=4000,PATH=(FD),UNIT=OSA           IODEVICE ADDRESS=(4000,016),CUNUMBR=(4000),UNIT=OSA           IODEVICE ADDRESS=(40FE,001),CUNUMBR=(4000),UNIT=OSAD
OSA-FEX QDIO	CHPID PATH=(FE),SHARED,                                        *                PARTITION=((LPAR1,LPAR2)),TYPE=OSD           CNTLUNIT CUNUMBR=5000,PATH=(FE),UNIT=OSA           IODEVICE ADDRESS=(5000,016),CUNUMBR=(5000),UNIT=OSA           IODEVICE ADDRESS=(50FE,001),CUNUMBR=(5000),UNIT=OSAD
OSA-GBX QDIO	CHPID PATH=(FF),SHARED,                                        *                PARTITION=((LPAR1,LPAR2)),TYPE=OSD           CNTLUNIT CUNUMBR=6000,PATH=(FF),UNIT=OSA           IODEVICE ADDRESS=(6000,016),CUNUMBR=(6000),UNIT=OSA           IODEVICE ADDRESS=(60FE,001),CUNUMBR=(6000),UNIT=OSAD

back to top

Native IPL

If you IPL native, you need your parmline to have an ipldelay of at least 3 minutes for the OSA cards to stabilize. If you IPL as an LPAR, this is not necessary. Remember to do a "silo" if you change any of these parameters.

"/boot/parmline" or "/boot/parmfile"

root=/dev/dasda1 noinitrd dasd=2000-201f ctc=0,0xfa48,0xfa49,escon0 mem=2048M ipldelay=3m

Note: "ctc", "mem" , and "ipldelay" are optional and I've coded them for reference.

back to top

Operational Notes

• Once an OSA has hung in an LPAR, you have to re-ipl that LPAR.
• Sometimes, the IPL will NOT fix the hang, so you must reset the OSA card CHP.
  • Caution: Resetting an OSA card or configuring the CHPID off/on may cause the Linux LPARS with OSA card genned to hang, whether you have specified it in /etc/modules.conf or not!
  • In most shops, this is handled by Field Support for the Processor. Some shops allow the systems programmer to do this procedure.
  • First, you must know the CHPID for the OSA card you want to reset
  • OSA cards can be displayed and reset from the HCD
    • Click on "Defined CPC's"
    • Highlight the CPC you want
    • Click on "Single Object Operations"
    • Click on "CPC"
    • right click on CPC to get CHPID's
    • Use "Advanced Facilities" to view, set the parameter on OSA2 cards and reset the cards
      • All cards should be full duplex
      • OSA2-ENTR - 10 Mbs
      • All others 100 Mbs
    • After you have finished in "Advanced Facilities", toggle the CHPID's off, then on under "Configure ON/OFF"
    • log out of "Single Object Operations" when you're done.
• Certain levels of the OCO drivers (2.2.16, May 2001), may not reboot correctly (shutdown -r). The system will shutdown, almost complete the IPL process, then hand in a multi-cp loop while trying to process the OSA card. A complete shutdown (shutdown -h now) and an ipl (ipl clear) will avoid this problem.
• Older levels of the HCD do not have the option to reset the OSA2 card to its default settings. We would recommend that, if you can, reset the card to default settings before you remove the card from the processor. This will simplify later installs for that card, especially if you do not have OSA/SF available..

back to top

References

• IBM Red Books (Search for "OSA")
• IBM Enterprise Library (Search for "OSA")
• S/390 VM OSA/SF web page

• Jim Sibley (jlsibley@us.ibm.com)
• Daiki Shimizu (Daili Shimizu/Japan/IBM@IBMJP
• Cristoph Arenz (arenz@de.ibm.com)

back to top