# Installing the eeE8 DDI8 Driver for UnixWare* 7 and Open UNIX* 8

- Installation

- Configuration, Teaming, and VLANs

- Technical Notes and Troubleshooting

- Contact Information

| NOTE: The instructions and documentation below reflect the process using the console mode, rather than a graphical interface. X Windows* commands may be worded somewhat differently, but the process for installation and configuration is fundamentally the same.

When installing and configuring network adapters, you may need to refer to Caldera* (formerly SCO*) UnixWare* 7 or Open UNIX 8 documentation. Have this documentation available during the installation process.

For maximum system stability, it is recommended that all network adapters be configured with the same driver type (eeE8 (DDI8)). To use the teaming and VLAN features of your Intel adapter, you need to install DDI8 on all adapters that use Intel® Advanced Networking Services (ANS).

## Installation

To ensure predictable behavior when installing network adapters, deconfigure all adapters and reboot the system prior to configuring new adapters or reconfiguring installed adapters.

1. Mount the CDROM on the UnixWare system. For example:

   > mkdir cdrom
   > chmod 777 cdrom
   > mount -F cdfs -r /dev/cdrom/cdrom1 /cdrom

2. Copy the /<cdrommount>/UNIX/UW7DDI8/eeE8.pkg file from the CDROM into any directory on the system, such as in the /tmp directory.

3. Make sure no other users are logged on and all user applications are closed.

4. Run netcfg and remove any configuration using the older drivers. Then, exit netcfg.

5. Verify that there is not an older version of either eeE or eeE8 driver on the system using ' pkginfo -l'. For example: pkginfo -l eeE. If there is, remove the driver using 'pkgrm'. For example: pkgrm eeE. Reboot the system using 'init 6'.

6. Install the new driver using 'pkgadd'. For example: pkgadd -d /tmp/eeE8.pkg.

7. If you intend to use advanced options such as teaming/VLAN, repeat steps 1 and 4 above to add the iANS.pkg file to your system.

8. If you install the iANS.pkg, reboot your system after installation, before beginning to configure your adapters.

9. Add and configure the NICs. See Configuration and Teaming below to add and configure adapters, configure ANS, and set up teaming and VLANs.

#### Notes:

- Hot Plug enabled systems require the eeE8 (DDI8) drivers. If you are not using ANS, these drivers must only be configured using the Hot Plug utility. For example: scoadmin hot. If you are using iANS, see the section Using Hot Plug and iANS.

- PCI Hot Plug capabilities also require a Hot Plug Controller driver and the appropriate PCI Hot Plug ptfs.

- Systems not requiring PCI Hot Plug capabilities, but wishing to use eeE8 (DDI8) drivers, should use 'netcfg'.

- Use 'netcfg' to configure eeE8 (DDI8) drivers, not 'scoadmin network'.

## Configuration, Teaming, and VLANs

Creating a team with ANS in UnixWare* 7 and Open UNIX 8 is a two-step process. The first step involves adding members to the team, during which process the team's configuration is established. During the second step, you add a virtual adapter (or several virtual adapters in VLAN mode) and choose the networking protocol you wish to use on the virtual adapter(s).

1. Copy the iANS.pkg file to any directory on the UnixWare and Open UNIX system.

2. Install ANS using 'pkgadd'. For example: #pkgadd -d /tmp/iANS.pkg. After installing ANS, reboot your system.

3. If the base driver (DDI eeE8) has not yet been installed, follow the instructions in Installation to install the driver. Make sure that all network adapters in the system are configured with the same driver type and that no protocols are configured on the adapters.

4. Start netcfg: # netcfg.

5. Netcfg presents the available adapters and drivers. Select the first adapter to configure, then select Continue.

   | NOTE: If you also have the DDI7 (eeE) driver on your system, it appears as a configuration option. Make sure you select only the adapters displaying the eeE8 driver.

6. Configure Advanced Options if you wish to set a default duplex speed (Half Duplex_10Mbps, Full Duplex_10Mbps, Half Duplex_10Mbps, or Full Duplex_100Mbps). For automatic duplexing, select OK.

   | NOTE: "Auto_Auto" is the default for this setting. Select Auto_Auto, not Cancel, to leave this screen without making changes. Cancel is nonfunctional in this screen, and may leave Netcfg in an unusable state if it is selected.

7. Select the initial protocol for this adapter. For teaming or VLAN operation, select ANS team "0", where "0" is the team ID given to the first team, then Add.

   | NOTE: You must select ANS before you select any other protocol in order to use this adapter as part of a virtual adapter. You are able to configure TCP/IP or IPX/SPX on the virtual adapter after you have successfully configured ANS on your adapters.

8. Choose the priority of the member just added, then select OK. Priority determines the manner in which the different NICs handle traffic or errors. The adapter is configured with the chosen settings.

9. The ANS Configuration Utility window prompts you to choose the role of the member in the team. The user determines the team configuration only once, during the addition of the first team member. For teaming configurations, see below.

10. You may now configure a second adapter from the Hardware menu by selecting Add New LAN Adapter. Repeat steps 5-9. If you are configuring more than one team, you must select another team ID for subsequent teams.

11. Add the virtual adapter(s) created for the team(s). Select the same protocol for all adapters you add to a team. Instructions for adding a virtual adapter are included below.

### Teaming

1. After adding the first member of the team, the ANS Configuration Utility screen appears. Identify the team, and select VLAN mode if you wish to include VLANs in your team. If you commit the team in "no VLAN" mode, you still must add a virtual adapter (Ethernet-ANS Virtual Adapter without Vlan-PCI Slot 0 Bus 0) and select a protocol to be installed over it. Instructions for adding a virtual adapter are included below.

2. Enter the number of VLANs that will be added.

3. Select the teaming mode for the team being configured (none, alb, aft, fec/la/802.3ad, or gec/la/802.3ad).

   | NOTE: Adaptive Load Balancing (ALB) encompasses Adapter Fault Tolerance (AFT) as well.

4. Some users may wish to configure Advanced Options for a team. Most users do not need to configure advanced options. Unless necessary, do not modify these settings. Configuration options include:

   | NOTE: one tick = 50 milliseconds

- Probe mode (on/off) - Determines whether ANS probes are used or not.

- Probes addr mode (Broadcast / Multicast) - Determines whether ANS probes will be sent as broadcast or multicast packets.

- Check timeout (ticks) - After sending a probe, n ticks expires before the system verifies that the probe was sent to the other members.

- Send time (ticks) - Interval time between probes.

- Max retry count - Number of probe retry bursts to transmit if the first probe was not received by team members.

- Receive timeout (ticks) - After sending a probe retry burst, n ticks expires before checking to see if the retry probe was transmitted to other team members.

- Rx back cycles - Number of probe back cycles to check (determines if probes from previous cycles are valid).

- Balance interval (ticks) - Refresh time of load balancing tables (ALB, FEC).

### Adding virtual adapters

Follow the steps below to add a virtual adapter.

1. In netcfg, select Add New LAN Adapter from the Hardware menu.

2. The virtual adapter created in the previous steps will be highlighted. For example: Ethernet-ANS Virtual Adapter with VLAN - PCI Slot 0 Bus 1. If several virtual adapters were added in the previous installation stage, they will appear with sequential bus numbers. The slot number increases for virtual adapters on different teams.

3. Choose the networking protocol to be installed on the virtual adapter.

   | NOTE: Although ANS is presented by netcfg as an available protocol, ANS cannot be installed and configured over itself on a virtual adapter.

### Examples

These examples summarize the actions needed to build two typical topologies:

For a team without VLAN and with several members:

1. Add the first member using Add New LAN Adapter.

2. Choose teaming modes as desired, and set VLAN mode to "off".

3. Add all the other members in the same manner.

4. Add the virtual adapter using Add New LAN Adapter.

For a team with VLAN and with several members:

1. Add the first member using Add New LAN Adapter.

2. Choose teaming modes as desired, and set VLAN mode to "on".

3. Enter the number of VLAN IDs you will use.

4. Add all the other members in the same manner.

5. Add the virtual adapters using Add New LAN Adapter from the Hardware menu. For each adapter added you are prompted for the virtual adapter's VLAN ID. Do not give two virtual adapters in the same team the same VLAN ID.

### Remarks

- Viewing a team's configuration can be done when selecting the team's protocol, and choosing Protocol > View protocol configuration from the menu.

- Modifying a team's configuration can be done when selecting the team's protocol, and by choosing Protocol > Modify protocol configuration from the menu. Some modifications require a reboot; others do not. See Modifying a Team's Configuration for further information about dynamic configuration options.

- Viewing a virtual adapter can be done when selecting the virtual adapter hardware, and choosing Hardware > View hardware configuration.

- Modifying a virtual adapter with VLAN can be done when selecting the virtual adapter hardware, and choosing Hardware > Modify hardware configuration. This allows you to change the virtual adapter's VLAN ID.

## Technical Notes and Troubleshooting

- Installation Issues

- Using PCI Hot Plug and ANS

- Modifying a Team's Configuration

- ANS Usage Guidelines

- Known Issues

### Installation issues

- ANS should only be installed over Intel’s DDI8 driver:
  Add an ANS team as a protocol over Intel’s DDI8 driver (PRO/100+ (eeE8)). Trying to add it over a different driver is possible in netcfg, but ANS will fail to work.

- Don’t add a protocol to an adapter that already has ANS configured above it:
  After adding an ANS team as a protocol over an adapter, netcfg will allow adding other protocols over that adapter. This should be avoided as unexpected results may occur (the system will not “know” which of the 2 protocols to use).

- It is recommended that you set configurations using the graphic interface (in an X environment). There are some limitations when configuring using the console-mode.

- It is recommended that you do not select "Cancel" when in the process of adding an adapter. To prevent Netcfg from hanging and to avoid rebooting, continue installing the adapter. After it's installed, manually remove it.

- Tunable parameters:
  When the ANS package is installed, two system tunable parameters are altered. This is needed in order to support the maximum number of VLANs that can be configured in ANS (64 VLANs).

  Parameter | Default value | Value set when ANS is installed
  FMOD_RESERVE | 50 | 114
  SFNOLIM | 64 | 172

  After removing the ANS package, these parameters retain the values set by ANS. This shouldn’t affect the user, but the parameters can nevertheless be restored to their default values or to any other value using the SCOAdmin System Tuner or using the “idtune” command. For more information, use “man idtune”.

### Using Hot Plug and ANS

#### Requirements:

- You need a Hot Plug controller driver, such as intelhped for Intel Hot Plug Controller, and the latest Unixware 7.1.x or Open UNIX 8 Hot Plug patch. For 7.1.1, use ptf7604c, and for 7.1.0, use ptf7428d or later.

- If you already have the above packages installed, remove them before installing new drivers.

- Install the necessary drivers (DDI8 drivers for adapters and ANS).

- Reboot the system.

- Configure your initial network configuration.

- Install the Hot Plug controller driver and the Unixware/Open UNIX Hot Plug patch.

- Reboot to rebuild the kernel.

#### Using Hot Plug with ANS Team Members:

- Set normal team configurations using netcfg.

- To replace a member adapter: remove it from the team using netcfg, power off the slot using the SCOAdmin Hot Plug manager, remove the adapter, insert the new adapter, power on the slot using the SCOAdmin Hot Plug manager, and add the adapter to the team using netcfg.

### Modifying a Team's Configuration

Most changes to an active team can be accomplished dynamically, without rebooting after the change. Other topology changes require a reboot. View the following list to determine whether your change requires that your system be rebooted before the change takes affect.

Changes that do not require reboot:

- Create a new team

- Add or remove a member to an existing team ("Hot Add")

- Add a virtual adapter to an existing team

- Remove a virtual adapter from an existing team

- Replace a member in an existing team

- Change VLAN ID

- Probe settings (on/off, or other probe parameters)

- Balance interval (in ticks)

Changes that do require reboot:

- Change teaming mode (none, alb, aft, fec/la/802.3ad, or gec/la/802.3ad)

- VLAN on/off

- Probe address mode (broadcast/multicast)

Limitations of dynamic configurations in Hot Plug computers:

- When ANS is used on a computer in which Hot Plug is enabled, the following limitations apply (due to the way netcfg works in these cases):

1. Building a team and adding virtual adapters require a reboot of the computer.

2. In order to configure the IPX/SPX protocol over a virtual adapter, the user should first configure TCP/IP over it, reboot the system, and only then add the IPX/SPX protocol (the TCP/IP protocol can then be removed).

- These limitations can be avoided by temporarily disabling Hot Plug before doing anything that creates virtual adapter resources in netcfg (i.e. creating a new team or adding vlans when modifying a team).

Hot Plug can be temporarily disabled using "modadmin -U <HPCD_MODULE>", where HPCD_MODULE is hpcd/intelhpcd or any other module name used for your Hot Plug Controller Driver.

Hot Plug can be enabled again after configuring all virtual adapters by using "modadmin -l <HPCD_MODULE>.

### ANS Usage Guidelines/Recommendations and Limitations

- Switching init states:
  When the system is in init state 1, do not go directly to init state 3 (using “init 3”). Instead reboot (using “init 6”) and bring the system up to init state 3. Going directly from init state 1 to init state 3 causes some adapters not to open when adding them to netcfg, and therefore hot-add failures might occur.

- Throughput limitations:
  There is a Caldera limitation regarding the maximum TCP/IP throughput of an adapter. Therefore, when using ALB or FEC mode for TCP/IP traffic, we recommend configuring no more than 4 members in a team PER virtual adapter. This is because the virtual adapter’s throughput will not exceed the mentioned Caldera limitation so more members will not improve the total throughput.

  If the configuration has more than one virtual adapter in a team (in VLAN mode), the throughput of each virtual adapter will vary according to the distribution of the traffic between the virtual adapters. As a rule of thumb, traffic that is equally distributed between all virtual adapters allows better average throughput per virtual adapter.

  Examples:

  - If you configure a non-VLAN ALB/FEC team (only one virtual adapter), the team should have a maximum of 4 members.

  - If you configure an ALB/FEC team with 2 VLANs, a maximum of 8 members should be configured, but their utilization depends on the distribution of traffic between the VAdapters.

- Hot-remove in FEC or GEC mode:
  When hot-removing a member in FEC or GEC mode, you should also disconnect the member’s link from the switch. If you fail to do this, the switch will keep transmitting to that port (the switch isn’t aware of the member’s removal from the team).

- Hot-removing a team’s original primary adapter:
  A team’s MAC address is taken from the MAC address of its original primary physical adapter (the adapter that was chosen as primary at boot-time). If this adapter is hot-removed from the team, both the team and the adapter will have the same MAC address and the following warning will be printed to the system log: “WARNING: the MAC Address ------ is still in use by Team _”. In such a case, the removed adapter should not be used anywhere in the network until the system is rebooted, or unexpected results may occur.

- Running /etc/nd stop on a member:
  /etc/nd stop should NOT be executed on a member’s net. If it is executed, the member will become non-operational and you will have to remove it from the team and add it back, or reboot, for the member to become operational again. The following warning will be printed to the system log in this case: “WARNING: Closed member [PCI Slot -, (---)] which is still attached to a team! It must be removed to become operational again.”.

### Known Issues

- “Error in handshake with adapter’s driver: adapter wasn't opened” error message at boot:
  Symptom: At boot time, a member fails to be added to a team, and the following error messages are seen on the screen: "Error in handshake with adapter’s driver: adapter wasn't opened.", "[FAILED]: anscfg cmd=add_lower team=… lower=… net=… low_attr=….".

  Cause: This is a known Caldera bug, in which the adapter’s driver isn’t opened.
  Possible workarounds: The problem usually disappears after a reboot. If it doesn’t you can try the following: Try to add a different member and reboot if necessary. Try removing some members from netcfg and then adding other members. Reboot if necessary. If none of the above helps, try to remove everything from netcfg, reboot and reconfigure everything.

  | NOTE: This error message can also be seen when hot adding a member.

- Hot-add failures with no apparent reason:
  Symptom: Hot adding a member to a team fails. The member will not be configured in netcfg and the user will see the following message: “Hot add of member ___ failed”. Look at the system log. It is possible that the hot-add failed because of a legitimate reason (no server adapter in team, adapter that doesn’t support vlan in a vlan-team, etc). In such cases, a relevant error message will be seen in the log. If there is no message in the log, or if the following message is seen: “Error in handshake with adapter’s driver: adapter wasn't opened”, then this is an anomaly which is a known issue. It happens without ANS as well (in that case, the adapter will be configured in netcfg but will not function).

  Possible workarounds:

  - Try to hot add the same member again (try several times).

  - Try to hot add a different member.

  - Try removing some members from netcfg and then adding other members.

  - Reboot.

  > If none of the above helps, try to remove everything from netcfg, reboot and reconfigure everything.

- Repeated messages of members being deactivated and rejoined in VLAN mode:
  Symptom: the following messages are printed to the system log repeatedly while in VLAN mode: “Secondary adapter ___ deactivated / isolated from other team members.”, "Secondary adapter ___ rejoined.".

  | NOTE: Messages of fail-overs might also appear.

  Cause: VLAN IDs are configured in an ANS team, but some of these IDs are not configured in the switch ports to which the team’s members are connected. This causes ANS probes transmitted on these VLAN IDs to be lost, which causes adapters to be deactivated.

  Solution: Configure all the switch ports to which the team’s members are connected to be 802.3ac/802.1Q VLAN tagged, with all the VLAN IDs corresponding to the ones configured in the team.

- Periodic messages of members being deactivated and rejoined in high-stress traffic:
  Symptom: the following messages are printed to the system log from time to time when transmitting/receiving high-stress traffic: “Secondary adapter ___ deactivated / isolated from other team members.” "Secondary adapter ___ rejoined.".

  | NOTE: Messages of fail-overs might also appear.

  Cause: ANS probes are getting lost.

  Possible workaround: Changing the team’s probe-settings may solve the problem or at least limit it. In netcfg, modify the team’s advanced options (choose team, Protocol -> Modify protocol configuration -> OK -> Advanced options) as follows:

  1. Change “Max retry count” to 20 (causes more probe retries).

  2. Change “Receive timeout” to 1 (less time between each retry).

- All of the ANS configuration commands fail, “Could not open ctrl device” message printed to the ANS log.(Very rare):
  Symptom: At boot time, ANS fails to be configured. The console is filled with [FAIL] messages describing the commands that ANS failed to run. In the ANS log (etc/*ans/data/log), the following messages are also printed: "[***] The last operation failed. Exited with status =…” and “Could not open ctrl device…”.

  Possible workaround:

  1. Remove everything from netcfg.

  2. Re-install the ANS package using “pkgadd –d” (no need to remove the package).

  3. Reboot.

- Error messages concerning a “wrong state” of an adapter’s driver, followed by “OS Configuration error: message received on the wrong queue. Try to remove the network configuration and reconfigure.” Could also cause “Member is not ready for destruction” messages (very rare):
  Symptoms: When booting the system, or when hot-adding a new member, the following messages are printed to the system log: “eeE8open: wrong state! “ and “OS Configuration error: message received on the wrong queue. Try to remove the network configuration and reconfigure.".
  The “problematic” member will not function correctly. Other system abnormalities might also occur. If the user doesn’t reconfigure the network, another problem might occur at shutdown or when removing the last virtual adapter from netcfg. The system can enter an infinite loop, printing the message: “Member is not ready for destruction”.

  Cause: a known Caldera bug preventing an adapter’s driver to be opened correctly, a thing that later causes problems in ANS.

  Workaround: The user should remove the entire network configuration from netcfg and then reconfigure it (as the error message says). If the user doesn’t reconfigure netcfg and enters an infinite loop, he should do so after resetting the system.

## Contact Information (for base driver support)

Caldera Systems, Inc.
240 West Center Street
Orem, Utah 84057 USA
801-765-4999 phone
801-765-1313 fax

http://www.caldera.com/

1. Go to Support

2. Go to "Installation Knowledge Center"

3. Search for "Intel gigabit" in search box

4. Select "AND keywords"

For Teaming ANS/VLAN support use the following email address:

unixteam.nic@intel.com

Copyright © 2002 Intel Corporation. Legal Information