irdma RDMA FreeBSD* driver for Intel(R) Ethernet Controller E810
================================================================
May 12, 2021

Contents
========

- Prerequisites
- Building and Installation
- Testing
- Configuration
- Interoperability
- Known Issues
- Support

================================================================================


Prerequisites
-------------

- FreeBSD version 11.4 or 12.1, 13.0 or later.
- Kernel configuration:
    Please add the following kernel configuration options:
        include GENERIC
        options OFED
        options OFED_DEBUG_INIT
        options COMPAT_LINUXKPI
        options SDP
        options IPOIB_CM

        nodevice ice
- For the irdma driver to work, an if_ice module with RDMA interface
  is required. The interface is available in if_ice version 0.28.2 or later.
  The RDMA interface may be turned on or off by using tunable of if_ice module:
    hw.ice.irdma
  It may be modified by putting:
    hw.ice.irdma=1
  to /boot/loader.conf file. Reboot is needed for the change to take effect.
  The RDMA interface is turned on by default (the value is 1).

Building and Installation
-------------------------

1. Untar ice-<version>.tar.gz and irdma-<version>.tar.gz
    tar -xf ice-<version>.tar.gz
    tar -xf irdma-<version>.tar.gz
2. Install the if_ice driver:
    cd ice-<version>/ directory
    make
    make install
3. Install the irdma driver:
    cd irdma-<version>/src/
    make clean
    make ICE_DIR=$PATH_TO_ICE/ice-<version>/
    make install

Testing
-------
1. To load the irdma driver, call:
     kldload irdma
   If if_ice is not already loaded, the system will load it on its own.
   Please check whether the value of
     sysctl hw.ice.irdma
   is 1, in case of irdma driver not loading. To change the value put:
     hw.ice.irdma=1
   to /boot/loader.conf file and reboot.
2. To validate the load of the driver, check:
     sysctl -a | grep infiniband
   A number of sys.class.infiniband should appear, provided at least one
   port of the E810 is up.
3. Each interface of the card may work in either iWARP or RoCEv2 mode.
   The iWARP mode is the default. To enable RoCEv2 compatibility, add:
     dev.irdma<interface_number>.roce_enable=1
   where <interface_number> is a desired ice interface number on which
   RoCEv2 protocol needs to be enabled to file:
     /boot/loader.conf

   for instance:
     dev.irdma0.roce_enable=0
     dev.irdma1.roce_enable=1
   will keep iWARP mode on ice0, and enable RoCEv2 mode on interface ice1.

   To check irdma roce status execute command:
     sysctl dev.irdma<interface_number>.roce_enable
   for instance:
     sysctl dev.irdma2.roce_enable
   with returned value of '0' indicate the iWARP mode, and the value of '1'
   indicate the RoCEv2 mode.

   Note: An interface configured in one mode will not be able to connect
   to a node configured in another mode.
   Note: RoCEv2 has currently limited support, for functional testing only.
   DCB and Priority Flow Controller (PFC) are not currently supported which
   may lead to significant performance loss or connectivity issues.
4. Enable flow control in the ice driver:
     sysctl dev.ice.<interface_num>.fc=3
   Enable flow control on the switch your system is connected to. See your
   switch documentation for details.
5. The source code for krping software is provided with the kernel in
   /usr/src/sys/contrib/rdma/krping/. To compile the software, change
   directory to /usr/src/sys/modules/rdma/krping/ and invoke the following:
     make clean
     make
     make install
6. Start krping server on one machine:
    echo size=64,count=1,port=6601,addr=100.0.0.189,server > /dev/krping
7. Connect client from another machine:
    echo size=64,count=1,port=6601,addr=100.0.0.189,client > /dev/krping

==============================================================================


Configuration
-------------

The following sysctl options are visible:
- hw.irdma.debug
    defines level of debug messages.
    Typical value: 1 for errors only, 0x7fffffff for full debug.
- dev.irdma<interface_number>.roce_enable
    enables RoCEv2 protocol usage on <interface_numer> interface.
    By default iWARP protocol is used.

Interoperability
----------------

Known Issues
------------
- The krping is unable to bind to an address belonging to vLAN interface.
  This appears to be a problem in rdma_copy_addr of ib_addr.c

Support
-------
For general information, go to the Intel support website at:
www.intel.com/support/

or the Intel Wired Networking project hosted by Sourceforge at:
http://sourceforge.net/projects/e1000

If an issue is identified with the released source code on a supported
kernel with a supported adapter, email the specific information related to the
issue to e1000-rdma@lists.sourceforge.net



================================================================================


License
-------

This software is available to you under a choice of one of two
licenses. You may choose to be licensed under the terms of the GNU
General Public License (GPL) Version 2, available from the file
COPYING in the main directory of this source tree, or the
OpenFabrics.org BSD license below:

  Redistribution and use in source and binary forms, with or
  without modification, are permitted provided that the following
  conditions are met:

  - Redistributions of source code must retain the above
    copyright notice, this list of conditions and the following
    disclaimer.

  - Redistributions in binary form must reproduce the above
    copyright notice, this list of conditions and the following
    disclaimer in the documentation and/or other materials
    provided with the distribution.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================================================


Trademarks
----------

Intel and Itanium are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and/or other countries.

* Other names and brands may be claimed as the property of others.


