PXE boot over routed network

I've bumped into interesting problem with PXE boot, while TFTP server resides on another routed (not NAT'ed) sub-net.

Setup:

Dell PowerEdge 640/740 servers with Intel x710 10g network cards.
Two routed networks: 10.0.1.0/24, 10.0.2.0/24.
Physical servers and DHCP server resides on network segment 10.0.1.0/24.
TFTP server resides on network segment 10.0.2.0/24.
TFTP server can be accessed from network 10.0.1.0/24. Files can be downloaded.

DHCP config:

default-lease-time 300;
max-lease-time 600;
allow booting;
allow bootp;
next-server 10.0.2.10;
subnet 10.0.1.0 netmask 255.255.255.0 { range 10.0.1.50 10.0.1.90; option subnet-mask 255.255.255.0; option routers 10.0.1.254; filename "bootx64.efi";
}

Problem:

Server network card boot message displays:

>> Start PXE over IPV4
Station IP address is 10.0.1.50
NBP filename is bootx64.efi
NBP filesize is 0 Bytes
PXE-E99: Unexpected network error

It looks like network card does not set default gateway address properly.
Tcpdump from DHCP server:

15:42:12.799841 IP (tos 0x0, ttl 64, id 30339, offset 0, flags [none], proto UDP (17), length 387) 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx, length 359, xid 0xd4d2c098, Flags [Broadcast] Client-Ethernet-Address xx:xx:xx:xx:xx:xx Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Request Server-ID Option 54, length 4: 10.0.1.1 Requested-IP Option 50, length 4: 10.0.1.50 MSZ Option 57, length 2: 65280 Parameter-Request Option 55, length 35: Subnet-Mask, Time-Zone, Default-Gateway, Time-Server IEN-Name-Server, Domain-Name-Server, Hostname, BS Domain-Name, RP, EP, RSZ TTL, BR, YD, YS NTP, Vendor-Option, Requested-IP, Lease-Time Server-ID, RN, RB, Vendor-Class TFTP, BF, GUID, Option 128 Option 129, Option 130, Option 131, Option 132 Option 133, Option 134, Option 135 GUID Option 97, length 17: 0.68.69.76.76.81.0.16.54.128.82.200.192.79.81.88.50 NDI Option 94, length 3: 1.3.16 ARCH Option 93, length 2: 7 Vendor-Class Option 60, length 32: "PXEClient:Arch:00007:UNDI:003016"
15:42:12.800819 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 10.0.1.1.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 300, xid 0xd4d2c098, Flags [Broadcast] Your-IP 10.0.1.50 Server-IP 10.0.2.10 Client-Ethernet-Address xx:xx:xx:xx:xx:xx file "bootx64.efi"[|bootp]

Tried running Linux live CD on one of the servers. Linux dhclient is able to receive default route from DHCP server.
So its either a DHCP misconfiguration or missing capabilities on network card.
Is it even possible to use PXE boot over routed network?
Thank you.

1 Answer

First, please doublecheck that routers are actually sent by DHCP server - I do not see them in the trace which you included (but you might have cut it).

If they are sent this looks like the issue with UEFI network stack which is a part of the platform BIOS, not an option rom on the NIC itself. The NIC in this case only delivers UEFI UNDI driver which is just shuffling ethernet frames between host and the NIC.

You can try to use different card which also provides UEFI UNDI driver to check if the issue still persists.

Please check if your motherboard vendor has updated BIOS available.