Linux unnumbered interface source IP behavior

In certain cases, (looking at you, Fortinet), interfaces make more sense without IP addresses. But the kernel must determine which source IP to use for sockets that don't specify one.

Linux maintains a scoping of IP addresses. Those exposed directly with iproute2 are host, link, and global.

When you intend on taking an unnumbered path, one of two things may happen:

1) A route exists for the unnumbered path. This route is populated with a src address when fib_create_info() is called  (such as in routing table update) (occurs via  fib_info_update_nh_saddr) or;

2) No route exists, but an interface is specified for the connection and the source is derived in ip_route_output_key_hash_rcu().

In both cases, inet_select_addr() is called to determine the source IP, but with slightly different parameters.

In the first case, the scope of the FIB table is taken into account. As an example, sources used for link-local routes will permit link-local or global scope.

In the second case, it will always allow link-local or better.

The way the address is selected is through a walk of the device list. This means that the first address on the first device (in ifindex order) that meets the scope will be selected.

Example from iproute2:
~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:ce:9e:1e brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.109/24 brd 10.10.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fece:9e1e/64 scope link 
       valid_lft forever preferred_lft forever
298: br-clouda: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 56:1d:96:29:0b:ab brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2c0b:e0ff:fe23:f59c/64 scope link 
       valid_lft forever preferred_lft forever
299: br-cloudb: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 36:27:e9:4b:3a:96 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8cc5:d4ff:fe2c:fe0b/64 scope link 

       valid_lft forever preferred_lft forever

lo is ALWAYS the first interface. Notice the number next to the interface? That is the internal ifindex. If we assign a global scope address to lo, it will be used for unnumbered interface source.

If you need to specify the source directly and you're using routes, there is a preferred source that can be attached to the route. For example:

ip route add 10.10.0.90/32 dev ens3 src 10.10.0.5

This would achieve the result of setting the source. Note: this source has to be bound to an interface to work!

Why the note on Fortinet? Well, Fortinet runs on linux kernel, but its configuration does not support directly attaching IPs to the real loopback (lo). So, in order for unnumbered tunnel interfaces to get the correct source IP for traffic, you either need a) luck with respect to the ifindex or 2) to manually set an IP on the tunnel. Similarly, it does not support preferred source for routes!

The behavior of Fortinet tunnel config is similar to "ip addr add 10.10.0.1/32 peer 10.10.0.2/32 dev tun0". This is unfortunate because the peer address cannot be blank or it errors out.

Comments

Post a Comment

Popular posts from this blog

FortiManager TCL scripts

I had a request sent to me the other day to make a script push some config with the hostname of the node included. Pretty basic thing to do. I was further constrained to use the FortiManager script runner to make this work. As usual, CLI-like scripting sucks and the magic behind the scenes made things more difficult. FortiManager's Role Big HA pile of shit to sync configs with templates. Also has a script runner (used by the config pushes). The scripts are handled through a component called "deployment manager". Relevant debugs: diagnose debug application depmanager -1 -> Not so useful for TCL, good under-the-hood look diagnose debug dpm conf-trace enable -> Will show you SSH connection stuff. More useful IMO. FortiManager holds SSH credentials for the device. It isn't like, even though FortiManager is trusted to play with config, FortiManager can magically login via SSH. This seems like quite a stupid design, but that is the situation we'...
Read more

But Ansible does it all!!

Most people running networks have a decent fleet of Cisco IOS devices (or devices that feature "industry-standard" *cough* ripoff *cough* CLI). Often times, the discussion of Ansible as a framework comes up as the solution. What does Ansible do? Ansible is designed to express desired state, compare against existing state and then calculate whether a change is needed. A great example is that we'll generate a config file with a template, push this to a server, and then signal the relevant service (perhaps with a restart). Cool! Easy! But what if the config file is already the same as what we generated from template? No problem, we'll calculate that the files are the same and do nothing. No restart, no problem.. its already good. This technique is highly effective for things like webservers where the service supports a consistent and suitable approach (in context, of course) for loading the new config. kk.. cool but what is wrong with IOS? IOS-like CLIs ar...
Read more