Posts

But Ansible does it all!!

Most people running networks have a decent fleet of Cisco IOS devices (or devices that feature "industry-standard" *cough* ripoff *cough* CLI). Often times, the discussion of Ansible as a framework comes up as the solution. What does Ansible do? Ansible is designed to express desired state, compare against existing state and then calculate whether a change is needed. A great example is that we'll generate a config file with a template, push this to a server, and then signal the relevant service (perhaps with a restart). Cool! Easy! But what if the config file is already the same as what we generated from template? No problem, we'll calculate that the files are the same and do nothing. No restart, no problem.. its already good. This technique is highly effective for things like webservers where the service supports a consistent and suitable approach (in context, of course) for loading the new config. kk.. cool but what is wrong with IOS? IOS-like CLIs ar...

Network Device Authentication

A lot of people outside of the network space are quite surprised to find out that network devices don't always support LDAP. There are convenient reasons for this. Doing Authentication Most devices will support RADIUS with varied levels of "support". For others, they will support TACACS+ as well as RADIUS. Finally, a subset of devices will support LDAP. RADIUS You have to map in the dictionary for the appropriate attributes. Example, Juniper, Cisco, etc have their own special attributes. FreeRadius keeps a good list, but it may require occasional additions for other vendors TACACS+ Is a pile of shit, but a convenient pile of shit for cisco environments. Most people use shrubbery.net's tac_plus or Cisco ACS. ACS is well known for being horrible at the worst times. Also, building your own TACACS+ box means it will last until the apocalypse unlike ACS. Why not just LDAP everywhere? Three easy reasons:  LDAP is usually maintained by someone els...

Big changes

So, you wanna begin in automation? Begin with your changes and how you develop and test. The biggest mistake network admins regularly make is to merely "think" about problems and then develop solutions by "thinking" and committee. The figuring is, it will all work if we just think enough about it. This is nonsense. You will make mistakes relying on eyes and perfection. Let machines do their thing. Change analysis I have a need to sling some routes to another part of the network. Will it break stuff? By some work in determining the scope of the proposed changes, one can note the assumptions that take place in the change. Example, I will be doing redistribution from BGP into OSPF and I assume that all prefixes from BGP from BGP today originate on two routers. How can I verify? Another (crappy) example, I believe RIP route distribution contains no filtering anywhere. How can this be checked? Decomposing the change and verifying state Consider a situa...

FortiManager TCL scripts

I had a request sent to me the other day to make a script push some config with the hostname of the node included. Pretty basic thing to do. I was further constrained to use the FortiManager script runner to make this work. As usual, CLI-like scripting sucks and the magic behind the scenes made things more difficult. FortiManager's Role Big HA pile of shit to sync configs with templates. Also has a script runner (used by the config pushes). The scripts are handled through a component called "deployment manager". Relevant debugs: diagnose debug application depmanager -1 -> Not so useful for TCL, good under-the-hood look diagnose debug dpm conf-trace enable -> Will show you SSH connection stuff. More useful IMO. FortiManager holds SSH credentials for the device. It isn't like, even though FortiManager is trusted to play with config, FortiManager can magically login via SSH. This seems like quite a stupid design, but that is the situation we'...

Tooling against proprietary CLIs

Proprietary CLIs are often considered the "normal" way of interacting with IP network devices. These CLIs allow for bad design with respect to automation. CLI transport methods and notes Vendors regularly shit the bed with their CLI behaviors. Some of the components worth looking at are transport, authentication, the TTY, and error handling. Transport Some things only do telnet? In 2017?! Meh, yes, some. Console You need a commserver. Better to write everything against this than trying to mess with shell scripts. Problem is: you have to adjust settings carefully for the serial link to work well. Console is the most manual idiotic shit to write for. It should be treated as such, used for basic provisioning tasks only. SSH This is usually what we want. Two basic modes of operation: pty allocation or an "exec" type. In PTY, shitty expect-like garbage will be necessary. In exec, you have the POSSIBILITY that return codes actually come back and d...

Linux unnumbered interface source IP behavior

In certain cases, (looking at you, Fortinet), interfaces make more sense without IP addresses. But the kernel must determine which source IP to use for sockets that don't specify one. Linux maintains a scoping of IP addresses. Those exposed directly with iproute2 are host, link, and global. When you intend on taking an unnumbered path, one of two things may happen: 1) A route exists for the unnumbered path. This route is populated with a src address when fib_create_info() is called  (such as in routing table update) (occurs via  fib_info_update_nh_saddr) or; 2) No route exists, but an interface is specified for the connection and the source is derived in ip_route_output_key_hash_rcu(). In both cases, inet_select_addr() is called to determine the source IP, but with slightly different parameters. In the first case, the scope of the FIB table is taken into account. As an example, sources used for link-local routes will permit link-local or global scope. In the seco...