Automatically provisioning baremetal/VM server

During my endavours I had a situation where I had to provision 10 servers (install all of them manually and configure same things on all of them, same files etc).

There is a nice alternative to it called Stacki from StackIQ (bought by Teradata last year). What it offers is specialized PXE server that is used to boot baremetal/VM servers (CentOS/Redhat/Ubuntu). 

Its architecture look as follows (Stacki server == Frontend, server to be provisioned == backend):

Firstly in CSV file you prepare a list of hosts with their MACs and as a next step you add puppet that will be used to provision the servers after booting.

More can be found here: 

https://github.com/Teradata/stacki

Frontend machine can be a VM – actually it worked pretty nice – tested with provisioning other VMs.

Kill a hanging machine in VMware ESXi

It might not be possible to kill a “running” (in reality hanging) VM from Vcenter GUI or using vSphere Client. 

In such situation a login via SSH to ESXi host is needed, list all VMs and kill the appropriate one.

Connecting over SSH and running a command over NETCONF

When HW/virtualized/containerized network element offers NETCONF interface to manage, it is extremely beneficial to use it for repetitive tasks (upgrades, sanity checking, route table checking etc).

NETCONF can be used over different transports as below:

In case SSH is used then it must be made sure that SSH subsystem is enabled in SSH config on a device. 

If a YANG model is available then TailF offers a client and Java class generator under:

https://github.com/tail-f-systems/JNC

Alternatively, as in the example below, Python can be used to manually connect over SSH and based on YANG model (if used) instruct the device to perform a specific task.

There is a library in Python for that purpose called ncclient (NETCONF client): 

https://github.com/ncclient/ncclient

Firstly we need to do the proper import in our client script after installing the library:

Let’s define method used for connecting to the device:

Create an object class that inherits from RPC class of ncclient library and define a method that will be compose an XML NETCONF message based on YANG model:

Connect to the device and perform requested action (probably not the safest way to use clear text password to connect):

The request in NETCONF formatted XML would look as follows:

And a corresponding YANG model:

Lastly in XML:

Juniper Contrail – how to check Next-Hop

One might need to check which MPLSoGRE/UDP/VXLAN tunnel is used to reach a specific prefix on a vrouter.

There are few steps needed to find it out after logging in to a compute node running a VM/contrainer with an interface of interest (its prefix or prefix reachable via it) :

    1. List all interfaces and see which vrouter VRF a specific interface maps to:

    2. Use the VRF id to dump routes in a routing table for this VRF and check a prefix in question and its next-hop id:

    3. Check next-hop id parameters with:

Inter-AS Option B between Juniper routers

Recently I stumbled on a topic related to interconnecting 2 Juniper routers with Inter-AS Option B. 

With this kind of connectivity it is not enough to have control plane working properly i.e. prefixes exchanged and visible in appropriate routing-instances on both ends – for data plane to work you need to have next-hop resolved – and it is implicit in case E-BGP peering is sourced with interface address. In case loopback is used as a source you need to have it resolved by:

  • either a LSP between the two ASBR (actually no label required, but it must appear in inet.3, it can even be a static dummy LSP)
  • or configure routing-options resolution rib bgp.l3vpn.0 resolution-ribs inet.0 which will allow L3VPN routes to be resolved in inet.0 instead of inet.3

More details can be found on the Juniper forum where I found the solution:

https://forums.juniper.net/t5/Routing/interprovider-l3vpn-option-B/td-p/255305

 

There are also 2 useful commands that can be used for troubleshooting next-hop connectivity

Heat templates with cinder volumes

While trying to launch several VMs using heat templates on RHEL OpenStack Liberty only 2 out of 8 launched. Rest failed because cinder volumes could not be created. 

heat stack status was CREATE_FAILED and cinder volume status was error

/var/log/cinder/volume.log

For a brief moment there was some log on a console related to haproxy failure

While checking the services I noticed that both haproxy and swift-proxy are not running so just decided to restart them

systemctl restart haproxy.service

systemctl start openstack-swift-proxy.service

After that I could again create cinder volumes via heat templates. So be careful – create VMs from templates with cinder one by one

SDN course for free on coursera.org

There is a great SDN introductory course available on coursera.org run by Dr. Nick Feamster from Princetone University. Enroll on current session – but in order to get a certificate of course completion tasks must be followed weekly for 8 weeks in a row. Now a 3rd week has just started so there is still some space to catch up as later on it might be too late.

Configure replace from terminal on IOS/IOS-XE

Today I stumbled on a problem that I wanted to replace VPN names in my whole config on CSR1000v. 

Instruction on how to do it I’ve found on invaluable Ivan Pepelnjak’s blog:

http://blog.ipspace.net/2008/01/copy-text-files-into-router-flash.html

Basically I created a file in Notepad++, replaced what I wanted then I copied the new content to new file on flash via ssh session using the instruction in the link provided above e.g.

Afterwards I just replaced current config with the one in file: 

 And voila!