Ticket: CM-11807
Reviewed By: Roopa, Daniel
Testing Done:
Loading the default template engine (mako) is adding an extra overhead: 100ms
We also import the template engine even if we are dealing with a json input
The overhead is noticable when using NCLU.
With this change we are now important the template engine when a keyword is
found in the configuration file.
Signed-off-by: Julien Fortin <julien@cumulusnetworks.com>
ifupdown2 can invoke scripts under /etc/network/if-*.d/*
in the required order with the required environment variables.
This patch includes fixes to execute these scripts.
The following attributes in /etc/network/ifupdown2/ifupdown.conf
can influence the execution behaviour of python-addon modules
and /etc/network/if-*.d/*
<ifupdown.conf>
addon_syntax_check=0
addon_scripts_support=1
addon_python_modules_support=1
</ifupdown.conf>
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
and validation
Ticket: CM-8101
Reviewed By: Roopa
Testing Done:
The goal of this commit is to provide the addons a new way to perform their
own specific syntax checks. By implementing `syntax_check` method.
The addons will have to make sure the provided ifaceobj is valid.
e.g. vxlan.py:syntax_check(ifaceobj) should check:
self._is_vxlan_device(ifaceobj)
Signed-off-by: Julien Fortin <julien@cumulusnetworks.com>
Ticket: None
Reviewed By: Roopa
Testing Done: smoke/testifupdown2.py:TestMakoJson
ifupdown2.TestMakoJson test was failing because the json values werent trimmed
Because of a trailing whitespace, a list of iface had an empty value, ifupdown2
was throwing an exception.
Signed-off-by: Julien Fortin <julien@cumulusnetworks.com>
Ticket: None
Reviewed By: CCR-4692
Testing Done: smoke + scale tests
If called with close_fds=True the subprocess module will try to close every fd
from 3 to MAXFD before executing the specified command. This is done in Python
not even with a C-implementation which truly affecting performances.
This patch aims to better handle the file descriptor used by ifupdown2. Either
by closing them after use or by setting the close-on-exec flag for the file
descriptor, which causes the file descriptor to be automatically
(and atomically) closed when any of the exec-family functions succeed.
With the actual patch all tests are passing, I can't think of any future issue
but if any a possible future modification might be to use the parameter
'preexec_fn', which allows us to set function which will be executed in the
child process before executing the command line. We can always manually close
any remaining open file descriptors with something like:
>>> os.listdir('/proc/self/fd/')
['0', '1', '2', ‘3’, etc..]
>>> for fd in os.listdir('/proc/self/fd/')
>>> if int(fd) > 2:
>>> os.close(fd)
This patch is also totally re-organising the use of subprocesses. By removing
all subprocess code redundancy.
brought up
Ticket: CM-10954, CM-10953
Reviewed By: julien
Testing Done: ran ifupdown2 smoke and vrf ifup/ifdown testing
- vrf master if not around is brought up when the first slave is brought up
by design (because we want the slaves to be
enslaved to a vrf master before addresses are configured).
and master is not brought up by first because interfaces
are brought up down to top in the dependency tree.
- This patch makes sure a slave brings up a vrf master only
when all interfaces are specified. When an individual interface
is brought up, skip master bring up and throw an error.
- Since the addon modules also need to use the ALL and WITH_DEPENDS
flags, this patch moves them to the existing global ifupdownflags
class
- vrf module uses the ifupdownflags.ALL flag to not bring up the master
when only the slave is being brought up
example:
ifup <vrf> # brings up the vrf device
ifup <vrf> --with-depends # brings up the vrf dev and
# its slaves
ifup <vrf_slave> # if vrf master is not around,
# an error is thrown
ifup <vrf_slave> --with-depends # will still not bring up the vrf master
ifdown <vrf> # deletes vrf dev and flushes the
# addresses on vrf slaves
ifup <vrf> # brings up vrf dev and does not
# up the adresses on the slaves
ifup <vrf> --with-depends # will bring up vrf and reapply config
# on slaves (including addresses)
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-8095
Reviewed By: julien, nikhil
Testing Done: Tested ifupdown2 sanity and unclean removal of interfaces from the
interfaces file
example: for the below interfaces file,
{noformat}
auto bond2
iface bond2 inet static
address 100.0.0.2/16
bond-slaves swp3 swp4
auto bridge
iface bridge
bridge-vlan-aware yes
bridge-ports swp1 swp2 bond2
bridge-vids 12 505
bridge-stp on
auto swp1
iface swp1
mtu 9000
{noformat}
when iface sections for bond2 and swp1 are removed from the interfaces file
but left as dependents of bridge, on an ifreload:
prior to this patch:
===================
- ifreload would throw a warning for both bond2 and swp1
warning: bond2: misconfig ? removed but still exists as a dependency of
['bridge']
warning: swp1: misconfig ? removed but still exists as a dependency of
['bridge']
after the patch:
================
- ifreload will now correctly throw a warning only for bond2
warning: bond2: misconfig ? removed but still exists as a dependency of
['bridge']
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-7840
Reviewed By: julien, nikhil
Testing Done: Tested ifquery with and without --with-defaults option
commit "6e16e5ae90f2" introduced --with-defaults for ifquery --check
output. But the base ifquery --with-defaults should also include
the default policy attributes we support. This patch adds infrastructure
to query default attributes --with-defaults in base ifquery output.
example:
{noformat}
$ifquery br0
auto br0
iface br0 inet static
bridge-ports swp1 swp2
$ifquery br0 --with-defaults
auto br0
iface br0 inet static
bridge-ports swp1 swp2
bridge-stp yes
{noformat}
Ticket: CM-10465
Reviewed By: julien, nikhil
Testing Done: Tested creating more than 64 vrf devices
example config:
{noformat}
%for v in range(1, 66):
auto vrf${v}
iface vrf${v}
vrf-table auto
%endfor
{noformat}
also, since vrf module already does a link up on vrf dev, remove
link up of vrf dev from ifupdownmain scheduler callback
Ticket: CM-7840
Reviewed By: Roopa Prabhu
Testing Done: yes, by installing ifupdown .deb file onto dell-s3000-02
This patch adds a new argument '--with-defaults' to 'ifquery'
when 'ifquery --with-defaults' is executed, running states of all interface
attributes are compared against respective configured attributes from
/etc/network/interfaces file, if configured. Otherwise, compared against
default attributes from policy file
This patch also: (1) fixes ifquery check failure for bridge-* stp attributes.
(2) removes vrf-default-route and vrf-cgroup attributes from ifupdown2 policy
and just have the vrf-helper attribute
Signed-off-by: Nikhil <nikhil@cumulusnetworks.com>
Ticket: CM-10178
Review: trivial
Testing: tested with failing testcase in the CM
This patch fixes a check in vrf map initialization code which did
not account for running vrfs correctly. This caused the case where
there were no running vrfs but stale map file to fail.
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
addons,ifupdown,sbin: adding ifquery --with-defaults option
Ticket: CM-7840
Reviewed By: Roopa Prabhu
Testing Done: yes, by installing ifupdown .deb file onto dell-s3000-02
This patch adds a new argument '--with-defaults' to 'ifquery'
when 'ifquery --with-defaults' is executed, running states of all interface
attributes are compared against respective configured attributes from
/etc/network/interfaces file, if configured. Otherwise, compared against
default attributes from policy file
Signed-off-by: Nikhil <nikhil@cumulusnetworks.com>
Ticket:
Reviewed By: julien
Testing Done: Tested ifquery -l and checked that it is compatible with
ifupdown
ifquery list lists all matching interfaces. By default that is all auto
interfaces:
example:
{noformat}
$cat /etc/network/interfaces
auto lo
iface lo inet loopback
allow-mgmt eth0
iface eth0 inet dhcp
vrf mgmt
allow-mgmt mgmt
iface mgmt
address 127.0.0.1/8
vrf-table auto
$ifquery -l --allow=mgmt
eth0
mgmt
$ifquery -a -l
lo
{noformat}
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: cleanup
Reviewed By:
Testing Done: Tested ifupdown sanity
This gets rid of some ugly previous flag handling which was
passed through modules. This creates a global instance of
flags that all addon modules and helper modules can use.
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: found during other testing
Reviewed By: trivial
Testing Done: Tested ifreload with dependency errors and dependencies
with upperiface being a master and also vlan device
This patch avoids false positives when set role is called twice
for an interface which is a vrf slave but also has a vlan device on
top of it.
Ticket: trivial
Reviewed By: wkok
Testing Done: Tested ifupdown2 sanity and checked log messages from rtnetlink api
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
This is cherry-pick of commit 269ff43d09ca4 from cumulus2.5
the same name
Ticket: CM-10051
Reviewed By: julien, nikhil
Testing Done: tested that bridge vlan attributes get applied correctly + ran ssim test testifupdown2.py
This was introduced by a patch that squashed multiple iface objects
into a single object.
That led to the below interfaces getting squashed into one. Which is not
the right thing to do:
{noformat}
auto Newbr.325
iface Newbr.325
address 24.0.0.22/30
address 3101:abc:bcad:1::3/64
auto Newbr.325
vlan Newbr.325
bridge-igmp-querier-src 194.31.10.45
{noformat}
The 'vlan Newbr.325' ifaceobject needs to be kept separately and it is
of type BRIDGE_VLAN. so, this patch just makes sure these interfaces are
kept separately in the squash function.
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-10188, CM-10061
Reviewed By: dsa, nikhil, julien
Testing Done: Tested static routes with vrf names for tables
This patch does the following:
- if a single vrf device is present in the config,
builds the vrf map by reading vrf interfaces from the kernel (with
existing link cache. Builds a shadow vrf only attribute cache)
- reads existing table map and adjusts it if required
- main change is the iproute2 map file on disk is updated
immediately on vrf creation, so that static routes used along with the
vrf slaves can use the vrf name for the table. This also helps dhclient dns
hook script which may use mgmt table name directly.
- cleans up default routes on down
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-10188, CM-9881
Reviewed By: nikhil, julien, dsa, daniel
Testing Done: Tested mgmt vrf bringup, teardown, static routes at boot
etc
This patch fixes a few things:
- kill existing ssh clients on enslavement change for mgmt vrf (original patch by NikhilG)
- bring vrf master up first during vrf slave enslavement if
master does not exist. This was originally done only for
vrf dhcp slaves. With this patch we do it for all vrf slaves.
needed for static routes on vrf slaves (CM-10188).
- cleanup: reorganize code and a few cleanups and corner case handling
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Signed-off-by: Nikhil Gajendrakumar <nikhil@cumulusnetworks.com>
Ticket: CM-10027
Reviewed By: julien, nikhil
Testing Done: Tested with an interfaces file with shared dependents
In the process of fixing this saw a few more issues with link kind
handing. Its better to separate kind from interface private flags
like bond slave and bridge port. this patch cleans up all that handling.
Example errors:
error: misconfig..? swp5.2 vrfslave is enslaved to multiple interfaces
['vrf1012', 'br2']
error: misconfig..? swp5.2 bridgeport is enslaved to multiple
interfaces ['vrf1012', 'br2']
This was bringing down vrf master earlier than required
leading to ssh session hangs in case of management vrf
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
This was bringing down vrf master earlier than required
leading to ssh session hangs in case of management vrf
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-10016
Reviewed By:
Testing Done: Tested with the upperiface bring up case described in
CM-10016
upperiface bringup is best effort and does not imply that the current
interface bring up has an error. It only means that the upperiface may
be in an incomplete state. Hence this patch resets scheduler error state
before returning from upperiface bringup.
Ticket: CM-9993
Reviewed By: julien
Testing Done: Tested with interfaces file and steps described in the CM
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-7168
Reviewed By: julien, nikhil, stannous
Testing Done: Tested with errors in interfaces file
This patch changes a few errors to warns. and propagates errors
on ifaceobjects to upperlayers.
- any exception passed to upper layers (/sbin/ifupdown) results in
exit code of 1
- It uses a global SCHED flag to flag a scheduler error (maybe there is
a better way). But traversing all the interfaces again to check status
is an overkill
- Changes a few errors to warns
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
default
Ticket:
Reviewed By: CCR-4268 (previous review)
Testing Done: Tested ifup/ifdown/ifreload/ifquery of multiple iface stanzas for
same interface
This patch is an extension to previous commit 99ce689411.
The previous commit squashes both external (ifquery) and internal
(ifup/ifdown/ifreload) representation of multiple iface stanzas into
one and it is off by default.
What we really want is internal representation to be squashed by
default. To that effect this patch introduces a new config flag
ifaceobj_squash_internal to only squash internal representation which is
used by ifup/ifdown/ifreload. ifquery forces this flag to off so that
external representations remain unsquashed and user does not see any
difference. This flag is on by default.
User can still get a squashed external representation if he sets
ifaceobj_squash=1 in ifupdown2.conf
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket:
Reviewed By: trivial
Testing Done:
This patch adds WARNING to ifaceStatus and adds a new method
iface:set_status
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket:
Reviewed By: CCR-4268
Testing Done: Tested squashing of interfaces with multiple iface stanzas
This is controlled by ifaceobj_squash config variable in
/etc/network/ifupdown2/ifupdown2.conf.
With ifaceobj_squash=1, ifquery and all commands will
output squashed interfaces.
$cat /etc/network/interfaces
auto swp3
iface swp3
mtu 9000
auto swp3
iface swp3 inet static
address 10.0.17.3/24
auto swp3
iface swp3 inet static
address 10.0.18.3/24
address 2000:1000:1000:1000:3::5/128
$ifquery -a
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto swp3
iface swp3
mtu 9000
address 10.0.17.3/24
address 10.0.18.3/24
address 2000:1000:1000:1000:3::5/128
When and why do we need this ?
- If we preserve multiple ifaceobjects for the same iface,
it gets tricky in some cases to set default policy
values because the addon module run methods are called
on each ifaceobject.
- Each ifaceobject belonging to the same interface
is treated as a separate interface. It is difficult
to remember things accross addon module run methods
- we have a few hacks in place which we would like to
get rid of
Why not turn it on by default ?
- still debating about it. Dont want to break existing
scripts with change of output. Will get some feedback before
I switch the default to squash.
We dont need to fail on saved state dependency graph generation errors.
Also dont check for shared dependents during dependency graph generation
on saved state (optimization)
Signed-off-by: Roopa Prabhu <roopa@cumulusnetworks.com>
Ticket: CM-9677
Reviewed By: nikhil
Testing Done: Tested with bond config file in CM-9677
This patch replaces the following attributes:
bond-ad-sys-priority with bond-ad-actor-sys-prio
bond-ad-sys-mac-addr with bond-ad-actor-system
The new attributes correspond to the new sysfs files below:
/sys/class/net/sidelink/bonding/ad_actor_sys_prio
/sys/class/net/sidelink/bonding/ad_actor_system
Old values will be accepted with a deprecated warning:
warning: attribute bond-ad-sys-priority is deprecated. Use bond-ad-actor-sys-prio instead
warning: attribute bond-ad-sys-mac-addr is deprecated. Use bond-ad-actor-system instead
