Major Processors rewrite (#8066)

* Extract DiscoveryItem and move some things to better places.
Extract model class
Fix up model construction.  I have problem with construction...
Makeshift model working.  Switch constructor to factory.  discover() and create()
Support legacy discovery.
Remove uneeded custom pollers
Remove netonix custom detection as we try ucd on all os now.
Add a few yaml procs.  Fix a couple things.
More processor discovery conversions
Move Calix e7 to standard hrProcessorLoad, but it doesn't fully implement the HR-MIB, move things around to make it work.
Add a few yaml procs.  Fix a couple things. Correct some stupid mib stuff.
Move more, drop php 5.3
Add netscaler which uses string indexes.  Port fiberhome to yaml and use skip_values
More conversions.  BroadcomProcessorUsage Trait
Serveriron and Ironware share some mibs.  Create a common abstract os for them.
Add yaml support for mib specification in each data entry
Make legacy discover_processor() set 0 for hrDeviceIndex

Untangle Dell switch OS processors

Use use shared OS for groups if they don't have a specific group.
fix silly mib mistake

Make index optional

Move HR and UCD to Traits and out of Processor.

* forgot to update the fortiswitch index

* Make sgos and avaya-ers match the old index.

* fix comware test data

* fix merge errors

* fix dsm and remove pointless empty modules

* file not found exception is in the wrong place.

* Updated processor development docs

LibreNMS/Util/ModuleTestHelper.php

@@ -26,6 +26,7 @@
 namespace LibreNMS\Util;
 
 use LibreNMS\Config;
+use LibreNMS\Exceptions\FileNotFoundException;
 use Symfony\Component\Yaml\Yaml;
 
 class ModuleTestHelper
@@ -59,13 +60,15 @@ class ModuleTestHelper
      */
     public function __construct($modules, $os, $variant = '')
     {
+        global $influxdb;
+
         $this->modules = $this->resolveModuleDependencies((array)$modules);
-        $this->os = $os;
-        $this->variant = $variant;
+        $this->os = strtolower($os);
+        $this->variant = strtolower($variant);
 
         // preset the file names
         if ($variant) {
-            $variant = '_' . $variant;
+            $variant = '_' . $this->variant;
         }
         $install_dir = Config::get('install_dir');
         $this->file_name = $os . $variant;
@@ -76,7 +79,9 @@ class ModuleTestHelper
 
         // never store time series data
         Config::set('norrd', true);
+        Config::set('hide_rrd_disabled', true);
         Config::set('noinfluxdb', true);
+        $influxdb = false;
         Config::set('nographite', true);
 
         $this->module_tables = Yaml::parse($install_dir . '/tests/module_tables.yaml');
@@ -456,10 +461,23 @@ class ModuleTestHelper
         }
     }
 
+    /**
+     * Run discovery and polling against snmpsim data and create a database dump
+     * Save the dumped data to tests/data/<os>.json
+     *
+     * @param Snmpsim $snmpsim
+     * @param bool $no_save
+     * @return array
+     * @throws FileNotFoundException
+     */
     public function generateTestData(Snmpsim $snmpsim, $no_save = false)
     {
         global $device, $debug, $vdebug;
 
+        if (!is_file($this->snmprec_file)) {
+            throw new FileNotFoundException("$this->snmprec_file does not exist!");
+        }
+
         // Remove existing device in case it didn't get removed previously
         if ($existing_device = device_by_name($snmpsim->getIp())) {
             delete_device($existing_device['device_id']);
@@ -549,7 +567,7 @@ class ModuleTestHelper
                 }
             }
 
-            file_put_contents($this->json_file, _json_encode($existing_data));
+            file_put_contents($this->json_file, _json_encode($existing_data) . PHP_EOL);
             $this->qPrint("Saved to $this->json_file\nReady for testing!\n");
         }