Comparing Your Database Schema
There is a new entry under menu → Admin → Database called Schema Compare. Running that will show you the schema as it is in your running database and compare it to the schema as shipped with Open-AudIT. If there are any differences, just post them to Questions and we can help you out. For supported customers, just log a support request and we’ll assist ASAP.
Change Log Improvements
Time has been spent to minimize false positive Change Logs being generated. As well as that, we have added two buttons on the Device Details screen (under the left side Actions menu) to remove Change Logs and remove Audit Logs. Using these may help improve database performance where these records are not required. Don’t forget you can always clear the entire tables using menu → Admin → Database → List Tables, clicking either table and hitting the Delete button. And don’t forget about our new configuration items for keeping non-current items and creating change logs. More information on these can be found – here.
There is now a configuration item named device_auto_delete. If set to ‘y’ (it is set to ‘n’ by default) when you change a device’s status (either individually or using Bulk Edit) you will get a regular warning “Are you sure?” and if you answer yes, the device and all it’s details will be completely removed from the database. Not just flagged with a status of deleted.
New Discovery Process and Improvements
With the coming release of Open-AudIT 3.3.0 we have implemented a new discovery process that scales even better than previously. Even faster discovery times!
The Discovery Queue
With 3.3.0 we have changed to using the discovery queue, not on a per discovery basis as previously, but on a per IP basis. From 3.3.0 onwards, when you click “Execute Discovery”, the following happens behind the scenes:
- The server starts a script that calls /util/queue and instantly returns to the web user (or the API user). It starts the shell script and does not wait for a response before returning.
- The user then continues on using the web/API as per normal.
- The shell script calls util/queue – this endpoint only accepts requests from localhost. The resulting function does the following:
- Check the config for the queue limit. If this has been reached, exit. If it has not been reached, continue.
- Pop an item from the queue (locking the queue table as it does so). The item is read from the database, then deleted. If no queue items exist, exit.
- Spawn another script to request util/queue.
- Execute the item – which on the first time is always “run discovery on subnet”.
- When finished, return to #1.
There are (currently) two types of queue entries. The overall discovery entry, and an entry for each IP to be scanned. The second entry is created by the first. So we run the initial discovery, and for each IP we need to scan (that responds, if that option is chosen), we create another entry to scan that device.
We no longer use the discover_subnet.sh or discover_subnet.vbs scripts at all. We now call Nmap directly from within the Open-AudIT code, which frees us up to have one and only one routine (versus a bash and vbscript). It also makes it easier to code – PHP has much easier to use text parsing than bash and vbscript (in my opinion).
Because of the above, we have created a new configuration item called “discovery_limit” and set it to 20 by default. This means when a discovery is run, it will spawn up to 20 processing instances in parallel. Because of this parallel processing of target IPs, discovery is $discovery_limit times faster. Well, not quite, but you get what I mean. The old way ran a discovery and for each target, sequentially, started a scan. Several scans were run at once, but it was still waiting for an Nmap scan, before handing off to PHP to complete the rest. The new way completes the initial scan and loads all resulting devices into the queue to be processed in parallel. At the end of the day, it’s just so much faster.
Sudo SSH Key and Password
Previously we only had support for an SSH Key that used a password, but where that password was also used for sudo. That is obviously not optimal, so as at 3.3.0 you can add a specific ssh key password and a sudo password.
Additional Nmap Option
We have also added a new option to discovery scan options – open|filtered. Previously we used the “filtered” column to check for open|filtered. This change aligns the discovery scan options with Nmap return strings.
Auditing Time Reduction When Using sudo
When auditing a device using sudo, we no longer have to wait for the configuration item discovery_ssh_timeout (previously 300 seconds) to timeout. We now check every 2 seconds for our response and when received, proceed. This has made another large difference to audit times when using sudo.
Windows Users Apache Service
As well as this, there has been a change targeted specifically to Windows Open-AudIT Servers. Because of the issue’s we have run into using the default service account, you will now get a big warning stating you should change the service account to a “real” account. This is because by default the service account cannot access network resources. IE – copy the audit script to thew target and run it. The “old” way of running the script on the Open-AudIT server itself and specifying the target still works and is enabled by a config item – discovery_use_vintage_service, which is set to ‘n’ by default. One reason for this is that the discovery script contains sections that do not and can not work remotely. Think starting an executable. That won’t work as WMI can target the remote machine, but running an executable from the audit script would run it where the script is running – the Open-AudIT server.
The Default Network Address
Because of our new way of running discovery, we no longer need to set the Default Network Address. The scripts are run on the target devices and create a file (as opposed to submit_online=y). That file is then copied to the server and processed, rather than submitted using the URL (that was created from the default_network_address). The only reason to set the Default Network Address for Discovery is if you’re using discovery_use_vintage_service. The now normal use of this is only for the “Audit My PC” functionality.
Auto Delete our Audit Script
Now when discovery runs, the audit script deletes itself on the target, hence we leave nothing present on the target device.
No More “New” Devices Where We Have No Information
We have added a new configuration option called match_ip_no_data. If we discover a device and that IP is already in the database and we have no audit data about that device, assume it is the same device, so do not create another (usually duplicate) device.
SNMP Route Retrieval
We now retrieve the first (configuration item discovery_route_retrieve_limit) routes from a device when using SNMP.
And there’s even more improvements. Make sure you read the Release Notes for Open-AudIT v3.3.0 to stay across it all.