Getting Started, Converting the OVF for VMware vCenter

Getting Started, Converting the OVF for VMware vCenter

Importing the OVA file to an ESXi Datacenter via vCenter using VMware OVF Tool

  • NOTE – We do not support OVA import to VMware ESXi directly or via VMware vCenter using a browser.
  • NOTE – We require the VMware OVF Tool if importing to VMware ESXi directly or via VMware vCenter.
    • Numerous attempts were made to import this OVA using the VMware vSphere HTML5 Client, with Chrome, Chromium and Firefox as browser: Every attempt timed out.
    • We have yet to experience a failed import of this OVF using VMware commandline tool ovftool: For this reason VMware commandline tool ovftool is recommended for importing OVAs’ via VMware vCenter

You can import the OVA file into a VMware ESXi Datacenter via VMware vCenter using VMware ovftool, from the PC or server you run ovftool.
Remember to set the Virtual Machine name you like, the datastore you want, the network you want, and point it at the server you want to deploy to.
Run commands like this:

  • In this example the VMware vCenter 6.5 install has created:

    • FQDN ‘photon-machine’ with

      • user ‘administrator,

      • sso domain name ‘vsphere.localdomain, and

      • sso password ‘Vagrant1!

        • which must be escaped when used in this context using ovftool:

          • escaped  password: ‘Vagrant1\!

  • VMware ovftool option that should be determined appropriate to this use case for this OVA import:
    • -ds=”datastore_name-number”
      • Required if the selected Datacenter has more than one datastore
    • -nw=”network_name-number”
      • Required if the selected Datacenter has more than one network
    • -dm
      –diskMode

      Specify the disk format. You can specify the following formats:
      • monolithicSparse
      • monolithicFlat
      • twoGbMaxExtentSparse
      • twoGbMaxExtentFlat
      • seSparse (vSphere target)
      • eagerZeroedThick (vSphere target)
      • thin(vSphere target)
      • thick (vSphere target)
      • sparse
      • flat
Determine the Datacenter name:

ovftool vi://administrator@vsphere.localdomain:Vagrant1\!@photon-machine

Error: Found wrong kind of object (Folder). Possible completions are:

Datacenter1/Determine the Datacenter available options:
ovftool vi://administrator@vsphere.localdomain:Vagrant1\!@photon-machine/Datacenter1/
Error: Found wrong kind of object (Datacenter). Possible completions are:
vm/
host/

Determine the Datacenter host IP or FQDN:
ovftool vi://administrator@vsphere.localdomain:Vagrant1\!@photon-machine/Datacenter1/host/
Error: Found wrong kind of object (Folder). Possible completions are:
172.16.37.128/

Import Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova which is in the current directory, in this example using vSphere option –diskMode=thin:
ls -la
-rw-rw-r– 1 user user 903864320 Jul 29 13:09 Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova

ovftool –acceptAllEulas –name=”omk_vm9_centos7″ –diskMode=thin Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova vi://administrator@vsphere.localdomain:Vagrant1\!@photon-machine/Datacenter1/host/172.16.37.128
Opening OVA source: Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova
Opening VI target: vi://administrator%40vsphere.localdomain@172.16.37.130:443/Datacenter1/host/172.16.37.128
Deploying to VI: vi://administrator%40vsphere.localdomain@172.16.37.130:443/Datacenter1/host/172.16.37.128
Transfer Completed
Warning:
– No supported manifest(sha1, sha256, sha512) entry found for: ‘omk-vm9-centos7-disk001.vmdk’.
– No supported manifest(sha1, sha256, sha512) entry found for: ‘omk-vm9-centos7-disk002.vmdk’.
– No manifest file found.
Completed successfully

Getting Started, Converting the OVF for VMware vCenter

Getting Started, Converting the OVF for VMware ESXi

NOTE – We do not support OVA import to VMware ESXi directly or via VMware vCenter using a browser.

NOTE – We require the VMware OVF Tool if importing to VMware ESXi directly or via VMware vCenter:

Importing the OVA file directly to ESXi using VMware OVF Tool

You can import the OVA file directly into ESXi using VMware ovftool, from the PC or server you run ovftool.
Remember to set the Virtual Machine name you like, the datastore you want, the network you want, and point it at the server you want to deploy to.
Run the command like this:

ovftool –name=”NMIS9″ -dm=”thin” -ds=datastore_name-number -nw=”network_name-number” Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova vi://user:escaped_password@host_ip_address
Opening OVA source: Opmantek-20200708-VM9-Centos7.2003-3-64bit.ova
Opening VI target: vi://user@host_ip_address:443/
Deploying to VI: vi://user@host_ip_address:443/
Transfer Completed
Warning:
– No supported manifest(sha1, sha256, sha512) entry found for: ‘omk-vm8-centos7-disk001.vmdk’.
– No supported manifest(sha1, sha256, sha512) entry found for: ‘omk-vm8-centos7-disk002.vmdk’.
– No manifest file found.
Completed successfully
Getting Started, Converting the OVF for VMware vCenter

Getting Started, Converting the OVF for VMware Desktop Applications

These VMware Desktop products  will import directly from an OVF/OVA file so you can just start up the software and select “File → Open” and select the OVA file for import.

VMware Fusion 11

Even though, there are no caveats in relation to importing the Virtual Appliance using VMware Fusion,
it is important to remind users to grant permission to VMware Fusion to fully execute on OSX, specially after installing it.

Broken Pipe MSG
The following message is displayed while running the VM without the permission granted.
Security Settings - 700

The permission should be granted on System Preferences > Security & Privacy

Upgrade the Virtual machine hardware version (optional)

The Opmantek Virtual Appliance ships with the virtual machine hardware version set very low for maximum compatibility, but you may want to upgrade the hardware version to a higher level (to leverage performance benefits, remove memory limitations, etc.) as described on the VMware website

Getting Started, Converting the OVF for VMware vCenter

Getting Started, Converting the OVF for VirtualBox

VirtualBox will import directly from an OVF/OVA file so you can just start up VirtualBox and select “Import Appliance…”
VM Guide 01 - 700
Browse to where you unzipped the file and select the ovf file, you should see a screen something like this:
vm-settings
The import process may take a couple of minutes to complete, just enough time to grab a cup of coffee I’d say.If you have any trouble the VirtualBox documentation is here: https://www.virtualbox.org/manual/ch01.html#ovf
NOTE – The virtual machine is configured to use 8GB memory and the network interface may appear as NAT. Please revise the memory to suit and change the NAT adapter to Bridged.
Important Note regarding Ext4:
If your Virtualbox host is Linux and your VMs are stored on XFS or Ext4 file systems, then it is absolutely necessary that you enable the “Host I/O Cache” for all virtual disk controllers after importing the OVF file, or you run the risk of file system corruption in your guest. On one of our test systems (Linux kernel 3.14.23, Virtualbox 4.3.14) the VM wouldn’t even fully get through the first boot before the virtual disks got corrupted – but with Host I/O Cache on everything is fine.

Below is an example of how to enable Host I/O Cache using VirtualBox:

SCSI-Host-I_O-Cache
Note – In some cases the system will log you out over and over due to the OVA by default having Automatic time synchronization enabled that may cause some authentication cookie issues. To avoid this issue simply check the “Hardware Clock in UTC Time” box in the settings menu under “System” before starting the appliance.
VM Guide 03 - 700
CentOS 7 on VirtualBox

It’s now highly unlikely that VirtualBox users will find themselves at the dracut emergency shell prompt when first booting our vm that runs CentOS 7.
We continue to provide this information in case needed:

rescueKernel - 700
When faced with this obstacle reboot the vm selecting the rescue kernel.
dracutCommand - 700

After it boots login as normal and try the try the following dacut command (you can find login credentials here).
dracut -f /boot/ <kernel_image> <kernel_name>

dracutPrompt - 700
If the command completes successfully reboot the vm normally. Be aware that future kernel updates may require this maneuver be done again.
Using Postman to query the Open-AudIT API

Using Postman to query the Open-AudIT API

I often utilise Postman to query the Open-AudIT API when developing. Just using a browser, it’s difficult to send anything other than a GET request – but Postman makes it simple to send a POST, PATCH or DELETE as required. You can get it from https://www.getpostman.com/downloads/ for Windows, Mac and Linux.

Install and start Postman. You can elect to create an account or not. You can also elect to create a new item using the wizard, or just close the modal and jump in. Let’s do that!

For the below, my Open-AudIT server is running on 192.168.84.4. You should substitute the IP address of your Open-AudIT server.

First, you need to make a post to /login to get a cookie. Set the dropdown to POST and the URL to http://192.168.84.4/omk/open-audit/login. Set the header Accept to application/json. Set the Body to form-data and provide the username and password keys, with values as appropriate for your installation. By default, it will look as below.  Now click the Send button.

Postman Open-AudIT API 1 - 650
Postman Open-AudIT API 2 - 650
You should see the JSON result saying you have been authenticated.

Once that’s done, it’s time to request some data. Make a GET request to http://192.168.84.4/omk/open-audit/devices and you should get a JSON response containing a list of devices. You can see the start of the JSON in the screenshot below.

Postman Open-AudIT API 3 - 650
What about changing the attribute of an item? Not too difficult. You’ll need the ID of the device you want to change, along with the attribute name from the database. You can see these in the application by going to menu → Admin → Database → List Tables and clicking on the “system” table. Let’s change the description for our device with ID 14.

You’ll need to create a JSON object and assign it to the “data” item to do this. It’s not too difficult. Your JSON object should look like below (formatted and indented for easy reading).
{
"data": {
"id": "14",
"type": "devices",
"attributes": {
"description": "My New Description"
}
}
}

 

It looks worse than it is. Normally you would use code to do this, so it’s a simple two line conversion. Because we’re using Postman, we’ll have to do it ourselves. A useful site is https://jsonlint.com/

So now you have your payload, let’s send it to Open-AudIT. Make a new PATCH request and use the URL http://192.168.84.4/omk/open-audit/devices/14.
Supply the data attribute in the body → x-www-form-urlencoded section and hit Send. You should see the request as below.

Postman Open-AudIT API 4 - 650
Deleting an item is even easier. Let’s delete an Org. In this case, our Org with ID 2. Make a new DELETE request to http://192.168.84.4/omk/open-audit/orgs/2. That’s it – easy!/span>

And if we want to read a specific entry, it’s just a GET request. Let’s get our default Org – ID 1. Just make a GET to http://192.168.84.4/omk/open-audit/orgs/1.

What about running a query? What’s the HTTP verb used to EXECUTE something? There is none. But we’ll make do by supplying /execute after the ID. So to execute a query, make a GET request to http://192.168.84.4/omk/open-audit/queries/1/execute. To execute a discovery, task or baseline, use the same format – ID/execute.

Remember we always receive the result in JSON as that is in our request header. We could receive it as HTML is we want – just remove that header item. Maybe more useful is a CSV output. Remove the Accept header and change the URL for a GET to http://192.168.84.4/omk/open-audit/queries/1/execute?format=csv. Done – CSV output you can copy and paste into Excel.

It really is that simple. The only one to watch is the PATCH request because you have to create your own JSON. Just about everything else is quite discoverable. Make sure you check the pages for Collections which detail the request formats. And don’t forget the Open-AudIT API page as well.

Onwards and upwards.
Mark Unwin.

Open-AudIT | Device SubSection Data Retention Options

Open-AudIT | Device SubSection Data Retention Options

With the release of Open-AudIT 3.1.0, we have massively expanded the options around keeping and processing data from devices. SubSections of a device within Open-AudIT refers to the many tables that hold specific data types – software, netstat ports, processors, memory, disks, users, groups, etc, etc. These options exist (for now at least) in the Configuration of Open-AudIT. The items of interest are create_change_log* . and delete_noncurrent*. We previously had these options for a couple of select couple of Subsections, but have expanded these to cover every subsection.

Create Change Logs

The items named create_change_log_* use the database table names to specify which subsection they apply to – so create_change_log_software and create_change_log_memory are both valid examples. You can override ALL items by setting create_change_log to “n” – this will stop any change logs being generated, regardless of the individual table setting. So if a device has a piece of software added (for example), a corresponding change log would not be inserted if create_change_log_software was set to “n”. This is set to “y” by default. This matches how Open-AudIT has always worked.

Special Items

We have also introduced three special configuration items for Netstat Ports. Because ports above 1024 are mostly designed to be dynamic, we now provide three options for keeping this data:

  • create_change_log_netstat_registered
  • create_change_log_netstat_well_known
  • create_change_log_netstat_dynamic

These options correspond to the ports 0-1023, 1024-49151 and 49152-65535. See this wiki list of TCP and UDP port numbers. In particular, Windows DNS servers open a LOT of ports high in the range that are (in my opinion) silly to keep track of, see here and here. By default, only create_change_log_netstat_registered is set to “y”. We may add to these options in the future for other subsections if required.

Delete NonCurrent Items

Along similar lines, the configuration items for delete_noncurrent* use the database table names to specify which subsection they apply to. If set to “y”, then no historical entries will be kept for that table, only the “current” items as at the last audit (or discovery). Again, these individual items can be overridden by the global “delete_noncurrent” item. If set to “y”, it will remove all noncurrent items from all tables. This is set to “n” by default. This matches how Open-AudIT has always worked.

Hopefully, these options provide some customisability for you to only keep the data you actually need.

Onwards and upwards.

Mark Unwin.