The Code described

hcisle is provided as a source distribution package to be installed using pip. In case the code shall be inspected, one can just un-zip the source package to find the folder structure outlined below.

Structure

 1hcisle-<version>.tar.gz:
 2    hcisle-<version>
 3    ├── DESCRIPTION.rst
 4    ├── MANIFEST.in
 5    ├── PKG-INFO
 6    ├── hcilib
 7    │   ├── __init__.py
 8    │   ├── __main__.py
 9    │   ├── base
10    │   │   ├── __init__.py
11    │   │   ├── exceptions.py
12    │   │   └── tools.py
13    │   ├── collect
14    │   │   └── __init__.py
15    │   │   └── hdidetails.py
16    │   ├── db
17    │   │   └── __init__.py
18    │   ├── loghandler
19    │   │   └── __init__.py
20    │   ├── version
21    │   │   └── __init__.py
22    │   └── ziphandler
23    │       └── __init__.py
24    ├── hcisle.egg-info
25    │   ├── PKG-INFO
26    │   ├── SOURCES.txt
27    │   ├── dependency_links.txt
28    │   ├── entry_points.txt
29    │   └── top_level.txt
30    ├── setup.cfg
31    └── setup.py

All non-emphasised entries are used by the package manager (pip) as installation instructions and to maintain the packages metadata.

Code flow

  • The entrance into the code is the file hcilib/__main__.py (line 8). It contains the program flow logic and calls in to all the other modules. It is called by a starter script (hcisle), auto-generated by the package manager during package installation.

  • As a first step, base/tools.py (line 12) is used to parse the command line arguments.

  • After that, loghandler/__init__.py (line 19) sets up logging. It also sets up a dedicated logger intended to send records to a syslog server, in case a syslog server IP address is defined using the --syslog-ip argument.

  • The database is opened and checked for validity, or created, if it doesn’t exist, using the Db object within db/__init__.py (line 17).

  • Given that --logpackage is not present as an argument, collect/__init__.py (line 14) is used to first call collect/hcidetails.py (line 15) to discover which Indexer service(s) is/are being active (this is btw. the only call that needs access to the HCI MAPI). Once the active indexer service(s) is/are known, the HCI log_download tool is called to create a tailored log package, including logfiles from all HCI instances.

  • The log package is then given to a Loghandler object defined in ziphandler/__init__.py (line 23). The loghandler walks the content of the log package, searching for the relevant logfiles. These are send to an extraction method, which scans for the log records of interest. For each record found, a call is made into a method of the Db object to store the record into the database. The records timestamp, as well as a hash calculated from the records content, are used to detect and discard duplicate records during this.

  • Once done with the log package, another Db method is called to select any not-yet sent record from the database, send it to the syslog server and finally mark it as sent in the database, so that it is not sent twice during subsequent runs. This only happens if a syslog server was defined, of course.

  • Then, database maintenance is performed: all records that have been sent to syslog and are older than 31 (tuneable) days are removed and the database is VACUUM’d. This makes sure that the database doesn’t grow too much, and that no security sensitive data is stored for longer than required. DB maintenance will trigger VACUUM if records were deleted, only.

  • Finally, the log package is deleted from disk (except the --logpackage argument is present).