Improve parsers/redhat_release to detect CentOS correctly #3832

PaulWay · 2023-06-29T00:48:30Z

All Pull Requests:

Check all that apply:

[Y] Have you followed the guidelines in our Contributing document, including the instructions about commit messages?
[Y] Is this PR to correct an issue?
[N] Is this PR an enhancement?

Complete Description of Additions/Changes:

The RedhatRelease parser already does a basic check of the operating system to set flags
for whether this is a RHEL, CentOS, Fedora or unknown system. We need to test that these
are handled correctly within the RedHatRelease combiner.

Part of this work simplifies the short product code detection in the
RedhatRelease parser, which we now advertise as a property.

Additionally, RHEV 3.5 now correctly detects as a RHEL operating system.

* also, skip symlinks for the filesystem_scan_* options Signed-off-by: Mark Huth <[email protected]> Signed-off-by: Mark Huth <[email protected]>

* Feat: New grub2_editenv_list parser Signed-off-by: Xinting Li <[email protected]> * Update default and init Signed-off-by: Xinting Li <[email protected]> * Update __init__ Signed-off-by: Xinting Li <[email protected]> * Correct docs-test error Signed-off-by: Xinting Li <[email protected]> * modify test content name Signed-off-by: Xinting Li <[email protected]> * Add doc entry Signed-off-by: Xinting Li <[email protected]> * Moving to grubenv Signed-off-by: Xinting Li <[email protected]> * Fix flake8 error Signed-off-by: Xinting Li <[email protected]> * Add base class Signed-off-by: Xinting Li <[email protected]> * modify self._error to self._errors Signed-off-by: Xinting Li <[email protected]> * Replace GrubEnv from handle ``/boot/grub2/grubenv`` file to ``grub2-editenv list`` command output Signed-off-by: Xinting Li <[email protected]> * Update insights_archive Signed-off-by: Xinting Li <[email protected]> * Update doc string Signed-off-by: Xinting Li <[email protected]> Signed-off-by: Xinting Li <[email protected]> Co-authored-by: Xinting Li <[email protected]>

* feat: New spec "auditctl -l" * Add a new module for all "auditctl" commands Signed-off-by: Huanhuan Li <[email protected]> * Replace the parent class "LegacyItemAccess" to "dict" * Raise "ParseException" when the line isn't in expected format instead of pass it, maybe we need to enhance the parser. update the module docstring. raise SkipException when there is no known status output Signed-off-by: Huanhuan Li <[email protected]> * Rename "AuditdStatus" to "AuditStatus" Signed-off-by: Huanhuan Li <[email protected]> Signed-off-by: Huanhuan Li <[email protected]>

Signed-off-by: ahitacat <[email protected]>

The examples and the release timeline will help developer to plan the function deprecation w.r.t the insights-core release cycle. Signed-off-by: Sachin Patil <[email protected]>

Add a new spec for authselect current and its parser Signed-off-by: Xiangce Liu <[email protected]>

* Feat: New parser for 'ls -lanL /etc/ssh' command New parser for 'ls -lanL /etc/ssh' command Signed-off-by: Xinting Li <[email protected]>

* feat: New spec and parser to get the satellite provision params * Currently it only collects the "package_upgrade" parameter for RHEL9 * Since more than one tables have columns in yaml format, move the yaml parsing part to the parent class. Signed-off-by: Huanhuan Li <[email protected]> * Update spec "satellite_provision_param_settings" to depend on "IsSatellite611" * Add component "IsSatellite611" Signed-off-by: Huanhuan Li <[email protected]> * Add "IsSatellite" and "IsCapsule" component * Update all the satellite and capsule specs with new "IsSatellite" and "IsCapsule" component Signed-off-by: Huanhuan Li <[email protected]> Signed-off-by: Huanhuan Li <[email protected]>

* Feat: New parser for 'journalctl --header' command Signed-off-by: Xinting Li <[email protected]> * Update doc string Signed-off-by: Xinting Li <[email protected]> * fix: change journalctl_header from parser to datasource Signed-off-by: Xinting Li <[email protected]> * move to parser and add filter Signed-off-by: Xinting Li <[email protected]> * Fix flake8 errors Signed-off-by: Xinting Li <[email protected]> * Fix python2 error Signed-off-by: Xinting Li <[email protected]> * fix python2 error Signed-off-by: Xinting Li <[email protected]> * Fix assert error Signed-off-by: Xinting Li <[email protected]> * Fix error Signed-off-by: Xinting Li <[email protected]> * Update doc string Signed-off-by: Xinting Li <[email protected]> * fix wrong doc string Signed-off-by: Xinting Li <[email protected]> * Change to journal_header Signed-off-by: Xinting Li <[email protected]> * change to 'from insights.core import Syslog' Signed-off-by: Xinting Li <[email protected]> * update doc string Signed-off-by: Xinting Li <[email protected]> Signed-off-by: Xinting Li <[email protected]> Co-authored-by: Xinting Li <[email protected]>

* When collect is called from the cli, the malware datasource calls client configuration which parses the collect args and fails. * This PR removes the args after parsing to avoid the failure. * Fix #3505 Signed-off-by: Bob Fahr <[email protected]> Signed-off-by: Bob Fahr <[email protected]>

* fix: Revert spec httpd_on_nfs * use ProcMounts parser instead of Mounts combiner * ensure vers can be checked Signed-off-by: Xiangce Liu <[email protected]> * fix test of py2 Signed-off-by: Xiangce Liu <[email protected]>

* fix: change cloud_cfg to Yaml and modify the filtering target - fix: #3482 - fix: #3478 Signed-off-by: Xiangce Liu <[email protected]> * forget the tests * fix doc test Signed-off-by: Xiangce Liu <[email protected]> * for python26 - 2 * add more doc for specs Signed-off-by: Xiangce Liu <[email protected]> * fix the doc tests Signed-off-by: Xiangce Liu <[email protected]>

Signed-off-by: Xinting Li <[email protected]>

Resolves: rhbz#2065233 Signed-off-by: ahitacat <[email protected]>

* util/subproc was already using which to get the full path for timeout, but the command spec was not. This change uses the which util to ensure the full path is obtained for the timeout command * Fix #1130 Signed-off-by: Bob Fahr <[email protected]>

* Feat: add secure spec to default.py Signed-off-by: Xinting Li <[email protected]> * Delete secure = simple_file("/var/log/secure") in sos_archive.py Signed-off-by: Xinting Li <[email protected]>

* feat: support InstanceType in saphostctrl Signed-off-by: Xiangce Liu <[email protected]> * fix and test the full_type Signed-off-by: Xiangce Liu <[email protected]> * Use full_type only to check function_instances Signed-off-by: Xiangce Liu <[email protected]> - Add 'Diagnostic Agents' * fix typos * revert the test cases of .data * revert the doc of attr 'instances' Signed-off-by: Xiangce Liu <[email protected]>

Signed-off-by: Jitka Obselkova <[email protected]>

Signed-off-by: ahitacat <[email protected]>

* feat: Add spec "satellite_enabled_features" back * It is needed by a new rule, so add it back Signed-off-by: Huanhuan Li <[email protected]> * Adjust the place of "satellite_enabled_features" Signed-off-by: Huanhuan Li <[email protected]> * Remove useless line since it depends on core collection Signed-off-by: Huanhuan Li <[email protected]> Signed-off-by: Huanhuan Li <[email protected]>

* Feat: Add spec and parser for cryptsetup luksDump Signed-off-by: daniel.zatovic <[email protected]> * Fix flake errors in cryptsetup parser test Signed-off-by: daniel.zatovic <[email protected]> * Rework luksDump parser and test. Signed-off-by: daniel.zatovic <[email protected]> * Fix documentation test and fix examples Signed-off-by: daniel.zatovic <[email protected]> * Add docs for cryptsetup parser and datasource Signed-off-by: daniel.zatovic <[email protected]> * Replace bash filtering by a custom datasource Signed-off-by: daniel.zatovic <[email protected]> * Add tests for the new cryptstup datasources Signed-off-by: daniel.zatovic <[email protected]> * Fix comment style Signed-off-by: daniel.zatovic <[email protected]> * Fix doc test inconsistency between Python 2 and 3 Signed-off-by: daniel.zatovic <[email protected]> * Fix title length Signed-off-by: daniel.zatovic <[email protected]> * Fix LUKS capitalisation Signed-off-by: daniel.zatovic <[email protected]> * Remove stray print command Signed-off-by: daniel.zatovic <[email protected]> * Fix names formatting according to PEP8. Signed-off-by: daniel.zatovic <[email protected]> * Change variable names in docs Signed-off-by: daniel.zatovic <[email protected]> * Filter tokens manually instead of using an option Signed-off-by: daniel.zatovic <[email protected]> * Add component to detect cryptsetup token support If the tokens are supported (cryptsetup package version at least 2.4.0 is installed), add --disable-external-tokens option to luksDump spec. Signed-off-by: daniel.zatovic <[email protected]> * Add tests for the cryptsetup component Signed-off-by: daniel.zatovic <[email protected]> * Enable dependent components in the config Signed-off-by: daniel.zatovic <[email protected]> * Extend test coverage Signed-off-by: daniel.zatovic <[email protected]> Signed-off-by: daniel.zatovic <[email protected]>

* Add CpuSMTControl parser Signed-off-by: Jitka Obselkova <[email protected]> * Rework current tests Signed-off-by: Jitka Obselkova <[email protected]> Signed-off-by: Jitka Obselkova <[email protected]>

* feat: New spec to get satellite logs table size and its parser Signed-off-by: Huanhuan Li <[email protected]> * Remove "pg_size_pretty" function in SQL * In this way, it outputs the bytes directly Signed-off-by: Huanhuan Li <[email protected]> * Remove useless line since it depends on core collection Signed-off-by: Huanhuan Li <[email protected]> Signed-off-by: Huanhuan Li <[email protected]>

- Removing extraneous space inserted during refactoring commit 894484a - Code was moved to a new function, and a space was added to the split. bz#2126155 Signed-off-by: Mike Kingsbury <[email protected]>

* Move results object to ReportItem Signed-off-by: Štěpán Tomsa <[email protected]> * Go classes Signed-off-by: Štěpán Tomsa <[email protected]> * Start the Glob File test Signed-off-by: Štěpán Tomsa <[email protected]> * Make tests saner Signed-off-by: Štěpán Tomsa <[email protected]> * Fix SOSCleaner for glob files Signed-off-by: Štěpán Tomsa <[email protected]> * Revert an unnecessary whitespace change Signed-off-by: Štěpán Tomsa <[email protected]> * Remove an unused import This fixes flake8 violations. Signed-off-by: Štěpán Tomsa <[email protected]>

Signed-off-by: Xiangce Liu <[email protected]> Signed-off-by: Xiangce Liu <[email protected]>

* Add combiner rhel_for_edge Signed-off-by: jiazhang <[email protected]> * Update test Signed-off-by: jiazhang <[email protected]> * Update return format Signed-off-by: jiazhang <[email protected]> * Remove SkipComponent Signed-off-by: jiazhang <[email protected]> * Update description Signed-off-by: jiazhang <[email protected]> * Update description format Signed-off-by: jiazhang <[email protected]> * Update description format Signed-off-by: jiazhang <[email protected]> * Add note Signed-off-by: jiazhang <[email protected]> * Update long condition Signed-off-by: jiazhang <[email protected]> * Update condition format Signed-off-by: jiazhang <[email protected]> Signed-off-by: jiazhang <[email protected]>

* feat: New spec "/etc/cron.d/foreman" and parser Signed-off-by: Huanhuan Li <[email protected]> * Add more examples in doc string Signed-off-by: Huanhuan Li <[email protected]> * Reimplement the parser "CronForeman" Signed-off-by: Huanhuan Li <[email protected]> * Abstract a base class for later use. * Call existing "keyword_search" in the search method Signed-off-by: Huanhuan Li <[email protected]> Signed-off-by: Huanhuan Li <[email protected]>

Signed-off-by: Sachin Patil <[email protected]> Signed-off-by: Sachin Patil <[email protected]>

Signed-off-by: Xiangce Liu <[email protected]>

* fix: support inhibitor entries with missing remediations Signed-off-by: Xiangce Liu <[email protected]> * update per feedback Signed-off-by: Xiangce Liu <[email protected]>

* Keep old files too Signed-off-by: Huanhuan Li <[email protected]>

Signed-off-by: Xinting Li <[email protected]>

This patch updates the logging file handler to prepare for the logrotate feature implementation in insights-client. insights-client versions older than 3.2.0 will use the legacy rotating file handler and newer versions will use logrotate. Signed-off-by: Jason Jerome <[email protected]>

…3812) * Also fix wrong database name in sql Signed-off-by: Huanhuan Li <[email protected]>

Signed-off-by: Huanhuan Li <[email protected]>

* Feat: new spec "/var/log/watchdog/*" and the parser Signed-off-by: Huanhuan Li <[email protected]> * Update "watchdog_logs" spec pattern to make it more specific Signed-off-by: Huanhuan Li <[email protected]> * Raise RuntimeError instead of SkipComponent Signed-off-by: Huanhuan Li <[email protected]>

Signed-off-by: Xiangce Liu <[email protected]>

- see: ESSNTL-4697 - and update the Parsers implemantion Signed-off-by: Xiangce Liu <[email protected]>

* fix: CI test for python27 Signed-off-by: Xiangce Liu <[email protected]> * Fix failing tests in malware-detection Signed-off-by: Mark Huth <[email protected]> --------- Signed-off-by: Xiangce Liu <[email protected]> Signed-off-by: Mark Huth <[email protected]> Co-authored-by: Mark Huth <[email protected]>

- To catch any potential changes in output/behaviour in newer yara releases Signed-off-by: Mark Huth <[email protected]>

Since CentOS 7 doesn't have pre-installed python 3.x we need to use this url `https://bootstrap.pypa.io/pip/2.7/get-pip.py` to get the right pip version, to avoid error on docker build process Signed-off-by: Fellipe Henrique <[email protected]>

- The cert_verify value may be set outside the malware-detection app Use its value if set, otherwise set it to True Signed-off-by: Mark Huth <[email protected]>

* feat: new spec and parser for eap runtime json reports Signed-off-by: Chenlizhong <[email protected]> * fix: remove filterable from eap_json_reports Signed-off-by: Chenlizhong <[email protected]> * fix: update tests and remove print Signed-off-by: Chenlizhong <[email protected]> * fix: update doc test for python 2.6 and 2.7 Signed-off-by: Chenlizhong <[email protected]>

- Use 'be removed from 3.#+1.0' instead of 'be removed after 3.#.25'. Since sometimes the minor version would be greater than `25`. Signed-off-by: Xiangce Liu <[email protected]>

The RedhatRelease parser already does a basic detection of RHEL, CentOS, Fedora or Unknown operating system. We need to test this with the RedHatRelease combiner. Part of this work simplifies the short product code detection in the RedhatRelease parser, which we now advertise as a property. Signed-off-by: Paul Wayper <[email protected]>

Signed-off-by: Paul Wayper <[email protected]>

xiangce

@PaulWay - This change looks good to me, but the update of the combiner will break some tests of the GSS-Rules, since these tests take the RedHatRelease for RHEL only. We'd better fix them before merging this PR.

PaulWay · 2023-06-29T03:00:50Z

Good point @xiangce - I'll look through the rules and write up a PR for that.

xiangce · 2023-07-13T03:21:52Z

insights/combiners/redhat_release.py

@@ -74,7 +74,7 @@ def __init__(self, uname, rh_rel):
            self.major = uname.redhat_release.major
            self.minor = uname.redhat_release.minor
            self.rhel = '{0}.{1}'.format(self.major, self.minor)
-        elif rh_rel and rh_rel.is_rhel:
+        elif rh_rel:


Hi @PaulWay - Other than making an MR to the GSS rules, it would be more suitable to keep this RedHatRelease combiner for RHEL only, As this combiner is designed for RHEL system only, See the docstring in L40

I agree with Xiangce, I wanted to say something about this previously but forgot. I feel like detection for centos, fedora, oracle, etc should be done in the os-release parser not the redha-release parser/combiner. Since os-release is used by every distro, and provides a lot of information that the redhat-release file doesn't provide I feel it's better fitted.

candlepin-bot · 2024-04-26T15:29:30Z

Can one of the admins verify this patch?

mhuth and others added 30 commits August 17, 2022 15:43

Automatically retry malware-detection downloads & uploads (#3493)

030176a

* also, skip symlinks for the filesystem_scan_* options Signed-off-by: Mark Huth <[email protected]> Signed-off-by: Mark Huth <[email protected]>

New location for temp directory and tests (#3489)

ed8f91d

Signed-off-by: ahitacat <[email protected]>

Add release timeline (#3500)

2eb0fb5

The examples and the release timeline will help developer to plan the function deprecation w.r.t the insights-core release cycle. Signed-off-by: Sachin Patil <[email protected]>

feat: New spec and parser for authselect current (#3490)

103f84c

Add a new spec for authselect current and its parser Signed-off-by: Xiangce Liu <[email protected]>

Feat: New parser for 'ls -lanL /etc/ssh' command (#3499)

3d8ae09

* Feat: New parser for 'ls -lanL /etc/ssh' command New parser for 'ls -lanL /etc/ssh' command Signed-off-by: Xinting Li <[email protected]>

fix: Revert the httpd_on_nfs datasource spec (#3509)

26d6155

* fix: Revert spec httpd_on_nfs * use ProcMounts parser instead of Mounts combiner * ensure vers can be checked Signed-off-by: Xiangce Liu <[email protected]> * fix test of py2 Signed-off-by: Xiangce Liu <[email protected]>

Fix: grubenv cannot be collected when error shown in output (#3511)

860c098

Signed-off-by: Xinting Li <[email protected]>

Added no_proxy autoconfiguration from rhsm conf and tests (#3507)

e56f7a6

Resolves: rhbz#2065233 Signed-off-by: ahitacat <[email protected]>

Feat: add secure spec to default.py (#3513)

92dec74

* Feat: add secure spec to default.py Signed-off-by: Xinting Li <[email protected]> * Delete secure = simple_file("/var/log/secure") in sos_archive.py Signed-off-by: Xinting Li <[email protected]>

feat: Update ls_systemd_units parser (#3518)

e4886d3

Signed-off-by: Jitka Obselkova <[email protected]>

Refractor cleanup local files for unregistration processes (#3520)

fd368b5

Signed-off-by: ahitacat <[email protected]>

New parser for CpuSMTControl and tests update (#3521)

73a472c

* Add CpuSMTControl parser Signed-off-by: Jitka Obselkova <[email protected]> * Rework current tests Signed-off-by: Jitka Obselkova <[email protected]> Signed-off-by: Jitka Obselkova <[email protected]>

fix: Removing extraneous space inserted in commit 894484a (#3523)

a316c02

- Removing extraneous space inserted during refactoring commit 894484a - Code was moved to a new function, and a space was added to the split. bz#2126155 Signed-off-by: Mike Kingsbury <[email protected]>

fix: bz-2126966: use SIGTERM for rpm instead of SIGKILL (#3524)

ecc5475

Signed-off-by: Xiangce Liu <[email protected]> Signed-off-by: Xiangce Liu <[email protected]>

Update docstring to make it more readable (#3531)

6732c34

Signed-off-by: Sachin Patil <[email protected]> Signed-off-by: Sachin Patil <[email protected]>

fix: bz-2130242, remove the print statement (#3535)

0c0a551

Signed-off-by: Xiangce Liu <[email protected]>

xiangce and others added 18 commits June 7, 2023 11:10

chore: remove the planned deprecated modules from 3.2.0 (#3811)

c9ad39a

Signed-off-by: Xiangce Liu <[email protected]>

fix: support inhibitor entries with missing remediations (#3809)

1c226b9

* fix: support inhibitor entries with missing remediations Signed-off-by: Xiangce Liu <[email protected]> * update per feedback Signed-off-by: Xiangce Liu <[email protected]>

Feat: new spec "ls -lan /etc/watchdog.d/" (#3813)

c710c43

* Keep old files too Signed-off-by: Huanhuan Li <[email protected]>

Enhance datasource kernel_module_list (#3816)

c23d539

Signed-off-by: Xinting Li <[email protected]>

feat: New spec to get repos which ingores source rpms on Satellite (#…

3ff0a1b

…3812) * Also fix wrong database name in sql Signed-off-by: Huanhuan Li <[email protected]>

Feat: new spec "/etc/sysconfig/stonith" and its parser (#3817)

7ee2905

Signed-off-by: Huanhuan Li <[email protected]>

fix: Build python26 instead of install (#3823)

18cee6f

Signed-off-by: Xiangce Liu <[email protected]>

feat: update Specs.yumlog to filterable=True (#3810)

2de336f

- see: ESSNTL-4697 - and update the Parsers implemantion Signed-off-by: Xiangce Liu <[email protected]>

Add malware-detection tests that use real yara (#3822)

5246e57

- To catch any potential changes in output/behaviour in newer yara releases Signed-off-by: Mark Huth <[email protected]>

Use the existing cert_verify value, if set (#3826)

abfc7fa

- The cert_verify value may be set outside the malware-detection app Use its value if set, otherwise set it to True Signed-off-by: Mark Huth <[email protected]>

chore: uniform the deprecation warnings and docstring (#3830)

6ded429

- Use 'be removed from 3.#+1.0' instead of 'be removed after 3.#.25'. Since sometimes the minor version would be greater than `25`. Signed-off-by: Xiangce Liu <[email protected]>

Detect RHEV 3.5 correctly as RHEL

f817149

Signed-off-by: Paul Wayper <[email protected]>

PaulWay self-assigned this Jun 29, 2023

PaulWay requested review from xiangce and bfahr June 29, 2023 00:48

PaulWay added 2 commits June 29, 2023 10:59

Correctly feed uname output to tests

37df9b6

Signed-off-by: Paul Wayper <[email protected]>

Merge branch 'master' into paulway_parsers_redhat_release_centos_checks

a8cc4b7

xiangce reviewed Jun 29, 2023

View reviewed changes

xiangce added the BREAK Rules The change breaks the test of some rules. label Jul 12, 2023

xiangce reviewed Jul 13, 2023

View reviewed changes

xiangce force-pushed the master branch from 3b668e9 to c7ff320 Compare September 5, 2024 11:35

xiangce force-pushed the paulway_parsers_redhat_release_centos_checks branch from c545756 to a8cc4b7 Compare September 6, 2024 01:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve parsers/redhat_release to detect CentOS correctly #3832

Improve parsers/redhat_release to detect CentOS correctly #3832

PaulWay commented Jun 29, 2023

xiangce left a comment

PaulWay commented Jun 29, 2023

xiangce Jul 13, 2023

ryan-blakley Jul 13, 2023

candlepin-bot commented Apr 26, 2024

Improve parsers/redhat_release to detect CentOS correctly #3832

Are you sure you want to change the base?

Improve parsers/redhat_release to detect CentOS correctly #3832

Conversation

PaulWay commented Jun 29, 2023

All Pull Requests:

Complete Description of Additions/Changes:

xiangce left a comment

Choose a reason for hiding this comment

PaulWay commented Jun 29, 2023

xiangce Jul 13, 2023

Choose a reason for hiding this comment

ryan-blakley Jul 13, 2023

Choose a reason for hiding this comment

candlepin-bot commented Apr 26, 2024