Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set OVAL version from 5.11 to 5.11.2 #11903

Closed

Conversation

marcusburghardt
Copy link
Member

@marcusburghardt marcusburghardt commented Apr 26, 2024

Description:

Investigating the issue #11891 it was noticed that OpenSCAP scanner was returning error for rule auditd_audispd_configure_sufficiently_large_partition because the OVAL in this rule was using an OVAL property only present in OVAL 5.11.2, but our content was defined as version 5.11 (without .2).

OVAL version 5.11.2 was released in 2016 and we are likely already using it for long time.

Rationale:

Review Hints:

First take a look in #11891

There is no test scenario or remediation for this rule because it is about partition and tests are not that simple to be created. Therefore, we should test it in the traditional way using a RHEL9 VM, for example.

./build_product rhel9

Copy the generated DataStream to this RHEL9 VM:
scp build/ssg-rhel9-ds.xml root@<ip or host>:

Start a SSH session to this VM and executes a scan for the rule auditd_audispd_configure_sufficiently_large_partition:
oscap xccdf eval --profile stig --rule xccdf_org.ssgproject.content_rule_auditd_audispd_configure_sufficiently_large_partition --results-arf /tmp/arf.xml --report /tmp/report.html --oval-results ssg-rhel9-ds.xml

After this PR, the result will be "Pass" or "Fail" based on the partition size mounted on /var/log/audit. Before this PR, the error shown in #11891 is reproduced.

A good way to confirm the check is accurate is examining the /tmp/arf.xml file.
For example, in the VM I used this was the relevant information about the partition table:

# df -h
Filesystem                     Size  Used Avail Use% Mounted on
...
/dev/mapper/VolGroup-audit     507M   32M  475M   7% /var/log/audit
...

and this was the result of the scan:

Title   Configure a Sufficiently Large Partition for Audit Logs
Rule    xccdf_org.ssgproject.content_rule_auditd_audispd_configure_sufficiently_large_partition
Ident   CCE-88173-0
Result  fail

To confirm the result, this was the relevant information in /tmp/arf.xml file:

                <system_data>
                  <ind-sys:variable_item id="1059257" status="exists">
                    <ind-sys:var_ref>oval:ssg-var_aacsflp_audit_partition_size:var:1</ind-sys:var_ref>
                    <ind-sys:value>531267584</ind-sys:value>
                  </ind-sys:variable_item>
                  <lin-sys:partition_item id="1059256" status="exists">
                    <lin-sys:mount_point>/var/log/audit</lin-sys:mount_point>
                    <lin-sys:device>/dev/mapper/VolGroup-audit</lin-sys:device>
                    <lin-sys:uuid>50934914-4c90-407b-bcee-268a5a76f641</lin-sys:uuid>
                    <lin-sys:fs_type>xfs</lin-sys:fs_type>
                    <lin-sys:mount_options>rw</lin-sys:mount_options>
                    <lin-sys:mount_options>seclabel</lin-sys:mount_options>
                    <lin-sys:mount_options>nosuid</lin-sys:mount_options>
                    <lin-sys:mount_options>nodev</lin-sys:mount_options>
                    <lin-sys:mount_options>noexec</lin-sys:mount_options>
                    <lin-sys:mount_options>relatime</lin-sys:mount_options>
                    <lin-sys:mount_options>attr2</lin-sys:mount_options>
                    <lin-sys:mount_options>inode64</lin-sys:mount_options>
                    <lin-sys:mount_options>logbufs=8</lin-sys:mount_options>
                    <lin-sys:mount_options>logbsize=32k</lin-sys:mount_options>
                    <lin-sys:mount_options>noquota</lin-sys:mount_options>
                    <lin-sys:mount_options>bind</lin-sys:mount_options>
                    <lin-sys:total_space datatype="int">129704</lin-sys:total_space>
                    <lin-sys:space_used datatype="int">9171</lin-sys:space_used>
                    <lin-sys:space_left datatype="int">120533</lin-sys:space_left>
                    <lin-sys:block_size datatype="int">4096</lin-sys:block_size>
                  </lin-sys:partition_item>
                </system_data>
...
               <test test_id="oval:ssg-test_aacsflp:tst:1" version="1" check_existence="all_exist" check="all" result="false">
                  <tested_item item_id="1059257" result="false"/>
                  <tested_variable variable_id="oval:ssg-var_aacsflp_audit_partition_size:var:1">531267584</tested_variable>
                </test>

In this case, 531267584 is the partition size in bytes, or 507M.

Minor OVAL updates in auditd_audispd_configure_sufficiently_large_partition regarding
readability only.
Version 5.11.2 was released in 2016 and we are likely using it for a
long time, although our files inform simply 5.11. There are rules
already using properties from 5.11.2, such as
auditd_audispd_configure_sufficiently_large_partition.
@marcusburghardt marcusburghardt added the OVAL OVAL update. Related to the systems assessments. label Apr 26, 2024
@marcusburghardt marcusburghardt added this to the 0.1.73 milestone Apr 26, 2024
@marcusburghardt marcusburghardt changed the title Oval version Set OVAL version from 5.11 to 5.11.2 Apr 26, 2024
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

rhel8 (from CTF) Environment (using Fedora as testing environment)
Open in Gitpod

Fedora Testing Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

Copy link

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11903
This image was built from commit: ac8dfe0

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11903

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11903 make deploy-local

Copy link

codeclimate bot commented Apr 26, 2024

Code Climate has analyzed commit ac8dfe0 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.4% (0.0% change).

View more on Code Climate.

@Mab879 Mab879 self-assigned this Apr 26, 2024
@dodys
Copy link
Contributor

dodys commented Apr 26, 2024

@marcusburghardt that's not entirely true depending on the vendor
currently ubuntu 20.04 and 22.04 support oval 5.11.1 only
that will also be true for anyone running oscap 1.2.17 or earlier

@Mab879
Copy link
Member

Mab879 commented Apr 26, 2024

@marcusburghardt that's not entirely true depending on the vendor currently ubuntu 20.04 and 22.04 support oval 5.11.1 only that will also be true for anyone running oscap 1.2.17 or earlier

Based on that we will need check that I removed in #11816 needs to be added back.
Then this PR will need to be updated for the project to understand OVAL version 5.11.0 (maybe), 5.11.1, 5.11.2.

@dodys
Copy link
Contributor

dodys commented Apr 26, 2024

@marcusburghardt that's not entirely true depending on the vendor currently ubuntu 20.04 and 22.04 support oval 5.11.1 only that will also be true for anyone running oscap 1.2.17 or earlier

Based on that we will need check that I removed in #11816 needs to be added back. Then this PR will need to be updated for the project to understand OVAL version 5.11.0 (maybe), 5.11.1, 5.11.2.

I didn't check the oval, but does it have a hard requirement on 5.11.2?

Copy link
Member

@Mab879 Mab879 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that on RHEL 9 that the rule now results in fail in my test box, as expected.

Until we address the comments about Ubuntu I'm holding off merging this for now.

@marcusburghardt
Copy link
Member Author

marcusburghardt commented Apr 26, 2024

@marcusburghardt that's not entirely true depending on the vendor currently ubuntu 20.04 and 22.04 support oval 5.11.1 only that will also be true for anyone running oscap 1.2.17 or earlier

Based on that we will need check that I removed in #11816 needs to be added back. Then this PR will need to be updated for the project to understand OVAL version 5.11.0 (maybe), 5.11.1, 5.11.2.

I didn't check the oval, but does it have a hard requirement on 5.11.2?

This relevant rule depends on a property (block_size) from partition_state that was introduced in 5.11.2. Without this property the rule assessment can't be accurate, unfortunately. Today I reviewed the changelog from 5.11.1 to 5.11.2 and didn't find too much technical changes. Most was about documentation, some changes were related to Windows probes and few other minor changes.

It is likely that we don't have any other case in the project where we use something only in 5.11.2 but not in 5.11.1.
But it would be interesting to better check and run some tests.

@dodys
Copy link
Contributor

dodys commented Apr 26, 2024

@marcusburghardt that's not entirely true depending on the vendor currently ubuntu 20.04 and 22.04 support oval 5.11.1 only that will also be true for anyone running oscap 1.2.17 or earlier

Based on that we will need check that I removed in #11816 needs to be added back. Then this PR will need to be updated for the project to understand OVAL version 5.11.0 (maybe), 5.11.1, 5.11.2.

I didn't check the oval, but does it have a hard requirement on 5.11.2?

This relevant rule depends on a property (block_size) from partition_state that was introduced in 5.11.2. Without this property the rule assessment can't be accurate, unfortunately. Today I reviewed the changelog from 5.11.1 to 5.11.2 and didn't find too much technical changes. Most was about documentation, some changes were related to Windows probes and few other minor changes.

It is likely that we don't have any other case in the project where we use something only in 5.11.2 but not in 5.11.1. But it would be interesting to better check and run some tests.

@mpurg would you be able to test this rule auditd_audispd_configure_sufficiently_large_partition on both focal and jammy, as we use it for STIG on both and I don't remember seeing issues with it in the past but I might have overlooked the 5.11.2 requirement.

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

@marcusburghardt Have you tried to use yamlfilecontent with OVAL version set to 5.11.2? It is technically defined in OVAL 5.11.3 rogue schema that comes with openscap. I have a vague memory of problems we had with this patch version component.

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

Yeah, that's what I got for oval version 5.11.2 in oscap test suite:

262: File '/home/ekolesni/Devel/openscap/tests/probes/yamlfilecontent/test_probes_yamlfilecontent_array.xml' line 49:
Element '{http://oval.mitre.org/XMLSchema/oval-definitions-5#independent}yamlfilecontent_test': This element is not
expected. Expected is one of ( {http://oval.mitre.org/XMLSchema/oval-definitions-5}test, {http://oval.mitre.org/
XMLSchema/oval-definitions-5#aix}interim_fix_test, {http://oval.mitre.org/XMLSchema/oval-definitions-5#aix}
fileset_test, {http://oval.mitre.org/XMLSchema/oval-definitions-5#aix}fix_test, {http://oval.mitre.org/XMLSchema/oval-
definitions-5#aix}no_test, {http://oval.mitre.org/XMLSchema/oval-definitions-5#aix}oslevel_test, {http://oval.mitre.org/
XMLSchema/oval-definitions-5#android}appmanager_test, {http://oval.mitre.org/XMLSchema/oval-
definitions-5#android}bluetooth_test, {http://oval.mitre.org/XMLSchema/oval-definitions-5#android}camera_test,
{http://oval.mitre.org/XMLSchema/oval-definitions-5#android}certificate_test ).

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

Oh, okay. The oscap oval eval behaves differently compared to oscap xccdf eval. And OpenShift tests are quite happy. Which is kinda strange.

@yuumasato @lbragstad Is the OCP4 CI actually trying to execute/interpet the content or it just builds it?

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

Apparently it does work. If I use 5.11 or 5.11.2 in a ds that is evaluated via oscap xccdf eval ... the scanner does not check if a test is defined the schema. The only thing that cares about the OVAL version in this case is the probe, if wanted to.

But! I wonder if it will make sense to have OVAL version set to 5.11.3 in that case?

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

Well, schematron validation dies on a data stream with OVAL 5.11.3 definition. So, disregard all of the above. Nice talking to you. 5.11.2 it is, but watch for yamlfilecontent problems, they still could appear in some corner case.

@dodys
Copy link
Contributor

dodys commented Apr 29, 2024

@marcusburghardt that's not entirely true depending on the vendor currently ubuntu 20.04 and 22.04 support oval 5.11.1 only that will also be true for anyone running oscap 1.2.17 or earlier

Based on that we will need check that I removed in #11816 needs to be added back. Then this PR will need to be updated for the project to understand OVAL version 5.11.0 (maybe), 5.11.1, 5.11.2.

I didn't check the oval, but does it have a hard requirement on 5.11.2?

This relevant rule depends on a property (block_size) from partition_state that was introduced in 5.11.2. Without this property the rule assessment can't be accurate, unfortunately. Today I reviewed the changelog from 5.11.1 to 5.11.2 and didn't find too much technical changes. Most was about documentation, some changes were related to Windows probes and few other minor changes.
It is likely that we don't have any other case in the project where we use something only in 5.11.2 but not in 5.11.1. But it would be interesting to better check and run some tests.

@mpurg would you be able to test this rule auditd_audispd_configure_sufficiently_large_partition on both focal and jammy, as we use it for STIG on both and I don't remember seeing issues with it in the past but I might have overlooked the 5.11.2 requirement.

@marcusburghardt I've tested it and with the change from #11816 the rule started to fail, while previously it would return Result notchecked

I think because we still need to build the profiles on versions that don't support 5.11.2, it would be nice to still maintain 5.11 and/or add 5.11.1 to the build system. Regarding the OVAL maybe reverting the PR mentioned above is a good idea, until we land a fix specific to Ubuntu.

@evgenyz
Copy link
Member

evgenyz commented Apr 29, 2024

It could be a product feature (property), actually. With 5.11.2 (the latest) being the default.

@marcusburghardt marcusburghardt marked this pull request as draft April 30, 2024 07:40
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Apr 30, 2024
@marcusburghardt
Copy link
Member Author

Thanks @Mab879 @dodys @mpurg @evgenyz and @jan-cerny for the review, comments and tests.

We are starting the stabilization phase for 0.1.73 and I concluded a change in the OVAL version deserves more investigation and tests. Doing so now would be premature. This is the reason I moved this PR to Draft now.
I plan to create a mechanism to make the OVAL version conditional to the product. I will keep this draft around for some days until I can work on this again.

In the meantime I believe it is reasonable to revert the #11816 , which basically means the auditd_audispd_configure_sufficiently_large_partition will continue without an OVAL in 0.1.73. I will quickly propose a PR for this.

marcusburghardt added a commit to marcusburghardt/content that referenced this pull request Apr 30, 2024
Thanks to ComplianceAsCode#11816 it were releaved more issues related to OVAL version.
After discussions in ComplianceAsCode#11903 it was concluded to revert this change for
now so we can better work a long-term solution.
marcusburghardt added a commit to marcusburghardt/content that referenced this pull request Apr 30, 2024
Thanks to ComplianceAsCode#11816 it were revealed more issues related to OVAL version.
After discussions in ComplianceAsCode#11903 it was concluded to revert this change for
now so we can better work a long-term solution.
@vojtapolasek vojtapolasek modified the milestones: 0.1.73, 0.1.74 Apr 30, 2024
@openshift-merge-robot openshift-merge-robot added the needs-rebase Used by openshift-ci bot. label May 1, 2024
@openshift-merge-robot
Copy link
Collaborator

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yuumasato
Copy link
Member

Oh, okay. The oscap oval eval behaves differently compared to oscap xccdf eval. And OpenShift tests are quite happy. Which is kinda strange.

@yuumasato @lbragstad Is the OCP4 CI actually trying to execute/interpet the content or it just builds it?

@evgenyz The OCP4 CI runs an e2e test, it executes scans with the profiles and assesses the rule results.
Do you have concerns that yamlfilecontent_test might get broken?

@marcusburghardt
Copy link
Member Author

Thanks @Mab879 @dodys @mpurg @evgenyz and @jan-cerny for the review, comments and tests.

We are starting the stabilization phase for 0.1.73 and I concluded a change in the OVAL version deserves more investigation and tests. Doing so now would be premature. This is the reason I moved this PR to Draft now. I plan to create a mechanism to make the OVAL version conditional to the product. I will keep this draft around for some days until I can work on this again.

In the meantime I believe it is reasonable to revert the #11816 , which basically means the auditd_audispd_configure_sufficiently_large_partition will continue without an OVAL in 0.1.73. I will quickly propose a PR for this.

This Draft is open for some time and I have no plans to work on this soon. I am also considering how worth would be to create this mechanism consider the issue affects only one rule. So, I am closing it now. It can be reopened if we have stronger demand for this in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Used by openshift-ci bot. needs-rebase Used by openshift-ci bot. OVAL OVAL update. Related to the systems assessments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

auditd_audispd_configure_sufficiently_large_partition reports unknown after #11816
7 participants