Given all in-model constraints IDs #2090
Conversation
aj-stein-gsa
force-pushed
the
2088-constraint-ids
branch
from
0432b04 to
a7b833a
Compare
aj-stein-gsa
force-pushed
the
2088-constraint-ids
branch
from
7629657 to
27eb4d7
Compare
aj-stein-gsa
force-pushed
the
2088-constraint-ids
branch
from
27eb4d7 to
4848333
Compare
aj-stein-gsa
changed the title
|
@wendellpiez - can you please assist with this PR. Robust profile resolution testing must be part of this PR. |
|
I saw your comment in the other PR, @iMichaela, which does not seem particularly related. What is the impact and how will we test adding constraint IDs break profile resolution? The constraints conceptually operate separate of constraints, no? |
|
With @aj-stein-gsa I wonder why profile resolution is implicated here. |
It came up in another comment in #2095 (comment) from @iMichaela, not I. It is why I wanted clarification.
I wanted to redirect that conversation here as it summarized possible concerns about this PR. EDIT: Updated to include the specific context from #2095, not just the top-level PR reference. |
|
Sorry, I accidentally posted this to #2095 thinking it was this PR (#2090 if you are reading email). We have a way of computing all the constraints and targets in liboscal-java, and weirdly just adding the constraint IDs caused some exceptions. That result is separate of the low-risk changes, but is interesting because it does mean I may have duplicated or missed something. So on that note, I will update will relevant information of that analysis. If NIST staff or community members want more details about methodology or the use of a consisten prefix (id="oscal-...), I am more than willing to discuss that. Let me know where, when, and how you would like that information. |
To make sure that when a profile is resolved, the IDs for constraints are carried on into the resolved profile correctly. I am not saying there is an issue, but rather wanted to make sure the profile resolver(s) oscal-cli and XSLT transformations work well, so the oscal-content can be updated and released. |
During testing with metaschema-framework/{metaschema-java,liboscal-java,
oscal-cli} harness, I was able to detect that I had erroneously prefixed
a constraint ID with `oscal=`, not the intended `oscal-` prefix, and
constraint builder function utilities picked this mistake up.
I have not touched the XSLT documentation pipeline in three to six months, but such a feature would be new and require additional development. There resulting constraint information is rendered into usnistgov/OSCAL-Reference, but not IDs. You can evidence by the fact a small minority of constraints, such as https://github.com/usnistgov/OSCAL/blob/v1.1.3/src/metaschema/oscal_ssp_metaschema.xml#L517C18-L522 |
|
The affected elements (within |
Re my commit message in c0eee8b, I have found the source of the error and it was accidental use of Again, @iMichaela or @wendellpiez, if you want a brief on the methodology or adhoc testing without sufficient automated testing in usnistgov/OSCAL, oscal-content, or related dependency repos, let us know. I am happy to summarize how I am doing this analysis for the benefit of you and others in the community, so others can follow suit. |
I have agreed several times, not sure how to make it more clear, but I did want to address a different but related issue I thought might be helpful to discuss. I can also test profile resolution but I am less concerned about that for that reason, @wendellpiez. |
The constraint with id="oscal-system-implementation-validated-by-index" had its corresponding index name erased in error, making this index-has-key constraint inoperable by tools conformant with the current version of the Metaschema spec. This change restores it back.
I was able to test constraint validation and profile resolution locally. I resolved profiles, both the NIST and FedRAMP profiles without issue. I cannot run the documentation generation pipeline in GitHub Actions from your org to do a full end-to-end test there, but this appears to be ready for review and possible merge. |
There was a problem hiding this comment.
With the qualification that I have not run the build or inspected outputs, I'm approving the PR. Having scanned it, it seems to me any problems would have been exposed and repaired, or will be exposed in CI/CD. TMK we have no resources that hardcode these ID values to point into the constraints, which would break. Almost by definition any other emergent problem would be a new bug or requirement discovery (good things). @aj-stein-gsa thanks for this!
|
I've verified that these constraint ID's cover all constraints in the repo Test run without AJ changes Test run with AJ changes |
Committer Notes
This PR closes #2088 once reviewed, approved, and merged.
All Submissions:
By submitting a pull request, you are agreeing to provide this contribution under the CC0 1.0 Universal public domain dedication.
(For reviewers: The wiki has guidance on code review and overall issue review for completeness.)
Changes to Core Features:
Have you written new tests for your core changes, as applicable?Have you included examples of how to use your new feature(s)?Have you updated the OSCAL website and readme documentation affected by the changes you made? Changes to the OSCAL website can be made in the OSCAL-Pages and OSCAL_Reference repositories.These changes will not be used in the document generation pipeline unless the current metaschema-xslt and oscal-xslt are adapted to generate references to the identifiers of constraints, but at this time the pipelines do not act on them and they are open flags for Metaschema definitions.