Add ValidatingAdmissionPolicy tests for RayClusters and AppWrappers #300
Conversation
| vapb.Spec.PolicyName = "none" | ||
| _, err = test.Client().Core().AdmissionregistrationV1().ValidatingAdmissionPolicyBindings().Update(test.Ctx(), vapb, metav1.UpdateOptions{}) | ||
| test.Expect(err).ToNot(HaveOccurred()) | ||
| time.Sleep(2 * time.Second) // Wait for the ValidatingAdmissionPolicyBinding to be updated |
There was a problem hiding this comment.
You should be able to use test.Eventually here and get the VAP and check for an expected updated value
There was a problem hiding this comment.
I've given this a try just now, with the debugger I can see it does find the resource to be updated with the updated value, but the tests that follow fail as if the resource had not yet been updated. Perhaps, k8s needs time to propagate these changes somewhere else too? - It seems to need an initial couple of seconds after updating the resource.
There was a problem hiding this comment.
I've tried implementing:
test.Eventually(func() bool {
vapb, _ = test.Client().Core().AdmissionregistrationV1().ValidatingAdmissionPolicyBindings().Get(test.Ctx(), vapb.Name, metav1.GetOptions{})
return vapb.Spec.PolicyName == "none"
}, 5*time.Second, 2*time.Second).Should(BeTrue())
The issue found is that the 'retry' never gets executed as the updated value is found immediately to be true:
}, 5*time.Second, 2*time.Second).Should(BeTrue())
Using a sleep is providing consistent results. - Given it's a test, perhaps this is okay.
There was a problem hiding this comment.
Additionally, @Bobbins228 and I looked for a status in both resources in case there was any other indicative of propagation readiness. - No indication found.
There was a problem hiding this comment.
What exactly needs to propagated?
There was a problem hiding this comment.
I removed the sleeps to avoid flaky tests in the future. I'm using Eventually from gomega to create RayClusters that should eventually be evaluating to True, ensuring changes have been propagated to the VAPB resource.
ChristianZaccaria
force-pushed
the
vap-test
branch
2 times, most recently
from
3d97c3d to
73029a8
Compare
| _, err = test.Client().Core().AdmissionregistrationV1().ValidatingAdmissionPolicyBindings().Update(test.Ctx(), vapb, metav1.UpdateOptions{}) | ||
| test.Expect(err).ToNot(HaveOccurred()) |
There was a problem hiding this comment.
@Bobbins228 thanks for taking the time to test Mark!
Tried out these tests a couple of times and the testing examples look sound.
I am not too sure how we can confirm that the VAP updates correctly rather than using sleep but on the testing side /lgtm
These lines ensure that the update happens without errors. Eventhough we ensure the update does happen, the test runs so quickly that subsequent tests fail unless we allow a couple of seconds for it to 'settle'.
ChristianZaccaria
force-pushed
the
vap-test
branch
from
73029a8 to
494f7ac
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: KPostOffice The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Description
Part of: https://issues.redhat.com/browse/RHOAIENG-14731
How Has This Been Tested?
For context, the VAP-B resources will be applied by the RHOAI Operator based on the OCP version it is installed on.
In the meantime, this can be tested by manually applying the resources:
>=v4.17oc applyboth resources from this directorygo test -timeout 1m ./tests/odh/validating_admission_policy_test.go ./tests/odh/support.go.Additional Thoughts:
sleepsin the tests when updating the VAP resource to allow sufficient time for changes to take effect. Thinking of a better way.Merge criteria: