Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hugepages 2Mi and 1Gi fields to ResourceDescription and pass them to the statefulset #2311

Merged
merged 13 commits into from
Jan 4, 2024
Merged

Add hugepages 2Mi and 1Gi fields to ResourceDescription and pass them to the statefulset #2311

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
33a7041
Add hugepages-2Mi and 1Gi to ResourceDescription type and crd (#1549,…
silenium-dev Apr 28, 2023
a5387a1
Fix typo
silenium-dev Apr 30, 2023
96c784e
Add tests for hugepages resource requests/limits
silenium-dev May 4, 2023
1643d10
Add tests for hugepages resource requests/limits on sidecars
silenium-dev May 4, 2023
9ccc78a
Remove additional fields from minimal-postgres-manifest
silenium-dev May 4, 2023
5bb950a
Add docs for hugepages support
silenium-dev May 4, 2023
59e777a
Cleanup diff for docs
silenium-dev Jun 26, 2023
1bdc7c7
Add link to kubernetes docs on hugepages
silenium-dev Oct 14, 2023
3e7ffa1
Add tests for hugepages not being set on container if not requested i…
silenium-dev Oct 14, 2023
06004a1
Add hugepages resources fields to manifest docs
silenium-dev Nov 19, 2023
e48163b
Add hugepages resources fields to complete manifest example
silenium-dev Nov 19, 2023
2d4eccf
Add hugepages resources fields to chart crd
silenium-dev Nov 19, 2023
ea04bb8
Update manifests/complete-postgres-manifest.yaml
FxKu Jan 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/user.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,6 +688,25 @@ manifest the operator will raise the limits to the configured minimum values.
If no resources are defined in the manifest they will be obtained from the
configured [default requests](reference/operator_parameters.md#kubernetes-resource-requests).

### HugePages support
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this comes with K8s 1.28

Can we also link https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The operator supports [HugePages](https://www.postgresql.org/docs/15/kernel-resources.html#LINUX-HUGEPAGES).
To enable HugePages, set the matching resource requests and/or limits in the manifest:

```yaml
spec:
resources:
requests:
hugepages-2Mi: 250Mi
hugepages-1Gi: 1Gi
limits:
hugepages-2Mi: 500Mi
hugepages-1Gi: 2Gi
```

There are no minimums or maximums and the default is 0 for both HugePage sizes,
but Kubernetes will not spin up the pod if the requested HugePages cannot be allocated.

## Use taints, tolerations and node affinity for dedicated PostgreSQL nodes

To ensure Postgres pods are running on nodes without any other application pods,
Expand Down
12 changes: 12 additions & 0 deletions manifests/postgresql.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,12 @@ spec:
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
# Note: the value specified here must not be zero or be higher
# than the corresponding limit.
hugepages-2Mi:
type: string
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
hugepages-1Gi:
type: string
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
requests:
type: object
properties:
Expand All @@ -448,6 +454,12 @@ spec:
memory:
type: string
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
hugepages-2Mi:
type: string
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
hugepages-1Gi:
type: string
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
schedulerName:
type: string
serviceAnnotations:
Expand Down
21 changes: 19 additions & 2 deletions pkg/apis/acid.zalan.do/v1/crds.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ package v1
import (
"fmt"

acidzalando "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do"
"github.com/zalando/postgres-operator/pkg/util"
apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

acidzalando "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do"
"github.com/zalando/postgres-operator/pkg/util"
)

// CRDResource* define names necesssary for the k8s CRD API
Expand Down Expand Up @@ -684,6 +685,14 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
"hugepages-2Mi": {
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
"hugepages-1Gi": {
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
},
},
"requests": {
Expand All @@ -697,6 +706,14 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
"hugepages-2Mi": {
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
"hugepages-1Gi": {
Type: "string",
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
},
},
},
},
Expand Down
6 changes: 4 additions & 2 deletions pkg/apis/acid.zalan.do/v1/postgresql_type.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,10 @@ type PostgresqlParam struct {

// ResourceDescription describes CPU and memory resources defined for a cluster.
type ResourceDescription struct {
CPU string `json:"cpu"`
Memory string `json:"memory"`
CPU string `json:"cpu"`
Memory string `json:"memory"`
HugePages2Mi string `json:"hugepages-2Mi"`
HugePages1Gi string `json:"hugepages-1Gi"`
}

// Resources describes requests and limits for the cluster resouces.
Expand Down
20 changes: 17 additions & 3 deletions pkg/cluster/k8sres.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ import (
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/intstr"

"golang.org/x/exp/maps"
batchv1 "k8s.io/api/batch/v1"
"k8s.io/apimachinery/pkg/labels"

acidv1 "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do/v1"
"github.com/zalando/postgres-operator/pkg/spec"
"github.com/zalando/postgres-operator/pkg/util"
Expand All @@ -28,9 +32,6 @@ import (
"github.com/zalando/postgres-operator/pkg/util/k8sutil"
"github.com/zalando/postgres-operator/pkg/util/patroni"
"github.com/zalando/postgres-operator/pkg/util/retryutil"
"golang.org/x/exp/maps"
batchv1 "k8s.io/api/batch/v1"
"k8s.io/apimachinery/pkg/labels"
)

const (
Expand Down Expand Up @@ -267,6 +268,19 @@ func fillResourceList(spec acidv1.ResourceDescription, defaults acidv1.ResourceD
}
}

if spec.HugePages2Mi != "" {
requests[v1.ResourceHugePagesPrefix+"2Mi"], err = resource.ParseQuantity(spec.HugePages2Mi)
if err != nil {
return nil, fmt.Errorf("could not parse hugepages-2Mi quantity: %v", err)
}
}
if spec.HugePages1Gi != "" {
requests[v1.ResourceHugePagesPrefix+"1Gi"], err = resource.ParseQuantity(spec.HugePages1Gi)
if err != nil {
return nil, fmt.Errorf("could not parse hugepages-1Gi quantity: %v", err)
}
}

return requests, nil
}

Expand Down
95 changes: 93 additions & 2 deletions pkg/cluster/k8sres_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@ import (
"fmt"
"reflect"
"sort"
"time"

"testing"
"time"

"github.com/stretchr/testify/assert"

Expand Down Expand Up @@ -2979,6 +2978,98 @@ func TestGenerateResourceRequirements(t *testing.T) {
ResourceLimits: acidv1.ResourceDescription{CPU: "1", Memory: "2Gi"},
},
},
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we amend the other test to test for no huge pages set? Given its K8s 1.28 feature lets make sure its not set if not enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a test to cover this

subTest: "test HugePages are passed through to the postgres container",
config: config.Config{
Resources: configResources,
PodManagementPolicy: "ordered_ready",
},
pgSpec: acidv1.Postgresql{
ObjectMeta: metav1.ObjectMeta{
Name: clusterName,
Namespace: namespace,
},
Spec: acidv1.PostgresSpec{
Resources: &acidv1.Resources{
ResourceRequests: acidv1.ResourceDescription{
HugePages2Mi: "128Mi",
HugePages1Gi: "1Gi",
},
ResourceLimits: acidv1.ResourceDescription{
HugePages2Mi: "256Mi",
HugePages1Gi: "2Gi",
},
},
TeamID: "acid",
Volume: acidv1.Volume{
Size: "1G",
},
},
},
expectedResources: acidv1.Resources{
ResourceRequests: acidv1.ResourceDescription{
CPU: "100m",
Memory: "100Mi",
HugePages2Mi: "128Mi",
HugePages1Gi: "1Gi",
},
ResourceLimits: acidv1.ResourceDescription{
CPU: "1",
Memory: "500Mi",
HugePages2Mi: "256Mi",
HugePages1Gi: "2Gi",
},
},
},
{
subTest: "test HugePages are passed through on sidecars",
config: config.Config{
Resources: configResources,
PodManagementPolicy: "ordered_ready",
},
pgSpec: acidv1.Postgresql{
ObjectMeta: metav1.ObjectMeta{
Name: clusterName,
Namespace: namespace,
},
Spec: acidv1.PostgresSpec{
Sidecars: []acidv1.Sidecar{
{
Name: "test-sidecar",
DockerImage: "test-image",
Resources: &acidv1.Resources{
ResourceRequests: acidv1.ResourceDescription{
HugePages2Mi: "128Mi",
HugePages1Gi: "1Gi",
},
ResourceLimits: acidv1.ResourceDescription{
HugePages2Mi: "256Mi",
HugePages1Gi: "2Gi",
},
},
},
},
TeamID: "acid",
Volume: acidv1.Volume{
Size: "1G",
},
},
},
expectedResources: acidv1.Resources{
ResourceRequests: acidv1.ResourceDescription{
CPU: "100m",
Memory: "100Mi",
HugePages2Mi: "128Mi",
HugePages1Gi: "1Gi",
},
ResourceLimits: acidv1.ResourceDescription{
CPU: "1",
Memory: "500Mi",
HugePages2Mi: "256Mi",
HugePages1Gi: "2Gi",
},
},
},
}

for _, tt := range tests {
Expand Down