Feat test report #1244

timhuynh94 · 2025-01-17T20:51:01Z

No description provided.

* refactor(pipeline): use server API types for pipeline and migrate compiler types * gci * feat: add sender rule for pipelines --------- Co-authored-by: David May <[email protected]>

* chore(lint): address existing linter issues * remove dupl from exclusion list

* enhance(build): add fork field for OIDC * fix test * integration test update

* enhance(yaml): allow for users to parse pipelines using old library * testing file for internal yaml * chore(compiler): convert unmarshaled buildkite to go-yaml * remove tests used in later PRs * lintfix * fix schema * gci

…1230) Co-authored-by: David May <[email protected]>

Co-authored-by: david may <[email protected]>

* init commit * feat(repo): add pending approval timeout * fix test * remove dead code --------- Co-authored-by: David May <[email protected]>

…r have dupl anchors in map (#1232)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

…ploy config (#1239)

…rams (#1242)

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

golangci

🚫 [golangci] _{reported by reviewdog 🐶}
unnecessary trailing newline (whitespace)

server/storage/storage.go

Line 56 in 14660f5

}

🚫 [golangci] _{reported by reviewdog 🐶}
unnecessary trailing newline (whitespace)

server/storage/minio/download.go

Line 95 in 14660f5

}

🚫 [golangci] _{reported by reviewdog 🐶}
expressions should not be cuddled with blocks (wsl)

server/api/admin/storage.go

Line 65 in 14660f5

l.Debugf("bucket name: %s", input.BucketName)

🚫 [golangci] _{reported by reviewdog 🐶}
return statements should not be cuddled if block has more than two lines (wsl)

server/api/admin/storage.go

Line 70 in 14660f5

return

🚫 [golangci] _{reported by reviewdog 🐶}
return statements should not be cuddled if block has more than two lines (wsl)

server/api/admin/storage.go

Line 393 in 14660f5

return

🚫 [golangci] _{reported by reviewdog 🐶}
return statements should not be cuddled if block has more than two lines (wsl)

server/api/admin/storage.go

Line 452 in 14660f5

return

🚫 [golangci] _{reported by reviewdog 🐶}
assignments should only be cuddled with other assignments (wsl)

server/api/admin/storage.go

Line 395 in 14660f5

err = storage.FromGinContext(c).Download(ctx, input)

🚫 [golangci] _{reported by reviewdog 🐶}
only one cuddle assignment allowed before if statement (wsl)

server/api/admin/storage.go

Line 396 in 14660f5

if err != nil {

🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

server/api/admin/storage.go

Line 390 in 14660f5

if input.FilePath == "" {

🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

server/api/admin/storage.go

Line 312 in 14660f5

if input.Bucket.BucketName == "" || input.ObjectName == "" {

🚫 [golangci] _{reported by reviewdog 🐶}
assignments should only be cuddled with other assignments (wsl)

server/api/admin/storage.go

Line 322 in 14660f5

err = storage.FromGinContext(c).Upload(ctx, input)

🚫 [golangci] _{reported by reviewdog 🐶}
only one cuddle assignment allowed before if statement (wsl)

server/api/admin/storage.go

Line 323 in 14660f5

if err != nil {

🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

server/api/admin/storage.go

Line 385 in 14660f5

if input.Bucket.BucketName == "" || input.ObjectName == "" {

🚫 [golangci] _{reported by reviewdog 🐶}
only one cuddle assignment allowed before if statement (wsl)

server/api/admin/storage.go

Line 67 in 14660f5

if err != nil {

🚫 [golangci] _{reported by reviewdog 🐶}
assignments should only be cuddled with other assignments (wsl)

server/api/types/test-report/code_coverage_file.go

Line 125 in 14660f5

c.DirectoryName = &v

🚫 [golangci] _{reported by reviewdog 🐶}
expressions should not be cuddled with blocks (wsl)

server/storage/storage.go

Line 43 in 14660f5

logrus.Debug("creating storage client from setup")

🚫 [golangci] _{reported by reviewdog 🐶}
declarations should never be cuddled (wsl)

server/storage/minio/create_bucket.go

Line 12 in 14660f5

var opts minio.MakeBucketOptions

🚫 [golangci] _{reported by reviewdog 🐶}
append only allowed to cuddle with appended value (wsl)

server/storage/minio/list_objects.go

Line 19 in 14660f5

objects = append(objects, object.Key)

🚫 [golangci] _{reported by reviewdog 🐶}
ranges should only be cuddled with assignments used in the iteration (wsl)

server/storage/minio/list_objects.go

Line 15 in 14660f5

for object := range objectCh {

🚫 [golangci] _{reported by reviewdog 🐶}
block should not end with a whitespace (or comment) (wsl)

server/storage/minio/minio.go

Line 89 in 14660f5

}

🚫 [golangci] _{reported by reviewdog 🐶}
block should not start with a whitespace (wsl)

server/storage/minio/minio.go

Line 112 in 14660f5

    
           func NewTest(endpoint, accessKey, secretKey string, secure bool) (*MinioClient, error) {

🚫 [golangci] _{reported by reviewdog 🐶}
directive //nolint:revive // ignore returning unexported client is unused for linter "revive" (nolintlint)

server/storage/minio/minio.go

Line 111 in 14660f5

//nolint:revive // ignore returning unexported client

github-actions · 2025-01-17T21:10:04Z

api/types/test-report/test_case.go

+func (t *TestCase) Duration() string {
+	// check if the test case doesn't have a started timestamp
+	if t.GetStarted() == 0 {
+		return "..."


🚫 [golangci] _{reported by reviewdog 🐶}
string ... has 3 occurrences, make it a constant (goconst)

github-actions · 2025-01-17T21:10:05Z

storage/minio/download.go

+
+	logrus.Debugf("getting object in bucket %s from path: %s", object.Bucket.BucketName, object.ObjectName)
+
+	logrus.Infof("%s to download", humanize.Bytes(uint64(objInfo.Size)))


🚫 [golangci] _{reported by reviewdog 🐶}
G115: integer overflow conversion int64 -> uint64 (gosec)

github-actions · 2025-01-17T21:10:05Z

storage/minio/download.go

+		return err
+	}
+
+	logrus.Infof("downloaded %s to %s on local filesystem", humanize.Bytes(uint64(stat.Size())), filename)


🚫 [golangci] _{reported by reviewdog 🐶}
G115: integer overflow conversion int64 -> uint64 (gosec)

github-actions · 2025-01-17T21:10:06Z

storage/minio/minio.go

+}
+
+// MinioClient implements the Storage interface using MinIO.
+type MinioClient struct {


🚫 [golangci] _{reported by reviewdog 🐶}
exported: type name will be used as minio.MinioClient by other packages, and that stutters; consider calling this Client (revive)

github-actions · 2025-01-17T21:10:06Z

storage/context.go

+
+// WithContext adds the minio Storage to the context
+func WithContext(ctx context.Context, storage Storage) context.Context {
+	return context.WithValue(ctx, key, storage)


🚫 [golangci] _{reported by reviewdog 🐶}
context-keys-type: should not use basic type string as key in context.WithValue (revive)

github-actions · 2025-01-17T21:10:10Z

api/types/test-report/test_suite.go

+//
+// When the provided TestSuite type is nil, it
+// will set nothing and immediately return.
+func (t *TestSuite) SetStartTs(v int64) {


🚫 [golangci] _{reported by reviewdog 🐶}
ST1003: method SetStartTs should be SetStartTS (stylecheck)

github-actions · 2025-01-17T21:10:10Z

api/types/test-report/test_suite.go

+//
+// When the provided TestSuite type is nil, or the field within
+// the type is nil, it returns the zero value for the field.
+func (t *TestSuite) GetStartTs() int64 {


🚫 [golangci] _{reported by reviewdog 🐶}
ST1003: method GetStartTs should be GetStartTS (stylecheck)

github-actions · 2025-01-17T21:10:10Z

compiler/types/pipeline/container.go

@@ -48,6 +48,7 @@ type (
 		Pull        string            `json:"pull,omitempty"        yaml:"pull,omitempty"`
 		Ruleset     Ruleset           `json:"ruleset,omitempty"     yaml:"ruleset,omitempty"`
 		Secrets     StepSecretSlice   `json:"secrets,omitempty"     yaml:"secrets,omitempty"`
+		TestReport  TestReport        `json:"test_report,omitempty"     yaml:"test_report,omitempty"`


🚫 [golangci] _{reported by reviewdog 🐶}
tag is not aligned, should be: json:"test_report,omitempty" yaml:"test_report,omitempty" (tagalign)

github-actions · 2025-01-17T21:10:10Z

compiler/types/pipeline/test_report.go

+	//
+	// swagger:model PipelineTestReport
+	TestReport struct {
+		Results     []string `yaml:"results,omitempty" json:"results,omitempty"`


🚫 [golangci] _{reported by reviewdog 🐶}
tag is not aligned, should be: yaml:"results,omitempty" json:"results,omitempty" (tagalign)

github-actions · 2025-01-17T21:10:11Z

storage/storage.go

+
+	return New(_setup)
+
+}


🚫 [golangci] _{reported by reviewdog 🐶}
unnecessary trailing newline (whitespace)

storage/minio/download.go

+		return err
+	}
+
+	stat, err := os.Stat(object.FilePath)


To fix the problem, we need to validate the object.FilePath to ensure it does not contain any path traversal characters or sequences that could lead to accessing unintended files or directories. We can achieve this by checking for the presence of path separators (/ or \) and parent directory references (..). If any of these are found, we should reject the input.

The best way to fix the problem without changing existing functionality is to add a validation step in the DownloadObject function in api/admin/storage.go before calling the Download method. This ensures that the object.FilePath is safe to use.

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

golangci

🚫 [golangci] _{reported by reviewdog 🐶}
directive //nolint:revive // ignore returning unexported client is unused for linter "revive" (nolintlint)

server/storage/minio/minio.go

Line 111 in 05a09bb

//nolint:revive // ignore returning unexported client

github-actions · 2025-01-20T19:43:59Z

compiler/types/pipeline/test_report_test.go

@@ -0,0 +1,29 @@
+package pipeline


🚫 [golangci] _{reported by reviewdog 🐶}
Missed header for check (goheader)

github-actions · 2025-01-20T19:43:59Z

storage/context.go

+package storage
+
+import (
+	"context"


🚫 [golangci] _{reported by reviewdog 🐶}
File is not properly formatted (gci)

github-actions · 2025-01-20T19:43:59Z

storage/service.go

+package storage
+
+import (
+	"context"


🚫 [golangci] _{reported by reviewdog 🐶}
File is not properly formatted (gci)

github-actions · 2025-01-20T19:43:59Z

storage/setup.go

+package storage
+
+import (
+	"fmt"


🚫 [golangci] _{reported by reviewdog 🐶}
File is not properly formatted (gci)

github-actions · 2025-01-20T19:44:00Z

storage/storage.go

+		return nil, fmt.Errorf("invalid storage driver provided: %s", s.Driver)
+	}
+
+}


🚫 [golangci] _{reported by reviewdog 🐶}
unnecessary trailing newline (whitespace)

github-actions · 2025-01-20T19:44:03Z

api/admin/storage.go

+		return
+	}
+	err = storage.FromGinContext(c).Upload(ctx, input)
+	if err != nil {


🚫 [golangci] _{reported by reviewdog 🐶}
only one cuddle assignment allowed before if statement (wsl)

github-actions · 2025-01-20T19:44:03Z

api/admin/storage.go

+
+		return
+	}
+	if input.Bucket.BucketName == "" || input.ObjectName == "" {


🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

github-actions · 2025-01-20T19:44:03Z

api/admin/storage.go

+		util.HandleError(c, http.StatusBadRequest, retErr)
+		return
+	}
+	if input.FilePath == "" {


🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

github-actions · 2025-01-20T19:44:03Z

api/admin/storage.go

+
+		return
+	}
+	if input.Bucket.BucketName == "" || input.ObjectName == "" {


🚫 [golangci] _{reported by reviewdog 🐶}
if statements should only be cuddled with assignments (wsl)

github-actions · 2025-01-20T19:44:03Z

api/admin/storage.go

+
+		return
+	}
+	l.Debugf("bucket name: %s", input.BucketName)


🚫 [golangci] _{reported by reviewdog 🐶}
expressions should not be cuddled with blocks (wsl)

TimHuynh and others added 30 commits December 12, 2024 09:41

ground work

a2042f3

feat: add sender rule for pipelines (#1206)

8fb48dc

* refactor(pipeline): use server API types for pipeline and migrate compiler types * gci * feat: add sender rule for pipelines --------- Co-authored-by: David May <[email protected]>

chore(lint): add struct tag align rule (#1216)

a3b1bff

chore(lint): address existing linter issues (#1218)

2245376

* chore(lint): address existing linter issues * remove dupl from exclusion list

enhance(build)!: add fork field for OIDC (#1221)

5a1f403

* enhance(build): add fork field for OIDC * fix test * integration test update

enhance(compiler): cache templates per compilation (#1224)

9a5bf00

ground work

58718a1

working admin endpoints for bucket management

8e9cd8e

working admin endpoints for object operations

3239b53

all endpoints for storage

8a95775

some working test

a4f29d0

working tests

41df509

keys for test report step

86d822d

feat: deployment config for expected params and targets (#1214)

83afd15

resolve conflicts with buildkite

8b5b79f

feat: opt-in gh app integration (#1217)

9328ecd

chore(yaml): add go-yaml types package (#1225)

b16622f

* enhance(yaml): allow for users to parse pipelines using old library * testing file for internal yaml * chore(compiler): convert unmarshaled buildkite to go-yaml * remove tests used in later PRs * lintfix * fix schema * gci

resolve conflicts

3156794

rebase

57f8f9a

enhance(yaml): allow for users to set version legacy for buildkite (#…

09da5df

…1230) Co-authored-by: David May <[email protected]>

refactor(db/build): drop source index, add event index (#1228)

114ae7b

Co-authored-by: david may <[email protected]>

feat(repo)!: add pending approval timeout (#1227)

fd16b7b

* init commit * feat(repo): add pending approval timeout * fix test * remove dead code --------- Co-authored-by: David May <[email protected]>

enhance(yaml): silent fix anchor merges in YAML maps (#1231)

68547d6

fix(webhook): use correct repo variable for enqueue (#1234)

c7c1d0a

feat(pipeline)!: add warnings to pipelines that use legacy YAML lib o…

f43e2d1

…r have dupl anchors in map (#1232)

feat: status endpoints (#1235)

f1a98ac

enhance(deploy): validate on CreateDeployment (#1236)

bf2313d

enhance(metrics): include route specific queue length (#1237)

9a532cf

chore(deps): update codecov/codecov-action action to v5 (#1222)

9f13f97

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

fix(deps): update module github.com/google/go-github/v65 to v68 (#1229)

3920eab

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

wass3rw3rk and others added 9 commits January 17, 2025 15:00

chore: bump go (#1238)

084fea3

fix(deployment): use Config instead of ConfigBackoff when fetching de…

52df828

…ploy config (#1239)

fix(build): use item build host (#1241)

4eee25d

fix(build): ensure count/pagination links account for filter/query pa…

df21b7a

…rams (#1242)

fix(yaml): improved messaging (#1243)

d24fd37

keys for test report step

a051afa

resolve conflicts with buildkite

5254aad

Merge branch 'main' into feat_test_report

a138127

mod tidy

14660f5

github-actions bot reviewed Jan 17, 2025

View reviewed changes

github-advanced-security bot found potential problems Jan 17, 2025

View reviewed changes

fields for buildkite and ymal

05a09bb

github-actions bot reviewed Jan 20, 2025

View reviewed changes

@@ -394,2 +394,8 @@
             	}
+            	// Validate the file path to prevent path traversal
+            	if strings.Contains(input.FilePath, "/") || strings.Contains(input.FilePath, "\\") || strings.Contains(input.FilePath, "..") {
+            		retErr := fmt.Errorf("invalid file path")
+            		util.HandleError(c, http.StatusBadRequest, retErr)
+            		return
+            	}
             	err = storage.FromGinContext(c).Download(ctx, input)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat test report #1244

Feat test report #1244

timhuynh94 commented Jan 17, 2025

github-actions bot left a comment

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

github-actions bot Jan 17, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

github-actions bot left a comment

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025

github-actions bot Jan 20, 2025


		logrus.Debugf("getting object in bucket %s from path: %s", object.Bucket.BucketName, object.ObjectName)

		logrus.Infof("%s to download", humanize.Bytes(uint64(objInfo.Size)))

Feat test report #1244

Are you sure you want to change the base?

Feat test report #1244

Conversation

timhuynh94 commented Jan 17, 2025

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot Jan 17, 2025

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment

github-actions bot Jan 20, 2025

Choose a reason for hiding this comment