Repository navigation
Closed
Labels
area/testingIssue related to testing the operator-sdk and subcomponentsIssue related to testing the operator-sdk and subcomponentsgood first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.Denotes an issue ready for a new contributor, according to the "help wanted" guidelines.help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.Indicates that an issue or PR should not be auto-closed due to staleness.
Milestone
Description
Bug Report
Memcached sample is not passing in the operator-sdk scorecard bundle test.
What did you do?
$ cd testdata/go/v3/memcached-operator
$ kind create cluster
$ operator-sdk scorecard bundle --output=json
What did you expect to see?
- All scorecard bundle tests are called
- The sample pass in all tests
- The e2e test with scorecard calling all available tests
What did you see instead? Under which circumstances?
See the errors faced:
"name": "olm-status-descriptors",
"log": "Loaded ClusterServiceVersion: memcached-operator.v0.0.1\nLoaded 1 Custom Resources from alm-examples\n",
"state": "fail",
"errors": [
"memcacheds.cache.example.com does not have a status descriptor"
"name": "olm-crds-have-resources",
"log": "Loaded ClusterServiceVersion: memcached-operator.v0.0.1\n",
"state": "fail",
"errors": [
"Owned CRDs do not have resources specified"
Following the full result:
$ operator-sdk scorecard bundle --output=json
{
"kind": "TestList",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"items": [
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"olm-crds-have-resources"
],
"labels": {
"suite": "olm",
"test": "olm-crds-have-resources-test"
}
},
"status": {
"results": [
{
"name": "olm-crds-have-resources",
"log": "Loaded ClusterServiceVersion: memcached-operator.v0.0.1\n",
"state": "fail",
"errors": [
"Owned CRDs do not have resources specified"
]
}
]
}
},
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"olm-status-descriptors"
],
"labels": {
"suite": "olm",
"test": "olm-status-descriptors-test"
}
},
"status": {
"results": [
{
"name": "olm-status-descriptors",
"log": "Loaded ClusterServiceVersion: memcached-operator.v0.0.1\nLoaded 1 Custom Resources from alm-examples\n",
"state": "fail",
"errors": [
"memcacheds.cache.example.com does not have a status descriptor"
]
}
]
}
},
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"basic-check-spec"
],
"labels": {
"suite": "basic",
"test": "basic-check-spec-test"
}
},
"status": {
"results": [
{
"name": "basic-check-spec",
"state": "pass"
}
]
}
},
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"olm-bundle-validation"
],
"labels": {
"suite": "olm",
"test": "olm-bundle-validation-test"
}
},
"status": {
"results": [
{
"name": "olm-bundle-validation",
"log": "time=\"2021-02-22T11:17:19Z\" level=debug msg=\"Found manifests directory\" name=bundle-test\ntime=\"2021-02-22T11:17:19Z\" level=debug msg=\"Found metadata directory\" name=bundle-test\ntime=\"2021-02-22T11:17:19Z\" level=debug msg=\"Getting mediaType info from manifests directory\" name=bundle-test\ntime=\"2021-02-22T11:17:19Z\" level=info msg=\"Found annotations file\" name=bundle-test\ntime=\"2021-02-22T11:17:19Z\" level=info msg=\"Could not find optional dependencies file\" name=bundle-test\n",
"state": "pass"
}
]
}
},
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"olm-spec-descriptors"
],
"labels": {
"suite": "olm",
"test": "olm-spec-descriptors-test"
}
},
"status": {
"results": [
{
"name": "olm-spec-descriptors",
"log": "Loaded ClusterServiceVersion: memcached-operator.v0.0.1\nLoaded 1 Custom Resources from alm-examples\n",
"state": "fail",
"errors": [
"size does not have a spec descriptor"
],
"suggestions": [
"Add a spec descriptor for size"
]
}
]
}
},
{
"kind": "Test",
"apiVersion": "scorecard.operatorframework.io/v1alpha3",
"spec": {
"image": "quay.io/operator-framework/scorecard-test:v1.4.0",
"entrypoint": [
"scorecard-test",
"olm-crds-have-validation"
],
"labels": {
"suite": "olm",
"test": "olm-crds-have-validation-test"
}
},
"status": {
"results": [
{
"name": "olm-crds-have-validation",
"log": "Loaded 1 Custom Resources from alm-examples\nLoaded CustomresourceDefinitions: [\u0026CustomResourceDefinition{ObjectMeta:{memcacheds.cache.example.com 0 0001-01-01 00:00:00 +0000 UTC \u003cnil\u003e \u003cnil\u003e map[] map[controller-gen.kubebuilder.io/version:v0.4.1] [] [] []},Spec:CustomResourceDefinitionSpec{Group:cache.example.com,Names:CustomResourceDefinitionNames{Plural:memcacheds,Singular:memcached,ShortNames:[],Kind:Memcached,ListKind:MemcachedList,Categories:[],},Scope:Namespaced,Versions:[]CustomResourceDefinitionVersion{CustomResourceDefinitionVersion{Name:v1alpha1,Served:true,Storage:true,Schema:\u0026CustomResourceValidation{OpenAPIV3Schema:\u0026JSONSchemaProps{ID:,Schema:,Ref:nil,Description:Memcached is the Schema for the memcacheds API,Type:object,Format:,Title:,Default:nil,Maximum:nil,ExclusiveMaximum:false,Minimum:nil,ExclusiveMinimum:false,MaxLength:nil,MinLength:nil,Pattern:,MaxItems:nil,MinItems:nil,UniqueItems:false,MultipleOf:nil,Enum:[]JSON{},MaxProperties:nil,MinProperties:nil,Required:[],Items:nil,AllOf:[]JSONSchemaProps{},OneOf:[]JSONSchemaProps{},AnyOf:[]JSONSchemaProps{},Not:nil,Properties:map[string]JSONSchemaProps{apiVersion: { \u003cnil\u003e APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources string nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e},kind: { \u003cnil\u003e Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds string nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e},metadata: { \u003cnil\u003e object nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e},spec: { \u003cnil\u003e MemcachedSpec defines the desired state of Memcached object nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[foo:{ \u003cnil\u003e Foo is an example field of Memcached. Edit memcached_types.go to remove/update string nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e} size:{ \u003cnil\u003e Size defines the number of Memcached instances integer int32 nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e}] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e},status: { \u003cnil\u003e MemcachedStatus defines the observed state of Memcached object nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] nil [] [] [] nil map[nodes:{ \u003cnil\u003e Nodes store the name of the pods which are running Memcached instances array nil \u003cnil\u003e false \u003cnil\u003e false \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e false \u003cnil\u003e [] \u003cnil\u003e \u003cnil\u003e [] \u0026JSONSchemaPropsOrArray{Schema:\u0026JSONSchemaProps{ID:,Schema:,Ref:nil,Description:,Type:string,Format:,Title:,Default:nil,Maximum:nil,ExclusiveMaximum:false,Minimum:nil,ExclusiveMinimum:false,MaxLength:nil,MinLength:nil,Pattern:,MaxItems:nil,MinItems:nil,UniqueItems:false,MultipleOf:nil,Enum:[]JSON{},MaxProperties:nil,MinProperties:nil,Required:[],Items:nil,AllOf:[]JSONSchemaProps{},OneOf:[]JSONSchemaProps{},AnyOf:[]JSONSchemaProps{},Not:nil,Properties:map[string]JSONSchemaProps{},AdditionalProperties:nil,PatternProperties:map[string]JSONSchemaProps{},Dependencies:JSONSchemaDependencies{},AdditionalItems:nil,Definitions:JSONSchemaDefinitions{},ExternalDocs:nil,Example:nil,Nullable:false,XPreserveUnknownFields:nil,XEmbeddedResource:false,XIntOrString:false,XListMapKeys:[],XListType:nil,XMapType:nil,},JSONSchemas:[]JSONSchemaProps{},} [] [] [] nil map[] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e}] nil map[] map[] nil map[] nil nil false \u003cnil\u003e false false [] \u003cnil\u003e \u003cnil\u003e},},AdditionalProperties:nil,PatternProperties:map[string]JSONSchemaProps{},Dependencies:JSONSchemaDependencies{},AdditionalItems:nil,Definitions:JSONSchemaDefinitions{},ExternalDocs:nil,Example:nil,Nullable:false,XPreserveUnknownFields:nil,XEmbeddedResource:false,XIntOrString:false,XListMapKeys:[],XListType:nil,XMapType:nil,},},Subresources:\u0026CustomResourceSubresources{Status:\u0026CustomResourceSubresourceStatus{},Scale:nil,},AdditionalPrinterColumns:[]CustomResourceColumnDefinition{},Deprecated:false,DeprecationWarning:nil,},},Conversion:nil,PreserveUnknownFields:false,},Status:CustomResourceDefinitionStatus{Conditions:[]CustomResourceDefinitionCondition{},AcceptedNames:CustomResourceDefinitionNames{Plural:,Singular:,ShortNames:[],Kind:,ListKind:,Categories:[],},StoredVersions:[],},}]\n",
"state": "pass"
}
]
}
}
]
}Enviroment:
- operator-sdk master branch: aafa2d5
Metadata
Metadata
Assignees
Labels
area/testingIssue related to testing the operator-sdk and subcomponentsIssue related to testing the operator-sdk and subcomponentsgood first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.Denotes an issue ready for a new contributor, according to the "help wanted" guidelines.help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.Indicates that an issue or PR should not be auto-closed due to staleness.
Activity
estroz commentedon Feb 22, 2021
These tests can be made to pass by CSV markers added by the sample generator to Go operator samples. Another way to make changes to make these pass needs implementing for Ansible/Helm samples, ex. kustomize patch files on the CSV.
camilamacedo86 commentedon Jun 21, 2021
I am removing myself as assigned from this one and adding help wanted and good first issue since follows what required to get done.
To achieve that we need to improve the mocks in https://github.com/operator-framework/operator-sdk/tree/master/hack/generate/samples to add what is required to pass in the scorecard for these test and then change the tests in
operator-sdk/test/common/scorecard.go
Lines 70 to 73 in 9fa13f3
Also, please feel free to ping me if you need help to work on that.
openshift-bot commentedon Sep 20, 2021
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting
/remove-lifecycle stale.Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting
/lifecycle frozen.If this issue is safe to close now please do so with
/close./lifecycle stale
openshift-bot commentedon Oct 20, 2021
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting
/remove-lifecycle rotten.Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting
/lifecycle frozen.If this issue is safe to close now please do so with
/close./lifecycle rotten
/remove-lifecycle stale
openshift-bot commentedon Nov 19, 2021
Rotten issues close after 30d of inactivity.
Reopen the issue by commenting
/reopen.Mark the issue as fresh by commenting
/remove-lifecycle rotten.Exclude this issue from closing again by commenting
/lifecycle frozen./close
openshift-ci commentedon Nov 19, 2021
@openshift-bot: Closing this issue.
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
camilamacedo86 commentedon Feb 14, 2022
/lifecycle frozen
ima1mai commentedon Feb 17, 2022
I could reproduce the error locally. I'll try looking into it.
add CSV markers to go/v3 testdata
update scorecard tests