Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenEBS 3.0.0 operator based install on OpenShift 4.8 #23

Open
patrickstrick opened this issue Nov 13, 2021 · 0 comments
Open

OpenEBS 3.0.0 operator based install on OpenShift 4.8 #23

patrickstrick opened this issue Nov 13, 2021 · 0 comments

Comments

@patrickstrick
Copy link

  • Installing through in-cluster OperatorHub on OpenShift 4.8.18 and 4.9.5
  • OpenEBS 3.0.0
  • Using a freshly installed OpenShift cluster on AWS (4.8) or bare-metal (4.9)
  • Logged in as a user with the cluster-admin role when performing all steps
  • Using the default settings for Operator install (channel=alpha; all namespaces; installed namespace=openshift-operators;update=automatic)
  • Using the default YAML for OpenEBS Install Template
  • openebs-maya-operator SA given privileged SCC using instructions in Operator description
    ** Side note, the instructions in the operator description differ slightly from the instructions on openebs.io which also differ from the instructions on GitHub. Differences are mainly in the namespace to install to, and I suspect have evolved over time. It would be good to make them consistent.

The Operator installs fine an creates the ServiceAccount openebs-operator.
However, when creating the OpenEBS Install Template instance, it fails with permissions errors for openebs-operator (NOT openebs-maya-operator) to create or destroy storage classes.

I have also tried giving the openebs-operator SA privileged SCC, but I get the same error. Also, tried installing the operator to a single namespace called openebs as per some of the other instructions pointed out above, but get the same failure.

open-ebs operator pod log

{"level":"info","ts":1636830926.867148,"logger":"cmd","msg":"Version","Go Version":"go1.16.9","GOOS":"linux","GOARCH":"amd64","helm-operator":"v1.14.0","commit":"78f08b4852faf344ad3ef457c54f86087aaa0a0a"}
{"level":"info","ts":1636830926.8682532,"logger":"cmd","msg":"Environment variable OPERATOR_NAME has been deprecated, use --leader-election-id instead."}
{"level":"info","ts":1636830926.868304,"logger":"cmd","msg":"Watching all namespaces.","Namespace":""}
I1113 19:15:27.921489       1 request.go:665] Waited for 1.021728884s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/authentication.k8s.io/v1?timeout=32s
{"level":"info","ts":1636830929.8728647,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1636830929.8740401,"logger":"helm.controller","msg":"Watching resource","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","namespace":"","reconcilePeriod":"1m0s"}
{"level":"info","ts":1636830929.8744724,"msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1636830929.874601,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting EventSource","source":"kind source: openebs.io/v1alpha1, Kind=OpenEBSInstallTemplate"}
{"level":"info","ts":1636830929.8746696,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting Controller"}
{"level":"info","ts":1636830929.9755356,"logger":"controller.openebsinstalltemplate-controller","msg":"Starting workers","worker count":16}
I1113 19:15:57.404478       1 request.go:665] Waited for 1.045015416s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operator.openshift.io/v1?timeout=32s
{"level":"error","ts":1636830965.3285844,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830965.3765109,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:07.963847       1 request.go:665] Waited for 1.039821531s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operators.coreos.com/v2?timeout=32s
{"level":"error","ts":1636830970.9624512,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830971.0066683,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830976.6246536,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830976.6556742,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:19.249108       1 request.go:665] Waited for 1.043298602s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/imageregistry.operator.openshift.io/v1?timeout=32s
{"level":"error","ts":1636830982.482207,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830982.519001,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830987.6638396,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830987.705868,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:30.279264       1 request.go:665] Waited for 1.04668003s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/helm.openshift.io/v1beta1?timeout=32s
{"level":"error","ts":1636830993.2904336,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830993.3285933,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830998.6364563,"logger":"helm.controller","msg":"Release failed","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830999.053493,"logger":"helm.controller","msg":"Failed to update status after install release failure","namespace":"openshift-operators","name":"oebs","apiVersion":"openebs.io/v1alpha1","kind":"OpenEBSInstallTemplate","release":"oebs","error":"Operation cannot be fulfilled on openebsinstalltemplates.openebs.io \"oebs\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1636830999.0535834,"logger":"controller.openebsinstalltemplate-controller","msg":"Reconciler error","name":"oebs","namespace":"openshift-operators","error":"failed installation (storageclasses.storage.k8s.io is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot create resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope) and failed rollback: uninstallation completed with 2 error(s): storageclasses.storage.k8s.io \"openebs-device\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope; storageclasses.storage.k8s.io \"openebs-hostpath\" is forbidden: User \"system:serviceaccount:openshift-operators:openebs-operator\" cannot delete resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
I1113 19:16:41.580579       1 request.go:665] Waited for 1.045188919s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant