Skip to content

Latest commit

 

History

History
363 lines (235 loc) · 13.4 KB

data-portability.md

File metadata and controls

363 lines (235 loc) · 13.4 KB
Raw
copyright lastupdated keywords subcollection
years
2024
2024-11-21
code engine, data portability
codeengine

{{site.data.keyword.attribute-definition-list}}

Understanding data portability for {{site.data.keyword.codeengineshort}}

{: #data-portability}

Data portability involves a set of commands that enable customers to export the digital artifacts that are needed to implement similar workload and data processing on different service providers or on-premises software. It includes commands for exporting the service customer content, including the related configuration that is used by the service to store and process the data, on the customer's own location. {: shortdesc}

Responsibilities

{: #data-portability-responsibilities}

{{site.data.keyword.cloud_notm}} services provide interfaces and instructions to guide the customer to copy and store the service customer content, including the related configuration, on their own selected location.

The customer is responsible for the use of the exported data and configuration for data portability to other infrastructures, which includes:

  • The planning and execution for setting up alternative infrastructure on different cloud providers or on-premises software that provide similar capabilities to the {{site.data.keyword.IBM_notm}} services.
  • The planning and execution for the porting of the required application code on the alternative infrastructure, including the adaptation of customer's application code, deployment automation, and so on.
  • The conversion of the exported data and configuration to the format that's required by the alternative infrastructure and adapted applications.

For more information about your responsibilities for {{site.data.keyword.codeengineshort}}, see Understanding your responsibilities when using {{site.data.keyword.codeengineshort}}.

Data export procedures

{: #data-portability-procedures}

{{site.data.keyword.codeengineshort}} provides mechanisms to build container images from source code. Source code can be stored either locally or on Github. {{site.data.keyword.codeengineshort}} does not provide commands or tools to download such container images. Instead, customers need to download and store those images following the data export procedures for {{site.data.keyword.registryfull_notm}}.

{{site.data.keyword.codeengineshort}} provides mechanisms to export settings and configurations that are used to process the customer's content.

Configuration data for the following {{site.data.keyword.codeengineshort}} entities can be exported using the commands specified later in this section:

  • Projects
  • Applications
  • Application revisions
  • Builds
  • Build runs
  • Configmaps
  • Secrets
  • Domain mappings
  • Event subscriptions ({{site.data.keyword.cos_full_notm}}, cron, and Kafka)
  • Functions
  • Jobs
  • Job runs

Tools required for exporting data

For exporting {{site.data.keyword.codeengineshort}} data, you need to download and install the following commandline tools:

  • The {{site.data.keyword.cloud_notm}} CLI client
    • Visit the {{site.data.keyword.cloud_notm}} CLI Getting started page to install the CLI client for your operating system.
  • The {{site.data.keyword.codeengineshort}} CLI plug-in
  • The Kubernetes CLI client

Exporting project configuration

A project in {{site.data.keyword.codeengineshort}} is a container of related configurations and workloads. If you want to export a project, use the following ibmcloud resource commands:

To list all {{site.data.keyword.codeengineshort}} projects use:

ibmcloud resource service-instances --service-name codeengine

To list all {{site.data.keyword.codeengineshort}} projects in a specific region use:

ibmcloud resource service-instances --service-name codeengine --location us-south

When you have identified the {{site.data.keyword.codeengineshort}} project you want to export use:

ibmcloud resource service-instance <project-name> -o json >project-name.json

Exporting a {{site.data.keyword.codeengineshort}} project only exports configuration of the project entity itself. Contained configuration entities and workload entities need to be exported separately. {:note}

Exporting a KUBECONFIG to export project entities

When you have identified a project you want to export, you have to export a KUBECONFIG environment variable to scope the kubectl commands described in this document to that project.

To obtain this KUBECONFIG environment variable, run the following commands

ibmcloud code-engine project select -n <project-name>
ibmcloud code-engine project current

This selects the project and prints a statement similar to the following one:

Getting the current project context...
OK

Name:       <project-name>
ID:         123a4cbc-567d-8e90-1f2b-032d33d931ba
Subdomain:  1abc2defi34
Domain:     us-south.codeengine.appdomain.cloud
Region:     us-south

Kubernetes Config:
Context:               1abc2defi34
Environment Variable:  export KUBECONFIG="/Users/<username>/.bluemix/plugins/code-engine/<project-name>-123a4cbc-567d-8e90-1f2b-032d33d931ba.yaml"

Copy the export KUBECONFIG=xxx statement to your terminal prompt and run it. All following kubectl commands are now scoped to this project.

Exporting application configuration

To export the configuration of all applications in your project, run the following kubectl command:

kubectl get service -o json >project-name_application-configurations.json

If you want to export the configuration of a single application, use the following kubectl command:

kubectl get service <application-name> -o json >project-name-application-name_application-configuration.json

Exporting application revision configuration

To export the configuration of all application revisions in your project, run the following kubectl command:

kubectl get revision -o json >project-name_application-revisions.json

If you want to export the configuration of a single application revision, use the following kubectl command:

kubectl get revision <application-revision-name> -o json >project-name-application-revision-name_revision.json

Exporting build configuration

To export the configuration of all builds in your project, run the following kubectl command:

kubectl get build -o json >project-name_build-configurations.json

If you want to export the configuration of a single build, use the following kubectl command:

kubectl get build <build-name> -o json >project-name-build-name_build-configuration.json

Exporting build run configuration

To export the configuration of all buildruns in your project, run the following kubectl command:

kubectl get buildrun -o json >project-name_buildrun-configurations.json

If you want to export the configuration of a single buildrun, use the following kubectl command:

kubectl get buildrun <buildrun-name> -o json >project-name-buildrun-name_buildrun-configuration.json

Exporting configmap configuration

To export the content of all configmaps in your project, run the following kubectl command:

kubectl get configmap -o json >project-name_configmaps.json

If you want to export the content of a single configmap, use the following kubectl command:

kubectl get configmap <configmap-name> -o json >project-name-configmap-name_configmap.json

Exporting secret configuration

Do not export the contents of secrets to a file on a persistent storage. {:note}

To export the content of all secrets in your project, run the following kubectl command:

kubectl get secret -o json >project-name_secrets.json

If you want to export the content of a single secret, use the following kubectl command:

kubectl get secret <secret-name> -o json >project-name-secret-name_secret.json

Values stored in {{site.data.keyword.codeengineshort}} secrets are usually copies of values managed in other systems, like {{site.data.keyword.secrets-manager_full_notm}}. Therefore, most of the time exporting secrets is not required. If you still require to export secrets from {{site.data.keyword.codeengineshort}}, you should choose exporting single secrets over exporting all secrets to avoid unintentional disclosure of sensitive information. {:note}

Exporting domain mapping configuration

To export the configuration of all domain mappings in your project, run the following kubectl command:

kubectl get domainmapping -o json >project-name_domainmappings.json

If you want to export the configuration of a single domain mapping, use the following kubectl command:

kubectl get domainmapping <domainmapping-name> -o json >project-name-domainmapping-name_domainmapping.json

Exporting {{site.data.keyword.cos_full_notm}} event subscription configuration

To export the configuration of all {{site.data.keyword.cos_full_notm}} event subscriptions in your project, run the following kubectl command:

kubectl get cossource -o json >project-name_cos-event-subscriptions.json

If you want to export the configuration of a single {{site.data.keyword.cos_full_notm}} event subscription, use the following kubectl command:

kubectl get cossource <cos-event-subscription-name> -o json >project-name-cos-event-subscription-name_cos-event-subscription.json

Exporting cron event subscription configuration

To export the configuration of all cron event subscriptions in your project, run the following kubectl command:

kubectl get pingsource -o json >project-name_cron-event-subscriptions.json

If you want to export the configuration of a single cron event subscription, use the following kubectl command:

kubectl get pingsource <cron-event-subscription-name> -o json >project-name-cron-event-subscription-name_cron-event-subscription.json

Exporting Kafka event subscription configuration

To export the configuration of all Kafka event subscriptions in your project, run the following kubectl command:

kubectl get kafkasource -o json >project-name_kafka-event-subscriptions.json

If you want to export the configuration of a single Kafka event subscription, use the following kubectl command:

kubectl get kafkasource <cron-event-subscription-name> -o json >project-name-kafka-event-subscription-name_kafka-event-subscription.json

Exporting function configuration

To export the configuration of all functions in your project, run the following kubectl command:

kubectl get function -o json >project-name_functions.json

If you want to export the configuration of a single function, use the following kubectl command:

kubectl get function <function-name> -o json >project-name-function-name_function.json

Exporting job configuration

To export the configuration of all jobs in your project, run the following kubectl command:

kubectl get jobdefinition -o json >project-name_jobs.json

If you want to export the configuration of a single job, use the following kubectl command:

kubectl get jobdefinition <function-name> -o json >project-name-job-name_job.json

Exporting job run configuration

To export the configuration of all jobruns in your project, run the following kubectl command:

kubectl get jobrun -o json >project-name_jobruns.json

If you want to export the configuration of a single jobrun, use the following kubectl command:

kubectl get jobrun <function-name> -o json >project-name-jobrun-name_jobrun.json

Exported data formats

{: #data-portability-data-formats}

{{site.data.keyword.codeengineshort}} supports the following data format and schema of the exported data, configuration, and application:

  • JSON

Example: Exporting a ConfigMap with name export-sample and format JSON results in the following JSON output:

  • Command: kubectl get configmap export-sample -o json

  • Output:

{
    "apiVersion": "v1",
    "data": {
        "APP_OWNER": "John Doe",
        "NUM_DATASETS": "24",
        "foo": "bar"
    },
    "kind": "ConfigMap",
    "metadata": {
        "creationTimestamp": "2024-10-15T09:12:47Z",
        "name": "export-sample",
        "namespace": "7yuw2furi55",
        "resourceVersion": "9162003517",
        "uid": "12962640-a294-4e1f-a006-a875f119df43"
    }
}

All commands that use the kubectl command with the -o json option, produce a JSON output that is compatible with Kubernetes. The JSON produced by other commands, like ibmcloud resource or ibmcloud code-engine cannot be assumed to be compatible with Kubernetes as is.

{{site.data.keyword.codeengineshort}} doesn't support the export of the following data format and schema of the exported data, configuration, and application:

  • YAML

Data ownership

{: #data-portability-ownership}

All exported data is classified as customer content. Apply the full customer ownership and licensing rights, as stated in the {{site.data.keyword.cloud_notm}} Service Agreement.