-
Notifications
You must be signed in to change notification settings - Fork 28
HOLD FOR MTV 2.10.1 -- MTV-2385-Storage copy offload #763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
RichardHoch
wants to merge
9
commits into
kubev2v:main
Choose a base branch
from
RichardHoch:MTV-2385_storage_copy_offload_2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
bad9d5f
MTV-2385-Storage copy ofload
RichardHoch 333a149
Nov 11
RichardHoch 4f7dd27
minor
RichardHoch 03679d2
formatting
RichardHoch 9a1cfae
after Roy
RichardHoch c3ce2e6
assembly titles
RichardHoch ccc5c9b
After Roy
RichardHoch 3c056b3
removed warning, rewrote procedures
RichardHoch 3a8ded5
Added limitation and tables for all secrets
RichardHoch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * documentation/doc-Planninig_your_migration/assembly_planning-migration-vmware.adoc | ||
|
|
||
| :_content-type: CONCEPT | ||
| [id="about-storage-copy-offload_{context}"] | ||
| = About migrating {vmw} virtual machines by using storage copy offload | ||
|
|
||
| [role="_abstract"] | ||
| You can migrate {vmw} virtual machines (VMs) that are in a storage array network (SAN) more efficiently by using a method called _storage copy offload_. You use this method to accelerate migration speed and reduce the load on your network. | ||
|
|
||
| {vmw}'s vSphere Storage APIs-Array Integration (VAAI) includes a command named `vmkfstools`. This command sends the `XCOPY` command, which is part of the SCSI protocol. The `XCOPY` commandS lets you copy data inside a SAN more efficiently than copying the data over a network. | ||
|
|
||
| {project-first} 2.10.0 leverages this command as the basis for storage copy offload, allowing you to delegate cloning your VMs' data to the storage hardware instead having it to transit it between {project-short} and {virt}. You do this by configuring the storage map in your migration plan to point to your storage array instead of the network you usually use for migration. When you start the migration plan, {project-short} migrates your VMs by copying them to the storage array you choose and using `XCOPY` to copy them directly to {virt}, instead of transmitting the contents of your VMs to {virt}. | ||
|
|
||
| The storage copy offload feature has some unique configuration prerequisites, which are discussed in xref:proc_storage-copy-offload_vmware[Migrating {vmw} vSphere VMs by using storage copy offload]. Once you configure your system, you can migrate plans using storage copy offload by using either the {project-short} UI or its CLI. Instructions for using storage offload have been integrated into the procedures for migrating {vmw} VMs for both the Ui and CLI. | ||
|
|
||
| [IMPORTANT] [VDDK mage OK, VM can't migrate using both VDDK and copy-offload] | ||
| ==== | ||
| A migration plan cannot mix VDDK mappings with copy-offload mappings. Because the migration controller copies disks either through CDI volumes (VDDK) or through Volume Populators (copy-offload), all storage pairs in the plan must either include copy-offload details (secret + product) or none of them must; otherwise the plan will fail. | ||
| ==== | ||
|
|
||
| Storage copy offload is available as a Technology Preview feature for {project-first} 2.10.0 for cold migration and as a Developer Preview feature for warm migration. | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| Storage copy offload for cold migration is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them | ||
| in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. | ||
|
|
||
| For more information about the support scope of Red Hat Technology Preview | ||
| features, see https://access.redhat.com/support/offerings/techpreview/. | ||
| ==== | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| Storage copy offload for warm migration is a Developer Preview feature only. Developer Preview software is not supported by Red{nbsp}Hat in any way and is not functionally complete or production-ready. Do not use Developer Preview software for production or business-critical workloads. Developer Preview software provides early access to upcoming product software in advance of its possible inclusion in a Red{nbsp}Hat product offering. Customers can use this software to test functionality and provide feedback during the development process. This software might not have any documentation, is subject to change or removal at any time, and has received limited testing. Red{nbsp}Hat might provide ways to submit feedback on Developer Preview software without an associated SLA. | ||
|
|
||
| For more information about the support scope of Red{nbsp}Hat Developer Preview software, see link:https://access.redhat.com/support/offerings/devpreview/[Developer Preview Support Scope]. | ||
| ==== | ||
|
|
||
| [id="how-storage-copy-offload-works_{context}"] | ||
| == How storage copy offload works | ||
|
|
||
| Without storage copy offload, {project-short} migrates a virtual disk as follows: | ||
|
|
||
| . {project-short} reads the disk from the source storage. | ||
| . {project-short} sends the data over a network to {virt}. | ||
| . {virt} writes the data to its storage. | ||
| + | ||
| This method can be slow and consume significant network and host resources. | ||
|
|
||
| With storage copy offload, the process is streamlined: | ||
|
|
||
| . {project-short} initiates a disk transfer request. | ||
| . Instead of sending the data, {project-short} instructs the storage array in which the Vsphere Virtual Machine File System (VMFS) datastore holds the source VMs to perform a direct copy from the source storage to the target volume, on the same array, in the correct storage class. | ||
| + | ||
| The storage array handles the cloning of the VM disk internally, often at a much higher speed than a network-based transfer. | ||
|
|
||
| The Forklift project, a key component of {project-short}, includes a specialized `vsphere-xcopy-volume-populator` that directly interacts with {vmw}'s VAAI. This allows {project-short} to trigger the high-speed, array-level data copy operation for supported storage systems. | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| The storage arrays must be the ones specified above. Otherwise `XCOPY` will perform a network copy on the ESXi. Although a network copy on the ESXi is usually considerably faster than a standard migration using an VDDK image, neither is as quick as a properly configured storage copy offload migration. | ||
| ==== | ||
|
|
||
| [role="_additional-resources"] | ||
| .Additional resources | ||
|
|
||
| * xref:proc_storage-copy-offload_vmware[Migrating {vmw} vSphere VMs by using storage copy offload] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * documentation/doc-Migration_Toolkit_for_Virtualization/master.adoc | ||
|
|
||
| :_content-type: PROCEDURE | ||
| [id="preparing-storage-copy-offload_{context}"] | ||
| = Preparing to use storage copy offload | ||
|
|
||
| [role="_abstract"] | ||
| You need to do the following to use storage copy offload: | ||
|
|
||
| Might be more than one module here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,219 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * documentation/doc-Migration_Toolkit_for_Virtualization/master.adoc | ||
|
|
||
| :_content-type: PROCEDURE | ||
| [id="proc_storage-copy-offload_{context}"] | ||
| = Migrating {vmw} vSphere VMs by using storage copy offload | ||
|
|
||
| [role="_abstract"] | ||
| You can use the storage copy offload feature of {project-first} to migrate {vmw} vSphere virtual machines (VMs) faster than by other methods. | ||
|
|
||
| .Prerequisites | ||
|
|
||
| In addition to the regular xref:vmware-prerequisites_mtv[{vmw} prerequisites], storage copy offload has the following additional prerequisites: | ||
|
|
||
| * A Hitachi Vantara or NetApp ONTAP storage system or one of the following: | ||
| ** A configured Dell PowerFlex Storage Data Client (SDC) or | ||
| ** A configured Internet Small Computer System Interface (iSCSI) storage system | ||
| ** A configured Fibre Channel storage system | ||
| * A working Container Storage Interface (CSI) driver connected to the above and to {virt} | ||
| * A configured {vmw} vSphere provider | ||
| * vSphere users must have a role that includes thw following privileges (suggested name: `StorgeOffloader`): | ||
|
|
||
| ** Global | ||
| *** Settings | ||
| ** Datastore | ||
| *** Browse datastore | ||
| *** Low level file operations | ||
| ** Host Configuration | ||
| *** Advanced settings | ||
| *** Query patch | ||
| *** Storage partition configuration | ||
|
|
||
| .Procedure | ||
|
|
||
| . In the {project-short} Operator, set the value of `feature_copy_offload` to `true` in `forklift-controller` by running the following command: | ||
| + | ||
| [source, terminal] | ||
| ---- | ||
| oc patch forkliftcontrollers.forklift.konveyor.io forklift-controller --type merge -p '{"spec": {"feature_copy_offload": "true"}}' -n openshift-mtv | ||
| ---- | ||
|
|
||
| . Create a `Secret` in the namespace in which the migration provider is set up, usually `openshift-mtv`. Include the following credentials in the `Secret`. Note that Hitachi has a different set of credentials: | ||
| + | ||
| [cols="4", options="header"] | ||
| .Credentials for a non-Hitachi storage copy offload Secret | ||
| |=== | ||
| |Key| Description| Mandatory?| Default | ||
|
|
||
| | `STORAGE_HOSTNAME` | ||
| | IP or URL of the host (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_USERNAME` | ||
| | The user's name (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_PASSWORD` | ||
| | The user's password (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_SKIP_SSL_VERIFICATION` | ||
| | If set to `true`, SSL verification is not performed (`true`, `false`). | ||
| | No | ||
| |`false` | ||
| |=== | ||
| + | ||
| [cols="4", options="header"] | ||
| .Credentials for a Hitachi storage copy offload Secret | ||
| |=== | ||
| |Key| Description| Mandatory?| Default | ||
|
|
||
| | `GOVMOMI_HOSTNAME` | ||
| | hostname or URL of the vSphere API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `GOVMOMI_USERNAME` | ||
| | User name of the vSphere API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `GOVMOMI_PASSWORD` | ||
| | Password of the vSphere API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_HOSTNAME` | ||
| | The hostname or URL of the storage vendor API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_USERNAME` | ||
| | The username of the storage vendor API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_PASSWORD` | ||
| | The password of the storage vendor API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_PORT` | ||
| | The port of the storage vendor API (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `STORAGE_ID` | ||
| | Storage array serial number (string) | ||
| | Yes | ||
| | NA | ||
|
|
||
| | `HOSTGROUP_ID_LIST` | ||
| | List of IO ports and host group IDs, for example. `CL1-A,1:CL2-B,2:CL4-A,1:CL6-A,1` | ||
| | Yes | ||
| | NA | ||
| |=== | ||
|
|
||
| . In either the UI or the CLI, complete the following steps: | ||
|
|
||
| .. In the UI, complete the following steps: | ||
|
|
||
| ... Create an ownerless storage map by using the procedure in xref:creating-form-based-storage-maps-ui-vmware_vmware[Creating ownerless storage maps using the form page of the {project-short} UI]. Use the *Offload plugin* named `vSphere XCOPY`. | ||
| ... Create a migration plan by using the procedure in xref:creating-plan-wizard-vmware_vmware[Creating a VMware vSphere migration plan by using the MTV wizard]. | ||
|
|
||
|
|
||
| .. In the CLI, complete the following steps: | ||
|
|
||
| ... Create a `StorageMap` custom resource (CR) according to the following example: | ||
| + | ||
| [source,yaml,subs="attributes+"] | ||
| ---- | ||
| apiVersion: forklift.konveyor.io/v1beta1 | ||
| kind: StorageMap | ||
| metadata: | ||
| name: copy-offload | ||
| namespace: openshift-mtv | ||
| spec: | ||
| map: | ||
| - destination: | ||
| accessMode: ReadWriteMany <1> | ||
| storageClass: <storage_class> <2> | ||
| offloadPlugin: | ||
| vsphereXcopyConfig: | ||
| secretRef: <Secret_for_the_storage_vendor_product> <3> | ||
| storageVendorProduct: <storage_vendor_product> <4> | ||
| source: | ||
| id: <datastore_ID> <5> | ||
| provider: | ||
| destination: | ||
| apiVersion: forklift.konveyor.io/v1beta1 | ||
| kind: Provider | ||
| name: host | ||
| namespace: openshift-mtv | ||
| uid: <ID_of_provider_host> | ||
| source: | ||
| apiVersion: forklift.konveyor.io/v1beta1 | ||
| kind: Provider | ||
| name: <name_of_vSphere_provider> | ||
| namespace: openshift-mtv | ||
| uid: <ID_of_vSphere_provider> | ||
| ---- | ||
| <1> Optional label. | ||
| <2> The storage class for the target Persistent Volume Claim (PVC) of the VM. | ||
| <3> `Secret` that contains the storage provider credentials. | ||
| <4> String that identifies the storage product. Valid strings are listed in the table that follows this CR. | ||
| <5> Datastore ID as set by {vmw} vSphere. | ||
| + | ||
| [cols="1,1",options="header"] | ||
| .Supported storage vendors and their identifying strings in the CLI | ||
| |=== | ||
| |Vendor | ||
| |Identifying string (Value of `storageVendorProduct` label) | ||
|
|
||
| |Hitachi Vantara | ||
| |`vantara` | ||
|
|
||
| |NetApp | ||
| |`ontap` | ||
|
|
||
| |Hewlett Packard Enterprise | ||
| |`primera3par` | ||
|
|
||
| |Pure Storage | ||
| |`pureFlashArray` | ||
|
|
||
| | Dell (PowerFlex) | ||
| |`powerflex` | ||
|
|
||
| | Dell (PowerMax) | ||
| |`powermax` | ||
|
|
||
| | Dell (PowerStore) | ||
| |`powerstore` | ||
|
|
||
| | Infinidat | ||
| |`infinibox` | ||
|
|
||
| | IBM | ||
| |`flashsystem` | ||
| |=== | ||
|
|
||
| ... Create a migration plan using the procedure in link:https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.10/html/migrating_your_virtual_machines_to_red_hat_openshift_virtualization/assembly_migrating-from-vmware_mtv#proc_migrating-virtual-machines-cli_vmware[Running a {vmw} vSphere migration from the command-line]. | ||
| ... In the `Plan` CR, modify the `spec:map:storage` portion of the CR as follows: | ||
| + | ||
| [source,yaml,subs="attributes+"] | ||
| ---- | ||
| spec: | ||
| map: | ||
| storage: | ||
| apiVersion: forklift.konveyor.io/v1beta1 | ||
| kind: StorageMap | ||
| name: <storage_map_in_StorageMap_CR> | ||
| namespace: <namespace> | ||
| ---- | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.