backup: veeam kvm integration by shwstppr · Pull Request #12991 · apache/cloudstack

shwstppr · 2026-04-09T17:09:43Z

Description

Design spec: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421954133

This PR introduces the initial implementation of Veeam integration support for KVM in CloudStack by adding a UHAPI-compatible server and image server components.

Veeam Backup & Replication interacts with virtualization platforms using its Universal Hypervisor API (UHAPI). To enable backup and restore workflows for CloudStack-managed KVM environments, this change introduces a UHAPI server that exposes CloudStack resources through a UHAPI-compatible interface.

In addition to the control plane APIs, an image server component is introduced to handle the data transfer operations required during backup and restore workflows.

Architecture

The integration consists of two main components:

UHAPI Server (Control Plane) named CloudStack Veeam Control Service

A lightweight UHAPI server runs inside the CloudStack management server and exposes endpoints under:

/ovirt-engine
    - /api - For APIs
    - /sso - For authentication
    - /services/pki-resource - For certificates

This server provides inventory discovery APIs required by Veeam and translates CloudStack resources into the structures expected by UHAPI.

The server:

exposes infrastructure inventory
handles authentication and session tokens
maps CloudStack resources to UHAPI-compatible representations

Image Server (Data Plane) named CloudStack Image Service

A separate image server component is introduced to handle backup and restore data transfer operations.

This component:

serves disk image data during backup
receives image data during restore operations
exposes endpoints used by Veeam worker components
integrates with CloudStack storage to read and write VM disk data

The separation between both these components server ensures that:

metadata APIs and control operations remain lightweight
bulk image transfer operations are handled independently

Documentation PR: apache/cloudstack-documentation#642

Co-authored by @abh1sar @weizhouapache

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

todo: patch (needed?)

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

jochemkalsbeek287 · 2026-04-20T07:57:03Z

+            return true;
+        }
+
+        resetService(unitName);


If the service is running (checkResult == null) but the control socket is not ready, the code falls through, calls resetService, and then skips the start block because checkResult != null is false. It then waits up to 10 seconds for a socket that will never become ready because nothing restarted the service. The service is left in a broken state.

Thanks for your comment @jochemkalsbeek287
If the service is running but the control socket is not ready, it means that it is an intermittent state - the service was just started, so the control socket is not ready yet.
It's bound to get ready after the 10 second wait.

If there is an error in the image server code which causes the control socket to not open, that will cause it to fail. Image server will be restarted in the next attempt.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2026-04-29T05:33:21Z

@blueorangutan package

blueorangutan · 2026-04-29T05:34:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-29T07:01:41Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17647

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2026-04-29T16:08:22Z

@blueorangutan test

blueorangutan · 2026-04-29T16:10:04Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

github-actions · 2026-04-30T03:29:36Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2026-04-30T07:07:29Z

[SF] Trillian test result (tid-15987)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50128 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12991-t15987-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

shwstppr · 2026-04-30T07:13:20Z

@blueorangutan package

blueorangutan · 2026-04-30T07:14:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-30T08:07:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17665

weizhouapache · 2026-04-30T07:33:28Z

+# Enable TLS for image server transfers. The keys are read from:
+# cert file = /etc/cloudstack/agent/cloud.crt
+# key file = /etc/cloudstack/agent/cloud.key
+image.server.tls.enabled=true


@shwstppr
is /etc/cloudstack/agent/cloud.ca.crt used ?

@abh1sar can tell better but I think yes we use cloud certificates for image server.
To the Veeam worker VM, we just pass the Root CA from the MS

that's right. Is it ok to show the file names?

got it, the control service and image service use the same Root CA (these key/cert are generated from it too), right ?

No, control service uses the certificate configured for the management server by the operator.
Image service uses the internal CA.
control service CA can be different for example in case when multiple management servers are behind LB and that LB has its own SSL config.

weizhouapache · 2026-04-30T07:42:11Z

+import com.cloud.utils.EnumUtils;
+
+@APICommand(name = "createImageTransfer",
+        description = "Create image transfer for a disk in backup. This API is intended for testing only and is disabled by default.",


This API is intended for testing only and is disabled by default.

this sentence exists in all APIs in this folder. is this correct ? @abh1sar

yes Wei, I don't want to expose these APIs to end users. Only Veeam control service uses them.
But they are useful in testing. For example, we can run integration tests using these even without veeam.
Any better way to handle this?

maybe add a global setting and expose them only when the setting is set to true ?

yes, the global setting is already there expose.kvm.backup.export.service.apis

weizhouapache · 2026-04-30T08:25:03Z

@@ -10075,4 +10113,33 @@ private void setVncPasswordForKvmIfAvailable(Map<String, String> customParameter
            vm.setVncPassword(customParameters.get(VmDetailConstants.KVM_VNC_PASSWORD));
        }
    }
+
+    protected boolean isBlankInstanceDefaultTemplate(VirtualMachineTemplate template) {
+        return KVM_VM_DUMMY_TEMPLATE_NAME.equals(template.getUniqueName());


maybe rename to KVM_BLANK_VM_TEMPLATE_NAME or so

weizhouapache · 2026-04-30T08:30:13Z

+    'listVmCheckpoints' : 'Backup and Recovery',
+    'deleteVmCheckpoint' : 'Backup and Recovery',
+    'ImageTransfer' : 'Backup and Recovery',
+    'VmCheckpoint' : 'Backup and Recovery',
    'UnmanagedInstance': 'Virtual Machine',


maybe only the last two are needed

'ImageTransfer' : 'Backup and Recovery', 'VmCheckpoint' : 'Backup and Recovery',

weizhouapache · 2026-04-30T08:30:30Z

@@ -87,6 +87,7 @@ export default {
    }
  },
  created () {
+    console.log('---------------', this.$route.meta.name)


remove it ?

weizhouapache

overall LGTM
left some minor comments and questions.

great job @shwstppr @abh1sar !

shwstppr and others added 30 commits April 9, 2026 09:38

[WIP] plugins: veeam control service

7c23b26

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

wip

065ec85

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

changes for discovery

a30eb28

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

changes

f52b114

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

changes

2784468

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

changes

81c3b5b

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Basic working version-1

f396c5c

Create volume on the given storage pool

73df3cb

Image server basic working version in SSVM.

23ecb1f

Image server with disk upload

5389fe6

upload fix-1

aae158b

image upload working

10f65b6

wip: changes for imagetransfer handling

7b45d2e

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Added progress to upload Image Transfers

2350661

get Options to return capabilities for upload

9ee9748

add license to image_server.py

f83fd00

fix precommit, license

2bc3114

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

veeam control changes

3460a5d

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

server changes

b926c74

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Support multiple disks and checkpoints

da62e9a

extents(zero/dirty) and capabilities - working

4173947

todo: patch (needed?)

Patch (zero, data) + Flush support in image_server.py

91a081b

Backup of stopped VMs

c36cd2c

api/server: create dummy KVM VM without volume and network is optional

ca4112e

worker vm deployment wip

a366929

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Support file backend for cow format: api and server

586134d

Image server support for file backend (qcow2 upload)

6ca1c9b

ut failure in UserVmManagerImplTest

fba7c63

wip

a89f872

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fixes to allow worker vm deployment

106fbdb

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix precommit

34960a0

jochemkalsbeek287 reviewed Apr 20, 2026

View reviewed changes

shwstppr added 4 commits April 29, 2026 01:15

fix nic attach warning during restore

6b8a725

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

schema fix

adb317d

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix eof

5710676

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

address comments

574f0ea

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

apache deleted a comment from blueorangutan Apr 29, 2026

minor cleanup and tests

568c1aa

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

github-actions Bot added the status:has-conflicts label Apr 30, 2026

shwstppr added 3 commits April 30, 2026 11:29

more tests, fix pre-commit

7bdd703

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Merge remote-tracking branch 'apache/main' into integration-veeam-kvm

eaab07d

fix datacenter href bug

905be92

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

github-actions Bot removed the status:has-conflicts label Apr 30, 2026

another pre-commit fix

1f306b7

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr changed the title ~~[WIP] backup: veeam kvm integration~~ backup: veeam kvm integration Apr 30, 2026

weizhouapache reviewed Apr 30, 2026

View reviewed changes

weizhouapache approved these changes Apr 30, 2026

View reviewed changes

abh1sar added 2 commits May 4, 2026 14:56

address review comments

1f9cbd4

fix ut and remove sudo from LibvirtDeleteVmCheckpointCommand

c65dfa1

Conversation

shwstppr commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

shwstppr commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

shwstppr commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abh1sar Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

shwstppr commented Apr 9, 2026 •

edited

Loading

abh1sar Apr 30, 2026 •

edited

Loading