Commit graph

3680 commits

Author SHA1 Message Date
Fiona Ebner
0210f0de17 mirror: code style: avoid masking earlier declaration of $op
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-07-03 10:46:44 +02:00
Thomas Lamprecht
57a41dd73c adopt perltidy also for executables without perl extension
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-07-02 22:47:05 +02:00
Fiona Ebner
75f6a16eb4 block job: factor out helper for common mirror QMP options
To be re-used by blockdev-mirror.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
5dcc2804dc test: migration: factor out common mocking for mirror
To be re-used for mocking blockdev_mirror.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
55978e0f95 drive mirror: add variable for device ID and make name for drive ID precise
Suggested-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
0ba0af6cc9 block job: add and use wrapper for mirror
In preparation for the switch to -blockdev.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
721db03bb5 drive: die in get_drive_id() if argument misses relevant members
Catch errors early instead of continuing with unexpected values.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
70058510b7 introduce BlockJob module
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
02700cb197 code cleanup: drive mirror: do not prefix calls to function in the same module
In preparation to move block job related helpers to a dedicated
module. Like this, moving the code will be clearly visible in the diff
without any changed lines sticking out in between.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
4bb2f44e0c introduce RunState module
For now, move only the vm_resume() and vm_suspend() functions. Others
like vm_stop() and friends, vm_reboot() and vm_start() would require
more preparation.

Apart from slightly improving modularity, this is in preparation to
add a BlockJob module, where vm_resume() and vm_suspend() need to be
called after drive-mirror, to avoid a cyclic dependency with the main
QemuServer module.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
5044ae35a7 introduce QemuMigrate::Helpers module
The QemuMigrate module is high-level and should not be called from
many other places, while also being an implementation of
AbstractMigrate, so using a separate module is much more natural.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
e05f608998 move find_vmstate_storage() helper to QemuConfig module
While not the main motivation, this has the nice side effect of
removing a call from QemuConfig to the QemuServer main module.

This is in preparation to introduce a RunState module which does not
call back into the main QemuServer module. In particular, qm_suspend()
will be moved to RunState which needs to call find_vmstate_storage().

Intuitively, the StateFile module seems like the most natural place
for find_vmstate_storage(), but moving find_vmstate_storage() requires
moving foreach_storage_used_by_vm() too and that function calls into
QemuConfig. Now, QemuConfig also calls find_vmstate_storage(), meaning
a cyclic dependency would result.

Note that foreach_storage_used_by_vm(), is related to foreach_volume()
and also uses foreach_volume(), so QemuConfig is the natural place for
that function.

So the arguments for moving find_vmstate_storage() to QemuConfig are:
1. most natural way to avoid cylcic dependencies.
2. related function foreach_storage_used_by_vm() belongs there too.
3. qm_suspend() and other functions relating to the run state already
   call other QemuConfig methods.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
bee71e195f agent: move qga_check_running() to agent module
Makes it possible to call into the module from the main QemuServer
module and other modules that are used by QemuServer without adding a
cyclic dependency.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
187685eff7 agent: avoid use of deprecated check_running() function
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
17966d24aa agent: avoid dependency on QemuConfig module
The QemuConfig module uses qga_check_running() which is planned to be
moved to the Agent module. Loading the config on the call-sites of
agent_cmd(), qemu_exec() and qemu_exec_status() makes it possible to
avoid introducing that cyclic dependency.

Note that the import for the QemuConfig module was already missing.

Also drops unused variables $write and $res from the 'file-write' API
endpoint implementation.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
9bfb6f5d94 agent: code style: order module imports according to style guide
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
76ee4cd234 agent: drop unused $noerr argument from helpers
Both agent_available() and agent_cmd() have no callers that use the
$noerr argument.

The current implementation was not ideal: agent_cmd() did not check
the return value of agent_available() in the $noerr scenario. It
should return early. In agent_available(), failure was silently
ignored in the $noerr scenario. Having a message or warning then would
have been more useful.

The agent_available() function is renamed to assert_agent_available()
and it is not exported anymore, the single caller outside the module
can just call it with the full module path.

The import of 'agent_available' in qm.pm was not used and is dropped.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
eac162a865 introduce Network module
Also gets rid of a cyclic dependency between the main QemuServer
module and the Cloudinit module.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
771a505b2a remove unused $nic_model_list_txt variable
The last usage of this was removed by commit 52261945 ("improve
documentation").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
e923bdeae7 schema: remove unused pve-qm-ipconfig standard option
Grepping in /usr/share/perl5/PVE, the standard option is never used
and it was accidentally assigned $netdesc rather than $ipconfigdesc.
This can be seen in commit 0c9a7596 ("implement cloudinit").

Can still be added correctly later if the need arises.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
8313607c62 assume that SDN is available
pve-manager >= 8.2.10 has a hard dependency on libpve-network-perl
which includes the required modules.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
6aa63bd0f4 cfg2cmd: ovmf: support print_ovmf_commandline() returning machine flags
This is in preparation for the switch to -blockdev, where it will be
necessary to specify the 'pflash0' and 'pflash1' machine flags.

Suggested-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
1d0818d6b5 ovmf: add support for using blockdev
Co-developed-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
21ad0c893d introduce OVMF module
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
1469b4c17d print ovmf commandline: collect hardware parameters into hash argument
Also avoids querying the AMD-SEV type again, inside the function.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-26 13:20:52 +02:00
Fiona Ebner
4bd4a6378d import disk: add missing include for QemuConfig module
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-25 15:49:05 +02:00
Daniel Kral
b5f27c00d6 import disk: fix wrong package prefix for QemuImage's convert helper
Fixes: bc097c94 ("introduce QemuImage module")
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Link: https://lore.proxmox.com/20250625125039.153187-1-d.kral@proxmox.com
2025-06-25 15:47:41 +02:00
Fiona Ebner
c7c19199ae blockdev: add support for 'size' option
For EFI disks in raw format, it is necessary to specify a precise size
and have no padding. See commit 818ce80e ("fix efidisks on storages with
minimum sizes bigger than OVMF_VARS.fd") for details.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 12:24:25 +02:00
Fiona Ebner
f2f2edcd7f blockdev: add workaround for issue #3229
Enforce the 'writeback' cache setting for an EFI disk with RBD driver
to work around issue #3229. See also commit 6aaad230 ("fix #3329: turn
on cache=writeback for efidisks on rbd").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 12:24:25 +02:00
Fiona Ebner
8ca64671a7 blockdev: re-use cache setting from child node
To ensure that the setting is always consistent between format and
file node.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 12:24:25 +02:00
Fiona Ebner
bc097c94c4 introduce QemuImage module
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fiona Ebner
c9aefa0ea9 move helper for iscsi initiator name to helpers module and improve name
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fiona Ebner
a36a17fb6a helpers: fix perlcritic warning about variables named $a and $b
> Using $a or $b outside sort() at line 193, column 13.  $a and $b are
> special package variables for use in sort() and related functions.
> Declaring them as lexicals like "my $a" may break sort(). Use
> different variable names.  (Severity: 4)

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fiona Ebner
56129974d4 helpers: add missing includes
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fiona Ebner
25ba3af4e7 qmp client: add default timeouts for more blockdev commands
For the HMP 'drive_add' command, the used timeout is 1 minute and for the
'drive_del' command, the used timeout is 10 minutes, because IO might
need to be finished. Use the same for 'blockdev-add' respectively
'blockdev-del'.

For 'drive-mirror', 10 minutes is used, so use the same for
'blockdev-mirror'.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fiona Ebner
ba37488cda fix #5985: qmp client: increase timeout for {device, netdev, object}_{add, del} commands
In the bug report, the user mentioned that 7 seconds was enough. For
the HMP 'drive_add' command, the used timeout is 1 minute and for the
'drive_del' command, the used timeout is 10 minutes, because IO might
need to be finished. While something similar might be true for certain
objects/devices, there were no issues reported with the *_del
operations using the default timeout until now and the callers can
still use a higher timeout if they know the specific device/object
requires it.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-24 09:48:12 +02:00
Fabian Grünbichler
d4dcef75b8 install PVE::Blockdev
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-06-20 14:58:17 +02:00
Fabian Grünbichler
7fe56548d5 tests: adapt to QEMU 10
it reset the pve-specific machine suffix to 0 again, and some preparatory
changes for the upcoming switch to blockdev are already visible in the
commandlines.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2025-06-20 14:52:28 +02:00
Fiona Ebner
2ea50f8bb4 blockdev: add helpers to generate blockdev commandline
The drive device and node structure is:
front-end device {ide-hd,scsi-hd,virtio-blk-pci} (id=$drive_id)
 - throttle node (node-name=$drive_id)
  - format node  (node-name=f$encoded-info)
   - file node   (node-name=e$encoded-info)

The node-name can only be 31 characters long and needs to start with a
letter. The throttle node will stay inserted below the front-end
device. The other nodes might change however, because of drive
mirroring and similar. There currently are no good helpers to
query/walk the block graph via QMP, x-debug-query-block-graph is
experimental and for debugging only. Therefore, necessary information
is encoded in the node name to be able to find it again. In
particular, this is the volume ID, the drive ID and optionally a
snapshot name. As long as the configuration file matches with the
running instance, this is enough to find the correct node for
block operations like mirror and resize.

The 'snapshot' option, for QEMU's snapshot mode, i.e. writes are only
temporary, is not yet supported.

[FE: split up patch
     expand commit message
     explicitly test for drivers with aio setting
     improve readonly handling
     improve CD-ROM handling
     fix failure for storage named 'nbd' by always using full regex
     improve node name generation
     fail when drive->{snapshot} is set]

Originally-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 14:44:38 +02:00
Fiona Ebner
2b026cc600 print drive device: set {r, w}error front-end properties starting with machine version 10.0
These properties cannot be specified via '-blockdev' like they could
with '-drive', because they are properties of the front-end drive
device.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
22b4036c55 print drive device: explicitly set write-cache starting with machine version 10.0
With the 'blockdev' command line option, the cache options are split
up. While cache.direct and cache.no-flush can be set in the -blockdev
options, cache.writeback is a front-end property and was intentionally
removed from the 'blockdev' options by QEMU commit aaa436f998 ("block:
Remove cache.writeback from blockdev-add"). It needs to be configured
as the 'write-cache' property for the ide-hd/scsi-hd/virtio-blk
device.

Table from 'man kvm':

┌─────────────┬─────────────────┬──────────────┬────────────────┐
│             │ cache.writeback │ cache.direct │ cache.no-flush │
├─────────────┼─────────────────┼──────────────┼────────────────┤
│writeback    │ on              │ off          │ off            │
├─────────────┼─────────────────┼──────────────┼────────────────┤
│none         │ on              │ on           │ off            │
├─────────────┼─────────────────┼──────────────┼────────────────┤
│writethrough │ off             │ off          │ off            │
├─────────────┼─────────────────┼──────────────┼────────────────┤
│directsync   │ off             │ on           │ off            │
├─────────────┼─────────────────┼──────────────┼────────────────┤
│unsafe       │ on              │ off          │ on             │
└─────────────┴─────────────────┴──────────────┴────────────────┘

Suggested-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Alexandre Derumier via pve-devel
9af3ef69c7 vm devices list: prepare querying block device names for -blockdev
Look at the 'qdev' value, because the 'device' property is not
initialized with '-blockdev'. This can be seen in the QEMU source code
(the device property is the name of the block backend and blk->name is
assigned a value only in a code path reached via drive_new()). This
most likely was done to avoid confusion/clashes, since with
'-blockdev', the node that's inserted for a front-end device can
change and then both the block backend and the node would be named
the same, but not connected.

[FE: fix commit message and comment - it does not depend on the presence of media
     escape dot in regex
     skip right away if qdev is undef to avoid warning when regex matching]

Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
25161b43bd qm: showcmd: never reserve PCI devices
Never reserve PCI devices when config_to_command() is called for
'showcmd'. Previously, choose_hostpci_devices() only skipped reserving
PCI devices when the VM was already running as a heuristic. Make it
explicit via a new parameter. Adapt the config-to-command test to
never expect actual reservation. There is a dedicated test for this
since commit "test: add tests for PCI reservations".

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
0b892485a9 cfg2cmd: collect optional parameters as a hash
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
770dc42145 pci: code cleanup: remove superfluous machine type paramater from print_pci_addr
The only supported machine for aarch64 is 'virt', so there is no need
to check if that is the machine. Also many (all?) other machines for
aarch64 in QEMU also don't have a 'pci' bus by default.

The parameter is also transitively removed from the functions:
1. print_hostpci_devices()
2. print_rng_device_commandline()
3. get_usb_controllers()
4. print_netdevice_full()
5. print_vga_device()

Should this ever be required in the future again, or also for the
$arch itself right now, it would be nicer to properly abstract this
away instead of passing the parameters along everywhere, e.g. by using
a class.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
3978960f94 cfg2cmd: print vga: fix call to print_pcie_addr()
The function print_pcie_addr() only takes a single parameter.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
f77e34c5b7 test: add tests for PCI reservations
This is currently tested as part of the config-to-command tests, but
the plan is to make command generation for 'qm showcmd' not actually
reserve anything. The reserve_pci_usage() function also serves as
checking what already is reserved by other VMs at the same time.
Thus, the config-to-command test will not be able to test whether
reservations are actually respected after the above-mentioned change.
Add a dedicated test for this.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
058e0f6520 pci: add missing includes
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
f7b63cc5e7 vm start/commandline: activate volumes before config_to_command()
With '-blockdev', it is necessary to activate the volumes to generate
the command line, because it can be necessary to check whether the
volume is a block device or a regular file.

Do not deactivate after commandline generation for 'qm showcmd',
because there can be concurrent operations acting on the volumes.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00
Fiona Ebner
b7ce0f7907 vm start/commandline: also clean up pci reservation when config_to_command() fails
The config_to_command() function already calls into
print_hostpci_devices() which makes reservations.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-06-20 13:50:54 +02:00