Vulkan Object Picking

Recently I’ve being trying to implement object picking on my custom vulkan renderer/engine but couldn’t find many resources on the subject, except these ones:

• Gav’s with storage buffer method
• sonoozetime’s pixel-perfect method
• naorliron26’s pixel-perfect method

and of course, a bunch of OpenGL alternatives. There isn’t any problems with them, I just find them rather incomplete since I don’t have a previous background in Engine development.

Why not use Physics?

If you want to have precision, the only way is to cast a ray from the camera’s view into the scene, testing for collisions with all triangles the meshes/objects are made of. But I only keep track of the object’s transformation matrices, center position, scale, and rotation vectors. I discard all vertices after loading to save memory; maybe I shouldn’t. I’ll leave this subject open for discussion.

I did, however, test ray-casting with a sphere that touched the nearest center-position vector of an object (if in range). This is very imprecise, since it only guarantees the correct selection if the cursor is near the center of the object. And finally, I also did AABB (Axis-Aligned Bounding Box) ray collision. This is a cube that encapsulates the object; when the ray intersects with the cube, the object is “selected”. The problem with that is, if the mesh is different than a cube, like a triangle, selecting the empty space that would turn it into a square will also select the triangle.

It seems like with physics, if the object is not mathematically described, I couldn’t achieve the precision I was looking for. So now I understand why Pixel-Perfect selection was the way to go. The problem with such method is redrawing the scene once more, writting an id to the pixel output instead of a color. Then read the id back at the mouse’s location when needed.

First things first

I’ll be using C++ and my custom-based framework, so you’ll sometimes find in the code some weird classes. Just go to it’s repo and check what they are, it’s heavelly based on SaschaWillems.

We’re going to need to:

1. Create a new renderpass, drawing the entire scene again, we’ll use this renderpass to read from latter;
2. Create a graphic pipeline almost identical with our mesh shader, but with different shaders;
3. Read the the pixel on the mouse coordinates from our new renderpass when we want (mouse press?);

1) Renderpass

This mostly match the normal creation of a render pass. Yes, we’ll need framebuffers, command buffers, and some objects. They are ususally related to each other and that’s why I have the class Renderpass, it’s just a convenient way to store them together for latter access. I also keep track of them with a unordered_map on the Renderer class;

Important things:

• mSurfaceFormat should match the layout of what your id is and fragment shader output will be. Mine is an uint64_t so the shader output should be an uvec2;
• msaa can be only 1 sample for this technique;
• I’m using Vulkan memory allocator but it should be easy to bring it back to the standart implementation;
• I’ll not pretend I understand all aspects of the renderpass creation, but for our pourpuse the following will sufice, I’m all ears for suggestions;

void Picking::CreateRenderpass()
{
	Shared<Renderer::Vulkan::Renderpass> renderpass = CreateShared<Renderer::Vulkan::Renderpass>(mDevice, "Picking", VK_SAMPLE_COUNT_1_BIT);
	mRenderpassesLib.Insert("Picking", renderpass);

	mSurfaceFormat = VK_FORMAT_R32G32_UINT;
	mImageSize = 2 * 8; // (RED + GREEN) * 8 bits
	mDepthFormat = mDevice->FindSuitableDepthFormat();

	// create render pass
	{
		std::vector<VkAttachmentDescription> attachments = {};
		attachments.resize(2);

		// color attachment
		attachments[0].format = mSurfaceFormat;
		attachments[0].samples = mMSAA;
		attachments[0].loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
		attachments[0].storeOp = VK_ATTACHMENT_STORE_OP_STORE;
		attachments[0].stencilLoadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;
		attachments[0].stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
		attachments[0].initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
		attachments[0].finalLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;

		// depth attachment
		attachments[1].format = mDepthFormat;
		attachments[1].samples = mMSAA;
		attachments[1].loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
		attachments[1].storeOp = VK_ATTACHMENT_STORE_OP_STORE;
		attachments[1].stencilLoadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
		attachments[1].stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
		attachments[1].initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
		attachments[1].finalLayout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL;

		VkAttachmentReference colorReference = {};
		colorReference.attachment = 0;
		colorReference.layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;

		VkAttachmentReference depthReference = {};
		depthReference.attachment = 1;
		depthReference.layout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL;

		VkSubpassDescription subpassDescription = {};
		subpassDescription.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
		subpassDescription.colorAttachmentCount = 1;
		subpassDescription.pColorAttachments = &colorReference;
		subpassDescription.pDepthStencilAttachment = &depthReference;
		subpassDescription.inputAttachmentCount = 0;
		subpassDescription.pInputAttachments = nullptr;
		subpassDescription.preserveAttachmentCount = 0;
		subpassDescription.pPreserveAttachments = nullptr;
		subpassDescription.pResolveAttachments = nullptr;

		// subpass dependencies for layout transitions
		std::vector<VkSubpassDependency> dependencies = {};
		dependencies.resize(2);

		dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
		dependencies[0].dstSubpass = 0;
		dependencies[0].srcStageMask = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT;
		dependencies[0].dstStageMask = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT;
		dependencies[0].srcAccessMask = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
		dependencies[0].dstAccessMask = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT;
		dependencies[0].dependencyFlags = 0;

		dependencies[1].srcSubpass = VK_SUBPASS_EXTERNAL;
		dependencies[1].dstSubpass = 0;
		dependencies[1].srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
		dependencies[1].dstStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
		dependencies[1].srcAccessMask = 0;
		dependencies[1].dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_READ_BIT;
		dependencies[1].dependencyFlags = 0;

		VkRenderPassCreateInfo renderPassCI = {};
		renderPassCI.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO;
		renderPassCI.attachmentCount = (uint32_t)attachments.size();
		renderPassCI.pAttachments = attachments.data();
		renderPassCI.subpassCount = 1;
		renderPassCI.pSubpasses = &subpassDescription;
		renderPassCI.dependencyCount = (uint32_t)dependencies.size();
		renderPassCI.pDependencies = dependencies.data();
		COSMOS_ASSERT(vkCreateRenderPass(mDevice->GetLogicalDevice(), &renderPassCI, nullptr, &renderpass->GetRenderpassRef()) == VK_SUCCESS, "Failed to create renderpass");
	}

	// command pool
	{
		Renderer::Vulkan::Device::QueueFamilyIndices indices = mDevice->FindQueueFamilies(mDevice->GetPhysicalDevice(), mDevice->GetSurface());

		VkCommandPoolCreateInfo cmdPoolInfo = {};
		cmdPoolInfo.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO;
		cmdPoolInfo.queueFamilyIndex = indices.graphics.value();
		cmdPoolInfo.flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT;
		COSMOS_ASSERT(vkCreateCommandPool(mDevice->GetLogicalDevice(), &cmdPoolInfo, nullptr, &renderpass->GetCommandPoolRef()) == VK_SUCCESS, "Failed to create command pool");
	}

	// command buffers
	{
		renderpass->GetCommandfuffersRef().resize(CONCURENTLY_RENDERED_FRAMES);

		VkCommandBufferAllocateInfo cmdBufferAllocInfo = {};
		cmdBufferAllocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
		cmdBufferAllocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
		cmdBufferAllocInfo.commandPool = renderpass->GetCommandPoolRef();
		cmdBufferAllocInfo.commandBufferCount = (uint32_t)renderpass->GetCommandfuffersRef().size();
		COSMOS_ASSERT(vkAllocateCommandBuffers(mDevice->GetLogicalDevice(), &cmdBufferAllocInfo, renderpass->GetCommandfuffersRef().data()) == VK_SUCCESS, "Failed to allocate command buffers");
	}
}

After this we can create all image resources we’re going to need, as well as the framebuffers. Why multiple framebuffers? Well, my implementation renders multiple frames at the same time. And while I do use multiple color images I only use one depth image (this may change, I was having syncronization erros when using one color image only). This function must be called again when a resize event happens, just like the swapchain images and framebuffers normally needs to.

void Picking::CreateImages()
{
	VkExtent2D size = mSwapchain->GetExtent();

	// depth buffer
	{
		mDevice->CreateImage
		(
			size.width,
			size.height,
			1,
			1,
			mMSAA,
			mDepthFormat,
			VK_IMAGE_TILING_OPTIMAL,
			VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT,
			VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT,
			mDepthImage,
			mDepthMemory
		);

		mDepthView = mDevice->CreateImageView
		(
			mDepthImage,
			mDepthFormat,
			VK_IMAGE_ASPECT_DEPTH_BIT
		);
	}

	// color images
	{
		size_t imageCount = mSwapchain->GetImagesRef().size();
		auto renderpass = mRenderpassesLib.GetRef("Picking");
		renderpass->GetFramebuffersRef().resize(imageCount);
		mColorImages.resize(imageCount);
		mColorMemories.resize(imageCount);
		mColorViews.resize(imageCount);

		for (size_t i = 0; i < imageCount; i++) {
			mDevice->CreateImage
			(
				size.width,
				size.height,
				1,
				1,
				mMSAA,
				mSurfaceFormat,
				VK_IMAGE_TILING_OPTIMAL,
				VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT, // for picking
				VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT,
				mColorImages[i],
				mColorMemories[i]
			);

			VkCommandBuffer command = mDevice->BeginSingleTimeCommand(renderpass->GetCommandPoolRef());

			mDevice->InsertImageMemoryBarrier
			(
				command,
				mColorImages[i],
				VK_ACCESS_TRANSFER_READ_BIT,
				VK_ACCESS_MEMORY_READ_BIT,
				VK_IMAGE_LAYOUT_UNDEFINED, // must get from last render pass (undefined also works)
				VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, // must set for next render pass
				VK_PIPELINE_STAGE_TRANSFER_BIT,
				VK_PIPELINE_STAGE_TRANSFER_BIT,
				VkImageSubresourceRange{ VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1 }
			);


			mDevice->EndSingleTimeCommand(renderpass->GetCommandPoolRef(), command);

			mColorViews[i] = mDevice->CreateImageView(mColorImages[i], mSurfaceFormat, VK_IMAGE_ASPECT_COLOR_BIT);

			std::vector<VkImageView> attachments = { mColorViews[i], mDepthView };
			attachments.resize(2);

			VkFramebufferCreateInfo framebufferCI = {};
			framebufferCI.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO;
			framebufferCI.renderPass = renderpass->GetRenderpassRef();
			framebufferCI.attachmentCount = (uint32_t)attachments.size();
			framebufferCI.pAttachments = attachments.data();
			framebufferCI.width = mSwapchain->GetExtent().width;
			framebufferCI.height = mSwapchain->GetExtent().height;
			framebufferCI.layers = 1;
			COSMOS_ASSERT(vkCreateFramebuffer(mDevice->GetLogicalDevice(), &framebufferCI, nullptr, &renderpass->GetFramebuffersRef()[i]) == VK_SUCCESS, "Failed to create framebuffer");
		}
	}
}

2) Graphics Pipeline

After this we can create a graphics pipeline with different shaders that output the id instead of the pixel color. Unfortunatelly my implementation of the graphics pipeline creation is a bit different than the standart tutorials I see around (You can check it here). But the only differences between the mesh and the picking graphics pipeline are the vertex inputs (now only position is required) and the shaders. Also, since each object has an id I’ve decided to use push constants to write the id only when it’s time to draw a given mesh.

The shaders are:

//////////////////////////////////////////////////////////////////// vertex shader
#version 450
#extension GL_ARB_gpu_shader_int64 : enable // this is for uint64_t

layout(push_constant) uniform constants
{
    uint selected; // this is not used on this shader, but would be used on a mesh shader to tint the output color if object is selected for example
    uint64_t id;
    mat4 model;
} pushConstant;

layout(set = 0, binding = 0) uniform ubo_camera
{
    mat4 view;
    mat4 proj;
} camera;

layout(location = 0) in vec3 inPosition;

void main()
{
    // set vertex position on world
    gl_Position = camera.proj * camera.view * pushConstant.model * vec4(inPosition, 1.0);
}

//////////////////////////////////////////////////////////////////// fragment shader
#version 450
#extension GL_ARB_gpu_shader_int64 : enable // this is for uint64_t

layout(push_constant) uniform constants
{
    uint selected; // this is not used on this shader, but would be used on a mesh shader to tint the output color if object is selected for example
    uint64_t id;
    mat4 model;
} pushConstant;

layout(set = 0, binding = 0) uniform ubo_camera
{
    mat4 view;
    mat4 proj;
} camera;

layout(binding = 1) uniform sampler2D colorMapSampler;

layout(location = 0) out uvec2 outColor;

void main()
{
    // we're going to separate our uint64_t into a vec2 of float, latter on CPU code we're going to read it back
    // extract lower 32 bits
    uint lowerBits = uint(pushConstant.id & 0xFFFFFFFFUL); 

    // extract upper 32 bits
    uint upperBits = uint(pushConstant.id >> 32);         

    // convert to uvec2 by packing the parts as floats
    outColor = uvec2(uint(lowerBits), uint(upperBits));
}

My choice of id may be questionable, since endianess play a part on it. It’s also not an UUID, uses an extension, etc. The id system I use around my engine is an uint64_t variable, and so it’s size is 8 bytes. However the fragment shader output does not accept this format directly, I must “separate” the lower and higher bits from the push constant (I use an extension that adds supports for this “object type”) and then output a uvec2 with the 2 variables. This way I can “safely” cast them back to an uint64_t on CPU side.

Before we write our equivalent of glReadPixel() function, we can’t forget to redraw the scene again, writting the id’s on the objects with the push constants.

void Picking::ManageRenderpass(uint32_t currentFrame, uint32_t swapchainIndex)
{
	std::vector<VkClearValue> clearValues(2);
	clearValues[0].color = { {0.0f, 0.0f, 0.0f, 0.0f} };
	clearValues[1].depthStencil = { 1.0f, 0 };

	// render pass
	{
		VkCommandBuffer& cmdBuffer = mRenderpassesLib.GetRef("Picking")->GetCommandfuffersRef()[currentFrame];
		VkFramebuffer& frameBuffer = mRenderpassesLib.GetRef("Picking")->GetFramebuffersRef()[swapchainIndex];
		VkRenderPass& renderPass = mRenderpassesLib.GetRef("Picking")->GetRenderpassRef();

		vkResetCommandBuffer(cmdBuffer, /*VkCommandBufferResetFlagBits*/ 0);

		VkCommandBufferBeginInfo cmdBeginInfo = {};
		cmdBeginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
		cmdBeginInfo.pNext = nullptr;
		cmdBeginInfo.flags = 0;
		COSMOS_ASSERT(vkBeginCommandBuffer(cmdBuffer, &cmdBeginInfo) == VK_SUCCESS, "Failed to begin command buffer recording");

		VkRenderPassBeginInfo renderPassBeginInfo = {};
		renderPassBeginInfo.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;
		renderPassBeginInfo.renderPass = renderPass;
		renderPassBeginInfo.framebuffer = frameBuffer;
		renderPassBeginInfo.renderArea.offset = { 0, 0 };
		renderPassBeginInfo.renderArea.extent = mSwapchain->GetExtent();
		renderPassBeginInfo.clearValueCount = (uint32_t)clearValues.size();
		renderPassBeginInfo.pClearValues = clearValues.data();
		vkCmdBeginRenderPass(cmdBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

		// set frame commandbuffer viewport
		VkExtent2D extent = mSwapchain->GetExtent();
		VkViewport viewport = {};
		viewport.x = 0.0f;
		viewport.y = 0.0f;
		viewport.width = (float)extent.width;
		viewport.height = (float)extent.height;
		viewport.minDepth = 0.0f;
		viewport.maxDepth = 1.0f;
		vkCmdSetViewport(cmdBuffer, 0, 1, &viewport);

		auto& boundaries = IContext::GetRef()->GetViewportBoundariesRef();
		glm::vec2 mousePos = glm::clamp(Platform::MainWindow::GetRef().GetViewportCursorPos(boundaries.position, boundaries.size), glm::vec2(0.0f, 0.0f), glm::vec2(extent.width, extent.height));
		
		VkRect2D scissor = {};
		scissor.offset = { (int32_t)mousePos.x, (int32_t)mousePos.y };
		scissor.extent = { 1, 1 };
		vkCmdSetScissor(cmdBuffer, 0, 1, &scissor);

		// render objects
		mApplication->OnRender(IContext::Stage::Picking);

		// end render pass
		vkCmdEndRenderPass(cmdBuffer);

		// end command buffer
		COSMOS_ASSERT(vkEndCommandBuffer(cmdBuffer) == VK_SUCCESS, "Failed to end command buffer recording");
	}
}

Since I use a custom viewport I must “offset” my cursor position to the relative position on the viewport (behind the scene whats happening is: I click on the viewport but the scene is rendered at the window’s entire size, so I keep track of the corrected position of the mouse cursor), but if you don’t have a viewport, just use raw coordinates (don’t forget to clamp it from 0.0f to window’s size, we don’t want negative numbers here).

If you do have a viewport and is having trouble to properly offset it, this is the code I use:

glm::vec2 MainWindow::GetViewportCursorPos(const glm::vec2& viewportPosition, const glm::vec2& viewportSize)
{
    // window size in screen coordinates
    glm::vec2 windowSize = GetWindowSize();

    // relative mouse position on viewport
    glm::vec2 relativePos = GetCursorPos() - viewportPosition;

    // normalize mouse position to viewport position
    glm::vec2 normalized = { relativePos / viewportSize };

    // final coordinates
    return { windowSize * normalized };
}

The vulkan viewport object(VkViewport) has the same size as the framebuffer/swapchain but the scissor is 1×1 pixel only, this is for better performance (The viewport is used to transform the matrices and the scissor to ensure that rasterized pixels are within a range, 1×1 in this case).

Now the OnRender function really depends on the implementation but what’s going to happen is the redrawing of the objects with the picking renderpass.

You can’t forget to submit our command buffers after ending our renderpass alongside the other command buffers you most-certanly have.

3) glReadPixels equivalent?

After that there’s the ReadPixel function that can be called on demand (like a mouse press). We create a stagging buffer, map and copy a pixel value(where our cursor position is) from our colorImage to our stagging buffer and reconstruct our uint64_t from a glm::vec2 (this is my id system, if you use an uint32_t as id your life will be easier).

void Picking::ReadImagePixels(glm::vec2 pos)
{
	auto renderpass = mRenderpassesLib.GetRef("Picking");
	VkCommandBuffer cmdBuffer = mDevice->BeginSingleTimeCommand(renderpass->GetCommandPoolRef());

	// convert the image into transferable
	mDevice->InsertImageMemoryBarrier
	(
		cmdBuffer,
		mColorImages[mSwapchain->GetImageIndexRef()],
		VK_ACCESS_TRANSFER_READ_BIT,
		VK_ACCESS_MEMORY_READ_BIT,
		VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
		VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
		VK_PIPELINE_STAGE_TRANSFER_BIT,
		VK_PIPELINE_STAGE_TRANSFER_BIT,
		VkImageSubresourceRange{ VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1 }
	);

	// create the staging buffer
	VkBuffer stagingBuffer;
	VmaAllocation stagingBufferMemory;

	mDevice->CreateBuffer
	(
		VK_BUFFER_USAGE_TRANSFER_DST_BIT,
		VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,
		mImageSize,
		&stagingBuffer,
		&stagingBufferMemory
	);

	// copy the image to the buffer
	VkBufferImageCopy region = {};
	region.bufferOffset = 0;
	region.bufferRowLength = 0;
	region.bufferImageHeight = 0;
	region.imageSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
	region.imageSubresource.mipLevel = 0;
	region.imageSubresource.baseArrayLayer = 0;
	region.imageSubresource.layerCount = 1;
	region.imageOffset = { (int32_t)pos.x, (int32_t)pos.y, 0 };
	region.imageExtent = { 1, 1, 1 };
	vkCmdCopyImageToBuffer(cmdBuffer, mColorImages[mSwapchain->GetImageIndexRef()], VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, stagingBuffer, 1, &region);

	// convert the image into shader optimal again
	mDevice->InsertImageMemoryBarrier
	(
		cmdBuffer,
		mColorImages[mSwapchain->GetImageIndexRef()], // we'll be testing with only one image for now
		VK_ACCESS_TRANSFER_READ_BIT,
		VK_ACCESS_MEMORY_READ_BIT,
		VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
		VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
		VK_PIPELINE_STAGE_TRANSFER_BIT,
		VK_PIPELINE_STAGE_TRANSFER_BIT,
		VkImageSubresourceRange{ VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1 }
	);

	mDevice->EndSingleTimeCommand(renderpass->GetCommandPoolRef(), cmdBuffer);

	// map the buffer memory and read the pixels
	void* data;
	vmaMapMemory(mDevice->GetAllocator(), stagingBufferMemory, &data);
        // it would be wise to test if these variables doesn't hold garbage values
	uint32_t* mappedData = reinterpret_cast<uint32_t*>(data);
        uint32_t x = mappedData[0]; // first component of vec2
        uint32_t y = mappedData[1]; // second component of vec2
        uint32_t lowerBits = (uint32_t)x; // first float as uint
        uint32_t upperBits = (uint32_t)y; // second float as uint
        uint64_t reconstructedValue = ((uint64_t)upperBits << 32) | lowerBits;
        COSMOS_LOG(Logger::Trace, "ID: %llu", reconstructedValue);

	vmaUnmapMemory(mDevice->GetAllocator(), stagingBufferMemory);

	// cleanup
	vmaDestroyBuffer(mDevice->GetAllocator(), stagingBuffer, stagingBufferMemory);
}

Then you can use this function when appropriate,like a mouse press event. In my case I have an event system and I call this function on a mouse click when it’ happens inside of the viewport.

Demonstration video:

Final thoughs

Maybe there is a way to achieve this in one sub-pass but I don’t quite now how so I’ll leave this to a more experienced person;

You could also only issue the renderpass whe a mouse click is made (currently we only read on demand, but always write);

If you have trouble and need source code, it’ll be available on this repo. Suggestions, corrections and questions are more than welcome.