2
votes

I've been at this for a better part of today and I'm at the end of my wits.

I'm running Vulkan SDK 1.2.131.2

I have a RTX 2080 Ti.

I have Windows 10 Education, version 1909, build 18363.657.

I'm using Vulkan.hpp instead of Vulkan.h directly.

Here is where I specify the API version I use:

appInfo.apiVersion = VK_API_VERSION_1_2;

This is the relevant part of the code that creates the device:

// bla bla
const std::vector<const char*> deviceExtensions = {
    VK_KHR_SWAPCHAIN_EXTENSION_NAME,
    VK_KHR_TIMELINE_SEMAPHORE_EXTENSION_NAME,
    VK_NV_RAY_TRACING_EXTENSION_NAME
};

deviceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(deviceExtensions.size());
deviceCreateInfo.ppEnabledExtensionNames = deviceExtensions.data();

m_logicalDevice = m_physicalDevice.createDeviceUnique(deviceCreateInfo);

I use the following validation layers:

"VK_LAYER_LUNARG_api_dump"
"VK_LAYER_KHRONOS_validation"

This is how I later try to create a timeline semaphore:

vk::UniqueSemaphore VulkanContext::createTimelineSemaphore(const uint32_t initialValue) const {
    vk::SemaphoreTypeCreateInfo timelineCreateInfo;
    timelineCreateInfo.semaphoreType = vk::SemaphoreType::eTimeline;
    timelineCreateInfo.initialValue = initialValue;

    vk::SemaphoreCreateInfo createInfo;
    createInfo.pNext = &timelineCreateInfo;

    return m_logicalDevice->createSemaphoreUnique(createInfo);
}

I get the following error:

vkCreateSemaphore(device, pCreateInfo, pAllocator, pSemaphore) returns VkResultVkCreateSemaphore: timelineSemaphore feature is not enabled, can not create timeline semaphores The Vulkan spec states: If the timelineSemaphore feature is not enabled, semaphoreType must not equal VK_SEMAPHORE_TYPE_TIMELINE (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkSemaphoreTypeCreateInfo-timelineSemaphore-03252)

This is even more infuriating because timeline semaphores are supposed to be part of the core Vulkan 1.2, but I get the same error even if I ommit it from the extension list. Swapchain extension does work and I haven't had the time to verify that ray tracing extension is enabled.

It gets even more stupid because the next message tells me this:

VK_SUCCESS (0):
device:                         VkDevice = 0000023AA29BD8B0
pCreateInfo:                    const VkSemaphoreCreateInfo* = 0000008D145ED538:
    sType:                          VkStructureType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO (9)
    pNext:                          VkSemaphoreTypeCreateInfo = 0000008D145ED4F8:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO (1000207002)
        pNext:                          const void* = NULL
        semaphoreType:                  VkSemaphoreType = VK_SEMAPHORE_TYPE_TIMELINE (1)
        initialValue:                   uint64_t = 0
    flags:                          VkSemaphoreCreateFlags = 0
pAllocator:                     const VkAllocationCallbacks* = NULL
pSemaphore:                     VkSemaphore* = AA989B000000001E

I have no idea if this creates the timeline semaphore or just creates a normal binary one.

When I later use it to submit to a transfer queue:

vk::CommandBufferBeginInfo beginInfo;
transferCmdBuffer->begin(beginInfo);
object->recordUploadToGPU(*transferCmdBuffer);
transferCmdBuffer->end();

vk::TimelineSemaphoreSubmitInfo timelineSubmitInfo;
timelineSubmitInfo.signalSemaphoreValueCount = 1;
timelineSubmitInfo.pSignalSemaphoreValues = &signalValue;

vk::SubmitInfo submitInfo;
submitInfo.pNext = &timelineSubmitInfo;
submitInfo.signalSemaphoreCount = 1;
submitInfo.pSignalSemaphores = &signalSemaphore;
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &*transferCmdBuffer;

vkCtx.m_transferQueue.submit(submitInfo, nullptr);

I get this error here:

vkQueueSubmit(queue, submitCount, pSubmits, fence) returns VkResultVkQueueSubmit: VkQueue 0x23aa2539500[] contains timeline sempahore VkSemaphore 0xaa989b000000001e[] that sets its wait value with a margin greater than maxTimelineSemaphoreValueDifference The Vulkan spec states: For each element of pSignalSemaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE the corresponding element of VkTimelineSemaphoreSubmitInfo::pSignalSemaphoreValues must have a value which does not differ from the current value of the semaphore or the value of any outstanding semaphore wait or signal operation on that semaphore by more than maxTimelineSemaphoreValueDifference. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkSubmitInfo-pSignalSemaphores-03244)

And just to be futher mocked, this is the next line:

VK_SUCCESS (0):
queue:                          VkQueue = 0000023AA2539500
submitCount:                    uint32_t = 1
pSubmits:                       const VkSubmitInfo* = 0000008D145ED370
    pSubmits[0]:                    const VkSubmitInfo = 0000008D145ED370:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_SUBMIT_INFO (4)
        pNext:                          VkTimelineSemaphoreSubmitInfo = 0000008D145ED318:
            sType:                          VkStructureType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO (1000207003)
            pNext:                          const void* = NULL
            waitSemaphoreValueCount:        uint32_t = 0
            pWaitSemaphoreValues:           const uint64_t* = NULL
            signalSemaphoreValueCount:      uint32_t = 1
            pSignalSemaphoreValues:         const uint64_t* = 0000008D145ED740
                pSignalSemaphoreValues[0]:      const uint64_t = 1
        waitSemaphoreCount:             uint32_t = 0
        pWaitSemaphores:                const VkSemaphore* = NULL
        pWaitDstStageMask:              const VkPipelineStageFlags* = NULL
        commandBufferCount:             uint32_t = 1
        pCommandBuffers:                const VkCommandBuffer* = 0000008D145EF408
            pCommandBuffers[0]:             const VkCommandBuffer = 0000023AA9CEC8E0
        signalSemaphoreCount:           uint32_t = 1
        pSignalSemaphores:              const VkSemaphore* = 0000008D145EF430
            pSignalSemaphores[0]:           const VkSemaphore = AA989B000000001E
fence:                          VkFence = 0000000000000000

I've also tried with VK_API_VERSION_1_1 and VK_API_VERSION_1_0, both with and without enabling the extension explicitly, none of them work.

The dumps are from the VK_LAYER_LUNARG_api_dump validation layer, while VK_LAYER_KHRONOS_validation validation layer is the one spewing out the errors. They seem to disagree.

So, what gives?

In what way am I stupid today?

EDIT:

Here is an example that you should be able to run yourself. I think I made it as minimal as I can:

#include <vulkan/vulkan.hpp>
#include <iostream>

VKAPI_ATTR VkBool32 VKAPI_CALL debugCallback(
    VkDebugUtilsMessageSeverityFlagBitsEXT messageSeverity,
    VkDebugUtilsMessageTypeFlagsEXT messageType,
    const VkDebugUtilsMessengerCallbackDataEXT* pCallbackData,
    void* pUserData) {

    std::cerr << pCallbackData->pMessage << std::endl;

    return VK_FALSE;
};

int main() {
    vk::ApplicationInfo appInfo;
    appInfo.apiVersion = VK_API_VERSION_1_2;

    vk::InstanceCreateInfo instanceCreateInfo;
    instanceCreateInfo.pApplicationInfo = &appInfo;

    std::vector<const char*> extensions;

    extensions.push_back(VK_EXT_DEBUG_UTILS_EXTENSION_NAME);

    instanceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(extensions.size());
    instanceCreateInfo.ppEnabledExtensionNames = extensions.data();

    const std::vector<const char*> validationLayers = {
        "VK_LAYER_LUNARG_api_dump",
        "VK_LAYER_KHRONOS_validation"
    };

    instanceCreateInfo.enabledLayerCount = static_cast<uint32_t>(validationLayers.size());
    instanceCreateInfo.ppEnabledLayerNames = validationLayers.data();

    vk::DebugUtilsMessengerCreateInfoEXT debugCreateInfo;
    debugCreateInfo.messageSeverity =
        vk::DebugUtilsMessageSeverityFlagBitsEXT::eInfo |
        vk::DebugUtilsMessageSeverityFlagBitsEXT::eVerbose |
        vk::DebugUtilsMessageSeverityFlagBitsEXT::eWarning |
        vk::DebugUtilsMessageSeverityFlagBitsEXT::eError;
    debugCreateInfo.messageType =
        vk::DebugUtilsMessageTypeFlagBitsEXT::eGeneral |
        vk::DebugUtilsMessageTypeFlagBitsEXT::eValidation |
        vk::DebugUtilsMessageTypeFlagBitsEXT::ePerformance;
    debugCreateInfo.pfnUserCallback = debugCallback;

    instanceCreateInfo.pNext = &debugCreateInfo;

    vk::Instance m_instance = vk::createInstance(instanceCreateInfo);
    vk::DispatchLoaderDynamic m_loader = vk::DispatchLoaderDynamic(m_instance, vkGetInstanceProcAddr);
    vk::DebugUtilsMessengerEXT m_debugMessenger = m_instance.createDebugUtilsMessengerEXT(debugCreateInfo, nullptr, m_loader);

    vk::PhysicalDevice m_physicalDevice = m_instance.enumeratePhysicalDevices()[0];

    std::vector<vk::DeviceQueueCreateInfo> queueCreateInfos;

    vk::DeviceQueueCreateInfo queueInfo;
    queueInfo.queueFamilyIndex = 0;
    queueInfo.queueCount = 1;
    queueCreateInfos.push_back(queueInfo);

    vk::PhysicalDeviceFeatures deviceFeatures;

    vk::DeviceCreateInfo deviceCreateInfo;
    deviceCreateInfo.pQueueCreateInfos = queueCreateInfos.data();
    deviceCreateInfo.queueCreateInfoCount = static_cast<uint32_t>(queueCreateInfos.size());
    deviceCreateInfo.pEnabledFeatures = &deviceFeatures;

    // This part can be omitted from here...
    const std::vector<const char*> deviceExtensions = {
        VK_KHR_TIMELINE_SEMAPHORE_EXTENSION_NAME
    };

    deviceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(deviceExtensions.size());
    deviceCreateInfo.ppEnabledExtensionNames = deviceExtensions.data();
    // ...to here. It doesn't work either way.

    vk::Device m_logicalDevice = m_physicalDevice.createDevice(deviceCreateInfo);

    vk::SemaphoreTypeCreateInfo timelineCreateInfo;
    timelineCreateInfo.semaphoreType = vk::SemaphoreType::eTimeline;
    timelineCreateInfo.initialValue = 0;

    vk::SemaphoreCreateInfo semaphoreCreateInfo;
    semaphoreCreateInfo.pNext = &timelineCreateInfo;

    m_logicalDevice.createSemaphore(semaphoreCreateInfo);
}
1
Thanks for the example setting up the semaphore. Can you continue to flush the example out with queueSubmit in a loop with and without using multiple threads? I think from there I can introduce presentation, but I's having trouble wrapping my head around where/when I'm supposed to do what. - Mike Mestnik
This goes a bit above this question. See here: github.com/SaschaWillems/Vulkan-Samples for various examples. - Karlovsky120
Forked repos are not searchable, but looking around it's not clear if any of those examples or branches address TLS. github.com/KhronosGroup/Vulkan-Samples/… searching the parent reveals the same. - Mike Mestnik
I don't know what to tell you. The basic principles are the same as with semaphores and fences (and you can find plenty of examples for that), just packaged more nicely and user friendly, about which you can read here: khronos.org/blog/vulkan-timeline-semaphores It should be enough to get you to where you need to be. - Karlovsky120
There are two things I believe need to be addressed that I don't see any of these examples tackling. Dynamic time/value. It's like for each submit the values for wait and signal should be calculated somehow. What way are ppl doing that? Presumably the submit happens in a function, does the function take the wait value as a parameter and add 1 for the signal, or does the submit function calculate the wait value and return the signal value? Also there is a need to handle resetting the value for overflow, that means re-creating the semaphore... So it's also possible that semaphores are just - Mike Mestnik

1 Answers

3
votes

The feature needs to be enabled explicitly as well:

vk:PhysicalDeviceVulkan12Features features;
features.timelineSemaphore = true;

vk::DeviceCreateInfo deviceCreateInfo;
deviceCreateInfo.pNext = &features;