# HG changeset patch # User sam # Date 1721838394 -25200 # Node ID c157707618659d75185fe6618c21f66dff0f6547 # Parent 3560893650767ae1f60064bf7b46a6f8b51828bd add: gltf loading test, gltf loading for materials diff -r 356089365076 -r c15770761865 semiconginev2/contrib/algorithms/texture_packing.nim --- a/semiconginev2/contrib/algorithms/texture_packing.nim Wed Jul 24 20:12:19 2024 +0700 +++ b/semiconginev2/contrib/algorithms/texture_packing.nim Wed Jul 24 23:26:34 2024 +0700 @@ -74,7 +74,7 @@ for x in 0 ..< rect.w: when T is Gray: assert result.atlas[rect.x + x, rect.y + y] == [0'u8], "Atlas texture packing encountered an overlap error" - elif T is RGBA: + elif T is BGRA: assert result.atlas[rect.x + x, rect.y + y] == [0'u8, 0'u8, 0'u8, 0'u8], "Atlas texture packing encountered an overlap error" else: {.error: "Unsupported type for texture packing".} diff -r 356089365076 -r c15770761865 semiconginev2/gltf.nim --- a/semiconginev2/gltf.nim Wed Jul 24 20:12:19 2024 +0700 +++ b/semiconginev2/gltf.nim Wed Jul 24 23:26:34 2024 +0700 @@ -1,9 +1,10 @@ type - GLTFMesh[TMesh, TMaterial] = object - scenes: seq[int] - nodes: seq[int] - meshes: seq[TMesh] - materials: seq[TMaterial] + GLTFMesh*[TMesh, TMaterial] = object + scenes*: seq[seq[int]] # each scene has a seq of node indices + nodes*: seq[seq[int]] # each node has a seq of mesh indices + meshes*: seq[TMesh] + materials*: seq[TMaterial] + textures*: seq[Image[BGRA]] glTFHeader = object magic: uint32 version: uint32 @@ -13,20 +14,41 @@ binaryBufferData: seq[uint8] MaterialAttributeNames = object + # pbr + baseColorTexture: string + baseColorTextureUv: string baseColorFactor: string - emissiveFactor: string + metallicRoughnessTexture: string + metallicRoughnessTextureUv: string metallicFactor: string roughnessFactor: string - baseColorTexture: string - metallicRoughnessTexture: string + + # other normalTexture: string + normalTextureUv: string occlusionTexture: string + occlusionTextureUv: string emissiveTexture: string + emissiveTextureUv: string + emissiveFactor: string + +#[ +static: + let TypeIds = { + int8: 5120, + uint8: 5121, + int16: 5122, + uint16: 5123, + uint32: 5125, + float32: 5126, + }.toTable +]# const HEADER_MAGIC = 0x46546C67 JSON_CHUNK = 0x4E4F534A BINARY_CHUNK = 0x004E4942 + #[ ACCESSOR_TYPE_MAP = { 5120: Int8, 5121: UInt8, @@ -35,6 +57,7 @@ 5125: UInt32, 5126: Float32, }.toTable + ]# SAMPLER_FILTER_MODE_MAP = { 9728: VK_FILTER_NEAREST, 9729: VK_FILTER_LINEAR, @@ -48,18 +71,8 @@ 33648: VK_SAMPLER_ADDRESS_MODE_MIRRORED_REPEAT, 10497: VK_SAMPLER_ADDRESS_MODE_REPEAT }.toTable - GLTF_MATERIAL_MAPPING = { - "color": "baseColorFactor", - "emissiveColor": "emissiveFactor", - "metallic": "metallicFactor", - "roughness", "roughnessFactor", - "baseTexture": "baseColorTexture", - "metallicRoughnessTexture": "metallicRoughnessTexture", - "normalTexture": "normalTexture", - "occlusionTexture": "occlusionTexture", - "emissiveTexture": "emissiveTexture", - }.toTable +#[ proc getGPUType(accessor: JsonNode, attribute: string): DataType = # TODO: no full support for all datatypes that glTF may provide # semicongine/core/gpu_data should maybe generated with macros to allow for all combinations @@ -95,6 +108,7 @@ case componentType of Float32: return Vec4F32 else: raise newException(Exception, &"Unsupported data type for attribute '{attribute}': {componentType} {theType}") +]# proc getBufferViewData(bufferView: JsonNode, mainBuffer: seq[uint8], baseBufferOffset = 0): seq[uint8] = assert bufferView["buffer"].getInt() == 0, "Currently no external buffers supported" @@ -107,6 +121,7 @@ raise newException(Exception, "Unsupported feature: byteStride in buffer view") copyMem(dstPointer, addr mainBuffer[bufferOffset], result.len) +#[ proc getAccessorData(root: JsonNode, accessor: JsonNode, mainBuffer: seq[uint8]): DataList = result = InitDataList(thetype = accessor.getGPUType("??")) result.SetLen(accessor["count"].getInt()) @@ -130,113 +145,60 @@ dstPointer = cast[pointer](cast[uint](dstPointer) + result.thetype.Size) else: copyMem(dstPointer, addr mainBuffer[bufferOffset], length) +]# -proc loadImage(root: JsonNode, imageIndex: int, mainBuffer: seq[uint8]): Image[RGBAPixel] = +proc loadTexture(root: JsonNode, textureNode: JsonNode, mainBuffer: seq[uint8]): Image[BGRA] = + + let imageIndex = textureNode["source"].getInt() + if root["images"][imageIndex].hasKey("uri"): - raise newException(Exception, "Unsupported feature: Load images from external files") + raise newException(Exception, "Unsupported feature: Cannot load images from external files") + let imageType = root["images"][imageIndex]["mimeType"].getStr() + assert imageType == "image/png", "glTF loader currently only supports PNG" let bufferView = root["bufferViews"][root["images"][imageIndex]["bufferView"].getInt()] - let imgData = newStringStream(cast[string](getBufferViewData(bufferView, mainBuffer))) - - let imageType = root["images"][imageIndex]["mimeType"].getStr() - case imageType - of "image/bmp": - result = ReadBMP(imgData) - of "image/png": - result = ReadPNG(imgData) - else: - raise newException(Exception, "Unsupported feature: Load image of type " & imageType) - -proc loadTexture(root: JsonNode, textureIndex: int, mainBuffer: seq[uint8]): Texture = - let textureNode = root["textures"][textureIndex] - result = Texture(isGrayscale: false) - result.colorImage = loadImage(root, textureNode["source"].getInt(), mainBuffer) - result.name = root["images"][textureNode["source"].getInt()]["name"].getStr() - if result.name == "": - result.name = &"Texture{textureIndex}" + result = LoadImage[BGRA](getBufferViewData(bufferView, mainBuffer)) if textureNode.hasKey("sampler"): let sampler = root["samplers"][textureNode["sampler"].getInt()] if sampler.hasKey("magFilter"): - result.sampler.magnification = SAMPLER_FILTER_MODE_MAP[sampler["magFilter"].getInt()] + result.magInterpolation = SAMPLER_FILTER_MODE_MAP[sampler["magFilter"].getInt()] if sampler.hasKey("minFilter"): - result.sampler.minification = SAMPLER_FILTER_MODE_MAP[sampler["minFilter"].getInt()] + result.minInterpolation = SAMPLER_FILTER_MODE_MAP[sampler["minFilter"].getInt()] if sampler.hasKey("wrapS"): - result.sampler.wrapModeS = SAMPLER_WRAP_MODE_MAP[sampler["wrapS"].getInt()] + result.wrapU = SAMPLER_WRAP_MODE_MAP[sampler["wrapS"].getInt()] if sampler.hasKey("wrapT"): - result.sampler.wrapModeT = SAMPLER_WRAP_MODE_MAP[sampler["wrapS"].getInt()] + result.wrapV = SAMPLER_WRAP_MODE_MAP[sampler["wrapT"].getInt()] +proc getVec4f(node: JsonNode): Vec4f = + NewVec4f(node[0].getFloat(), node[1].getFloat(), node[2].getFloat(), node[3].getFloat()) proc loadMaterial[TMaterial]( root: JsonNode, materialNode: JsonNode, mainBuffer: seq[uint8], - mapping: MaterialAttributeNames + mapping: static MaterialAttributeNames ): TMaterial = - let pbr = materialNode["pbrMetallicRoughness"] - for glName, glValue in fieldPairs(mapping): - if glValue != "": - for name, value in fieldPairs(result): - when name == glName: - value = - - #[ - - # color - if defaultMaterial.attributes.contains("color"): - attributes["color"] = InitDataList(thetype = Vec4F32) - if pbr.hasKey(GLTF_MATERIAL_MAPPING["color"]): - attributes["color"] = @[NewVec4f( - pbr[GLTF_MATERIAL_MAPPING["color"]][0].getFloat(), - pbr[GLTF_MATERIAL_MAPPING["color"]][1].getFloat(), - pbr[GLTF_MATERIAL_MAPPING["color"]][2].getFloat(), - pbr[GLTF_MATERIAL_MAPPING["color"]][3].getFloat(), - )] - else: - attributes["color"] = @[NewVec4f(1, 1, 1, 1)] - - # pbr material values - for factor in ["metallic", "roughness"]: - if defaultMaterial.attributes.contains(factor): - attributes[factor] = InitDataList(thetype = Float32) - if pbr.hasKey(GLTF_MATERIAL_MAPPING[factor]): - attributes[factor] = @[float32(pbr[GLTF_MATERIAL_MAPPING[factor]].getFloat())] - else: - attributes[factor] = @[0.5'f32] + result = TMaterial() - # pbr material textures - for texture in ["baseTexture", "metallicRoughnessTexture"]: - if defaultMaterial.attributes.contains(texture): - attributes[texture] = InitDataList(thetype = TextureType) - # attributes[texture & "Index"] = InitDataList(thetype=UInt8) - if pbr.hasKey(GLTF_MATERIAL_MAPPING[texture]): - attributes[texture] = @[loadTexture(root, pbr[GLTF_MATERIAL_MAPPING[texture]]["index"].getInt(), mainBuffer)] - else: - attributes[texture] = @[EMPTY_TEXTURE] + let pbr = materialNode["pbrMetallicRoughness"] + for name, value in fieldPairs(result): + for gltfAttribute, mappedName in fieldPairs(mapping): + when gltfAttribute != "" and name == mappedName: + if pbr.hasKey(gltfAttribute): + when gltfAttribute.endsWith("Texture"): + value = typeof(value)(pbr[gltfAttribute]["index"].getInt()) + elif gltfAttribute.endsWith("TextureUv"): + value = typeof(pbr[gltfAttribute[0 ..< ^2]]["index"].getInt()) + elif gltfAttribute in ["baseColorFactor", "emissiveFactor"]: + value = pbr[gltfAttribute].getVec4f() + elif gltfAttribute in ["metallicFactor", "roughnessFactor"]: + value = pbr[gltfAttribute].getFloat() + else: + {.error: "Unsupported gltf material attribute".} - # generic material textures - for texture in ["normalTexture", "occlusionTexture", "emissiveTexture"]: - if defaultMaterial.attributes.contains(texture): - attributes[texture] = InitDataList(thetype = TextureType) - # attributes[texture & "Index"] = InitDataList(thetype=UInt8) - if materialNode.hasKey(GLTF_MATERIAL_MAPPING[texture]): - attributes[texture] = @[loadTexture(root, materialNode[texture]["index"].getInt(), mainBuffer)] - else: - attributes[texture] = @[EMPTY_TEXTURE] - # emissiv color - if defaultMaterial.attributes.contains("emissiveColor"): - attributes["emissiveColor"] = InitDataList(thetype = Vec3F32) - if materialNode.hasKey(GLTF_MATERIAL_MAPPING["emissiveColor"]): - attributes["emissiveColor"] = @[NewVec3f( - materialNode[GLTF_MATERIAL_MAPPING["emissiveColor"]][0].getFloat(), - materialNode[GLTF_MATERIAL_MAPPING["emissiveColor"]][1].getFloat(), - materialNode[GLTF_MATERIAL_MAPPING["emissiveColor"]][2].getFloat(), - )] - else: - attributes["emissiveColor"] = @[NewVec3f(1'f32, 1'f32, 1'f32)] - ]# - +#[ proc loadMesh(meshname: string, root: JsonNode, primitiveNode: JsonNode, materials: seq[MaterialData], mainBuffer: seq[uint8]): Mesh = if primitiveNode.hasKey("mode") and primitiveNode["mode"].getInt() != 4: @@ -296,6 +258,7 @@ # TODO: getting from gltf to vulkan system is still messed up somehow, see other TODO Transform[Vec3f](result[], "position", Scale(1, -1, 1)) + proc loadNode(root: JsonNode, node: JsonNode, materials: seq[MaterialData], mainBuffer: var seq[uint8]): MeshTree = result = MeshTree() # mesh @@ -349,30 +312,30 @@ # result.transform = Scale(1, -1, 1) result.updateTransforms() + ]# -proc ReadglTF*[TMaterial, TMesh]( +proc ReadglTF*[TMesh, TMaterial]( stream: Stream, - attributeNames: MaterialAttributeNames, - baseColorFactor = "", - emissiveFactor = "", - metallicFactor = "", - roughnessFactor = "", - baseColorTexture = "", - metallicRoughnessTexture = "", - normalTexture = "", - occlusionTexture = "", - emissiveTexture = "", + baseColorFactor: static string = "", + emissiveFactor: static string = "", + metallicFactor: static string = "", + roughnessFactor: static string = "", + baseColorTexture: static string = "", + metallicRoughnessTexture: static string = "", + normalTexture: static string = "", + occlusionTexture: static string = "", + emissiveTexture: static string = "", ): GLTFMesh[TMesh, TMaterial] = - let mapping = MaterialAttributeNames( - baseColorFactor: baseColorFactor - emissiveFactor: emissiveFactor - metallicFactor: metallicFactor - roughnessFactor: roughnessFactor - baseColorTexture: baseColorTexture - metallicRoughnessTexture: metallicRoughnessTexture - normalTexture: normalTexture - occlusionTexture: occlusionTexture - emissiveTexture: emissiveTexture + const mapping = MaterialAttributeNames( + baseColorFactor: baseColorFactor, + emissiveFactor: emissiveFactor, + metallicFactor: metallicFactor, + roughnessFactor: roughnessFactor, + baseColorTexture: baseColorTexture, + metallicRoughnessTexture: metallicRoughnessTexture, + normalTexture: normalTexture, + occlusionTexture: occlusionTexture, + emissiveTexture: emissiveTexture, ) var header: glTFHeader @@ -402,9 +365,40 @@ debug "Loading mesh: ", data.structuredContent.pretty - var materials: seq[MaterialData] - for materialnode in data.structuredContent["materials"]: - result.materials.add loadMaterial[TMaterial](data.structuredContent, materialnode, data.binaryBufferData, mapping) + if "materials" in data.structuredContent: + for materialnode in items(data.structuredContent["materials"]): + result.materials.add loadMaterial[TMaterial](data.structuredContent, materialnode, data.binaryBufferData, mapping) + + if "textures" in data.structuredContent: + for texturenode in items(data.structuredContent["textures"]): + result.textures.add loadTexture(data.structuredContent, texturenode, data.binaryBufferData) - for scenedata in data.structuredContent["scenes"]: - result.add data.structuredContent.loadScene(scenedata, materials, data.binaryBufferData) + echo result + # for scenedata in data.structuredContent["scenes"]: + # result.add data.structuredContent.loadScene(scenedata, materials, data.binaryBufferData) + # +proc LoadMeshes*[TMesh, TMaterial]( + path: string, + baseColorFactor: static string = "", + emissiveFactor: static string = "", + metallicFactor: static string = "", + roughnessFactor: static string = "", + baseColorTexture: static string = "", + metallicRoughnessTexture: static string = "", + normalTexture: static string = "", + occlusionTexture: static string = "", + emissiveTexture: static string = "", + package = DEFAULT_PACKAGE +): GLTFMesh[TMesh, TMaterial] = + ReadglTF[TMesh, TMaterial]( + stream = loadResource_intern(path, package = package), + baseColorFactor = baseColorFactor, + emissiveFactor = emissiveFactor, + metallicFactor = metallicFactor, + roughnessFactor = roughnessFactor, + baseColorTexture = baseColorTexture, + metallicRoughnessTexture = metallicRoughnessTexture, + normalTexture = normalTexture, + occlusionTexture = occlusionTexture, + emissiveTexture = emissiveTexture, + ) diff -r 356089365076 -r c15770761865 semiconginev2/image.nim --- a/semiconginev2/image.nim Wed Jul 24 20:12:19 2024 +0700 +++ b/semiconginev2/image.nim Wed Jul 24 23:26:34 2024 +0700 @@ -1,11 +1,14 @@ type Gray* = TVec1[uint8] - RGBA* = TVec4[uint8] - PixelType* = Gray | RGBA + BGRA* = TVec4[uint8] + PixelType* = Gray | BGRA Image*[T: PixelType] = object width*: uint32 height*: uint32 - interpolation*: VkFilter = VK_FILTER_LINEAR + minInterpolation*: VkFilter = VK_FILTER_LINEAR + magInterpolation*: VkFilter = VK_FILTER_LINEAR + wrapU: VkSamplerAddressMode = VK_SAMPLER_ADDRESS_MODE_REPEAT + wrapV: VkSamplerAddressMode = VK_SAMPLER_ADDRESS_MODE_REPEAT data*: seq[T] vk*: VkImage imageview*: VkImageView @@ -13,18 +16,16 @@ isRenderTarget*: bool = false samples*: VkSampleCountFlagBits = VK_SAMPLE_COUNT_1_BIT -proc LoadImage*[T: PixelType](path: string, package = DEFAULT_PACKAGE): Image[T] = - assert path.splitFile().ext.toLowerAscii == ".png" +proc LoadImage*[T: PixelType](pngData: seq[uint8]): Image[T] = when T is Gray: let pngType = 0.cint - elif T is RGBA: + elif T is BGRA: let pngType = 6.cint - let indata = loadResource_intern(path, package = package).readAll() var w, h: cuint var data: cstring - if lodepng_decode_memory(out_data = addr(data), w = addr(w), h = addr(h), in_data = cstring(indata), insize = csize_t(indata.len), colorType = pngType, bitdepth = 8) != 0: + if lodepng_decode_memory(out_data = addr(data), w = addr(w), h = addr(h), in_data = cast[cstring](pngData.ToCPointer), insize = csize_t(pngData.len), colorType = pngType, bitdepth = 8) != 0: raise newException(Exception, "An error occured while loading PNG file") let imagesize = w * h * 4 @@ -32,10 +33,20 @@ copyMem(result.data.ToCPointer, data, imagesize) nativeFree(data) - when T is RGBA: # converkt to BGRA + when T is BGRA: # converkt to BGRA for i in 0 ..< result.data.len: swap(result.data[i][0], result.data[i][2]) +proc LoadImage*[T: PixelType](path: string, package = DEFAULT_PACKAGE): Image[T] = + assert path.splitFile().ext.toLowerAscii == ".png" + when T is Gray: + let pngType = 0.cint + elif T is BGRA: + let pngType = 6.cint + + result = LoadImage[T](loadResource_intern(path, package = package).readAll()) + + proc toPNG[T: PixelType](image: Image[T]): seq[uint8] = when T is Gray: let pngType = 0 # hardcoded in lodepng.h diff -r 356089365076 -r c15770761865 semiconginev2/rendering/renderer.nim --- a/semiconginev2/rendering/renderer.nim Wed Jul 24 20:12:19 2024 +0700 +++ b/semiconginev2/rendering/renderer.nim Wed Jul 24 23:26:34 2024 +0700 @@ -470,7 +470,12 @@ image.vk = svkCreate2DImage(image.width, image.height, format, usage, image.samples) renderData.images.add image.vk - image.sampler = createSampler(magFilter = image.interpolation, minFilter = image.interpolation) + image.sampler = createSampler( + magFilter = image.magInterpolation, + minFilter = image.minInterpolation, + addressModeU = image.wrapU, + addressModeV = image.wrapV, + ) renderData.samplers.add image.sampler let memoryRequirements = image.vk.svkGetImageMemoryRequirements() diff -r 356089365076 -r c15770761865 semiconginev2/rendering/shaders.nim --- a/semiconginev2/rendering/shaders.nim Wed Jul 24 20:12:19 2024 +0700 +++ b/semiconginev2/rendering/shaders.nim Wed Jul 24 23:26:34 2024 +0700 @@ -36,7 +36,9 @@ elif T is TMat4[float32]: "mat4" elif T is TMat4[float64]: "dmat4" elif T is Image: "sampler2D" - else: {.error: "Unsupported data type on GPU".} + else: + const n = typetraits.name(T) + {.error: "Unsupported data type on GPU: " & n.} func VkType[T: SupportedGPUType](value: T): VkFormat = when T is float32: VK_FORMAT_R32_SFLOAT diff -r 356089365076 -r c15770761865 tests/test_gltf.nim --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/test_gltf.nim Wed Jul 24 23:26:34 2024 +0700 @@ -0,0 +1,95 @@ +import std/os +import std/sequtils +import std/monotimes +import std/times +import std/options +import std/random + +import ../semiconginev2 + +proc test_gltf(time: float32) = + var renderdata = InitRenderData() + + type + Material = object + color: Vec4f + colorTexture: int32 = -1 + metallic: float32 = -1 + roughness: float32 = -1 + metallicRoughnessTexture: int32 = -1 + + normalTexture: int32 = -1 + occlusionTexture: int32 = -1 + emissive: Vec4f = NewVec4f(-1, -1, -1, -1) + emissiveTexture: int32 = -1 + MainDescriptors = object + material: GPUValue[Material, UniformBuffer] + Shader = object + position {.VertexAttribute.}: Vec3f + color {.VertexAttribute.}: Vec4f + uv {.VertexAttribute.}: Vec2f + fragmentColor {.Pass.}: Vec4f + fragmentUv {.Pass.}: Vec2f + outColor {.ShaderOutput.}: Vec4f + descriptors {.DescriptorSets.}: (MainDescriptors, ) + # code + vertexCode: string = """ +void main() { + fragmentColor = color; + fragmentUv = uv; + gl_Position = vec4(position, 1); +}""" + fragmentCode: string = """void main() { outColor = fragmentColor;}""" + Mesh = object + position: GPUArray[Vec3f, VertexBuffer] + color: GPUArray[Vec4f, VertexBuffer] + uv: GPUArray[Vec2f, VertexBuffer] + + let gltfMesh = LoadMeshes[Mesh, Material]( + "town.glb", + baseColorFactor = "color", + baseColorTexture = "colorTexture", + metallicFactor = "metallic", + roughnessFactor = "roughness", + metallicRoughnessTexture = "metallicRoughnessTexture", + normalTexture = "normalTexture", + occlusionTexture = "occlusionTexture", + emissiveTexture = "emissiveTexture", + emissiveFactor = "emissive", + ) + var mesh = gltfMesh.meshes[0] + renderdata.AssignBuffers(mesh) + renderdata.FlushAllMemory() + + var pipeline = CreatePipeline[Shader](renderPass = vulkan.swapchain.renderPass) + + var start = getMonoTime() + while ((getMonoTime() - start).inMilliseconds().int / 1000) < time: + + WithNextFrame(framebuffer, commandbuffer): + + WithRenderPass(vulkan.swapchain.renderPass, framebuffer, commandbuffer, vulkan.swapchain.width, vulkan.swapchain.height, NewVec4f(0, 0, 0, 0)): + + WithPipeline(commandbuffer, pipeline): + + Render(commandbuffer = commandbuffer, pipeline = pipeline, mesh = mesh) + + # cleanup + checkVkResult vkDeviceWaitIdle(vulkan.device) + DestroyPipeline(pipeline) + DestroyRenderData(renderdata) +when isMainModule: + var time = 1'f32 + InitVulkan() + + var renderpass = CreateDirectPresentationRenderPass(depthBuffer = true, samples = VK_SAMPLE_COUNT_4_BIT) + SetupSwapchain(renderpass = renderpass) + + # tests a simple triangle with minimalistic shader and vertex format + test_gltf(time) + + checkVkResult vkDeviceWaitIdle(vulkan.device) + vkDestroyRenderPass(vulkan.device, renderpass.vk, nil) + ClearSwapchain() + + DestroyVulkan()