-
Notifications
You must be signed in to change notification settings - Fork 15
Complete bxdf tests #236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Complete bxdf tests #236
Conversation
…ng/Nabla-Examples-and-Tests into bxdf_unit_tests
…ng/Nabla-Examples-and-Tests into bxdf_unit_tests
66_HLSLBxDFTests/main.cpp
Outdated
| #include "nlohmann/json.hpp" | ||
|
|
||
| using json = nlohmann::json; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AnastaZIuk is this ok ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, its ok
66_HLSLBxDFTests/main.cpp
Outdated
| int main(int argc, char** argv) | ||
| { | ||
| std::cout << std::fixed << std::setprecision(4); | ||
| IApplicationFramework::GlobalsInit(); | ||
| auto system = IApplicationFramework::createSystem(); | ||
| const system::path CWD = path(argv[0]).parent_path().generic_string() + "/"; | ||
| smart_refctd_ptr<ILogger> logger = make_smart_refctd_ptr<CColoredStdoutLoggerANSI>(ILogger::DefaultLogMask()); | ||
| logger->log("Logger Created!", ILogger::ELL_INFO); | ||
| auto assetManager = make_smart_refctd_ptr<IAssetManager>(smart_refctd_ptr(system)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, if you use the application framework you shouldn't have an int main anymore
66_HLSLBxDFTests/main.cpp
Outdated
| fprintf(stderr, "[ERROR] could not open config file\n"); | ||
| logger->log("could not open config file\n", ILogger::ELL_ERROR); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unformatted String Vulnerability, see https://en.wikipedia.org/wiki/Uncontrolled_format_string
|
|
||
| float bin(float a) | ||
| { | ||
| float diff = std::fmod(a, stride); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nitpick, use the hlsl namespace's tgmath (or intrinsics, I'm sure that @Przemog1 added such a function) fmod function for this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think there's a fmod function. tgmath only has modf, which is different I think.
| if (!s.isValid()) | ||
| continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it should be an error to not generate a valid sample if 100% VNDF is guaranteed by the NDF trait
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#236 (comment)
I guess I'll work on differentiating it in the sample_type::makeInvalid() function
| if (pdf.pdf == bit_cast<float>(numeric_limits<float>::infinity)) | ||
| buckets[bucket] += 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
explain this logic ?
Why are only infinite PDF samples added to a bucket !?
66_HLSLBxDFTests/tests.h
Outdated
|
|
||
| sample_t s; | ||
| quotient_pdf_t pdf; | ||
| float32_t3 bsdf; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unused variable ?
66_HLSLBxDFTests/tests.h
Outdated
| buckets[bucket] += 1; | ||
| } | ||
|
|
||
| #ifndef __HLSL_VERSION |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove the ifndef, you're in C++ header now
| if (!selective || b.second > 0) | ||
| { | ||
| math::Polar<float> polarCoords; | ||
| polarCoords.theta = b.first.x * numbers::pi<float>; | ||
| polarCoords.phi = b.first.y * 2.f * numbers::pi<float>; | ||
| const float32_t3 v = polarCoords.getCartesian(); | ||
| base_t::errMsg += std::format("({:.3f},{:.3f},{:.3f}): {}\n", v.x, v.y, v.z, b.second); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need some documentation about this test.
I don't get why you're only counting inf PDF samples, and why your only error is an inf PDF.
Also why the error message build is done here, and the back buckets are not appended to a list to format and print later closer to cb.__call
| inline float adaptiveSimpson(const std::function<float(float)>& f, float x0, float x1, float eps = 1e-6, int depth = 6) | ||
| { | ||
| int count = 0; | ||
| std::function<float(float, float, float, float, float, float, float, float, int)> integrate = | ||
| [&](float a, float b, float c, float fa, float fb, float fc, float I, float eps, int depth) | ||
| { | ||
| float d = 0.5f * (a + b); | ||
| float e = 0.5f * (b + c); | ||
| float fd = f(d); | ||
| float fe = f(e); | ||
|
|
||
| float h = c - a; | ||
| float I0 = (1.0f / 12.0f) * h * (fa + 4 * fd + fb); | ||
| float I1 = (1.0f / 12.0f) * h * (fb + 4 * fe + fc); | ||
| float Ip = I0 + I1; | ||
| count++; | ||
|
|
||
| if (depth <= 0 || std::abs(Ip - I) < 15 * eps) | ||
| return Ip + (1.0f / 15.0f) * (Ip - I); | ||
|
|
||
| return integrate(a, d, b, fa, fd, fb, I0, .5f * eps, depth - 1) + | ||
| integrate(b, e, c, fb, fe, fc, I1, .5f * eps, depth - 1); | ||
| }; | ||
|
|
||
| float a = x0; | ||
| float b = 0.5f * (x0 + x1); | ||
| float c = x1; | ||
| float fa = f(a); | ||
| float fb = f(b); | ||
| float fc = f(c); | ||
| float I = (c - a) * (1.0f / 6.0f) * (fa + 4.f * fb + fc); | ||
| return integrate(a, b, c, fa, fb, fc, I, eps, depth); | ||
| } | ||
|
|
||
| inline float adaptiveSimpson2D(const std::function<float(float, float)>& f, float32_t2 x0, float32_t2 x1, float eps = 1e-6, int depth = 6) | ||
| { | ||
| const auto integrate = [&](float y) -> float | ||
| { | ||
| return adaptiveSimpson(std::bind(f, std::placeholders::_1, y), x0.x, x1.x, eps, depth); | ||
| }; | ||
| return adaptiveSimpson(integrate, x0.y, x1.y, eps, depth); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replace std::function with a generic template functor, right now you're doing virtual calls on every invocation.
Also it would make sense to move this to Nabla somwhere in https://github.com/Devsh-Graphics-Programming/Nabla/tree/master/include/nbl/builtin/hlsl/math/quadrature
because we have a similar function to compute integrals https://github.com/Devsh-Graphics-Programming/Nabla/blob/3f5b2c33e07d43c6bfd78193a8c75ffa11842ecf/include/nbl/builtin/hlsl/math/quadrature/gauss_legendre/gauss_legendre.hlsl#L22
You can do recursion in HLSL a slightly cursed way -> https://godbolt.org/z/6rvE8rEqW but please do compile-time depth so we don't kill register occupancy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Erfan-Ahmadi look at the new toys you're getting to play with
| double RLGamma(double a, double x) { | ||
| const double epsilon = 0.000000000000001; | ||
| const double big = 4503599627370496.0; | ||
| const double bigInv = 2.22044604925031308085e-16; | ||
| assert(a >= 0 && x >= 0); | ||
|
|
||
| if (x == 0) | ||
| return 0.0f; | ||
|
|
||
| double ax = (a * std::log(x)) - x - std::lgamma(a); | ||
| if (ax < -709.78271289338399) | ||
| return a < x ? 1.0 : 0.0; | ||
|
|
||
| if (x <= 1 || x <= a) | ||
| { | ||
| double r2 = a; | ||
| double c2 = 1; | ||
| double ans2 = 1; | ||
|
|
||
| do { | ||
| r2 = r2 + 1; | ||
| c2 = c2 * x / r2; | ||
| ans2 += c2; | ||
| } while ((c2 / ans2) > epsilon); | ||
|
|
||
| return std::exp(ax) * ans2 / a; | ||
| } | ||
|
|
||
| int c = 0; | ||
| double y = 1 - a; | ||
| double z = x + y + 1; | ||
| double p3 = 1; | ||
| double q3 = x; | ||
| double p2 = x + 1; | ||
| double q2 = z * x; | ||
| double ans = p2 / q2; | ||
| double error; | ||
|
|
||
| do { | ||
| c++; | ||
| y += 1; | ||
| z += 2; | ||
| double yc = y * c; | ||
| double p = (p2 * z) - (p3 * yc); | ||
| double q = (q2 * z) - (q3 * yc); | ||
|
|
||
| if (q != 0) | ||
| { | ||
| double nextans = p / q; | ||
| error = std::abs((ans - nextans) / nextans); | ||
| ans = nextans; | ||
| } | ||
| else | ||
| { | ||
| error = 1; | ||
| } | ||
|
|
||
| p3 = p2; | ||
| p2 = p; | ||
| q3 = q2; | ||
| q2 = q; | ||
|
|
||
| if (std::abs(p) > big) | ||
| { | ||
| p3 *= bigInv; | ||
| p2 *= bigInv; | ||
| q3 *= bigInv; | ||
| q2 *= bigInv; | ||
| } | ||
| } while (error > epsilon); | ||
|
|
||
| return 1.0 - (std::exp(ax) * ans); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is https://en.wikipedia.org/wiki/Incomplete_gamma_function
Which in Boost is in this file https://www.boost.org/doc/libs/latest/libs/math/doc/html/math_toolkit/sf_gamma/igamma.html
C++ doesn't have incomplete gamma functions, but other "special functions" like Bessel, would live in tgmath (cmath, but we always template everything for good measure)
For now you can constrain the implementation to a scalar.
| smart_refctd_ptr<ICPUImage> writeToCPUImage() | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nitpick, instead of burying this function a test, I'd rather have it in main.cpp as a function that does
smart_refctd_ptr<ICPUImage> writeAsCPUImage(const TestChi2<BxDF,aniso>&)| const auto format = E_FORMAT::EF_R32G32B32A32_SFLOAT; | ||
|
|
||
| IImage::SCreationParams imageParams = {}; | ||
| imageParams.flags = static_cast<asset::IImage::E_CREATE_FLAGS>(asset::IImage::ECF_MUTABLE_FORMAT_BIT | asset::IImage::ECF_EXTENDED_USAGE_BIT); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nitpick, you shouldn't care about this when you're not making GPU images from the CPU image via asset converter or multiple views with different formats
| // write sample count from generate, top half | ||
| for (uint64_t j = 0u; j < thetaSplits; ++j) | ||
| for (uint64_t i = 0; i < phiSplits; ++i) | ||
| { | ||
| float32_t3 pixelColor = mapColor(countFreq[j * phiSplits + i], 0.f, maxCountFreq); | ||
| double decodedPixel[4] = { pixelColor[0], pixelColor[1], pixelColor[2], 1 }; | ||
|
|
||
| const uint64_t pixelIndex = j * phiSplits + i; | ||
| asset::encodePixelsRuntime(format, bytePtr + pixelIndex * asset::getTexelOrBlockBytesize(format), decodedPixel); | ||
| } | ||
|
|
||
| // write values of pdf, bottom half | ||
| for (uint64_t j = 0u; j < thetaSplits; ++j) | ||
| for (uint64_t i = 0; i < phiSplits; ++i) | ||
| { | ||
| float32_t3 pixelColor = mapColor(integrateFreq[j * phiSplits + i], 0.f, maxIntFreq); | ||
| double decodedPixel[4] = { pixelColor[0], pixelColor[1], pixelColor[2], 1 }; | ||
|
|
||
| const uint64_t pixelIndex = (thetaSplits + j) * phiSplits + i; | ||
| asset::encodePixelsRuntime(format, bytePtr + pixelIndex * asset::getTexelOrBlockBytesize(format), decodedPixel); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd just write first image into RGB, and second into Alpha.
Since you're paying for RGBA full fp32 eacvh channel, and one can easily turn off certain channels or view just alpha in Renderdoc and GIMP (Can leave this as a comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| float32_t3 mapColor(float v, float vmin, float vmax) | ||
| { | ||
| float32_t3 c = float32_t3(1, 1, 1); | ||
| float diff = vmax - vmin; | ||
| v = clamp<float>(v, vmin, vmax); | ||
|
|
||
| if (v < (vmin + 0.25f * diff)) | ||
| { | ||
| c.r = 0; | ||
| c.g = 4.f * (v - vmin) / diff; | ||
| } | ||
| else if (v < (vmin + 0.5f * diff)) | ||
| { | ||
| c.r = 0; | ||
| c.b = 1.f + 4.f * (vmin + 0.25f * diff - v) / diff; | ||
| } | ||
| else if (v < (vmin + 0.75f * diff)) | ||
| { | ||
| c.r = 4.f * (v - vmin - 0.5f * diff) / diff; | ||
| c.b = 0; | ||
| } | ||
| else | ||
| { | ||
| c.g = 1.f + 4.f * (vmin + 0.75f * diff - v) / diff; | ||
| c.b = 0; | ||
| } | ||
|
|
||
| return c; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd add this to nbl::hlsl::visualization::false_color and keep our Sci-viz gradient there
@AnastaZIuk same thing for your IESes
Some color mappings have particular names in scientific literature https://docs.paraview.org/en/latest/ReferenceManual/colorMapping.html
https://pmc.ncbi.nlm.nih.gov/articles/PMC4959790/
https://www.youtube.com/watch?v=tJ6aEIB61hY & https://www.fabiocrameri.ch/colourmaps/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea we could have nbl::hlsl::visualization:: stuff
mine is blue-magenta, I don't think there is a canonical name for it (cool-like in scientific literature)
| float32_t3 falseColor(float32_t v) |
and almost monotonic in luminance (mapColor does not appear to be https://www.kennethmoreland.com/color-advice/)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It corresponds to jet colormap in matplotlib
https://www.mathworks.com/help/matlab/ref/jet.html
| if (!s.isValid()) | ||
| continue; | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same comment as always about VNDF
66_HLSLBxDFTests/tests.h
Outdated
| if (write_frequencies) | ||
| writeToEXR(assetManager); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we're liklely to drown in temp files, maybe have an option to only write failed tests' frequencies instead of always ?
basically turning write_frequencies into an enum
Also if you want the tests to be thread safe and parallizable, you'd need to somehow document that you expect that all base_t::rc.halfSeed to be unique (with a wide enough seed, good enough rng you shouldn't get collisions in rng output until veeeeeery large test instance counts)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
halfSeed right now is the iteration number. Goes into a PCG32 hash to produce uint32_t2 seed for Xoroshiro rng, which is what is used throughout the tests. Is that enough?
66_HLSLBxDFTests/main.cpp
Outdated
| #include "nlohmann/json.hpp" | ||
|
|
||
| using json = nlohmann::json; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, its ok
| #define GLM_ENABLE_EXPERIMENTAL | ||
| #include <glm/gtx/hash.hpp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no conflict, gtx stuff is opt-in and not included in nabla, as long as we are not planning to use this experimental feature in our public headers it should stay like this right there
if we ever do then we just move
#define GLM_ENABLE_EXPERIMENTAL
#include <glm/gtx/hash.hpp>to nbl/builtin/hlsl/cpp_compat/vector.hlsl
| float32_t3 mapColor(float v, float vmin, float vmax) | ||
| { | ||
| float32_t3 c = float32_t3(1, 1, 1); | ||
| float diff = vmax - vmin; | ||
| v = clamp<float>(v, vmin, vmax); | ||
|
|
||
| if (v < (vmin + 0.25f * diff)) | ||
| { | ||
| c.r = 0; | ||
| c.g = 4.f * (v - vmin) / diff; | ||
| } | ||
| else if (v < (vmin + 0.5f * diff)) | ||
| { | ||
| c.r = 0; | ||
| c.b = 1.f + 4.f * (vmin + 0.25f * diff - v) / diff; | ||
| } | ||
| else if (v < (vmin + 0.75f * diff)) | ||
| { | ||
| c.r = 4.f * (v - vmin - 0.5f * diff) / diff; | ||
| c.b = 0; | ||
| } | ||
| else | ||
| { | ||
| c.g = 1.f + 4.f * (vmin + 0.75f * diff - v) / diff; | ||
| c.b = 0; | ||
| } | ||
|
|
||
| return c; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea we could have nbl::hlsl::visualization:: stuff
mine is blue-magenta, I don't think there is a canonical name for it (cool-like in scientific literature)
| float32_t3 falseColor(float32_t v) |
and almost monotonic in luminance (mapColor does not appear to be https://www.kennethmoreland.com/color-advice/)
66_HLSLBxDFTests/main.cpp
Outdated
| options.preprocessorOptions.includeFinder = compiler->getDefaultIncludeFinder(); | ||
| const IShaderCompiler::SMacroDefinition WorkgroupSizeDefine = { "WORKGROUP_SIZE", WorkgroupSizeAsStr }; | ||
| options.preprocessorOptions.extraDefines = { &WorkgroupSizeDefine,&WorkgroupSizeDefine + 1 }; | ||
| if (!(shader = compiler->compileToSPIRV((const char*)shaderSrc->getContent()->getPointer(), options))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
precompile https://github.com/Devsh-Graphics-Programming/Nabla/blob/master/docs/nsc-prebuilds.md
you are injecting WORKGROUP_SIZE but value you use is constexpr, you can move this one to your example's CMake. Normally you would just pass -D to NSC compile options (optionally to cpp compiler if required) but since we have a bug (I assigned @Przemog1 to fix) temporary you need to create "proxy" input in which you put your define and include your original input
set(OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/auto-gen")
file(MAKE_DIRECTORY "${OUTPUT_DIRECTORY}")
# ..
# read docs/nsc-prebuilds.md
set(WORKGROUP_SIZE 256u) # you might turn into option to toggle in GUI
set(PROXY_TEST_COMPILE_COMP_HLSL "${OUTPUT_DIRECTORY}/proxy_test_compile.comp.hlsl")
set(TEST_COMPILE_VIEW
[=[
#define WORKGROUP_SIZE @WORKGROUP_SIZE@
#include "app_resources/test_compile.comp.hlsl"
]=])
string(CONFIGURE "${TEST_COMPILE_VIEW}" TEST_COMPILE_CONTENT @ONLY)
file(WRITE "${PROXY_TEST_COMPILE_COMP_HLSL}" "${TEST_COMPILE_CONTENT}")
# ..
# remember to add -I "${CMAKE_CURRENT_SOURCE_DIR}" to COMPILE_OPTIONS
# ${PROXY_TEST_COMPILE_COMP_HLSL} is your input to precompile which you load at runtimewhen NSC's bug is fixed you just add -D WORKGROUP_SIZE=${WORKGROUP_SIZE} to COMPILE_OPTIONS and don't write proxy file
66_HLSLBxDFTests/main.cpp
Outdated
| BOOST_PP_REMOVE_PARENS(TEST_TYPE)::run(INIT_PARAMS, cb __VA_OPT__(,) __VA_ARGS__);\ | ||
| }\ | ||
|
|
||
| int main(int argc, char** argv) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you want to create class and inherit from application_templates:: + BuiltinResourcesApplication, check other examples for usage, no int main
AnastaZIuk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left some comments, please address
…hould already be invalid

No description provided.