WL

DDS Compression Format Code Issue

Share

ScreenCapture.cpp

715 lines

txt

ScreenCapture.h

80 lines

txt

WL

I am getting some strange results from some of the paramiters sent to the screen capture loop. In Particular the Code line:case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM appears to be unreachable.

Edit

Looking at your code, the issue is that in the GetDXGIFormat function, the case DDSCompression::RGBA appears unreachable because there's a default case before it that returns the same value (DXGI_FORMAT_BC7_UNORM).

Here's the problematic section in your ScreenCapture.cpp:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM; default: return DXGI_FORMAT_BC7_UNORM; } }

The problem is that in C++, once a return statement is executed in any case, the function exits. However, the compiler might be warning you that the DDSCompression::RGBA case is unreachable if there's some other issue.

Looking more closely, I suspect the issue might be with your enum definition. In your header file, you have:

cpp

enum class DDSCompression { BC1, BC2, BC3, BC4, BC5, BC6H, BC7, RGBA };

This looks correct. However, there could be a few potential issues:

  1. Compiler Warning: Some compilers might warn about unreachable code after the default case, even though it's actually reachable.
  2. Enum Value Collision: Make sure there are no duplicate values in your enum.

Here's the corrected version that should resolve any potential issues:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM; default: return DXGI_FORMAT_BC7_UNORM; } }

If you're still getting warnings about unreachable code, you could also reorganize it like this:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM; } // Default case outside switch return DXGI_FORMAT_BC7_UNORM; }

To debug this further, you could add some logging to see what values are actually being passed to the function:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { logger::debug("GetDXGIFormat called with compression: {}", static_cast<int>(compression)); switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: logger::debug("Using RGBA format"); return DXGI_FORMAT_R8G8B8A8_UNORM; default: logger::warn("Unknown DDS compression type, defaulting to BC7"); return DXGI_FORMAT_BC7_UNORM; } }

This will help you determine if the RGBA case is actually being reached during execution.

Retry

WL

The problem comes from a mismatch between These enumerations and the coded vallues sent from the calling Papyrus routine. In Papyrus I defined these values as a series of strings UnCOMPRESSED, BC1, BC2, BC3, BC4, BC5, BC7_Fast and BC7.

Edit

Ah, I see the issue! There's a mismatch between your Papyrus string values and your C++ enum values. Your Papyrus is sending string values like "UNCOMPRESSED", "BC7_Fast", etc., but your C++ StringToDDSCompression function isn't handling all of these cases properly.

Looking at your StringToDDSCompression function:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc6h") return DDSCompression::BC6H; if (lower == "bc7") return DDSCompression::BC7; if (lower == "rgba" || lower == "uncompressed") return DDSCompression::RGBA; return DDSCompression::BC7; // Default }

You need to update this function to handle the Papyrus string values. Here's the corrected version:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc6h") return DDSCompression::BC6H; if (lower == "bc7" || lower == "bc7_fast") return DDSCompression::BC7; if (lower == "rgba" || lower == "uncompressed") return DDSCompression::RGBA; return DDSCompression::BC7; // Default }

However, I notice you don't have a BC6H option in your Papyrus list. You might want to either:

  1. Update your Papyrus enum to match your C++ enum:

    UNCOMPRESSED, BC1, BC2, BC3, BC4, BC5, BC6H, BC7_Fast, BC7

  2. Or update your C++ enum to match your Papyrus values:

    cpp

    enum class DDSCompression { UNCOMPRESSED, // or RGBA BC1, BC2, BC3, BC4, BC5, BC7_FAST, BC7 };

For the second option, you'd also need to update the GetDXGIFormat function:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC7_FAST: case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::UNCOMPRESSED: return DXGI_FORMAT_R8G8B8A8_UNORM; default: return DXGI_FORMAT_BC7_UNORM; } }

And the string conversion function:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc7_fast") return DDSCompression::BC7_FAST; if (lower == "bc7") return DDSCompression::BC7; if (lower == "uncompressed") return DDSCompression::UNCOMPRESSED; return DDSCompression::BC7; // Default }

The DDSCompression::RGBA case was unreachable because Papyrus was never sending "RGBA" or "uncompressed" - it was sending "UNCOMPRESSED" (which your function converts to lowercase "uncompressed", so that part should work).

To debug this, you could add some logging to see what strings are actually being received:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { logger::debug("Converting DDS compression string: '{}'", str); std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); logger::debug("Lowercase version: '{}'", lower); // ... rest of your function }

This will help you verify exactly what strings Papyrus is sending to your C++ code.

Retry

WL

I am also concerened with the default value of BC7 since this is the slowest possible mode of operation. I would prefer BC1. So all I need to to fix this is change the BC7 to BC1. This occurs in the GETDXGIFormat function as well as StringTODDSCompression .

Edit

You're absolutely right! BC7 is indeed much slower than BC1, so changing the default to BC1 makes perfect sense for better performance.

You need to change the default return value in two places:

  1. In StringToDDSCompression function:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc6h") return DDSCompression::BC6H; if (lower == "bc7" || lower == "bc7_fast") return DDSCompression::BC7; if (lower == "rgba" || lower == "uncompressed") return DDSCompression::RGBA; return DDSCompression::BC1; // Changed from BC7 to BC1 }

  1. In GetDXGIFormat function:

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM; default: return DXGI_FORMAT_BC1_UNORM; // Changed from BC7 to BC1 } }

You might also want to update the default value in your CaptureParams structure in the header file:

cpp

struct CaptureParams { std::wstring basePath; ImageFormat format = ImageFormat::PNG; float jpegQuality = 85.0f; float gifDuration = 3.0f; TiffCompression tiffMode = TiffCompression::None; DDSCompression ddsMode = DDSCompression::BC1; // Changed from BC7 to BC1 std::atomic<bool>* cancelFlag = nullptr; };

This will ensure that BC1 (the fastest compression) is used as the fallback whenever an unrecognized string is passed or when the default constructor is used.

Retry

WL

I also note that there does not appear to be a BC&_Fast DDS format as the screencapture returns this param as simply BC7 I probably defined this in error.

Edit

WL

Still there?

Edit

Yes, I'm still here! You're absolutely correct about BC7_Fast - there isn't actually a separate "BC7_Fast" DDS format. BC7 is just BC7, and the "fast" vs "slow" distinction typically refers to different compression quality/speed settings during the compression process, not different formats.

In DirectX, BC7 compression can be done with different quality levels (affecting compression speed), but the resulting format is still DXGI_FORMAT_BC7_UNORM. So your Papyrus enum probably shouldn't have both "BC7_Fast" and "BC7" as separate options.

You should simplify your Papyrus enum to just:

UNCOMPRESSED, BC1, BC2, BC3, BC4, BC5, BC6H, BC7

And then update your StringToDDSCompression function to remove the BC7_Fast handling:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc6h") return DDSCompression::BC6H; if (lower == "bc7") return DDSCompression::BC7; // Removed bc7_fast if (lower == "rgba" || lower == "uncompressed") return DDSCompression::RGBA; return DDSCompression::BC1; // Default }

If you want to control compression speed/quality for BC7, you'd typically do that through the compression flags passed to DirectXTex's Compress function, not through different format enums.

Retry

WL

Can the program be modified in such a way that addional enums are handled so that they are parsed to provide compression control? say BC7 is uncompressed, BC7_fast is BC7 with a specified compression level or method? Whaat specific levels if compression are availiable?

Edit

Yes, absolutely! DirectXTex provides several compression flags that control the speed vs quality tradeoff for BC7 compression. You can modify the program to handle different compression levels.

Here's how you can implement this:

1. First, update your enum to include compression variants:

cpp

enum class DDSCompression { BC1, BC2, BC3, BC4, BC5, BC6H, BC7_QUICK, // Fastest compression BC7_FAST, // Fast compression BC7_NORMAL, // Default compression BC7_SLOW, // High quality compression RGBA };

2. Update your string parsing function:

cpp

DDSCompression StringToDDSCompression(const std::string& str) { std::string lower = str; std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower); if (lower == "bc1") return DDSCompression::BC1; if (lower == "bc2") return DDSCompression::BC2; if (lower == "bc3") return DDSCompression::BC3; if (lower == "bc4") return DDSCompression::BC4; if (lower == "bc5") return DDSCompression::BC5; if (lower == "bc6h") return DDSCompression::BC6H; if (lower == "bc7_quick") return DDSCompression::BC7_QUICK; if (lower == "bc7_fast") return DDSCompression::BC7_FAST; if (lower == "bc7_normal" || lower == "bc7") return DDSCompression::BC7_NORMAL; if (lower == "bc7_slow") return DDSCompression::BC7_SLOW; if (lower == "rgba" || lower == "uncompressed") return DDSCompression::RGBA; return DDSCompression::BC1; // Default }

3. Update GetDXGIFormat (all BC7 variants use the same format):

cpp

DXGI_FORMAT GetDXGIFormat(DDSCompression compression) { switch (compression) { case DDSCompression::BC1: return DXGI_FORMAT_BC1_UNORM; case DDSCompression::BC2: return DXGI_FORMAT_BC2_UNORM; case DDSCompression::BC3: return DXGI_FORMAT_BC3_UNORM; case DDSCompression::BC4: return DXGI_FORMAT_BC4_UNORM; case DDSCompression::BC5: return DXGI_FORMAT_BC5_UNORM; case DDSCompression::BC6H: return DXGI_FORMAT_BC6H_UF16; case DDSCompression::BC7_QUICK: case DDSCompression::BC7_FAST: case DDSCompression::BC7_NORMAL: case DDSCompression::BC7_SLOW: return DXGI_FORMAT_BC7_UNORM; case DDSCompression::RGBA: return DXGI_FORMAT_R8G8B8A8_UNORM; default: return DXGI_FORMAT_BC1_UNORM; } }

4. Create a new function to get compression flags:

cpp

DWORD GetCompressionFlags(DDSCompression compression) { switch (compression) { case DDSCompression::BC7_QUICK: return DirectX::TEX_COMPRESS_BC7_QUICK; case DDSCompression::BC7_FAST: return DirectX::TEX_COMPRESS_PARALLEL | DirectX::TEX_COMPRESS_BC7_USE_3SUBSETS; case DDSCompression::BC7_NORMAL: return DirectX::TEX_COMPRESS_PARALLEL; case DDSCompression::BC7_SLOW: return DirectX::TEX_COMPRESS_PARALLEL | DirectX::TEX_COMPRESS_BC7_USE_3SUBSETS | DirectX::TEX_COMPRESS_UNIFORM; default: return DirectX::TEX_COMPRESS_DEFAULT; } }

5. Update your compression code in CaptureScreen:

cpp

// In the DDS saving section, replace the compression call: if (DirectX::IsCompressed(targetFormat)) { DWORD compressionFlags = GetCompressionFlags(params.ddsMode); hr = DirectX::Compress( convertedImage.GetImages(), convertedImage.GetImageCount(), convertedImage.GetMetadata(), targetFormat, compressionFlags, // Use custom flags instead of TEX_COMPRESS_DEFAULT DirectX::TEX_THRESHOLD_DEFAULT, compressedImage ); // ... rest of the code }

6. Add the function declaration to your header file:

cpp

// Add this to ScreenCapture.h DWORD GetCompressionFlags(DDSCompression compression);

Available DirectX compression flags include:

  • TEX_COMPRESS_DEFAULT - Default settings
  • TEX_COMPRESS_BC7_QUICK - Fastest BC7 compression (lower quality)
  • TEX_COMPRESS_BC7_USE_3SUBSETS - Use 3 subsets for better quality (slower)
  • TEX_COMPRESS_PARALLEL - Use multiple threads
  • TEX_COMPRESS_UNIFORM - Uniform color weighting (slower but better quality)
  • TEX_COMPRESS_SRGB_IN - Input is sRGB
  • TEX_COMPRESS_SRGB_OUT - Output is sRGB

Recommended Papyrus enum values:

UNCOMPRESSED, BC1, BC2, BC3, BC4, BC5, BC6H, BC7_QUICK, BC7_FAST, BC7_NORMAL, BC7_SLOW

This gives you a good range from fastest (BC7_QUICK) to highest quality (BC7_SLOW), with the compression time and quality scaling accordingly.

Retry

WL

Is TEX_COMPRES_PARALLEL of any utility or does it overly complicate the code for little gain.

Edit

TEX_COMPRESS_PARALLEL can provide significant performance benefits, especially for BC7 compression which is computationally expensive. However, whether it's worth the complexity depends on your use case.

Benefits of TEX_COMPRESS_PARALLEL:

  • Can reduce compression time by 2-4x on multi-core systems
  • Most beneficial for BC7, BC6H, and BC5 (the slower compression formats)
  • Little to no benefit for faster formats like BC1-BC3
  • Screenshot capture is typically a one-time operation where speed matters

Potential complications:

  • Adds thread overhead for very small images
  • Could theoretically cause issues if your application has strict threading requirements
  • Minimal code complexity (just an additional flag)

My recommendation: Keep it for BC7 variants since the code complexity is minimal and the performance gain is substantial. Here's a simplified approach:

cpp

DWORD GetCompressionFlags(DDSCompression compression) { switch (compression) { case DDSCompression::BC7_QUICK: return DirectX::TEX_COMPRESS_PARALLEL | DirectX::TEX_COMPRESS_BC7_QUICK; case DDSCompression::BC7_FAST: return DirectX::TEX_COMPRESS_PARALLEL; case DDSCompression::BC7_NORMAL: return DirectX::TEX_COMPRESS_PARALLEL; case DDSCompression::BC7_SLOW: return DirectX::TEX_COMPRESS_PARALLEL | DirectX::TEX_COMPRESS_BC7_USE_3SUBSETS; default: return DirectX::TEX_COMPRESS_DEFAULT; // No parallel for other formats } }

If you want to keep it even simpler, you could just use parallel for all BC7 variants and skip the complexity:

cpp

DWORD GetCompressionFlags(DDSCompression compression) { switch (compression) { case DDSCompression::BC7_QUICK: return DirectX::TEX_COMPRESS_PARALLEL | DirectX::TEX_COMPRESS_BC7_QUICK; case DDSCompression::BC7_FAST: case DDSCompression::BC7_NORMAL: case DDSCompression::BC7_SLOW: return DirectX::TEX_COMPRESS_PARALLEL; default: return DirectX::TEX_COMPRESS_DEFAULT; } }

The performance gain usually outweighs the minimal complexity, especially for screenshot capture where users expect fast operation. But if you prefer to keep things simple, you can always start without it and add it later if compression speed becomes an issue.

Retry

Claude can make mistakes. Please double-check responses.

 

Research

Sonnet 4