How do I get Visual Studio 2017 / msbuild to use the latest DirectX shader compiler?

Question

I'm trying to compile a C# game project which utilizes HLSL shaders. These shaders are part of the .csproj project, and therefore Visual Studio is supposed to compile them along with the code. The shader properties should be correct - shaders are of the right type, the shaders use recent enough shader model (5.1) etc. Despite that, Visual Studio doesn't build all of the shaders, and instead outputs two unhelpful errors:

error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #2 of opcode #107 (counts are 1-based).
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Can't continue validation - aborting.

My first guess was that the shaders must be faulty in some way. I opened the VS2017 command prompt and as expected, manually running fxc produced the same errors for certain pixel shaders. However, I then decided to check if there are other versions of fxc installed:

C:\koodia\OpenSAGE\src\OpenSage.DataViewer.Windows\Shaders>where fxc
C:\Program Files (x86)\Windows Kits\10\bin\x86\fxc.exe
C:\Program Files (x86)\Windows Kits\10\bin\10.0.15063.0\x86\fxc.exe

The latter one is from a more recent Windows SDK - fxc /? reports the version as Direct3D Shader Compiler 10.1, while the first one is Direct3D Shader Compiler 10.0.10011.16384. I verified with a Powershell script that all of the shaders compile succesfully with the newer compiler. Then I compiled the project from command line with msbuild, which confirmed that msbuild (and therefore Visual Studio) is invoking the older fxc:

FxCompile:
  C:\Program Files (x86)\Windows Kits\10\bin\x86\Fxc.exe /nologo /Emain /T ps_5_1 /Fo obj\Debug\Shaders\MeshPS.cso /Zi /enable_unbounded_descriptor_tables Shade
  rs\MeshPS.hlsl
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Invalid operand type for operand #2 of opcode #107 (counts are 1-based). [C:\koodia\OpenSAGE\src
\OpenSage.DataViewer.Windows\OpenZH.DataViewer.Windows.csproj]
error X8000 : D3D11 Internal Compiler error : Invalid Bytecode: Can't continue validation - aborting. [C:\koodia\OpenSAGE\src\OpenSage.DataViewer.Windows\OpenZH
.DataViewer.Windows.csproj]

Finally, to the question itself: How do I make sure msbuild uses the shader compiler from the latest installed Windows SDK?

I have installed the latest Windows SDK, the latest Windows updates and I've upgraded to Visual Studio 15.3.5, with no effect. I couldn't find anything online directly related to this issue. Visual Studio doesn't seem to have any settings related to Windows SDK or DirectX versions.

What is the file version of bin\x86\fxc.exe? Maybe you still have some older SDK installed (e.g. 10.0.10240) and it serves as a base one or your project targets older SDK? — user7860670
@VTT fxc.exe is 10.0.10240.16384. The newer one is 10.0.15063.468. — paavohtl
@VTT I uninstalled both versions of the SDK, and reinstalled the latest one. Msbuild now uses an even older version of fxc, from the Windows 8 SDK (C:\Program Files (x86)\Windows Kits\8.0\bin\x86\Fxc.exe). — paavohtl
Did you retarget your project? Msbuild should figure out correct SDK paths from project settings. — user7860670
As mentioned in the original question, this is a C# project. VS doesn't seem to support (re)targetting for them. This is the context menu for the project: i.imgur.com/4Wwnp70.png — paavohtl

paavohtl paavohtl · Accepted Answer · 2017-10-08T19:57:30

I managed to temporarily work around this issue with a dumb trick. I replaced fxc.exe and d3dcompiler_47.dll in C:\Program Files (x86)\Windows Kits\8.1\bin\x86 with the ones from C:\Program Files (x86)\Windows Kits\10\bin\10.0.15063.0\x86 (and backed up the originals).

It's seldom a good idea to hand-modify Windows SDKs, but in my use case it's likely that it's not going to break anything, as FXC should be backwards compatible.

I'm still looking for a better solution, or at least a confirmation whether my Windows + VS installation is somehow messed up, or if this is an actual bug / missing feature in msbuild.

How do I get Visual Studio 2017 / msbuild to use the latest DirectX shader compiler?

1 Answers