Introducing OpenCL™ and OpenGL® on DirectX
For the last few months, we have been working on two exciting new projects at Collabora, and it’s finally time to share some information about them with the world:
We are partnering with Microsoft DirectX engineers to build OpenCL and OpenGL mapping layers, in order to bring OpenCL 1.2 and OpenGL 3.3 support to all Windows and DirectX 12 enabled devices out there!
This work builds on a lot of previous work. First and foremost, we are building this by using Mesa 3D, with the Gallium interface as the base for the OpenGL layer, and NIR as the base for the OpenCL compiler. We are also using LLVM and the SPIRV-LLVM-Translator from Khronos as the compiler front-end.
In addition, we are taking advantage of Microsoft’s experience in creating their D3D12 Translation Layer, as well as our own experience from developing Zink.
What is Mesa 3D?
Mesa 3D is an open source implementation of several graphics technologies, including OpenCL and OpenGL. The OpenGL implementation in Mesa is robust and is used as the base for several industry-strength OpenGL drivers from multiple GPU vendors.
Among other things, Mesa consists of several API implementations (called state-trackers) as well as the Gallium low-level driver interface. The Gallium interface hides a lot of the legacy OpenGL details and translates OpenGL calls into something that looks more like modern GPU primitives.
Why translate APIs?
Not all Windows-powered devices have consistent support for hardware-accelerated OpenCL and OpenGL. So in order to improve application compatibility, we are building a generic solution to the problem. This means that a GPU vendor only has to implement a D3D12 driver for their hardware in order to support all three APIs.
This mapping layer is also expected to serve as a starting point in porting older OpenCL and OpenGL applications over to D3D12.
In addition, we believe this is good for the wider open source community. A lot of the problems we are solving here are shared with other drivers and translation layers, and we hope that the code will be useful beyond the use cases listed above.
Implementation
The work is largely split into three parts: an OpenCL compiler, an OpenCL runtime, and a Gallium driver that builds and executes command-buffers on the GPU using the D3D12 API.
In addition, there is a shared NIR-to-DXIL shader compiler that both components use. For those not familiar with NIR, it is Mesa’s internal representation for GPU shaders. Similarly, DXIL is Microsoft’s internal representation, which D3D12 drivers will consume and translate into hardware-specific shaders.
OpenCL compiler
The OpenCL compiler uses LLVM and the SPIRV-LLVM-Translator to generate SPIR-V representations of OpenCL kernels. These, in turn, are passed to Mesa’s SPIR-V to NIR translator, where some optimizations and semantical translations are done. Then the NIR representation is finally passed to NIR-to-DXIL, which produces a DXIL compute shader and the needed metadata so it can be executed on the GPU by the runtime using D3D12.
Here’s a diagram of the complete process, including NIR-to-DXIL, which will be described below:
OpenCL runtime
While Mesa provides an OpenCL implementation called Clover, we are not using it for this project. Instead, we have a new OpenCL runtime that does a more direct translation to the DirectX 12 API.
NIR-to-DXIL
DXIL is essentially LLVM 3.7 bitcode with some extra metadata and validation. This was a technical choice that made sense for Microsoft because all the major driver vendors already used LLVM in their compiler toolchain. Using an older version of the LLVM bitcode format gives good compatibility with drivers because the LLVM bitcode format is backwards compatible.
Because we depend on a much more recent version of LLVM for the compiler front-end, we sadly cannot easily use the DirectX Shader Compiler as a compiler back-end. The DirectX Shader Compiler is effectively a fork of LLVM 3.7, and we are currently using LLVM 10.0 for the compiler front-end. Using DirectX Shader Compiler as that would require us to link two different versions of LLVM into the same binary, which would have led to problems.
We also cannot easily use LLVM itself to generate the bitcode. While the LLVM bitcode format is backwards compatible, LLVM itself is not forward compatible. This means that newer versions of LLVM cannot produce a bitcode format that is understood by older versions. This makes sense from LLVM’s point of view because it was never meant as a general interchange format.
So instead, we have decided to implement our own DXIL emitter. This is quite a bit harder than it looks because LLVM bitcode goes to great lengths to try to make the format as dense as possible. For instance, LLVM does not store its bitcode as a sequence of bytes and words, but rather as variable-width bitfields in a long sequence of bits.
There are a lot of tricky details to get right, but in the end we have a compiler that works.
D3D12 Gallium driver
The D3D12 Gallium driver is the last piece of the puzzle. Essentially, it takes OpenGL commands and, with the help of the NIR to DXIL translator, turns them into D3D12 command-buffers, which it executes on the GPU using the D3D12 driver.
There are a lot of interesting details that makes this tricky as well, but I will save those details for later.
But to not leave you empty-handed, here’s a screenshot of the Windows version of the famous glxgears, wglgears:
Source code
In the short term, the source code can be found here. We intend on upstreaming this work into the main Mesa repository shortly, so it is not a permanent home.
Next steps
This is just the announcement, and a whole lot of work is left to be done. We have something that works in some cases right now, but we are just starting to scratch the surface.
First of all, we need to get up to the feature-level that we target. Our goals at the moment is to pass conformance tests for OpenCL 1.2 and OpenGL 3.3. We have a long way to go, but with some hard work and sweat, I am sure we will get there.
Secondly, we need to work on application compatibility. For now we will be focusing on productivity applications.
We also want to upstream this in Mesa. This way we can keep up with fixes and new features in Mesa, and other drivers can benefit from what we are doing as well.
Acknowledgments
It is also important to point out that I am not the only one working on this. Our team consists of five additional Collabora engineers (Boris Brezillon, Daniel Stone, Elie Tournier, Gert Wollny, Louis-Francis Ratté-Boulianne) and two Microsoft DirectX engineers (Bill Kristiansen, Jesse Natalie).