Dev Diary #3: Interaction between XAML and DirectX 11

Introduction

Foot sliding is a common problem in computer animation. In short, for example, if you create a “walking” animation, it is expected that the feet stick to the ground on every other step as the character moves forward. If this character does not move forward at the same speed as the feet move, it gives the impression that the feet are “sliding” over the floor. The solution of this problem is to adjust the absolute translation speed of the character by adding the relative speed of the sliding foot to the floor. In other words, if the foot slides backwards then increase the speed of the character, or, if the foot slides forward then decrease the speed of the character.

A walking animation is a rather straight forward fix. However, for much more complex animations like a spinning hook kick, then the adjustment is not that easy to define.

Most animation tools come with a way to fix this problem automatically… or at least some good developer has come with a solution and he has been kind enough to share it with the world in the form of a script, plug in or add-on. However, since I’m doing my animations with my very own animation studio, then it seems that I’m out of luck.

No matter, though, for if there is a will then there is a way. So I went ahead and created a custom tool to solve this very special need (after all, that is what I do for a living) by trying a very special type of project in Visual Studio where XAML and DirectX coexist together. I won’t lie, this was not an easy experience: The learning curve was very steep and unfortunately the documentation is very difficult to follow. However, once the tool was done, I was able to grasp the main idea behind this kind of projects, and now I can see the benefits of combining XAML and DirectX.

I’m surprised no one has published anything about this type of project. I’m not talking about marketing brochures or news releases (there are plenty of those): I’m talking about the coding side of the story, expressed in terms that are easy to understand. So I decided to create a blog about it.

XAML and DirectX

For those who are really old school (like me), I need to make a quick parenthesis and define XAML as a markup language created by Microsoft, kind of like the language used to create web pages. With this language, we can create quite impressive User Interfaces and implement them using a wide range of technological platforms to create business oriented applications (msdn – 2017).

With the release of the Windows Store and the concept of Universal Windows Platform Applications, XAML has had a huge momentum. It’s been such a success that Microsoft thought about mixing XAML and DirectX. Ever since Visual Studio 2012 was released, if you want to create a videogame or a multimedia application for the Windows Store using DirectX, you will find that this API comes in two main flavors: With and without XAML.

Now, what I learned from the process of creating this custom tool is that the combination of XAML and DirectX is great for Multimedia Applications, but it is my conclusion that this combination may not be quite optimal when building action games.

Before getting to the reasons behind this conclusion, let me show a video that illustrates the interaction between DirectX and XAML so we can have a graphical reference about the topic.

To understand what just happened, we need to look at the block diagram of the inside of this application, and compare it to the “traditional” approach, which I mean, the one without XAML.

Traditional Block Diagram

When you create a DirectX 11 App (Universal Windows) Project using Visual Studio 2015, the template created (excluding the “Game” class – that one is on me) has the following block diagram:

DirectXClass Diagram

  • The App class is the starting object, and it is in charge of interacting with the Operating System. Windows will load, unload, activate and deactivate the game by accessing this object.
  • The Device Resources class is a wrapper that will help us deal with DirectX API. This frees us from dealing with the most obscure objects from DirectX.
  • The Renderer class is the object that will load all our graphic resources and keep them in buffers, and will wait patiently until it gets the chance to draw them.
  • The Main class is, as it sounds, the main object of the game.
  • All the previous classes come included in the DirectX project template. What I like to do is to create a “Game” class and encapsulate all my routines there. I do that for portability purposes so, if I ever need to change versions of Visual Studio, I just need to copy the game class and beyond, knowing that the main hooks will apply for any other template (this was a life saver when I migrated from Metro style apps in Windows 8 and Visual Studio 2012 to UWP in Windows 10 and Visual Studio 2015).

The dynamic here has some similarity with a typical XNA project:

  • There is an infinite loop in the App class that triggers the “Update” routine, then the “Draw” routine and finally the “Present” routine. This is what most people call the “Game Loop”.
  • The “Update” routine will call the “Update” method of the Game class, putting all components in motion.
  • The “Draw” routine will call the “Draw” method of the Game class, rendering all graphics.
  • However, all input data is handled by the “App” class. What I do is that I populate input information in structure variables that the Game class will interpret during the “Update” call.

XAML and DirectX Block Diagram

When you create a DirectX and XAML App (Universal Windows) Project using Visual Studio 2015, the template created (excluding the “Game” class – that one is also on me) has the following block diagram:

XAML and DirectX Diagram

  • The “App” class is still the starting object, although that’s pretty much all its functionality.
  • The “DirectXPage” class is the actual XAML page. This is where we would implement all our user control elements for input data. This class also takes over of almost all input events.
  • The Device Resources class is still the wrapper that will help us deal with DirectX API, freeing us from dealing with the most obscure objects from DirectX.
  • The Renderer class is still the object that will load all our graphic resources and keep them in buffers, and will wait patiently until it gets the chance to draw them.
  • The Main class is still the main object of the game.
  • All the previous classes come included in the DirectX project template. As in the traditional approach, I also recommend creating a “Game” class and encapsulate all game routines there, for portability purposes.

The biggest, meanest, most complex and most important difference that for some odd reason nobody has ever remarked, is that this new approach is a true multi-thread application.

Here is how things get on fire:

  • The “App” class creates an instance of the “DirectXPage” class, which in turn creates and launches two independent threads, each one with its own infinite loop:
    • The first loop is launched to link input events from pointing devices, like the mouse or a touch screen.
    • The second loop is the actual Game loop, triggering the Update call, the Draw call and the Present call.
  • Aside to all this, the DirectXPage, as a XAML page, has its own thread, usually referred as the “UI Thread”.

To describe in colloquial terms the implied challenge, we can compare the threads of a multi-thread application with the spoiled children of a highly dysfunctional family:

  • Every single one of them hates to share.
  • The communication between them is reduced to the bare minimum.
  • If one of them ever breaks protocol, we get such a drama that will last until the next Thanksgiving dinner.

It gets more volatile if we consider the nature of these threads:

  • No one but the UI Thread can interact with UI elements in an XAML page. That’s almost a cardinal rule.
  • DirectX has some roots from the good ol’ COM API. This means that objects created by one thread can only be accessed by this very same thread. If a different thread touches it, the whole thing crashes – I learned that the hard way.
    • Ok, granted, there is a way to make DirectX be able to handle multiple threads, but the procedure to follow is waaaaaaaaaaay above my level.
  • The input thread is a wee bit more flexible. However, this is the one in charge of communicating input data to the other two drama princesses. It’s still doable, but it is like walking on eggshells.
  • The class hierarchy doesn’t help either:
    • The “DirectXPage” class knows the “Main” and the “Game” classes. However none of these last two classes know the “DirectXPage” class. Adding an include entry in the header file as a reference to the DirectXPage creates a circular reference that generates an error at compiling time. I’m sure there is a way to fix this, but at this point I have absolute no idea how.
    • Even if the circular reference is solved, only the UI Thread can access UI elements, so there is not much benefit here.

The solution I found was to abandon the idea of a direct communication between all components and, instead, I implemented an indirect communication through the use of messages stored in a common repository (a bunch of variables known by everybody).

  • The “Game” class is known by everybody. As such, it becomes the perfect place to store our messages.
  • The “Main” class template comes already with a “Critical Section” object, which is the one to use to regulate access to our shared repository to read and store messages.
    • For those who are not familiar with a multi-thread application approach, this “Critical Section” object is also known as “The Bathroom Key” (Purwar – 2013): In order to use the premises, you need to get a hold of this key. Once you finish your business, you release this key for the next person to use.
    • Likewise, if any thread wants to access the common repository, it needs to get a lock of the “Critical Section” object, and release it once it is done.
    • I assume there is no need to say that in both cases you need to clean the aftermath mess on every use.
  • Following this approach:
    • Every time the UI thread wants to say something to the “Game” or the “Main” class, it needs to get a lock of the Critical Section object to avoid any conflict. I have labeled these messages in the diagram as “BG Events”. In my case, I’m using an enumeration. I know, it doesn’t really fit the formal definition of an “event”, but it is close enough, and it kind of works for our purposes.
    • The main game loop needs to get a hold of the Critical Section object every single time it calls the “Update” and “Draw” methods of the “Game” class, ensure that only one thread ever access objects created using the DirectX API. This gives the “Game” class exclusive access to the common repository.
    • Now, on every “Update” call, the “Game” class checks for any messages (or “BG Events”) that need to be processes. If so, it does what the message requests.
    • If the “Game” class needs to send information to the UI Thread, it leaves a message (what I have labeled as a “UI Event” in the diagram) in the common repository.
    • The UI Thread seldom runs as fast as the main game loop. However, it has an event that is called on every Draw call (called “CompositionTarget::Rendering”). This is the perfect place to get a hold of the Critical Section object and process any message coming from the Main Game loop.
    • Likewise, the Input thread will get a hold of the Critical Section object when it wants to communicate input data to any of the other two threads.

Benefits of using XAML

After reading all this rambling, you may be wondering why should we use this template if the implementation of XAML and DirectX is that complex. The answer is in the benefits provided by XAML:

  • A XAML page grants access to some very useful controls, like Textboxes and the File Open dialogs. This saves A LOT of work.
  • XAML runs on its very own UI Thread. By doing so, we don’t need to worry about a stutter in the UI controls, knowing that the Main Game loop will still run at the desired speed of 60 fps.
  • DirectX has always had a problem displaying text: Processing a long string is a tall order, considering that it is processed and drawn 60 times per second. If XAML is left with the task of displaying text, we reduce the risk of dropping from the desired 60 frames per second.
  • Sometimes, informational data doesn’t need to be updated at the same speed as the game loop. For example, the health bar of the player could be updated just a couple of times per second, making this element a perfect candidate to be displayed by the XAML page.
  • As a game developer, if you want to split the screen in two (one half for each player), a XAML page should be able to accomplish that easily… although I have to confess that I have never done this – no one to play with in a one-man team.

There are some reasons that XAML may not be an important component:

  • Based on the diagram shown previously, the game loop will get exclusive access to the computer resources on every “Update” and “Draw” call, leaving only a very small percent to the other two threads during the “Present” call. This could make the UI thread even slower, especially for hi level processing games.
  • Messages sent between the game loop and the UI threads are at least one frame behind. This could be the difference between winning and losing if the controls have been coded in XAML.

Conclusion

After I created my custom tool, I was able to grasp the “big picture” of mixing both XAML and DirectX. It wasn’t an easy project, and the development strategy described here about messages stored in a common repository may not have been the most optimal approach. I worked before with XAML during the Silverlight days and I have to say that I didn’t find much difference when used for UWP applications.

However, I found controls like the “Open File” and “Save File” dialogs to be essential for Multimedia applications (for me, it was a project saver). Also, I can see the performance benefit that RPG games could get when using text and input controls implemented in an XAML page (for example, sending messages between players using a XAML textbox while still rendering their fancy graphics on the background).

Still, for action games where control response is paramount, I’d still code everything within the main game loop, which makes a XAML page kind of an optional component. Then again, that is my personal point of view.

References

  1. “What is XAML?” Microsoft Development Network. 2017. Web. https://msdn.microsoft.com/en-us/library/cc295302.aspx
  2. “Multithread: How to Use the Synchronization Classes” Microsoft Development Network. 2017. Web. https://msdn.microsoft.com/en-us/library/thdxkfx9.aspx
  3. “Understanding Thread’s Synchronization Objects with a Real Life Example”, Priyank Purwar, 23 Jun 2013. Code Project. Web. https://www.codeproject.com/Articles/610566/Understanding-Thread-s-Synchronization-Objects-wit
Advertisements