Monthly Archives: November 2017

Dev Diary Entry #6: Emergency Upgrade

Introduction

I’ve been interacting with many fellow Indie developers through on-line media regarding games running as a UWP App on the Xbox One. Mostly, I just bought my Xbox One a month ago (Black Friday in Canada is in October) and I wanted to “catch up” regarding game development for this console. The most outstanding issue I noticed in most posts I read was a performance concern.

As developers, we all hate surprises during the last phase of a project. Aiming to diffuse the pressure of the last weeks, I went ahead and started doing performance tests. What I found was rather upsetting: the performance of my game engine as a UWP app on the Xbox One was awful: it could only run fluently at 30 fps. For a state-of-the-art device, that was a rather alarming find.

After a thorough analysis (and a lot of cursing), I found out that the source of my problem was the console’s GPU restrictions and the outdated development environment I was using.

Part of my stubbornness about keeping the tools I’m familiar with is due to the fact that I’m one of the few indie developers that use (and praise) DirectX 11. I know of other cases, but all of them are very, very, very quiet about what they do and how they do it. As a result, when I have a critical problem, I’m pretty much on my own.

However, when I chatted with other fellow developers about my findings, Shahed Chowdhuri, a friend and technical evangelist, pointed out that, although old versions of the Windows 10 SDK restricted access to the GPU by half, the latest SDK, known as the “Creator Fall Update”, granted full access to the GPU. As plain as it sounds, this was the solution to my performance problem. However, since the latest SDK runs only on the latest version of Windows 10 and can be used only by the latest version of Visual Studio, this meant that I had to update absolute everything, from my console and PC to the core of my home made game engine.

Here is the article that was pointed out:

https://blogs.windows.com/buildingapps/2017/09/15/resources-universal-windows-platform-games-fall-xbox-one-update/#pEZORfPbzlQU9ft7.97

Upgrading the Development Environment

I was lucky enough to be able to reserve a second development environment with Windows 10. The roadblock here was that the Windows 10 Fall Creators Upgrade is deployed by regions, and my system hadn’t been selected yet. However, I was able to get the upgrade at the following link:

https://support.microsoft.com/en-us/help/4028685/windows-10-get-the-fall-creators-update

I downloaded a fresh copy of Visual Studio 2017, which included the latest SDK. Here is the link I used:

https://www.visualstudio.com/downloads/

One day, without planning, my Xbox One S console updated itself to version October 2017. That was an easy upgrade.

Upgrading my Game to Visual Studio 2017

The next step was to upgrade my games to the new development environment. Here is where this procedure got painful: Although I was able to upgrade my game from Visual Studio 2015 to Visual Studio 2017, I was unable to re-target the target and minimum Windows 10 SDK version to the Fall Creators SDK: Simply put, the option for that version was not available in the drop down list.

I was kind of expecting a surprise like that. No matter, though, for I had a backup plan: Since all my code was encapsulated XNA style, I created a brand new project targeting the latest Windows 10 SDK and copied my code from the old project to the new one. That was what I did when I migrated from Visual Studio 2013 to 2015, so I was very familiar with the process.

I was happy to see that all the changes and enhancements in the new Visual Studio 2017’s “DirectX 11 App (Universal Windows)” Application Template were encapsulated and properly implemented. For instance, all the DirectX managers and handler classes are new and improved, but encapsulated in the DeviceResources wrapper class (see my previous post about the inside of a DirectX application). This made the upgrade so easy to implement.

Once this process was finished, I ran my tests one more time and I noticed a considerable improvement in the Xbox One console: I was now able to run my games at the desired frame rate of 60 fps, although it still stuttered frequently. The improvement was good, but not good enough to fulfill the expectation of gamers around the world.

Shader Compilation

After running the Graphic Debugger, I identified the bottleneck of my game engine as a particular pixel shader that draws in batch the background elements of the game (part of the DrawIndexedInstanced function implementation). For example, this is the one in charge of drawing the buildings in my third-person shooter. I re-wrote this shader so the code would be even more efficient, but I still couldn’t reach the desired performance. I was looking at some branching warning notes when I noticed something weird in the Visual Studio environment: By default, projects in “Debug” mode are compiled in such a way that all shader optimization options are disabled. Also, some debug functions are added to the shader. Likewise, projects in “Release” mode are compiled in such a way that all shaders are compiled with the optimization features enabled and all debug features are excluded from this code. In other words, projects need to be compiled in “Release” mode prior deploying it to the console for good performance.

I was about to test this theory when my Xbox One suddenly upgraded itself to Windows 10 NOVEMBER 2017 version. These upgrades are not exactly optional if we want to continue on-line, so I went ahead and upgraded my console.

Once I got all these changes done, I can say that my home made game engine now flies on the Xbox One S: It runs smoothly at 60 fps, it doesn’t stutter and when it does is because of my code, not the GPU. I have the capture video features on and I can record without jeopardizing performance.

Conclusion

For a desired 60fps game experience, we only have 16 milliseconds to draw a frame. Having half of that was a huge challenge for amateur developers like me. However, the Fall Creators Upgrade granted us full access to the GPU, and that is a huge improvement. Still, to be fair and objective, I have to be accountable for the lame HLSL code I wrote, along with the compilation configuration settings of the deployed executable.

Performance drop is a very complex issue and takes a lot of man power to solve. Most of the time, you don’t know you have a performance issue until you see your game stutter. I cannot say which upgrade / optimization (or combination thereof) fixed my game engine, but I’m really happy that it did. Still, this dev log entry has the steps I took documented, hoping that it will help other Indies with their projects.

Dev Diary Entry #5: Afterthoughts about Project Strategy

Introduction

It’s mid November, we are heading towards the final weeks before the due date and, despite my greatest efforts, I am once again behind schedule. Things are moving forward, though, and, as a token, I’ve created a work-in-progress video, captured straight from my brand new Xbox One S.

The small detail that amazes me regarding Microsoft’s UWP technology is that, in this particular case, the very same executable works for both PCs and Xbox consoles – the Xbox control can be used in both cases, although gamers will most likely use the keyboard when it comes to PC. For my little tablet, though, I’m afraid I still need to recompile it for a 32 bit processor, but that will do the trick regarding touch screen control.

Once shown graphic proof of how the game is going to look, I can finally talk about the technology and tools used without raising serious credibility doubts. However, I don’t want to write a sad blog entry overly justifying the use of tools that have become so unpopular on this day and age, using arguments that nobody is going to believe anyway – that won’t give any added value to my readers, and I already have enough hate mail to deal with. Instead, let me address that same topic from a different point of view by talking about Project Strategy, using “Antimatter Instance” as an example.

What I mean by “Project Strategy” is the list of steps taken to identify the development path to follow in order to deliver a given project. Following a structured path will ensure that all efforts made are done congruently, efficiently and towards a common objective, scheduled within the given time frame. Sure, we can always “wing it”, but that usually leads to an unproductive and expensive learning experience rather than a successful implementation – no doubt it is fun, but most of the time is fruitless.

Note that these steps are based on my personal experience as an “Indie” rather than a serious Project Management research.

Step 1: Identify what you want to accomplish

Every single project I’ve work with, as an Indie and as an IT Professional, starts by identifying what exactly is what we want to accomplish. A work-related project focuses on fulfilling a need and making the life of our users as easy and simple as possible. A videogame, however, focuses on the “dream game” that we have always wanted to create.

No doubt this step is the best phase of project planning, as usually we have long, frequent brainstorms of the videogame we want to create. We get inspired from other games, we get ideas of our own, we read on the Internet what can be accomplished on this day and age, we put all of this together and define what we want to accomplish. The more we define it, the more we want to put everything aside and start working on it. It is this vision (along with unlimited supply of caffeine) that will fuel all the long nights ahead.

In my case, I always wanted to create a 3D fighter / action game of the likes of the blockbuster “God of War”, although with much less gore, no racy scenes and a wee bit more cheerful story. I even had the perfect name (which I’m keeping close to my vest for now), as well as the main characters design. There were five particular scenes that kept me inspired, and I even wrote down the dialogues of each one so I could remember them as detailed as possible.

Step 2: Identify the resources and skills at hand

Once we have defined what we want to accomplish, then we identify what resources and skills we have that can be of use. Each of us is unique and our “skill set” as independent videogame developers vary from case to case. This step also includes defining the budget we currently count on, as well as a timeline, if any.

In my particular case, I am a software developer /.NET programmer. I have some experience as a web developer and, about 20 years ago, I used to provide technical support to the image centre of a famous newspaper. These two last entries in my resume gave me enough knowledge to create and handle digital images.

Here is where things get interesting: About 10 years ago a huge project for a Silverlight solution presented itself, and my day job boss asked me to learn and master C#. As the obedient employee I’ve always been, I downloaded XNA and published three games (not on her time and dime, though I have to say that the Silverlight application was a success). To do so, I created a game engine that used differential frame animations for 3D models. It is like a script language that can be extrapolated to generate in-between frames (e.g. change the animation speed to anything and still be a 60 frames per second animation clip) and allows me to add (e.g. play two animation clips on the same model at the same time) and subtract frames (e.g. inserting a set of frames right in the middle of an animation clip). As any language script, the best tool to create these animation clips is Notepad. Eventually, I created a set of in-house tools to aid in the creation and editing of animation sequences. I call these my “in-house animation studio tools”.

Later on, when Windows 8 was launched, I migrating this engine of mine to C++ as a Metro-style application by copying and pasting code… although I did have to translate some classes like the “Generic.List” to native arrays (commonly known as “vectors”), as well as the “.” notation To “->” for object references. Also, I had to implement the “DrawIndexedInstanced” function in order to substitute XNA’s SpriteBatch class with something that would run on DirectX 11.

About the budget available for the Dream Build Play challenge, well, there is none. I’m actually running on a shoestring so there is no way I can incur in an unplanned expense.

The timeline is defined by the Dream Build Play 2017 Challenge. Originally, we were given 6 months to create an entry. Granted, a finished product is not required, and even a running prototype is acceptable. Still, this is a contest, and 6 months is a tight timeline to create anything from scratch.

Step 3: Identifying what we require to reach our goal

This is the discouraging part of Project Planning: This is where we identify everything that we don’t have that we will need to create our dream game. Unless we know exactly what we are doing, we realize more often than not that, unfortunately, we are not the right element for our very own game. No matter, though, for where there is a will then there is a way. Here are some alternative paths to acquire what we are missing:

  • Purchase: Game assets that we cannot create in-house can be purchased in an asset store.
  • Outsource: certain tasks that cannot be done in-house can be outsourced to an external team. This is a very dangerous option since most of the external teams bill by the hour, which is an approach that more often than not drains our fiscal resources providing very few results for our buck.
  • Learn: If the budget is limited and the project’s timeline is somewhat flexible, we can purchase a book or take an on-line course to acquire / hone the skills that we require to create our dream game.
  • Team-up with another indie: When a single project is too big for one single person team then it is best to create a team of people with skill sets that complement each other. If all members have the same skills then all team meetings can be described as a match about who’s right or wrong, and the whole project is reduced to following the instructions of the alpha member. On the other hand, if all skill sets complement each other then each member can focus on his/her area of expertise and create a masterpiece altogether.

In my particular case, here was my first reality check: I looked at my dream game vision, I listed my skill sets, I counted the resources at hand and I came to the sad conclusion that there was absolute no way I could materialize my dream game within the given timeframe. The only way I could still participate in the challenge was to reduce the scope of the game and start planning all over. So, I created a “spin-off” of my dream game in such a way that, instead of being a truly 3D fighting / action game, I focused on creating a side scrolling brawler which would be a 2D game but with actual 3D models. I created a new story and a whole new set of characters. It is not what I always wanted to build and the story is nowhere as appealing as the original one, but it has some interesting ideas that I’m looking forward to implement. That is how “Antimatter Instance” was conceived.

The advantage of putting my dream game aside and creating a spin-off instead is that I leave the biggest vision aside, hoping that one day I could revisit the project and implement it in the near future, perhaps even in the following DBP challenge. In the meantime, everything I create for the spin-off can be of use for the bigger project.

Step 4: Get the tools to use for the project

Once we know what we have and what we are missing, it is time to round up all resources, get the tools we need, install them and start working for our dream game.

Talking about “Antimatter Instance”, here is a quick list of the Microsoft tools chosen for this project:

  • Game Engine: The game engine selected for this project is my home-made game engine running as UWP application on DirectX 11. The main reason for this choice is because I am very familiar with this engine, and I’m quite confident that the animation tools I have for it can be used to create an entry worthy to participate on the Dream Build Play 2017 Challenge.

    To elaborate a little bit, all game engines available have its advantages and drawbacks. One of my previous games called “A Snowy Slalom” (proudly participant of the DBP 2012 contest) had a generation algorithm that created slalom tracks at run-time, which required the use of skinned meshes for the tracks, assembled in an array similar to a chessboard. Due to performance challenges, it was imperative that these track components should be drawn in batch mode, which was a need that reduced the number of commercial game engines that could be used at the time. Due to this technical roadblock, it made complete sense to create an in-house game engine using DirectX 11. Despite popular knowledge, it was not that difficult to create, and the blue print can be found in this other article that I wrote some time ago for the Microsoft’s tech wiki knowledge base. This same approach was implemented in my third-person shooter by generating mazes at run-time.

    However, a side scrolling brawler does not need to have the background drawn in batch since the number of individual meshes is way much smaller than in an open world game (to throw some numbers, 600 meshes in my slalom game compared to 50 in my side scrolling brawler, both at a 60 fps). Instead, the main feature to use is an animation engine with a sophisticated hierarchy and programmable flow that can emulate a sparring match. In this regard, I’m inclined to believe that my home-made game engine has reached this level (as seen in the featured video of this article), although I have to admit that it would take me a few months before I can reach the quality of the user interface that commercial game engines provide.

    There is one big advantage about using my good ol’ game engine: The model editor, included in the tools I created during my XNA days, allows me to mix and match mesh parts and bake them in a single model embedded in an X-File. For example, “Anabelle”, the main character of “Antimatter Instance”, is a re-incarnation of “Alice”, the lieutenant in “Battle for Demon City”, who in turn is a re-incarnation of “Danika”, the main skier in “A Snowy Slalom”, who in turn is a re-engineered character of “Aida”, the mermaid from “Merball Tournament” (well, at least half of her).

    The other advantage is that I can implement special animation effects to be computed at run-time, as seen on the work-in-progress video shown at the beginning of this article, like the animation of the hair, the skirt and the eyes, among other elements.

  • Development Environment: For this project, I have a Visual Studio 2015 development environment with Windows 10 SDK Anniversary Edition. I thought about upgrading to Visual Studio 2017 with the Creator’s Club SDK (edit: now there is a 2017 Fall edition), but the truth is that this kind of upgrades always come with an unexpected technical challenge and, because of the aggressive timeline provided, I don’t really have time to spend in any surprise.

    For example, when I migrated from Visual Studio 2013 to 2015, the internal class structure for a DirectX game changed, which was expected when changing from a Metro-style to a UWP application. What I was not expecting is that the helper class, used to load game assets in the background, changed as well, which broke my implemented object pool managers. This meant that no shaders, textures or sounds could be loaded anymore. It took me a while to fix them since I am not a C++ expert.

    Also, when I changed Windows 10 SDK versions from the UWP version (10240) to the Creators version (15063), the DirectX SDK could not be compiled anymore. I don’t have the exact error message, but it was something like a particular function within the DirectMath library was returning a result in an unsupported format, and therefore the DirectMath could not be compiled. Later on I found out that there was a hot fix for the header file (it’s documented in GitHub), but I felt rather nervous about changing official libraries. At the end, I went one version behind and decided to stay with the Windows 10 Anniversary Edition.

  • Test Environment: As a UWP game, I’m targeting consoles, PCs and mobile devices. To test the game, I have an Xbox One S, a PC with an NVidia GeForce 8500 and a Tablet with an integrated Intel HD Graphic adapter. To elaborate, the Xbox console’s technical specs are standard. However, the technical specifications of PC’s are widely different from unit to unit. Same problem is present with mobile devices. The Microsoft Development Network suggests using an Atom device (that would be my tablet) to test the game. The line of thought I’m following is that, if the game runs on my tablet, then it can pretty much run anywhere.

  • Sound: The tool I’m using for background music and sound effects is Microsoft’s DirectMusic Producer. This is one of the hidden treasures that failed to get the recognition it deserved when it was published – personally, I’d have given a standing ovation. It can no longer be used to create the “mood” at run-time (since DirectSound is now deprecated), but it can still be used to create and edit midi files. In fact, I created the background music in the work-in-progress video with this tool. It was one of the first pieces I wrote back in the XNA days.

Conclusion

I am well aware that this should have been the very first entry in my developer blog. After all, it is rather pointless to list the tools to be used for a project when the project itself is half way done. On my defense, if I have stated on day one that I was going to use Notepad as my primary tool, I would have lost all my readers right then. On the other hand, If I create a work in progress video showing how the game is probably going to look like and then, and only then, I state that I was going to use Notepad as my primary tool then I would have kept my readers… at least for the duration of the video.

As an “indie”, the skill set I have puts me in a unique position:

  • I am a software developer that makes his living by creating custom applications that fulfills very specific needs. That is why, when I don’t have the tools to achieve something, I create them. It’s like second nature to me, and I have to admit that it is part of the fun about being an independent developer.
  • On the other hand, I really want to be an artist. I know, my art sucks, but that doesn’t mean I don’t enjoy creating masterpieces. “Antimatter Instance” has given me the opportunity to create some quite fluid animation clips. True, the in-house approach means that only my game engine will be able to play those clips, but that’s ok since I don’t think anybody would hire me as an animator anyway.

These two roles I play as an “indie” complement each other, and have been the factors that defined the technical framework and the tools I use to create videogames. Everybody is unique, and I don’t expect that the same approach may work for someone else. But I really hope this utterly long post may provide a good insight for other game developers.