Rotorscope: The secret of the endless energy

Finally, we are submitting to the DreamBuildPlay contest, in a few hours. We are running the final tests against the DBP build, and uploading the submission video everywhere.

Check it out, from Youtube:

3RROD won't stop us!

After a week and a half of hard work at coding the new serialization framework for Tomahawk engine (a.k.a SerialBoost), finally everything was ready for a first try on the Xbox. I was supposed to measure the new loading times against the old one. So I went to my Xbox, to test and pre-SerialBoost game build agains my stopwatch. I turn the Xbox on and... surprise! I got a beautyful 3 red lights ring of dead on my box :'(

I got really dissappointed by the fact that we are less than 20 days away from the DreamBuildPlay deadline, and I really wanted to check if my coding efforts of the past weeks were worthy or not. Instead of crying, or even launching the console out the window (which was one of the first thing that came into mi mind at first) I jump into surgery mode and opened the xbox for an emergency X-Clamp-Fix.

Finally, and luckyly, at 1:30 A.M. I got my Xbox back to life, here is picture of myself holding and hateing the X-Clamp piece removed:

Introducing Tomahawk's SerialBoost

The problem:

Currently our Engine, and so our game too, heavyly relay on Xml serialization. Tomhawks has a very powerful and flexible archetype system, that allows to instantiate logic objects using different initial value sets for their properties.

Currently archetype system allows placing any public variable of the logic object class in the archetype, as well as inlining the archetype of a smaller child object directly inside the archetype of the father one. The archetype has an open format (the order of the elements inside doesn't have to match the code, or have any particular order) and the number of fields specified is also optional (not all fields must be specified). This flexibility comes in part thanks to the use of XmlSerialize.

XmlSerialize is well know to work very slowly on PC, and terribly slow in the Xbox. This is because XmlSerialize requires a lot of Reflection, which also know to be not very fast. When we started the development of Rotoroscope, and it's engine, that wasn't a problem, because the number of logic objects and the size of its archetypes was small. Now, after 6 month of adding feature after feature, the number of archetypes has multiplies, as well as their size.

So, we had long benefit from the convenience of XmlSerialize during the development process, but now we have a puzzle game with a loading screen that takes longer that a AAA FPS game... and that's a problem :D

The Solution:

Fortunately, Tomahawk is modulated designed and all the serialization system can be replaced without touching any logic code, in theory at least. But... what can use instead of XmlSerialize?

For the PC, we found a tool by Microsoft called SGEN. This tool takes a .NET assembly as input, and outputs another assembly with a newly created class for each one of the serializable class in the input assembly. These new classes contain specialized code that eliminate the XmlSerialize use of Reflection, boosting up the speed of XmlSerialize. We added a single line to the compile script in the PC project, and the loading times were reduced by almost 85%.

Whoa! That's a hell of an optimization. But what about the Xbox?

I tried to use SGEN in the Xbox solution, but it turns out that it couldn't reflect the Xbox assemblies, and consequently generate the serialization optimizers. Then I tried to decompile the PC serialization optimizers, to recompile for Xbox, that didn't work too because the classes generated by SGEN were subclass of a .NET class that is not present in the Compact Framework. That was like hitting a wall, SGEN won't ever work for the Xbox, so I abandoned that way.

Anyway SGEN philosophy was the way to go, so if I can't use SGEN directly, I will have to replicate it's effects in some way. So I decided to create a similar approach from scratch, specially optimized for the Xbox.

SerialBoost:

I've started working in a framework that allow me to create a serialization specialized class for every logic class that requires serialization in Tomahawk and Rotoroscope. This framework will supply a base class compatible with Xbox (not like SGEN) that will simplify and reduce at a minimum the lines of code required for each serialization class.

Those specialized classes will be auto-generated with a command-line tool, at design time, and then deployed to the Xbox with the rest of the game. They will be indexed at startup, so no subsequent use of Reflection will be needed.

Then, when serializing or deserializing an object to/from Xml, the framework will look up for a specialized class to do so. If it's found, it will be used, and if it's not, it will fall back to standard XmlSerialize, making the presence of these generated classes optional.

I called this framework (and so the singleton class to access it): BoostSerialize. This class provides methods to create objects from a memory stream (containing the Xml) and viceversa. The interface is simple, you supply a memory stream containing Xml and the Type you wan to create with it. It will give you an object in return created with the fastest method available per each class.

Current State:

The first version of the framework is currently working in a isolated solution, so it's not currently integrated with the rest of the engine. As soon as I had the first working pieces of code, I created a small benchmark for the Xbox, to make sure the efforts will pay in the final game.

For testing purposes I took the SaveGame object from Rotoroscope, and I artificially extended it to cover all serialization cases (base types, enums, structs, other arquetipable objects, etc.). Then created a sample application that will load lots of them from disk, measuring the time spent using both XmlSerialize and SerialBoost.

These are the results, for 500 objects loaded:

* XmlSerialize took 3.17 sec.

* BoostSerialize took 0.72 sec.

And these are for 5.000 objects loaded:

* XmlSerialize took 30.16 sec.

* BoostSerialize took 7.14 sec.

Conclusion

These results makes it really worthy to spend a few more days creating the code generator, and integrating the whole system into Tomahawk. It will be risky to do such a massive change at this point of the development, but it will pay with a really noticeable improvement on loading times, which will sure produce a better playing experience for the players.

So, stop talking, and keep coding! More on Tomahawk's SerialBoost soon, I will publish the final effects on the loading time as soon as I finish and integrate it into the game.

Regards!

Adding video support to the game

This past weekend, we added a video aperture sequence for the puzzles.

It turns out that the WMV support recently added to XNA, it's not as good as one may expect. We experienced some troubles trying to get the video in the exact format required by the content pipeline video importer, without ruinning the quality.

IMPORTANT: It's mandatory to use CBR compression and a single audio track in your video. VBR and no-audio is not supported by XNA right now.

The first attempt was to use my video converter tool of choice, FormatFactory. This freeware program works great for all kind of formats and portable devices. So I presumed it'd work for XNA too. Sadly, I was wrong. This tool is unable to produce a WMV video compatible with the content pipeline.

Then, I tried the simplest option, and converted the video with Windows Movie Maker. That worked, but this tool is "too simple" and you just can't configure the video output beyond the built-in profiles. So the resulting videos created with it, are plain horrible.

Finally, I installed Windows Media Encoder. It comes with a non intiuitive but very flexible interface application, that allows you to configure every aspect of the compression and the output file. So after getting used with this application, I started experimenting with the different options, first one was resolution.

It turns out that there's no way to move a 720p video smoothly in XNA. So forget about 720p :'( as start. Or maybe you can, but with zero cpu expenses in your game logic side, which isn't the case for us.

800x450 is the highest resolution I got to work without heavy framedrops. The good news is that you can increase up the bitrate as much as you need, without hitting performance problems. So we'll have to go with a not-so-high-res, but at least with a generous bitrate.

Those are the settings we're using right now:

Unfortunatelly, I can't show you the results, because the gfx are not quite definitive yet. Anyway I hope this information is useful to anyone trying to add video to theirs games.

Adding vibration support

I just added vibration support to both Tomhawk and Rotoroscope. This concrete feature was really simple to implement. I wanted to cover these basic requirements:

• The support must be offered via Tomahawk's InputManager.
• I need to create and mantain the vibration effects as separated "assets".
• The vibration itselft must be defined using two curves, one for each motors in the gamepad.

Given those requirements, this is what I did:

First, I added the RumbleEffect class, which will contain the definition of a vibration effect, including:

• A name
• A duration
• A curve for the left motor
• A curve for the right motor

Note: For curves, I used the XNA Curves, to be able to use the Curve Editor that's available in the Creators website. We already used these curves in the engine for other subsystems, and this tools itself is included in the Tools folder of the Tomahawk engine.

Thanks to the archetyping feature of the Tomahawk engine, creating a RumbleEffect is as easy as creating the following XML:

Then I added a VibrationManager to the game, in which I defined the VibrationPackage concept. It's nothing more that a collection of RumbleEffects, to be able to define several effects in a single XML file (inlining arquetypes it's supported by Tomahawk).

Also put a public method to invoke a concrete effect, on a concrete gamepad, and voilá... vibration support done!

Now creating the effects itself, it's really easy, just edit the left and right curves using the Curve Editor. The following image shows the curves for the vibration produced when a "bomb" piece explodes:

User content sharing up and running!

Today I've been incorporating the design made by Qapitan (Enrique Cabeza) for the content sharing notifications. I made this small video to demonstrate the finished feature in the game.

What you see in the background is the "Puzzle Library", one of the game modes available in Rotoroscope (I'll post a complete overview of this soon). Whenever the content sharing service contacts another party, a popup will show up to notify this. A few seconds later, another popup will show indicating the results of the exchange session if any (the other party may not have new contents for us to retrieve).

Sorry about the lack of sound, the sound fxs aren't quite definitive yet, so I preffer not to show them right now.

Today, I've been porting the whole engine and game to XNA 3.1 too. The port was kick and painless, everything worked just fine. Kudos for the XNA team, as allways xD

User Content sharing in Rotoroscope

User Content sharing is a good way to give your game a longer life and a better experience for players. Tipically in a game you would like to share the player's progression in the game (like scores or medals/awards obtained). But also if your game support user created content (using a built-in editor) like user created levels, you'd probbably want to allow the players to share them too.

A few Live Indie Games (formerly Live Community Games) support this already. There's even a free component by enchantedage that allows exchanging highscore data. This component is being used in a few titles available now in the Community Games channel. There is also some other games that support this feature, like Zoomaroom (a very nice game, btw!). Anyway, there's a catch with current implementation on all games i've seen right now: They require you to stop playing the game and enter a special lobby designed for content exchange only.

As the only way to exchange content right now is to do it using network games (using a special multiplayer mode, that instead of exchanging and synching game object state, it exchanges game data like scores or user levels), all current implementation on content sharing require a players signed with Gold account privileges to work. This is mandatory, and there's no workaround. Only Gold Live members can enjoy content sharing, and that's a fact.

So current state of the art, is as mentioned: Gold is required and Lobby based sharing. We can do nothing about the Gold membership, we'll just have to live with that. But what about the lobby? There's two major problems with this approach:

1) First, is not fun. You don't want the player to stop playing to be able to share content.

2) The chances of two or more players to be sharing content simultenously is going to be really low, resulting in getting new user content for other players being to rare.

So we decided to make implement our content sharing as a "background service" that will be always running, transparently for the player, while he is playing the game. There's a couple of problems also with this approach, which are the following:

1) While playing the game, all Marketplace downloads you may have queued in your xbox will be paused.

2) When content sharing takes place in the backgrund, you probably need to access the storage device to save/load the content you're sharing, and that will cause the game to freeze sometimes, which can bother the player.

Our game content sharing is currently working, but is suffers from thoose two problems. I'm currently working on minimizing them.

For the queued donwloads, we have just provided a mechanism to allow the user to control wheter the content sharing is enabled or not, in the game options screen. This way, if you have downloads that you don't want to be stopped, just go to options menu and disable content sharing.

For the small freezes, we have already implemented a few of things:

1) We are packaging all files (objects) to be sent into a single transfer, to provide a single storage access per content exchange session.

2) Limit the amount of files being transfered per exchange session

3) Display a cool animated indicator for "storage device being accesed right now" while the files are being retrieved.

4) Implemented zip compression to reduce the time of the exchange session or increase the amount of files.

5) Implemented a cool animated notification to show the user that new content has arrived.

So currently, while playing Rotoroscope, if you have a Gold Live account, you will eventually see a popup animated notification (similar in size and behaviour to the xbox built-in notice for achievement and system events) that will indicate that another player had been contacted for content sharing. Then you continue playing, and a few seconds later the game will freeze for a second or two, showing the "storage being accesed" icon (only if the other party wants any of your avalable contents). Then you'll continue playing and sometime later you'll see a new popup notification indicating: "X new user puzzles are now available!"

Putting this thing to work has been a very painfull task... but I can't say that I'm really happy with the results. This implementation is already available in the lastest source code version of Tomahawk (sorry, no time to pack releases for the engine so far).