About two weeks ago I mentioned I was looking into using the open source library, gameswf to integrate into our little 2D engine for iOS. I was at the point where I needed a UI solution for Outwitters, and I had a few options:

  1. Keep doing what I was doing with Tilt to Live and create the UI by handing coding everything and procedurally animating it
  2. Create a UI editor to help automate some of the UI features I had created in Tilt to Live
  3. Look into gameswf and see if I could go for the whole she-bang and just use Flash for the UI.

I’m happy to report the gameswf seems to be doing just dandy on iOS. When researching, I was hearing a lot of complaints on performance and compatibility (it’s only compatible with actionscript 2.0), and if you’re using a lot of complicated vector shapes you can bog down the FPS. Well reading the post date on a lot of those articles I found that some were pre-dating the 3GS! So in the end, I just put aside a week to mess around with gameswf on an ipad 2 and try to convert some of the ‘hand coded UI' in our prototype to using flash.

Exhibit A

Just a disclaimer, all that you see in this video is a work-in-progress and extremely early. It was mostly a mock up to play around with the UI.

So in the video you see the main menu with the Outwitters logo, some buttons, a blue blade, and a background illustration being animated. This is entirely made in flash with pngs and some static text as buttons (just a full screen swf). As I press a map to load a pop up blinks into existance asking if I want to resume a previous game or not. This pop up doesn’t have any external bitmaps or pngs in it, it was made as vector shape rectangle with curved corners and some static text (which actually are bitmaps, but this is handled internally in gameswf). And finally, the mock up game board screen you see a flashy banner come across the screen announcing the player’s turn and shimmering, and then pulling away when the player taps the screen.

The animations weren’t anything too special as my flash skills are pretty horrible these days, but hopefully one can see that in the right hands this can be an incredibly useful tool to making games look that much prettier and be more engaging.

The best thing that’ll come out of this, is now the artist, Adam, has a large amount of creative control of how the UI will look and feel. Before it was heavily on the programmer side to make things work, and the fidelity suffered for it.

The Downside

Now for the things that might be problematic going forward:

Memory

Nothing has changed about this as far as I know, but images are required to be sized at powers of 2. When your working with the lower level api’s this isn’t a limitation that is difficult to overcome because you have so much control over what’s being loaded and how it’s organized in memory (for example, texture atlases). With gameswf, images are handled internally and they do 1 of two things:

  1. If the image is a non-power of 2 size, it will be resampled into a power 2 size using a software-based resizing algorithm. This can be a rather expensive performance hit if you don’t take this into account. The side effect as well is images that go down this pathway end up being rather blurry
  2. If the image’s height and width are a power of 2 then it uses the image directly.

My current solution? Just making image sizes a power of 2 before importing them into the Flash library. Yea it’s a lot of wasted space, but given how the UI functions currently, they aren’t long-lived objects so the memory is freed shortly after the user dismisses the screen or moves on. Of course, in the actual gameplay screen any flash-based UI elements have to be optimized. Sprite sheets aren’t exactly feasible in the traditional sense, but you can emulate sprite sheet functionality in flash by creating a movie clip that is masked differently on each of it’s frames. It’s tedious, but I imagine if you’re needing the extra memory savings this could be the way to go. There are a few other memory saving techniques I haven’t explored, like shared libraries that might provide some benefit as well.

In any case, if you’re targeting really low hardware (3G and below), you’ll most likely run into memory issues. The iPad 2 handled it just fine, but I imagine once I get down to the 3GS model and iPad 1 I’ll have to tighten my belt on the memory usage a bit, but it certainly seems doable.

Resolution

I had spent about a week coming up with a small body of photoshop scripts in conjunction with TexturePacker to help streamline our process for creating assets at different native resolutions. The upshot is we have the assets. The downside is now they can’t be used as-is. I don’t have a ‘swf generator' to generate iphone, retina, and ipad resolution swfs. So right now it’s looking like it’ll be a completely manual process of us creating separate swfs for each platform/resolution. Of course, if our swfs were 100% vector graphics, this wouldn’t be an issue :).

I’ve pondered some possible solutions, but haven’t experimented with any yet. One idea I had after browsing some of the source was using the ‘scaling' feature of gameswf to scale the rendering of swfs, but inside the actual flash files do some actionscript magic to display the correctly sized graphics in the UI. Not sure how much tedium this would save in the long run without some sort of automated way to do this. On one hand, Adam or I would be laying out 3 individual flash files to be used. On the other, we’d be laying out a single flash file but then I would be scripting the resolution changes in the file itself. I would have to come up with some sort of ‘utility' style movie clips in the flash library to help with this, but maybe it could work better than managing 3 files for every UI screen?

Scalability

Given the tests I’ve done so far, there’s nothing there that has lead me to believe that doing a commercial quality game on the iphone/ipad with gameswf isn’t possible. In fact, I’m sure it’s probably been done by now and it just hasn’t been talked about? Which is kind of strange, considering how awesome it is. But maybe that’s just me geeking out. This isn’t so much as a concrete downside as it is more of an ‘unknown'. My concern is once we’ve got a lot of in-engine animations/particles running how will it fair with a flash player running the UI. The other caveat is Outwitters lends itself really well to this type of setup as it’s not a very fact paced game. So this solution might only be feasible for particular types of games (in the same way that using UIKit and CoreGraphics instead of OpenGL is feasible for some types of games). I guess only time will tell.

The Upside

Some of the positives of using gameswf particularly:

Lazy loading of images

When a bitmap is being rendered inside gameswf, the texture isn’t sent to openGL until the first time it is drawn. It’s a nice thing to know. We can kind of ‘stuff' our flash files with image assets and not worry about them taking up tons of memory at runtime as long as we don’t render it all at once. This is useful for HUDs that are themed, in our case, by team colors, shapes, and logos. A point of caution is the the image *isn’t* unloaded from memory (even if you call an unloadMovie() from ActionScript on an externally loaded flash file).

Auto-rotation support seems feasible

Prior to this development I haven’t really considered doing anything too extravagant with support for auto-rotate. If we did support it, it would simply switch to the correct orientation and be done with it. But now it seems pretty trivial to have the UI do a fun but subtle animation as it stretches, squashes, or moves over to accomodate the different screen real-estate. It’ll be a ‘nice touch' if we get that working.

It isn’t an all-or-nothing setup

Giving the sample project of gameswf for iOS a quick look it seemed at first that running flash was an all-or-nothing deal. It was initializing in the main method and pretty much took over how things were rendered onto the screen. This part was time consuming but after experimenting around with it a few days, I got gameswf to play nice with my already established framework for game screens and UI. Each screen in our game is just a subclass of a ‘Screen' class that houses ui widgets and any logic particular to that screen. I was able to refactor a few things, and now I have just a subclass called ‘FlashUIScreen' that takes in a flash file name and a ‘controller' class as constructor parameters. It now functions and renders identically to anything I previously did. The point being, that if the need arose for something to render in-engine (perhaps using game assets already in memory for some sort HUD) I can still fall back to my previous method of hand coding UI. Hopefully that will be the rare exception than the norm.

Some Things To Know

It’s been a quick 2 weeks since I decided to investigate it and there’s still a few things left to answer, but currently I’m moving forward with the assumption we’ll be using gameswf for this project, and that is a fun thought to entertain :). For anyone else that is curious on getting gameswf up and running here are just some things I ran into while working with it:

If you’re loading separate movie swf files you’ll need a separate gameswf::player instance. I first went with the approach of having a single player and calling load_file() on anything I needed to render because I didn’t completely understand the relationship between a player and a gameswf::movie isntance. But the player tends to house the actionscript VM for a “root movie”, so loading 2 movies from a single player gave me some wonky actionscript behavior.

If you delete all the gameswf::player instances at any point in time, the library goes through a ‘complete clean up' phase of gameswf. This game me some issues when I would transition from a main menu to an all OpenGL view (no flash UI) so the swf players were delete, and when I went back to a flash-based menu the app would crash with an EXC_BAD_ACCESS error. I got around this by instantiating a single gameswf::player instance that didn’t load any files and is held in my singleton service-based class. It’s only ever deleted when the app shuts down.

Getting the GL state setup correctly was probably the most time consuming part. The library worked great in isolation, but when combined with the framework I was working with, there were all sorts of states I had to track down in order to render a flash file and still be able to render openGL correctly right after. Below is just the snippet of the code I wrapped around a flash movie’s display() method.

Somethings to note, it likes a blank projection matrix (had to reset it to an identity matrix) and you have to disable a few client states in order to render flash properly. Once, you leave the display() method you need reset your gl color, blend functions, and possibly any other states you have may used with the assumption that they are enabled:

const int w = GetDisplayWidth();
const int h = GetDisplayHeight();

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
movie->set_display_viewport(0, 0, w, h);
movie->set_background_alpha(0);

glDisableClientState(GL_TEXTURE_COORD_ARRAY);
glDisableClientState(GL_COLOR_ARRAY);
movie->display();
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnableClientState(GL_COLOR_ARRAY);
glEnable(GL_TEXTURE_2D);
glEnableClientState(GL_VERTEX_ARRAY);
SetGLColor(1, 1, 1, 1);
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);

And with that, I’m off to do some more work on Outwitters. Adam and I just played a few games over lunch and I actually WON. Mark this day, for this is the first day I didn’t get completely rolled in my own game.