abi.exdream.com - Benjamin Nitschke's Blog - Lost Squadron

A new more effective keyboard layout for programmers

Wed, 04 May 2005 00:16:20 GMT

Hey now, I'm back again. Lets just pretend I didn't stop blogging the last couple of months ^^ From now on I will try to post stuff every other week, I still got some interessting topics.

Quick links

About Lost Squadron
Todays topic
Dvorak keyboard layout
CountMostUsedKeys tool
Own keyboard layout
Useful links

About Lost Squadron

The project was paused because I got noone modeling the missing models (my intern Ivo quit in the middle of the project because he had to find a job), most of the engine and game logic is pretty much finished (and has gotten way more complicated than planed). I will post some screens and info from time to time, usually I will finish up projects like this some later time when I got the missing resources or time (maybe next chrismas). Sorry for not informing you about the latest progress, but I talked myself into continuing the project and realised to late I can't finish it on my own now.

Currently we at exDream have finally started working on the next project and will produce a nice graphic demo/prototype in the next 3 month. I will try to sneak in a couple of screenshots of it from time to time.

Todays topic

A new effective keyboard layout for programmers, especially for c# and c++:

Download AbiKeyboardV9 installer
Download the CountMostUsedKeys program (see below)
CountMostUsedKeys source code for anyone interessted.

2 years ago I was talking with a friend about my english keyboard layout and discussing why it is way better than the german keyboard layout for programming (for some keys you even need a special alt gr key combination). The english keyboard is better because the symbol keys like ()[]{}'":;/ etc. are positioned more effective. This is the english keyboard for comparsion with Dvorak and my own layouts below:

We talked a bit about other keyboard layouts like Dvorak, which is way more effective and relaxing than any qwerty layout, but I got no free time back then. You will need around a week to learn to type at low speeds and at least a month to get back to a better typing speed.

Dvorak

This is the Dvorak us keyboard layout (you can simply select it in windows, it is pre installed and can be used like any other keyboard layout):

After Arena Wars was finished last year I got finally some time to start learning Dvorak. Dvorak seems very nice (see the links below for additional information) and has a lot of advantages for typing "normal" text (but some people exaggerate "a little").

Anyway, while playing around with Dvorak (us default layout) I really found some things VERY annoying, especially for coding and playing games:

It is fairly hard to learn a new layout and getting really back to the old speed again. Additionally hotkeys you typical need on a computer are totally crazy and unusable. Hard to learn when you used QWERTY all your life, learning can be done in a week or so, but really being able to get back to your old speed can take months (and is painful).
Ctrl+X, Ctrl+C, Ctrl+V can't be used as easy. I always use my left hand while my right hand can use the mouse, especially for browsing, code comparing, getting text from one place to another, etc. and all this happens VERY often! I know you can also use Shift+Insert, Ctrl+Insert, Shift+Delete, but these shortkeys are just plain unuseable, maybe for lefties, but the keys still suck.
Hotkeys can't be used the way they used to be, for some games its possible to change hotkeys, for others not (like starcraft, at least its not easy to change) This does not only apply to games, but also a lot for applications, especially when you remembered just the location where to press and not the name of the hotkey, e.g. when using photoshop, you have to re-learn the hotkeys. Also stuff like Ctrl+S, Ctrl+O, Ctrl+F, Ctrl+R, etc. is usually remembered by the positions on the keyboard and not by the key and relearning this keys is pretty hard!
Dvorak is optimized for typical english words, but not for programming, some keys are even worse than the german keyboard layout. But what I really don't like is the way Dvorak handles symbols like []{};:?+-/, etc.
Its not that QWERTY is really good for coding, especially when typing stuff like () or -, = or +, but overall it performs still better than Dvorak (counting only symbols).

I've also collected some interessting facts when comparing Dvorak and QWERTY considering c#/c++/java code. Click this link to test it yourself.

When pasting random stuff into the Keyboard Compare Applet, QWERTY performs almost twice as good as Dvorak (Note: We are not random key generators, usually we type text or code which makes sense ^^).
When just pasting c# code stuff without much text like
if ( a == 0 )
{
doIt( a!=0?a:0 );
} // if
QWERTY still outperforms Dvorak, which can't be good!

I wanted to code faster and not slower, I don't know anyone using Dvorak anyway and I could not care less if others can use my keyboard layout or not (since I use an english keyboard layout anyway and other (german) people can't even use that ^^).

I created my own keyboard layout to solve all the problems (yeah, now its getting real geeky), but I based it on the Dvorak layout, which is really efficient and nice for typing "normal" text (which is still required a lot, even by crazy coders).
To solve the Ctrl+X, Ctrl+C, Ctrl+V problem I simply moved that keys to the QWERTY position and exchange the other keys accordingly. This didn't matter much because there isn't much of a difference between X,C,V and Q,J,K according to the key usage (see below).

At this time I was already using Dvorak for about a week or two and had spend a lot of effort into learning and thinking about improvement, I wanted a perfect solution. I checked typical c# and c++ code and wrote a program to count how often each key is used. Now I got the most important keys and asigned the best spot for each symbol using the distance (see the Keyboard Compare Applet) and how much it is used. Using shift is also a major impact IMO, especially when keys are far away.

So I wrote a little test program to count the used keys in all my *.cs files in my code folder, check out the results below.

Screenshot of the program (I improved it yesterday, it was a console tool before):

Download the CountMostUsedKeys program.

And the source code for anyone interessted
(may be useful if you want to write some line count tool yourself)

CountMostUsedKeys results:
Search mode=Everything, File type=*.cs, Directory=C:\code\
Number of files=4041, Number of lines=2031908, Number of keys=67829575

Type:	Keycode: 32	10,1%	(6851259 times used)
Type: e	Keycode: 101	7,21%	(4887698 times used)
Type: t	Keycode: 116	5,2%	(3525648 times used)
Type: a	Keycode: 97	4,13%	(2802961 times used)
Type: r	Keycode: 114	4,06%	(2754960 times used)
Type: i	Keycode: 105	3,93%	(2663137 times used)
Type: n	Keycode: 110	3,68%	(2494441 times used)
Type: o	Keycode: 111	3,39%	(2298022 times used)
Type: s	Keycode: 115	3,24%	(2195958 times used)
Type: l	Keycode: 108	2,61%	(1769538 times used)
Type: /	Keycode: 47	2,52%	(1710099 times used)
Type: u	Keycode: 117	1,99%	(1349478 times used)
Type: m	Keycode: 109	1,91%	(1298573 times used)
Type: c	Keycode: 99	1,77%	(1201477 times used)
Type: d	Keycode: 100	1,77%	(1198823 times used)
Type: .	Keycode: 46	1,42%	(964040 times used)
Type: p	Keycode: 112	1,41%	(956931 times used)
Type: g	Keycode: 103	1,23%	(832686 times used)
Type: f	Keycode: 102	1,22%	(827820 times used)
Type: y	Keycode: 121	1,15%	(783356 times used)
Type: h	Keycode: 104	1,11%	(753548 times used)
Type: )	Keycode: 41	1,09%	(739711 times used)
Type: (	Keycode: 40	1,09%	(736841 times used)
Type: ;	Keycode: 59	0,86%	(580753 times used)
Type: b	Keycode: 98	0,79%	(532778 times used)
Type: ,	Keycode: 44	0,76%	(512629 times used)
Type: =	Keycode: 61	0,74%	(503579 times used)
Type: w	Keycode: 119	0,62%	(419356 times used)
etc.

Note: This stuff does NOT represent every key pressed, hotkeys and errors and corrections are obviously not saved in the text files.

Same results without space and letters, very useful to assign the most important symbol keys:
Search mode=ExceptLetters

Type: /	Keycode: 47	2,52%	(1710099 times used)
Type: .	Keycode: 46	1,42%	(964040 times used)
Type: )	Keycode: 41	1,09%	(739711 times used)
Type: (	Keycode: 40	1,09%	(736841 times used)
Type: ;	Keycode: 59	0,86%	(580753 times used)
Type: ,	Keycode: 44	0,76%	(512629 times used)
Type: =	Keycode: 61	0,74%	(503579 times used)
Type: "	Keycode: 34	0,38%	(255445 times used)
Type: <	Keycode: 60	0,32%	(213786 times used)
Type: >	Keycode: 62	0,3%	(203429 times used)
Type: }	Keycode: 125	0,28%	(187354 times used)
Type: {	Keycode: 123	0,28%	(187017 times used)
Type: +	Keycode: 43	0,27%	(181705 times used)
Type: -	Keycode: 45	0,24%	(159515 times used)
Type: ]	Keycode: 93	0,19%	(130828 times used)
Type: [	Keycode: 91	0,19%	(130687 times used)
Type: _	Keycode: 95	0,13%	(89213 times used)
Type: *	Keycode: 42	0,12%	(83203 times used)
Type: :	Keycode: 58	0,11%	(77487 times used)
Type: !	Keycode: 33	0,1%	(64444 times used)
Type: &	Keycode: 38	0,09%	(58918 times used)
Type: #	Keycode: 35	0,07%	(47258 times used)
Type: \|	Keycode: 124	0,05%	(34348 times used)
Type: '	Keycode: 39	0,05%	(31284 times used)
Type: ?	Keycode: 63	0,03%	(19709 times used)
Type: \	Keycode: 92	0,02%	(15340 times used)
Type: %	Keycode: 37	0,01%	(7019 times used)
etc.

Based on that I created my own keyboard layout, which was updated 9 times until I was happy with it.

Version 1 looked like this (pretty much like dvorak, some cosmetic changes at the right side):

Version 2: The next thing I did was moved the Z X C V keys back and copied some keys around, () are moved to normal keys, () is used much more than [] or {}.

In Version 3-6 I tried moving some keys around based on the distance and effectiveness, but I moved most of them back because they felt really uncomfortable and were bad for hotkeys. One bigger change was moving the . , () keys back down, which improves typing speed a lot because we really need more keys on the left side when using the mouse with the right hand and when typing the right hand can easily handle most of the symbol keys on the right side.

Version 7 was pretty much the final version, I trained this version a week and except some minor cosmetic changes in the upper row everything stayed this way till the last version:

Version 8 and 9 changed only the positions of J D F K B and the ? | and + - keys, at first I didn't like the position for b at the right side, but b is used a lot in combination with vocal keys unlike j, it is much better to have it on the right side. The + - key on the upper left is now the same as the key on the lower right (which is not present on all keyboards).

This is the final Version 9 I am using for nearly 6 months now, I got up to my old speed in the first 1-2 months and now I type even faster and more relaxed than before (maybe around 400 keystrokes per minute for normal text if I try hard). I wouldn't suggest changing your keyboard layout just because you might be a little faster with Dvorak or my keyboard layout, I think the comfortablity is at least as important as the speed. In my option the greatest thing about my keyboard layout (or dvorak for normal text) is the fact you can type most of the time without moving your fingers much (90% of the most used keys are directly on the center row).

Download AbiKeyboardV9 installer

I was not trying to create the best keyboard layout ever or to make Dvorak obsolete, I just want to improve it for me and other interessted programmer. If you get started with Dvorak or my keyboard layout you should use some keyboard training tool and spend a couple of hours (but not much longer) each day training it. I found this website very useful: Learning Dvorak

The main reason I wrote this article now was the nagging of some people I told about this and who wanted to know more, so keep nagging if you want to see articles like this :)

If you got comments or some ideas to improve this even more, let me now!

Useful links

Introducing Dvorak
Keyboard Compare Applet (nice)
Best keyboard layout creation tool, hidden and hard to find, but this is the only free useful keyboard layout creation tool.

The Dvorak Keyboard and You
Changed Dvorak for similar symbol keys
Learning Dvorak
Pic of Dvorak default Layout (us)
Critik to Dvorak
Keyboard Layout Manager (allows to change keyboard layouts) It's shareware.
Some discussions about Dvorak (coder related):
Dvorak discussion 1 Dvorak discussion 2 Dvorak discussion 3

Lost Squadron, Day 19: Shoot'em'up

Sat, 08 Jan 2005 10:10:36 GMT

Day 19:
Some effects in action, the shoot line helps you to see where you shooting at (will be improved with a special texture, its just a line now). Some 3D to 2D converting is also not correct and needs some fixing. In the next days I will try to bring together all the completed parts and start with the mission design in the beginning of the next week.

Ratta ratta, peng peng, boom!

Shoot'em'up

Yeah, hey now. This time I had way to much fun when generating this screenshot, now it is actually possible do aim with the tank and shoot at stuff (with all the new effects and sounds). It is a lot of fun.

Warning, this is a long post (I've written it in the last 2 days), but I have a nice idea: Content links, just click on them to jump to a section:

Shoot'em'up
Troubles with IIS
RenderState errors
Light effects in DirectX
OutOfVideoMemory in Unit Tests
VS2005 needs a lot of RAM
Links for today
New StarCraft site: www.GosuGamers.net
Tropical-Islands resort

Troubles with IIS

I had some trouble the last few days with my IIS (Internet Information Server, the thing producing ASP.NET web sites), it keept crashing and hanging up with out of memory errors (this page was down a couple of times, sorry!) Really strange, that thing is running for over an year now and I never had much troubles with it. The memory usage was also pretty high (total ~900MB, IIS worker process alone had ~400 MB), I guess this has something to do with dasBlog using .NET 2.0, dunno (there are no errors logged anywhere). I had a lot of page hits this week after all the posts on other blogs about me.
The memory consumption of IIS goes constantly up and I had over 200MB mem usage again after a couple of hours, hmm? Don't wounder if the site is down or you get some errors, may happen because of these memory errors ...

I also optimized the page from 50 entries max to 10 and cut down the number of entries you will see on the start page. I hope that helps a bit, my server is really crappy and old and needs a lot more memory (had only 256MB, but I upgraded to 512MB recently, but its slower memory. I guess I need a gig or 2, lots of other services are also running on that server). I also changed the app pool to a special one and set some timeouts for requests and processing, maybe this will help (had no problems since that change).

RenderState errors

Another annoying thing is the RenderState property of the DirectX device, it will tell you you can "get and set" some state like FogEnabled, well you can set it, but if you try to get the state it will just throw an exception. A lot of RenderState gets will throw an exception actually, I'm really impressed by that. Great work not to mention any of these things in the docs (the docs are pretty useless anyway, most descriptions are trival anyway and IntelliSense helps you just as much).

Light-Effects in DirectX

The light effects for the effect engine were pretty easy in DirectX, I remember this was very hard to do in OpenGL (had to invert the alpha color, and with different color settings everything got messed up, so a lot of special cases had to catched and handled).

Basically this code is used (the light effect is a white texture with a spotlight effect in the middle as the alpha channel):

Device.RenderState.AlphaBlendEnable = true;
Device.RenderState.SourceBlend = Blend.DestinationColor;
Device.RenderState.DestinationBlend = Blend.SourceAlpha;

Now make sure both the color channel and the alpha channel will modulate from the texture color and the diffuse color. Switching back to normal SourceAlpha - InvSourceAlpha causes also no troubles with that. I like that :)

DirectX OutOfVideoMemory

Another problem I did have the last days when unit testing was running out of video memory all the when creating the DirectX device (getting D3DERR_OUTOFVIDEOMEMORY exceptions) after ~10 tests. When restarting VS, everything worked again for some time.

I worked with that problem all week and found no solution, I changed a lot of debug settings and changed some parts of the code, but nothing helped and there wasn't any solution for out of video memory exceptions posted anywhere (my card has 128MB, how can I run out of vid mem after a couple of tests?). Then I saw that some part of the screenshot generation code wasn't finished and I saw the presentParams.SwapMode was set to Flip instead of Discard (Discard is somewhat faster, but Flip allowed me to easily catch the back buffer for the screenshot generation). I changed that, reduced the back buffer count to double-buffering and reduced the resolution for tests from 1024x768 to 800x600 and never had any Out of Video Memory exceptions since then. Maybe they happen after over 50 or 100 unit test runs, but then I can just restart VS and it is not as annoying as before.

VS2005 needs a lot of RAM

But VS2005 consumes still a lot of RAM (>800MB, I got only 512MB ram on my dev machine). I guess I have to buy some more RAM soon, everything gets very slow and crashes more often when all the ram is used (and I can't work without VS2005 anymore). Restarting VS helps, but with that much complex unit tests the memory comsumption goes up very fast again. For example todays screenshot is actually just a unit test with around 50 lines of code, making very simple unit tests helps a lot to force you to refactor and simplyfy your code constantly.

Todays Links

The links for today are The Z Buffer and Managed world, both great Managed-DirectX sites.

The Z Buffer Managed DirectX resources by
Managed World great site with tutorials and articles by Jason Olson, check out his blog.

GosuGamers.net

If any StarCraft fan reads this, the guys from www.StarCraftGamers.com did have a lot of troubles with their hosting partner and left the partner. StarCraftGamers.net was one of the greatest StarCraft community sites ever IMO, the new site is called www.GosuGamers.net. Looks like the crew will continue their great work there, so check it out (the site has still some bugs because it is new): Good luck www.GosuGamers.net

Tropical-Islands

I'm now going to visit the Tropical-Islands resort (new big thing here in germany) with my brothers and some friends :) Development will continue tomorrow.

Lost Squadron, Week 4, Day 2: Getting back to the game.

Wed, 05 Jan 2005 06:36:35 GMT

Welcome to 2005 :) Last week I didn't get much code done and run in a couple of problems with Lua and the hardest part about it: debugging lua scripts called from c#. Over the weekend I even tested if it might be worth to switch to Python and tested out the IronPython implementation and written some Python test programs. Python has definitely much more powerful libraries than lua and you can do great stuff with it, the OOP is also much nicer than writing Lua code and having to worry about every . : ] ) and } symbol much more than in any other script language I know of. But Python and especially IronPython is not only a total overkill for a small application and trying to get something quickly to work, the language has also a lot of disadvantages when comparing to Lua. Its not only very large and complex and therefore much harder to learn, but to get something working is much more complicated on the c# side. I also think Lua is much more customable and allows modifying much lower level methods and calls.

Ok, so I skipped that idea and returned back to Lua and finished the EffectManager on the Lua side making only a couple of calls to c#. This did not only help to learn more about Lua and scripting, but I also improved the c# engine very much and spend a lot of time simplifying code of the engine. Creating textures, making unit tests, error handling, etc. all got much easier (this comes when working with Lua and then looking at c# code and thinking everything is overly complicated). Anyways, Ivo and Fabian didn't do much either on the game and so we got stuck in the development and started today again and made a new schedule to finish the game at the end of next week. We will include some left out ideas and have gotten some new improvements on the game, the editor and some units. Ivo did also start some new objects (buildings and enemy units), there might be some fresh graphics for that soon.

Over the chrismas and new years eve weekends I read some more of my Lua and Rapid development books and yesterday I started reading Code Generation in .NET by Kathleen Dollard, which is also very interessting and got me in a lot of thinking how to improve coding in general (not only using scripts and code generation techiques, but going 1 step ahead and create new ways to handle coding and reusing existing code).

Well, first of all Lost Squadron has to be finished :-)

Useful links today:

Free game libs (mostly c++ stuff)

Day 16:
This screenshots shows my Lua EffectManager rendering effects with help of c# classes (this is just a small unit test testing some effects), it is very fast and calling Lua up to 20 times a frame produces still over 500 fps in debug mode and won't slow down the game at all (using Lua is goooood ^^). I wasn't posting updates since the last week because there wasn't much going on to produce fancy screenshots and I worked at other stuff (internal parts of the engine and Lua scripting). I will now try to post a screenshot at least every 2 days or so. Tomorrow I will hopefully finish the unit shooting and exploding part and post a ingame screenshot again.

Lost Squadron, Week 3, Day 2: Getting back on track

Wed, 29 Dec 2004 11:42:35 GMT

Omg. I have become a total slacker the last couple of days, no updates here :( I didn't had much time over christmas to code, but I read some more of my Rapid Development book (see last post) and it is still interessting and insightful. Yesterday I still was in vacation land and had to force myself to get back to coding ^^

I also reconsidered some choices I made and came to the conclusion that this project was originally started to make a quick little game with DirectX9 in c# and also to write a new base engine for upcoming DirectX9 projects. I did obviously spend too much time researching shaders (DirectX9 fx high shader language) and saw last week there is no way to implement that nicely in 2 weeks (when the game is running with all the other parts finished I will reconsider implementing shaders). For the engine part of the deal this is still great work and I have a lot of base classes for any shader effect development ready, but for the "quick little game" Lost Squadron part it is not looking so good. It is a conflict of interests because finishing up something quickly does not work hand in hand with researching and implementing new techniques. For this project I think it is better to focus on developing a solid base engine, next project will be planed much straighter.

Yesterday I collected a lot of effect graphics and sounds and made some new ones. I got now a nice collection of effects with graphics and some funny sounds for the effect engine and made a couple of effect types (parameters, testing, etc.), but when adding more and more effects it became very obvious that this would be a great think for scripts. Instead of hating myself later for not using scripts right away, I just went crazy and implemented the whole effect system in LUA scripts and can add new effects there without changing any code of the engine. I did the base code for that a month ago (check out LUA.NET for doing that in c#) and have read the great book Programming in Lua by Roberto Ierusalimschy.

LUA links in case you are interessted:

The Programming Language LUA The online version of the book by Roberto I. is also available there for free.
LUA.NET for implementing Lua in .NET Sadly the LUA IL project isn't public yet, but LUA lib is ok too.
LUnit Unit Testing with LUA.
QDE THE IDE for LUA development!
LuaLite Syntax highlighting support for Visual Studio, but does only work for VS2003.
Lua-users.org Everything else you need for your LUA development

The effect engine needs still some testing and I haven't worked with lunit yet (will do that tomorrow), everything should be up and running tomorrow and I will post a new screenshot with some nice effects then. The rest of the base classes except wayfinding and enemy ai is also pretty much complete, so I hope I can finish up a small demo till New Years Eve (driving around, shooting at some stupid enemies).

Sorry for no screenshot today and sorry for the long pause. Fabian is ill (my intern helping me out a bit with Lost Squadron), hope you get better till next week :) Tomorrow the LUA effect engine should work and I will post a screenshot of the actual gameplay finally (I want to kill some stuff).

Lost Squadron, Day 14: Effects and Game Unit Testing

Fri, 24 Dec 2004 09:43:25 GMT

First of all: Merry christmas to everyone and thanks for reading all this stuff, my page views increased a lot since I do this daily series.

I spend the last couple of hours trying to get Unit Testing on the engine level to work and I finally got now everything up and running. This is really one of the greatest things ever in Game Development IMO! Normally I would spend a lot of time changing code to just test stuff or even write small test application only to check the behaviour and graphical output of effects, newly imported textures or models or new game features. Now I can just write a VERY short Unit Test for that INSIDE my project, no more extra projects, no more messing around in the code.

Just to make this clear: This is not a usual Unit Test like in other parts of the engine, like when checking file input/output, internal methods or some logic issues. These Game Unit Tests (thats how I will call them from now on, or just GUT, which is german for good ^^) give you all the features and easy testing possibilities like normal unit tests, but you can initialize the whole engine, see visually whats going on and shut down everything automatically or by user input. Let me give you an example how this looks:

Note: This code uses .NET 2.0, if you want to read about Anonymous Methods click here.

Same thing is of course possible with a couple of lines more code when not using Anonymous Methods (using Events or just duplicating the code in TestInitEngine). This code will create all the stuff required for the game engine, then execute the initialization code to create the effectManager. Then it will proceed to the game loop and execute the render code each loop there, everything will be shut down if the user quits the form, presses some key or if we set a timeout to quit automatically. There is a lot of hidden code behind all that, but this is exactly the code executed when just running the game and testing that is always good :-)

In the next 2 days I will not code much (visiting family, chrismas, etc.), but I will to read 2 nice book from Steve McConnell: Code Complete 2 and Rapid Development. I already read a couple of pages of Rapid Development and it is really written good and very interessting.

The next Lost Squadron screenshot will be posted presumably in 2 days.

Day 14:
The game itself did only get some minor improvements today, I spend most of the time on the engine. As you can see on the screenshot, there are now effects like that explosion effect (there are already over 20 effect types), but the EffectManager isn't complete yet. Additionally to the road it is now possible to set railroads and I imported some new objects and new graphics into the editor (not on the screenshot).

Lost Squadron, Day 13: One week more plz ^^

Thu, 23 Dec 2004 09:19:14 GMT

Hey everyone. Today I made some improvements on the graphical side, mainly because I received some mixed feedback about the current graphics. Doesn't look too pretty right now. For that reason and because a lot of classes for the game handling are not finished yet (and only 1 day left till chrismas) I think its much better to give this project 1 week more. Maybe this time will be enough to finally implement shaders and shadows for all objects. Especially shadows will add so much to the game, it is pretty much impossible to tell how height an object is when looking straight down. Shadows would not only make the scene more realistic but would also improve the gameplay (you would see what your unit can and can't see).

As you can see on the screen I completed the roads stuff and added some new objects. Also as planed yesterday the moving around and mouse aiming does work now, but I'm screwed again with those X files, there are totally messed up (rotation wrong, each object is rotated differently, meshes do not fit on top of each other, the upper part of the tank does not have the same pivot point, really annoying), grr, but I don't have the time to write an own 3d-model format (or import any format into the engine, 3ds would suck at a similar level).

I also started the game message classes and most classes needed for gameplay are existing now, but most of the functionallity is missing. I will try a new approach this time to write that code: Unit Testing. Many smaller methods of the internal engine are already checked, but I never done large Unit Tests for a whole game. The tests should be able to run the whole game (creating units, moving around, blowing stuff up, doing graphic effects, winning, losing, etc.). I will report back how that works. Later today (when I slept a couple of hours ^^) I will start that stuff with my intern Fabian :-)

So the new date for the first Demo version of Lost Squadron is 2004-12-31. It might be playable tomorrow already a bit, but I don't think it is wise to release it so early. The first experience should be very nice and for a quick look you got the screenshots here ^^

Here is a fun link about Programming Languages and why some of them failed and others not:
why-microsoft-can-blow-off-with-c.html

Day 13:
Some new models, but still a lot of problems with x files (man they suck, I can't imagine anyone is actually using them for anything except test projects). Note: Don't expect that this screenshots will fully represent the final game, everything can change (just check the screenshots of the first days, there have nothing to do with the current state).

Lost Squadron, Day 12: Back to the game

Wed, 22 Dec 2004 09:52:57 GMT

Yes! Finally I got back to the game code after all this Editor and Shader coding and run into some problems when rendering some models with transparency. First of all the models had to be rendered from back to front, but even then the models with multiple alpha layers were completly messed up. So I had to split up the alpha layers to multiple meshes and tryed to render them back to front, but DirectX tries to render per materials and not per meshes, so that didn't work easily.

I had another idea to just test the alpha values for 50% alpha and then skip writing to the frame buffer and that didn't work either (framebuffer is full of false pixels anyway). Instead of banging my head into the wall I just steped back and thought about needing this feature at all and no, there is no need for that in this game: I can just delete or don't use anything with multiple alpha layers. For example the palm had 3 layers of palm leaves, but I killed 2 of them and no everything is working perfectly (I still have to sort objects and to render first solid, then alpha materials, but thats pretty fast). I'm happy I got this solved :-)

The Editor was also improved a bit and everything needed for a full mission can be set there. As you can see on the screenshot there are a couple of new features for the map: Roads, some more neutral objects and some enemy units. Shadows and shaders would be nice too, but there is no time for that right now. Maybe I will be able to put that into the engine next week.

Moving around with the tank and aim and shoot on stuff should work tomorrow. Effects and AI won't be complete till christmas, there will be most likely only a couple of standard effects. Also collision detection is another big issue.

Day 12:
Lots of dummy objects (especially the red car), but lighting works, loading maps from editor works and roads are going to work soon too ^^ Also a lot of gameplay classes done today. More objects will be made and imported soon to improve the game feeling.

Lost Squadron, Day 11: Another day with the shaders.

Tue, 21 Dec 2004 05:42:51 GMT

Well, this shader stuff is really annoying. I got a new shader for the landscape in a test app working and implemented some test textures there. But when changing the light to a spotlight I ran into many problems, not so easy as I thought. At least I know now how to pass all that vertex buffer data to the vertex shader, but for a ground shader with works nice with all the lights there is still some work to do and I don't have the time for that. So I will skip now all the normal mapping stuff until later when the game is complete. For meshes normal maps are working (at least with directional light), maybe I will use some of that.

Anyways, time is running out, I wrote down all the weapons, powerups and enemies today and created some classes for that, but there is nothing finished yet, so no screenie today. Sorry. Worked the last couple of hours on shader stuff anyway and I told myself it I don't get it working today, I will leave shaders out and add them maybe later.

Lost Squadron, Day 10: Editor finished

Mon, 20 Dec 2004 08:13:54 GMT

I was working still on the Editor today, not that much editor features were missing, but a lot of other stuff eat up a lot of development time. Drawing lines for example is really bad in DirectX, either you just draw some primitives and get slow when drawing a lot of them and have a lot of overhead each time setting all states (and recovering from some crazy states the caller might have left, nothing is working as espected) or you use this stupid Line class wrapper thing, which will cut your framerate by the factor 2 or 3 (really great, no wonder noone uses it). Or you can just draw stuff with vertex buffers, but thats also very annoying because you have to create the vertex buffer and handle all that vertex creation, and also obviously not the right approach for a single line.

So I ended up writing a nice LineManager class, which collects all lines draw each frame and put them in a vertex buffer and because the lines are only changed when some big change happens, the vertex buffer can stay the same for a long time and thats really fast. Everthing gets rendered at the end of the frame (used for on screen controls and other stuff anyways). Same thing for filled boxes, color blending effects, etc. lots of helper classes.

The texture blending (as you can see in the image) was also not a piece of cake, but the blending over 5x5 vertices is now a nice technique. The textures are not really fitting good together, when you just put them besides each other, but with this blending technique it looks really good (or maybe I'm just to tired to tell anymore ^^). Another bonus is my new vertex format, which is compatible with a test shader I wrote 2 days ago (which didn't run correctly back then, I only managed to do some basic stuff). Seems like some arguments where missing and the vertex shader wasn't complaining, but crashing some time later (after couple of calls, not really funny to debug).

There isn't much time left, so maybe some features might have to be put on hold because I really want to finish the base game this week (just 1-2 missions, only basic features to see if its really as much fun as I imagine). If its playable a small demo or some sort of beta-version will be available before chrismas this week. Anyways, till new-year the game and the final demo-version should be finished, lets see. Maybe in the next couple of days things can be done more quickly because all the base classes and the complete framework are now completed.

Day 10:
Yup, yesterday no screen, I was too tired. The editor is now finished, only setting objects and the roads is missing, but that should be easy. All the other sub menus and UI is now complete too, even the multiplayer stuff has a UI now waiting for implementation (dunno if there is enough time this week for that, would be nice though). Time for some gameplay this week :-)

Lost Squadron, Day 8: Editor and more UI controls.

Sat, 18 Dec 2004 04:21:08 GMT

Today was Editor-Day. Our intern Fabian helped my out a little with the map file and we implemented all the UI controls for the Editor, all of them are working now, at least on the UI side. The Map Editor itself will be finished tomorrow, still a lot of smaller things to do before everything can run smoothly.

The UI needed also some improvements, so I made completly new graphics for them and updated all UI controls and added some we needed for the Editor. All this little things, like writing UI code, writing smaller classes for the ground textures, the road logic, neutral objects and the enemy units can really keep someone busy. I still hope the basic engine for this game can be finished this week, so we got next week to make some cool missions and the advanced stuff (shooting, effects, multiplayer).

About all this shader stuff from the last couple of days: I managed to get some basic shaders to work, which is really nice now, but still no clue how to implement all that spotlight normal mapping with alpha blended textures. Maybe I will just use some sort of directional light for the bump effects (or maybe a point light if I can pull that off, but then all the lighting calculation has to be done inside the shader as well, uhh complicated).

Day 8:
Half-time, 1 week has gone by for this project. This is a image of the editor, you can easily create new maps or change existing maps with this nice tool :)

Lost Squadron, Day 7: Still struggeling with shaders.

Fri, 17 Dec 2004 04:54:06 GMT

Some internal work for the Editor was done today and I wanted to finish up all this shader stuff, but run again into a lot of problems. Vertex Shaders are really a pain in the ass to debug, everytime anything goes wrong the whole app crashes (I catch exceptions everywhere, but DX will just fail at everything and the app has to be shut down). There are also a lot of problems with the shaders to support spot lights and some other stuff in the engine like blending textures over each other.

Because of all this problems there is no real screenshot this day (with all this shader errors Lost Squadron isn't looking pretty today ^^), but you can see my first shader version of normal mapping on the right. Tomorrow will be the last day I will try out to get this goddamn shaders to work, else I will just leave them out and finish the rest without any high level shader support and maybe add them later ...

Day 7:
Shader test (extra program I wrote), works ok after lot of testing finally, but shader needs lot of improvements to work correctly with spotlights.

Lost Squadron, Day 6: And there were lights.

Thu, 16 Dec 2004 06:24:42 GMT

I completed the landscape rendering class and object loading today and played around with light effects. Directional lights were pretty easy, but very boring for this top-view-camera. So I tried out Point lights and they are nice, but why not test Spot lights? Well, now I know why, they are very hard to maintain. Setting all values properly is not that easy. But I like the effect (you can even rotate the light direction).

Other than that I tried also to implement the ground normal map shader I developed yesterday for the last couple of hours, but I can't get it working. Some mul operation always crashes inside my shader and is very hard to debug. Maybe I need to debug everything with DEBUG_PS and DEBUG_VS. Well, I hope I get that working tomorrow.

Day 6:
A big spot light effect on the landscape. The objects are not final yet and have no effects. But lighting does work well, maybe there is even some time to implement shadow maps later this week. We will see.

Lost Squadron, Day 5: Exciting stuff soon ^^

Wed, 15 Dec 2004 05:23:04 GMT

Day 5:
A boring ground texture. You might guess there is still a lot missing and yeah, you are right! At least a lot of things are happening in the background: 6 different high-detail ground textures with normal map support are implemented into the engine right now, all the map, menu and game base classes are working now. Let's see what will be finished tomorrow. The shaders and the light effects are not implemented yet, so everything is at 100% brightness and normal mapping isn't working yet (but it will soon ^^).

Yep, thats right, I started implementing shader support and programming a normal mapping shader for the ground textures. You might ask, why the hell do you do this for a 14-day project? Well, I'm crazy, that's why :-) I finished up testing some shader effects and writing a fx file with normal maps and multiple lights, but I didn't manage to get that into the engine today. Tomorrow I will try to import some objects, test out all that normal mapping and will play around with lights and light effects (dunno if a night mission with a lot of lights isn't total overkill and if I should stick with a simple directional light and maybe 1-2 point lights for the tanks).

There was also a noticeable delay when loading all ground textures (6 512x512 textures with 6 512x512 normal map textures and some other smaller textures), and thats not nice when debugging and testing out everything every few seconds. So I converted all textures from .jpg, .png, etc. to .dds format and used compressed formats like dxt1 and dxt5. Now the loading is lighting fast and I guess when not using the retail DX9 runtime the delay will not even be noticeable (remember, over 10mb of pixel data are loaded here).

For models I will see how long I can stick with .X files. Usually a format like this will make me angry after using it a couple of days because it does not support some little things and is most likely not extensible and I end up writing my own format (which can't obviously not happen in the short timeframe of this project).

It is also important to use the proper tools for DDS files, normal map generation and writing fx shaders (or other formats, but I like fx), which are:

FxComposer by NVIDIA too for testing out .fx effect files and writing your own
RenderMonkey by ATI is a really cool tool to try out shaders and test stuff, making shaders is not a piece of cake, even with all this tools around there is a lot of testing and modifying going on even if you got some nice shaders working.
NVIDIA_Photoshop_Plugins for saving DDS files and making normal maps from heightmaps or textures
DDSViewer by NVIDIA for viewing DDS as thumbnails, but thats not really helping that much (because windows thumbnails just suck). The DXViewer or so by the DXSDK is also really useless because its so user unfriendly. Also ACDSEE, IrfanView and Google Image viewer (Picasa Photo Organizer) and any other Pic-View program I tried do not support DDS files and don't even have plugins for that. But I found another viewer, which pretty much can't do anything but converting and displaying every format ever invented called: Konvertor. This thing is really crazy, don't ever try to read the full list of supported file formats (its 840 or so). Well, navigating suxx with this tool, but at least you can quickly display DDS files and zoom around. Konvertor has also another really nice feature: When you right-click on a file you will see a small preview, thats really one of the coolest thing I ever saw in a popup-menu :-) Sad thing is: For DDS files this does not work, only .bmp, .jpg, etc. files previews are generated, grr!
There are also a lot of other tools involved when saving stuff in 3D Studio Max, Maya, etc., but I won't get into that. Has nothing to do with programming ^^

I also underestimated all the classes I had to create for just rendering the landscape, there are of course still a lot of things missing as you can see on the screenshot. The basic layout for all the menu and game classes is now more or less complete and I'm pretty happy with it. Everything is still very easy to maintain and I got a good overview of the classes. The game has to be filled up with a lot of objects, roads, enemies, etc. in the next days. Hopefully all of this and the basic game engine and gameplay will be finished by the end of this week, so we have next week to test and balance the game, make some cool missions and bugfix the hell out of it (well, the engine is only a couple of days old now and only tested by me and my unit testing stuff).

Btw: VS2005 will get REALLY slow when you have a lot of files opened and try to compile or work on some older file. I think some part of the auto-recovery of files when crashing is not really optimzied. Anyways, when you close all documents and open only a couple (lets say no more than 10-20), VS will responde quickly again. Little annoying, but I'm now much better of knowing that ^^

I also registered the domain www.LostSquadronGame.com, which will be the official site for Lost Squadron. But when its up and running it will just link to this site until we got some finished page (which is not planed before chrismas, but as I know our graphic guy Leif, he can't help himself and will make a page design when I'm not looking).

Lost Squadron, Day 4: Why can't I just press start and play?

Tue, 14 Dec 2004 02:42:06 GMT

Today there was a lot going on, but I managed to finish up some things. First of all I finished all my UI classes and got finally rid of the DirectX Utility Toolkit classes. Not that they are bad or anything, but they are total overkill for me right now, around 15000 lines are in this utility classes and there is not much fun going on when you want to customize anything there.

Ok, so I just wrote my own UI classes, I only need a very simple support for buttons, textboxes and listboxes, thats it. But I definatly encourage you to take a look at the utility toolkit and use some nice functions there, which help you to initialize DirectX and find the correct Device configuration programmatically, thats good stuff and saves you a lot of testing time :)

Instead of starting with the game engine and rushing into creating some incomplete code for today and leaving the menu unfinished as well, I decided to finish up all the menu stuff. Some base classes were improved, like sound and music playback, and I wrote the complete concept for the editor. I got also another new intern helping me a bit with the editor, thank for your support Fabian.

Day 4:
Another menu screen, graphics are also improved a little, but not finished yet. But why not show them off? :) Also all sub menus are working now, maybe some tweaking here and there, but the menu stuff is now kinda complete.

Lost Squadron, Day 3: No screenshot today.

Mon, 13 Dec 2004 04:59:28 GMT

Well, there is no screenshot today for several reasons. First of all I hadn't much time to program today because I visited my grandparents (brithday and stuff) and then was visiting my parents (all this family crap ^^). And secondly I was developing via remote today and for graphics this can be a pain in the ass because everything is so slow (software rendering with remote desktop suxx (slow and no effects are working) and hardware rendering with vnc will never display whats going on ^^), so I have finished some base classes and all the Sound and Music classes required later on. I also started writing my own UI handling, the DirectX9 utility toolkit seems to be a bit of an overkill for my little game.

I promise, there won't be any more days without screenshots.

Lost Squadron, Day 2: Looking good right now.

Sun, 12 Dec 2004 05:05:30 GMT

I'm back to VS2005, I just can't live with VS2003 after using VS2005 the last couple of months, too much features are missing. Intellisense is just not working, which is very annoying if you don't know anything about the DirectX classes and have to search the stupid managed DirectX help files (takes forever to find anything useful there. Btw: Why is everything "preliminary" there when parts of the docs are already 2-3 years old?)

In VS2005 it's really nice in that respect and has a lot of other useful stuff when writing new code, so I converted the project back to VS2005 (I'm getting an expert in converting stuff from generics back and forth).

The graphics for the controls are still pretty dummy (as you can see in the pic, pretty ugly right now) and the controls are not working yet correctly (oh oh, I'm already behind in schedule). Hopefully tomorrow I can finish loading some 3d objects so I can start with the real gameplay.

Day 2:
Programmed some of the User Interface, texture display and tryed out some effects. I also implemented the game manager, keyboard shortcuts for a lot of stuff (fps, making screenshot, etc.), cursors, the main menu and some animation effects for the menu.

Lost Squadron Part 1: Daily development screens

Sat, 11 Dec 2004 04:51:19 GMT

Day 1:
Finished loading textures, displaying text, game loop, imported lot of useful classes from older projects, tryed out dds and x file formats, etc. but we still got a boring black window. I promise tomorrow there will be something more fancy :-)

Hey now! Today I started the game and engine of a new smaller game project I will hopefully finish till chrismas. Our intern at exDream Ivo will help me out with some models and textures, other than that its only me ^^ The game will be called Lost Squadron and is a kind of an advanced shoot'n'up game, you drive around with a tank (and later on other vehicles) and blow up stuff (this might get you an impression on how I feel right now ^^)

I will use DirectX9 with c# for my first time (used only OpenGL before), but I already know how to program with DirectX (worked with DirectX in my c++ times) and I am pretty aware of some uncool factors when DirectX just returns a stupid error code -420529345 without any reference and no documentation about this (maybe when you search long enough you will find out that this translated to "some error occurred" or "some of the freaking 100 parameters you passed is invalid", like that tells you anything useful).
Don't believe me? Then just try to load a texture or create a empty texture. If any parameter is wrong, the file does not exists or the color of the sky is not an exact RGB value, the app crashes (an exception is thrown with an error message that tells you nothing). DirectX won't tell you whats going on, just that you have to try out for half an hour to find the correct combination (or you might find out that just some file was missing).

I always woundered why nobody bothered to tell Mircosoft that this is bullshit, any robust framework needs to handle that way better, you can't assume everyone passes only 100% correct stuff to you.

Whatever, maybe it's normal to be pissed, when dealing with this again after 2-3 years of OpenGL development (in OpenGL on the other hand you will sometimes not give you any error message at all, which can also be annoying).

To top that of I fought several hours to get DirectX9 native debugging enabled with VS2005, which resulted always in an IDE look up and crashes if any native exceptions occur (if nothing goes wrong or you use the release builds it works, but hell, I want to debug the freaking DirectX stuff). Anyways, I'm back to VS2003 for this project and had also to rewrite all my generics code back to work with VS2003 and .NET 1.1 (well, I had to do that anyways because this game should not require .NET 2.0).
Later I found out that any project (including all the c# tutorial and example files) will crash when quitting in debug version them because some memory is not freed properly. I just deactivated any Direct3D memory exceptions (with DxDiag) and now finally I can work without all this stupid exceptions (maybe it has something to do with bad graphic drivers or a corrupt dx version) ...

Well that aside, DirectX9 has a lot of good stuff. I really like the way you can now implement easily user interface graphics, buttons, textboxes, etc.

This game is not about writing some ugly spagetti code as fast as possible in this 14 days. I will try to produce the base code for my new engine and just develop the game to check out its basic features.
The engine will use a lot of Unit Testing, base classed for all the grunt work and also a lot of error handling and generalization.

I thought about using some existing engine for this smaller project, but I don't the way most engines just focus on first person shooter development and I also want to use my base classes I am comfortable with.