Tag: Software Development

Archeology vs. Engineering: contrasting approaches to long-term maintenance of IT Systems

development, programming
Comments

Thinking about programming and maintaining a system in a team environment, where the system has a long life and the team supporting it experiences turnover.

When I program I know exactly wtf I mean by . I understand its purpose and design, and this makes me fairly confident that I know what is and what it does or is supposed to do. My ability to solve problems when I code from scratch is limited by my ability to understand the problem and to conceive of a solution. Often what I can come up with is inferior to what the greatest minds can come up with, but for many problems I can come up with something acceptable, and code it in such a way that it is very easy to understand the code, because I like to write code that is written in a way that lends itself to understanding.

When I look at someone else’s , I don’t know wtf they meant when they coded . I have all kinds of questions about what was in the programmer’s mind, how they understood the problem, what they handled in and what they elected not to handle in it, and so on. It helps immensely if I know the language and any frameworks or libraries well, but often when you’re inexperienced that’s not the case. And, especially with larger, older things that have been built up over time, that becomes a very steep learning curve. Until I’ve had enough exposure and experience to , I feel very uncomfortable, uncertain, and unconfident that I understand any part of . And, if is sufficiently large, I never get over this, and it ends up limiting and paralyzing me in my efforts to become a better programmer.

If I built it, then I know why, and I can be in a position to know better later when I’ve learned more and maybe decide to change my mind based on some revelation. 

But if someone else coded it, unless they coded it in such a way that the code clearly expresses its intent, and/or they’ve commented it extensively, or documented it somewhere, explaining their rationale in detail, I can only speculate, and depending on whether I feel like I am smarter and more knowledgeable than they are, I might or might not feel comfortable making a change. I would at the very least feel uncomfortable making a change that I could not roll back quickly/easily if something broke, ideally in a test environment where there would not be serious consequences. But I might not feel confident that a change would be instantly and obviously noticeable; often things break in very subtle ways. Having a suite of unit tests is very useful here, but it’s often the case that there are no tests written.

Even when a system has extensive documentation, there’s no guarantee that it is up to date, or accurate, or correct. Other people often have very different ideas about what and how to document, and how much detail to include, and where to put what information. All systems of documentation seem to involve significant tradeoffs, and there is no silver bullet solution to documenting adequately.

I call such situations IT Archeology, rather than IT Engineering. It’s very much like discovering a lost culture’s artifacts and trying to figure out how their civilization must have been structured and how daily life must have been lived, and then trying to adopt those ways and live by them yourself in order to understand them better. By contrast, IT Engineering is what you do when you have a solid foundation of understanding of the problem domain that provides the context that it works within, and knowledge of the system and the technology stack that it is built upon.

At the moment, I am wrestling with how one moves from an Archeology paradigm to an Engineering paradigm. But it’s an observation I’ve noted many times in my experience, and it seems quite common. I am interested in advice from developers who have to deal with this sort of thing about what approaches actually work.

Game Jam postmortem++…

games, programming
Comments

I haven’t looked at my Global Game Jam project since the Jam ended, but I wanted to continue development and turn it into a more finished game. Over the weekend, I started.

Rushing and long hours hurt projects. (Duh.)

My intent was to refactor some things that I knew needed to be, but as I got into it, I ended up noticing more bugs that had escaped me during the Jam. Oh, how embarrassing. I’ll embarrass myself further by dishing the details here.

Bugs

  1. I thought I had fixed the Endian-ness of the Byte in the demo level, but as it turns out I didn’t. I fixed it in one place — the Create event — and forgot to make the same correction in the Step event — where the game spends 99.9999% of its time running. So, effectively I had it fixed only for the first 1/30th of a second of the game, and left it broken otherwise.Oops. See how code duplication can hurt you? If I’d refactored properly as I went along, the code would have existed in only one place, and fixing it the first time really would have fixed it.
  2. The victory condition check seemed to be broken. After I fixed the byte so that it was little-endian, I went back and played the level again, and sure enough found it easier (though it’s still difficult and largely luck-based) to get the Byte to output the ASCII character that matches the Clue. Only, something’s still not right… When I go hit the Executor switch, it’s still telling me they’re not matched. What gives?Well, it turns out the font that I picked, Joystix, renders identical glyphs for capitals and lowercase letters, making it a poor choice for my purpose. I might have noticed this if I’d been paying closer attention. The victory condition check actually does work properly, but since you have no way of discerning whether the clue is a capital or lower case letter, when you think you’ve matched it, you actually have just a 50-50 chance of being right.
  3. For some reason the logic switches sometimes get stuck “off”. This took the longest for me to figure out, because I kept looking in the wrong place. It turned out to be due to an improperly written switch statement which had one unhandled case that I didn’t account for.

Causes

So, these bugs really weren’t all that deep. Once found, the bugs were simple to fix. They were just mistakes that are typical of hastily written code that hasn’t been properly tested. Which isn’t to say that I didn’t do testing, just that I wasn’t able to do proper testing. I’m not sure whether to blame myself, the 48 hour structure of a Game Jam event, or Game Maker.

Well, the first two bugs were all me. They were logic errors on my part. I can blame the sleep deprivation and general rush of the Game Jam for contributing to my errors, but I really can’t have expected any tool I could have used to catch them. When I am exhausted, I make errors like this much more frequently than I normally do, and they become far harder for me to catch when I do make them.

I think this is a valid criticism of the Game Jam concept, but it’s also necessary to point out that Game Jam’s primary focus is not on producing quality code, or even quality games. So, to a certain extent, while it’s a valid point to make that building software this way isn’t the best way to do it, that doesn’t really diminish what is good about participating in a Game Jam. It’s foremost about experiencing the fun of building something cool. It’s also about showing that it’s possible to iterate very rapidly in game development. It’s about creativity and learning how to work effectively with small teams and limited resources. While code quality is important, and a development team could choose to make it a focus, it’s clearly an optional part of the weekend. Still, the better projects are more than likely going to be built with better quality code.

The third bug… well, ultimately, it was a user error with how I wrote my switch cases, but the lack of good debug tools in Game Maker do make it harder to find and fix errors. Game Maker really does not have proper testing and debugging features, apart from some very rudimentary variable tracing, and aborting and logging runtime errors, that’s all you get.

Prevention?

For the most part, the way you test in Game Maker is to run the project and try to set up the circumstances that you want to test, and watch closely as it executes and if something unexpected happens, try to infer from the behavior what is happening. If the program doesn’t crash outright when a bug is executed, it’s basically a perception check on the part of the tester to see if they bug is blatant enough to be noticed.

Subtle bugs, or rare occurrences, can be very difficult to detect. On top of that, it’s rather time consuming. And when you fix it, how you confirm that it’s fixed is more of the same trial-error process.

I’m not sure whether it’s possible to do Test Driven Development in the Game Maker environment. It might be, after a fashion. Like, there’s no way to set up a separate test suite project and rapidly have all events pass. But you could implement some functions designed to test your code, and then run them by sticking them into a special test room. It is possible to build yourself a “test room” where your test cases are set up in some defined, controlled manner and you can check each time you build to see if your latest changes to the project have broken any existing functionality. In fact, a good test room is a must with any complex game. But even then the test room will only help you to notice that a test fails, not tell you why it failed.

But even then, you’re probably not going to have total coverage. In some cases, a lot of of a game program’s behavior is actually emergent, rather than explicitly programmed, and I’m not sure how you could write a test routine for something like that.

Can bugs be good for the game development process?

Even if it is possible to reduce defects by using Test Driven Development in Game Maker, I’m not sure how feasible it is. Games are very complicated, and ultimately the real test of a game is how it plays, so to some extent all that time-consuming trial-and-error testing serves a purpose. It’s unlikely that any number of test rooms that you can build are going to be able to set up every single permutation that could possibly come to pass at runtime.

It’s easy to design certain aspects of game software, but designing “fun” into the game is not so easy. Often, what is fun is what surprises us, and what surprises us more than a program behaving in an unexpected manner? I’m sure countless times in the annals of game development, some unexpected program behavior was embraced as a feature or enhancement rather than classified as a bug and removed from the program.

Sometimes, Bug = serendipity

A good example of that from ASCIIboros is the way the player’s bit re-randomizes when you land from a stationary jump. The idea that the player even should carry a bit value around with him was something that came to me fairly late in the weekend. At some point, I realized that most of my bitwise operations required there to be some other value, since you can’t very well do Or, Xor, And, Add, or Sub with just a single value.

And where would that come from? Well, the easiest and most obvious thing to do was to make it a value that the player carries with them. I quickly implemented this, simply assigning a bit value randomly when the Player object is created.

I didn’t think about how the Player would change its bit value at first, but before it really even hit me that this would be useful and even necessary, the solution to this problem presented itself by way of a “bug” that I chose to adopt as a designed feature: When you jump vertically, upon landing the Player’s State Machine re-calls the bit assignment routine, resulting in a re-randomized value. Problem solved. Only, it was never a part of any design process.

I’m not saying that the bug was creative. But encountering this bug presented an opportunity which my creativity recognized, and found useful. Probably 99% of the bugs I fix don’t do this, but once in a while, they do, and sometimes they can be better ideas than the ones you I of deliberately. Now that I’ve adopted it and included it into my design, it’s no longer a bug.

Now, not all types of software projects want to have these kind of “accidental” design innovations come up during development. With game design, I imagine it’s much more allowable, since the game really doesn’t have to meet any specific real-world requirements; a game program’s output is the experience you have when you play the game, and as long as it’s more fun, that’s all that matters.

Of course you were writing business software, getting the output right is a lot more important, and probably needs to be defined in the designed much more carefully and precisely. It’s hard to imagine serendipity making payroll processing better, for example. Even so, if you’re too rigid with requirements and design, I think you’re closing yourself off to possibilities which are worth exploring when it comes time to muck about with code.

There are times when you can know ahead of time exactly what you want the program to do, and there are no questions that need to be answered during programming, or which you couldn’t have thought of ahead of time if you were just really, really smart. But perhaps those times are not as frequent as we might think. Particularly if “we” are Project Managers or Business Analysts.

Programmers are more than just adept at manipulating symbols to perform arcane calculations with mathematical precision. The good ones are often very creative thinkers in their own right. It’s important to recognize those types of developers and allow them freedom and to incorporate their creative process into your project management style.

And it’s not enough to include your programmers in the design phase of your development cycle (which you should be doing anyway). Sometimes design ideas present themselves during the process of building; we should be open to embracing good ideas whenever they present themselves. I’m not saying that all design innovations are going to be discovered through accidental defects in coding, either — oftentimes programmers will have good ideas come to them simply because they’re playing around with pieces of a project, and it’s a lot more stimulating to be playing with building blocks than it is to be looking at a blank design template.

If you haven’t played it yet, now’s as good a time as any to grab the ASCIIboros Demo from the Releases page and try it out. Windows only, sorry:/

Download ASCIIboros

About my process

programming
2 Comments

One thing that I think I am starting to realize about my programming process is how important it is to me to be able to complete a task in a single session.

Each evening when I sit down to work on my game project, I don’t usually have a specific idea about exactly what I want to work on or accomplish. I do have a lot of general ideas. I have a list of things, a backlog of features that I have brainstormed but have yet to implement, or have yet to implement to my complete satisfaction, and a rough idea of the priority I’d like to give on each item in that list. When I sit down to being my development session, I review the list and pick something. As I continue to develop, I continually update the list, usually removing things from it that I’ve completed (it makes me happy/inspired to see the list shrink), but sometimes adding new things if an idea comes to me or I realize a new problem has come up as a result of something I’ve been working on.

I am not strict about working on items in priority order. To some extent I am taking things in order of importance, but mostly I try to work on the most important thing that is also easy to accomplish quickly — the so-called low-hanging fruit type stuff that is easy to do and provides a high value to the project. To a larger extent, I work on things in the order that I can work on them — some things are dependent upon other things and I have to do those things first in order to be able to do the next thing. Or sometimes I need to learn more about something in order to do it, and doing something else first will give me the opportunity to start to get into that topic in a more lightweight and manageable fashion.

Sometimes I just work on something that I happen to be interested in at the moment — it is important to recognize the power that having an interest in something has. If you’re interested in something, engaging with it is easy and natural. If you’re not interested in something, it’s hard. If you’re actually interested in something else, it can be all but impossible.

Having a sense of progress and momentum is extremely important to maintaining my morale and motivation.

I like to work in long, mostly uninterrupted sessions. I’ll start around 6-7pm and stop around midnight-1am. I don’t just work on the project during this session, though — I read my usual websites, do email, make dinner, chat with friends over IM. I find that this does not interfere or impede my progress or ability to focus, and any of these “distractions” makes for a good punctuation mark that allows me to take a breather and come back to the project with renewed energy.

I don’t even spend most of my project time developing and coding. The coding part of the work just doesn’t take that much time. I spend more time building and playing, then going back and tweaking something, building, adding a tiny little bit, and replaying, again and again, until I get what I wanted, or, often, figure out what I wanted. A lot of times, an idea I had at first is not really as good as I thought once I have built it, but something else that is close to that is better. By carefully testing each little change iteratively as I build, I ensure that I move in a steady direction toward a successful design. This is much better than trying to design everything out on paper before I put any time into building it, by far.

When I first sit down to work for the evening, I have in mind a goal to improve some aspect of it, and by the time I am ready to put the project down again, leave the game in a state where it is both playable and improved from the last iteration. I sit down and look at my list, mull things over for a bit, do a little bit of planning and thinking, and then start building. As I build, I document what I’m doing.

The first thing I do with each session is to create a new version of my project. I don’t stop working until I’m satisfied that what I built works the way I wanted it to, and results in a better game than what I started with.

I’m not sure how sustainable this is, this approach of having tiny tasks and completing them in one session, but as much as possible, I want it to be. I find that I am far more productive in my project as I am working if I can complete* some aspect in one sitting, as it were. If I don’t make it, I run the risk of losing my thought process and not being able to pick up where I left off. I think of it like writing a book — you don’t want to just cut off in mid sentence. And ideally, you’d like to be able to complete a chapter, or at least a thought.

*When I say “complete”, I don’t mean “final”. I will likely go back again and make further refinements when the time is right. For example, when I started implementing my enemy objects, at first I just placed them into the game as immobile objects. I next focused on interactions with the player object. In a later session I added very basic movement, and spent a lot of time tweaking that. In upcoming sessions I plan to further refine enemy movement, adding different modes and coming up with some AI routines that determine how the enemy “decides” what mode to be in depending on what’s going on in the game.

My guess is that if you architect your program the right way, you should always, or nearly always be able to use this approach of tiny steps over many iterations. To extend the architecture metaphor a bit, if you’re laying bricks, to build a bigger building you just use more bricks — you don’t suddenly have to switch to bigger bricks that become unmanageable for one person to wrangle. That may not always hold true, but my feeling is that it should most of the time.