Archeology vs. Engineering: contrasting approaches to long-term maintenance of IT Systems

Thinking about programming and maintaining a system in a team environment, where the system has a long life and the team supporting it experiences turnover.

When I program I know exactly wtf I mean by . I understand its purpose and design, and this makes me fairly confident that I know what is and what it does or is supposed to do. My ability to solve problems when I code from scratch is limited by my ability to understand the problem and to conceive of a solution. Often what I can come up with is inferior to what the greatest minds can come up with, but for many problems I can comeĀ up with something acceptable, and code it in such a way that it is very easy to understand the code, because I like to write code that is written in a way that lends itself to understanding.

When I look at someone else’s , I don’t know wtf they meant when they coded . I have all kinds of questions about what was in the programmer’s mind, how they understood the problem, what they handled in and what they elected not to handle in it, and so on. It helps immensely if I know the language and any frameworks or libraries well, but often when you’re inexperienced that’s not the case. And, especially with larger, older things that have been built up over time, that becomes a very steep learning curve. Until I’ve had enough exposure and experience to , I feel very uncomfortable, uncertain, and unconfident that I understand any part of . And, if is sufficiently large, I never get over this, and it ends up limiting and paralyzing me in my efforts to become a better programmer.

If I built it, then I know why, and I can be in a position to know better later when I’ve learned more and maybe decide to change my mind based on some revelation.Ā 

But if someone else coded it, unless they coded it in such a way that the code clearly expresses its intent, and/or they’ve commented it extensively, or documented it somewhere, explaining their rationale in detail, I can only speculate, and depending on whether I feel like I am smarter and more knowledgeable than they are, I might or might not feel comfortable making a change. I would at the very least feel uncomfortable making a change that I could not roll back quickly/easily if something broke, ideally in a test environment where there would not be serious consequences. But I might not feel confident that a change would be instantly and obviously noticeable; often things break in very subtle ways. Having a suite of unit tests is very useful here, but it’s often the case that there are no tests written.

Even when a system has extensive documentation, there’s no guarantee that it is up to date, or accurate, or correct. Other people often have very different ideas about what and how to document, and how much detail to include, and where to put what information. All systems of documentation seem to involve significant tradeoffs, and there is no silver bullet solution to documenting adequately.

I call such situations IT Archeology, rather than IT Engineering. It’s very much like discovering a lost culture’s artifacts and trying to figure out how their civilization must have been structured and how daily life must have been lived, and then trying to adopt those ways and live by them yourself in order to understand them better. By contrast, IT Engineering is what you do when you have a solid foundation of understanding of the problem domain that provides the context that it works within, and knowledge of the system and the technology stack that it is built upon.

At the moment, I am wrestling with how one moves from an Archeology paradigm to an Engineering paradigm. But it’s an observation I’ve noted many times in my experience, and it seems quite common. I am interested in advice from developers who have to deal with this sort of thing about what approaches actually work.

Leave a Reply